You might be touching a wall. When the swimmer is touching a wall they can not drift down but they also can't swim up so all they do is float until struck by an undead.
Alternatively you might be clicking too often, as trying to spam will result in a penalty that slows you down.
Thank you for playing, I'm glad you liked it.