Interesting idea, this is original and with clever use of those few mechanics.
It does seem a bit rough indeed, the art is all over the shop and it cheapens the experience a bit.
There's a good nugget of a game in there, keep up the good work!