Wow this is a really clever mechanic! It feels like some sort of lost classic game. I'd love to see it fleshed out :D
