Yea, the buttons wouldn't be clickable, but I'm pretty sure you could read arrow keys & enter from stdin- so it'd need to be keyboard controlled.
But it would be really fun to have as a lil' terminal applet. :P