I want to toy heavilly with this concept on my game. I'm wondering if that's ok.
It's not like the game is  intended for audiences of a different language.  It's a very deliberate choice, and the parts of the game that do not toy with language that way will be in english either way.

I think as long as the overall important things that need to be known are in english or obviously telegraphed, like things for progression or how to use controls it might be fine. At the very least its throughpoint can be understood even with your gimmick so we know how to judge it in full.