Mmm the controls are very tricky, maybe simplifying them, adding more feedback to the user and adjust the hitbox of the interactable objects it would help a lot.
I like the concept and after discovering how the controls works, I was able to enjoy it :D