I think you understood the core of the feedback very well: the mouse has more than one utility and they can be at odds with each other. If the mouse were primarily for drawing sigils, and had no other utility, it would be cleaner. You would have to make your own choices about how to achieve the other functions.