We ran into a couple of really difficult bugs with colliders because of the way we set up object interactions. However, we think we've solved all the issues and we're pretty happy with the current game state. So we've decided as is