I'm just playing the browser version. But I figured it out so its all good. 

Btw, you might want to explain somehow that servants can't be in your party if you want to assign them. Either that or make it so that they automatically get kicked out of the party if you assign them.

Love the game btw, strikes a good balance between fun and hard.