Yup right now that is normal although in the next update since there will be more to do at each location that should be changing. In the next update, time will move forward only if you interact with one of the characters or do any of the location specific actions like working at the cafe.