I thought I had that issue at one point. Just keep working on all the girls' quests and you'll make progress
Got it! totally forgot about the part of the map beyond the Graveyard
