There are some ways, either give him a beer in the Entrance at fridays 3 times until you confess what you've been doing with Anna, or have sex with Isabella and after it go tell Josef that you did it, you'll have a threesome with him and Isabella, after that you can tell him to confess to Anna and he'll leave, those are the two ways I know you can make him leave