There is story significance later on. But if the minigame frustrates you, you can always turn it off for a more classical VN choice experience.
THAT is the minigame?? I left the minigame option on in the settings cause I didn't think "percent chance of dying" could possibly be considered a "minigame".
