No, I don't think the button confused me, just I would have gone faster if I knew I was being timed (I really took my time as-is). I would personally still lean toward an in-game timer, but I also think it's fine as-is. I like your idea of having it as a menu option; Celeste, for example, has the timer off by default, and it's never a bad thing to borrow ideas from Celeste :)
The falling blocks didn't get me, but they almost did; that's a place where a little extra sfx and/or a slightly bigger visual cue could signal the danger. For me, the wobble was enough for me to jump back to the side. But I could also imagine if you land on them at the wrong time, even if you notice the wobble, you might be frozen in place and doomed to fall. Maybe making them visually distinct even before landing on them would help that? Just a little signal to the player that "hey, something's different about these" might be enough to encourage caution on that first playthrough.