I thought that was default. Oh well.
In the options, go to "Display" where you'll find the Window Mode setting. Switch to fullscreen from there.
