I actually did that by mistake. The default EB icon comes with a command line already in it, I forgot what exactly, but if you haven't deleted it it should still work. I think all you have to do now is set the resolution to 800x600 and instead of showing it fullscreen with that setting, it'll be in window mode.