Check to see if the Framerate counter is showing in the corner (F12 by default I think). Generally if it's visible, you can take screenshots and record movies. Sometimes the counter won't show up, and you might not be able to capture anything. If the screenshots can be grabbed in fullscreen mode, then I see little reason why in windowed mode you can't do the same thing. Is there perchance a conflict in buttons between FRAPS and the software in question? If so, change the button used to grab screenshots/movies in fraps and try again.