It sounds as though some sort of process was trying to take over. Maybe your AV software decided to do a scan? Usually an unexplained system-wide slowdown is due to something like that.

If you're really stumped, you could always go into task manager the next time you see this behavior, and check what's taking your processor's attention.