Okay, it would be nice to hear some official word from CA or the Steam folks about whether it's a resident process when the game isn't being played.
Steam will remain running once you close down the game unless you exit it yourself. But it's easy to close down as it will be on the taskbar.

Is there an option for right-clicking the Steam icon in the taskbar and telling it not to load on every system boot? Do I have to do that every time Steam updates itself again, like Quicktime?
You can set Steam to not load on system start easily, and it remembers that setting every time it updates.