Back when I had a crappy XFX NVidia card I got similar troubles, but never since I went over to an ATI one.

I have a 4600+ Athlon with 3GB RAM and a Radeon X1900 Crossfire, so I'm probably not the best one to compare specs with...