For about the first week after meeting them - physical appearance.
Beyond that - intelligence and personality (though physical appearance is still nice )

Both are important, but you're nuts if you say physical appearance isn't a driving factor in asking someone out the first time. Well, I guess if you're dating a long-term friend, but that's not generally a good idea anyway.