On foot or by foot.. I wouldn't have been able to tell you why that's what is it though..

Just another part of the quirky English language