Uhhh... Late Byz are possibly the hardest you can get.
No up to date cav
No up to date infantry
No anti-cav worth mentioning
No friends...
And lastly the deadly Turks and their Jannisaries to the east.

Yup, they are tough

The only other faction I can see as being as hard must be the Late Almohads.