I've played through most of the factions and I've decided that Portugal and Sicily provide the most challenge for the early-to-mid game. Money is scarce, the kingdoms consist of far flung provinces and they're surrounded by enemies.

What factions does everyone else think are the most difficult?