The turks are on of those factions that do not require castles to field decent armies since you can hire your horse archers from cities. I personally wouldn't try to go for all of africa and would stop once i get to the egyptian provinces since the cities are really far apart and not extremely useful. It would be better to expand in two directions at once with one army southwards and one westwards. You can wipe out the Byzantines fairly quickly if you call a jihad at a good time and get multiple jihad armies.