Actually the Fall of Constantinople was a lengthy process. The fall started along with 1204. That really marked a turning point.Slowly, the Turks gained territory as the emperors neglected the middle class, turning their attention to Stratiots or Pronoias.
Edyz, as far as I understood you your point is that Byzantium went into a permanent decline since 1204. However some Nicaean emperors like John III Duca Vatatzes ran very successful policy that made Nicaea stable. At that time the Turkish threat was ellimainated because of the system of border forts and special kind of units Akritois. This emperor tolerated the minor pronoais who were the soldiers of the empire. The decline started when Michael Palaelogus usurped the throne. He was the leader of the major nobility and as a result he tolerated them too much. That was the beginning of the end because he made the Akritois and minor nobility vulnerable to the attempts of the major aristocracy to take their lands. So IMHO the turning point was the reign of Michael VIII Palaelogus.