The origins of the idea of the 'Middle Ages' can be traced back to the Renaissance scholars (although the actual words 'medieval' and Middle Ages date to a bit later). The scholars of the Italian renaissance saw everything that wasn't classical or consciously imitating classical forms as backwards, and their own age as reviving classical culture. Hence anything not classical or neo-classical was bad. In some ways, they made some good points; in other respects, they were just letting their own prejudices get in the way. Take 'Gothic' architecture, for example. To the scholars of the renaissance, it was barbaric, and so had to be denigrated and named after a 'barbaric' Germanic tribe. But of course now we can see that Gothic was one of the great styles of architecture in any historical period.

Now, there is no denying the Middle Ages was 'backwards' in some ways. In terms of military training, Roman troops were better trained because they could live collectively and train in common. Few medieval troops could do this. On the other hand, the Middle Ages developed technologies that helped make up for such deficits. Medieval cavalry had better saddles, horseshoes, bigger horses and stirrups. So which is more 'backwards', the Roman or the medieval army? To me, its a debatable point.

Was the Middle Ages 'backwards' economically? At the beginning of the period, yes: economies had turned inwards, becoming local, and trade declined. However, at the end of the Middle Ages one could make the argument that the Middle Ages had made more advances than Rome ever did. There was a burgeoning middle class (something that Rome never really had) of traders, merchants and artisans; thriving fairs and international trade; new techniques of credit and banking, new alliances between government and capital, etc. etc.

Politicall, Europe broke up after the fall of Rome. so looking at a map, it seems Europe declined. But of course, the fall of Rome also meant more competition, more experimentation. Europe was far more diverse politically at the end of the Middle Ages than it was in late antiquity: you had everything from absolute monarchies to oligarchies to republics and communes. Also, whereas the Romans gave up their republic for the security only despotism could provide, medieval people developed the parliamentary system that we hold so near and dear today.

If the Middle Ages was backwards, then so are we.