If you want to see some real History visit the Fort of Eger. (about 1-2 hours from Budapest by train).

Budapest has also got some places worth visiting (although its a little cramped).
You should visit Ópusztaszer - its near Szeged on the southern Borders - accoding to Legend this was the place the Magyar tribes made there first Capital - supposedly on the same site Attila the Hun had his Capital while the Huns used the Carpathian basin as a base of operation. (Some legends claim Attila was buried not far from there in the River Tisza - but there is no way tp prove it).
The famous "Feszty - Körkép" - is also there. Its original title (A magyarok bejövetele) can be translated as "the coming of the magyars" or something like that. It depicts the arrival of the Magyar tribes in the Carpathian basin under the Leadership of Árpád, Grand Prince of the Magyars (more info: http://en.wikipedia.org/wiki/%C3%81rp%C3%A1d)

There are many beautiful cities aswell like Kecskemét, Debrecen, Sopron (the city of loyalty) and many more.

If you need more information just Pm me.