A fort garrisoned by a military unit has a 1 square zone of control, just like an army does. They can march past it, but only in a certain manner. If you place it in a good choke point, the enemy can't go around without hitting that ZOC. At that point they stop. The next turn, they can move normally, but if they enter another square of the ZOC they stop again. In a 3 square wide pass, it will take them 4 turns to bypass the fort, 1 to move next to it, 2 more turns to move 1 square at a time past it, then the final turn to escape the ZOC and keep moving. You'll have bought time this way. If they attack it, they have to lay siege to the fort and either assault it after building rams (assuming they have no artillery) or starve it out, which takes 3 turns.

If you use a 1 unit army, you have to worry about it revolting, plus they can just slam into it and either make you retreat or slaughter it. Neither one slows them down.