In my tests phalanx without guard mode always lost against the same unit with guard mode because it kept pushing forward into the pikes and suffered much more casualties. I would use guard mode off when facing non-phalanx units or when attacking the flanks or rear of an enemy phalanx.

Another observation was with formation: useing a square (15 deep on hughe size) instead of a line (6 deep) always resutled in a loss in custom battle (1 vs. 1 with identical phalanxes). That was due to the fact that the AI phalanx managed to get a couple of units behind my phalanx; what looked very odd BTW. I had then given Thorakitai Phalanx long pikes and the phalanx ability and used them in a square (10 deep) against various successor phalanxes. This time the result was as expected: the Thorakitai was able to push thorugh the center of the enemy phalanx very fast. I had then disbanded phalanx and guard mode and ordered them to mob up the enemy with swords. This worked very well against low and medium enemy phalanx.