Org Mobile Site
Originally Posted by :
"Balancing" with autocalc strikes me as a bit of an oxymoron. A game "balanced" on autocalc is not going to be a balanced game because a decent player tends to achieve significantly better results than autocalc.
This might also help explain the AI's under defending cities and penchant for futile attacks. Those are probably wins on autocalc.
Partly, but the Auto-Calc engines AI should be equally stupid on both sides so it's a level playing feild. Thus in theory all the have to do is create a better battle map AI and check their is nothing the player can "exploit" based on the comments about animations having no effect in RTW I suspect that they didn't expect any problems beyond that. Thats still a big job, but it's much less than getting the basic balance right. Using Auto-Calc for that allows you to do it much faster, and with a greater amount of testing as with a simple auto graber program to record the resuls you can program 50 diffrent army combinations and test each a 1000 times in a matter of a couple of hours, somthing that would be nearly impossibile with Humans."Balancing" with autocalc strikes me as a bit of an oxymoron. A game "balanced" on autocalc is not going to be a balanced game because a decent player tends to achieve significantly better results than autocalc.
This might also help explain the AI's under defending cities and penchant for futile attacks. Those are probably wins on autocalc.
If you want an anolagy:
Somone is making a big metal item, they have 2 ways of making it, eithier do it by hand in a blacksmith, or cast it and then polish it to specification.
The Casting (auto-calc balancing), is reletivlly quick and can be reproduced on-mass. It then needs polishing, (the Graphical testing for exploits and to get a good AI going).
The Hand Crafting in the Blacksmith, (test it with humans throughout), has the advantages of being ore precice than basic casting and polishing, but only slightly and it takes a LOT more time and efort.
It's my belif that SEGA forced CA to rush it out the door so fast they never got the polsihing even started on really.
Originally Posted by :
It would have taken one developer like 15 minutes to retry a battle 10 times to see that yes, the uber peasant exists. Yes, most of the 2H units does something wrong and it needs to be looked at etc.
Much of this isn't even the deep down stuff, but stuff that is obviously bugged. After just a few hours the player notices odd stuff like peasants and artillery crew being very very tough, 2 handers are acting strange, the broken AI etc. It took some time before the shield-bug was properly named, but we all knew SOMETHING was wrong with the combat calculations.
As you note, it took a while to figure out what was wrong, it dosen't matter if you know somthing is wrong, if you don't know what the problem is you can't fix it, and CA's reaction to the sheild bug discoveriy leads me to belive CA didn't know what the problem was even then.It would have taken one developer like 15 minutes to retry a battle 10 times to see that yes, the uber peasant exists. Yes, most of the 2H units does something wrong and it needs to be looked at etc.
Much of this isn't even the deep down stuff, but stuff that is obviously bugged. After just a few hours the player notices odd stuff like peasants and artillery crew being very very tough, 2 handers are acting strange, the broken AI etc. It took some time before the shield-bug was properly named, but we all knew SOMETHING was wrong with the combat calculations.
I agree they should have snuck some graphical engine testing in amongst the auto-calc testing to check auto-calc and the graphical engine where matching up, but in the end the real blame, (but not necesserilly full blame), lies with SEGA for deciding to publish the game when they did, CA didn't have any control over that.
P.S. PAY ATTENTION TO LUSTED
.Originally Posted by :
It's my belif that SEGA forced CA to rush it out the door so fast they never got the polsihing even started on really.
Reminds me of RTw(though that was published by Activision). If anyone dares claim thaty RTw 1.0 was polished i will hit them. It's not, it was at least 1.3 before RTW could be considered polished.It's my belif that SEGA forced CA to rush it out the door so fast they never got the polsihing even started on really.
Originally Posted by Lusted:
Reminds me of RTw(though that was published by Activision). If anyone dares claim thaty RTw 1.0 was polished i will hit them. It's not, it was at least 1.3 before RTW could be considered polished.
So maybe things have improved a little then: from what I'm hearing, M2TW might resemble polished with the 1.2 release. At the very least I think Lusted has been more than clear that we expect the most game-breaking bugs will be gone, and the game without them is fairly close to polished at that point IMO. I think everyone will admit that it has the look and feel of a polished product, until something absolutely broken smacks you up side the head and forcefully removes you from the happy land of all things polished. The game primarily without those things, then, ought to be an immersing and thoroughly enjoyable experience. Perhaps I'm anticipating a little bit too much, but what can I say, I've got a sunshiny disposition at the moment Reminds me of RTw(though that was published by Activision). If anyone dares claim thaty RTw 1.0 was polished i will hit them. It's not, it was at least 1.3 before RTW could be considered polished.
Originally Posted by Carl:
Thats still a big job, but it's much less than getting the basic balance right. Using Auto-Calc for that allows you to do it much faster, and with a greater amount of testing as with a simple auto graber program to record the resuls you can program 50 diffrent army combinations and test each a 1000 times in a matter of a couple of hours, somthing that would be nearly impossibile with Humans.
Thats still a big job, but it's much less than getting the basic balance right. Using Auto-Calc for that allows you to do it much faster, and with a greater amount of testing as with a simple auto graber program to record the resuls you can program 50 diffrent army combinations and test each a 1000 times in a matter of a couple of hours, somthing that would be nearly impossibile with Humans.
The presence of the Elephant Artillery crash probably means the whole Timurid invasion received zero playtest as the player would play it. I can't imagine this is "best practice".
In the old days, playtesters would literally be charged with finding creative ways to get the software to produce weird outcomes.
If you aren't playing the game as a user will play it and actively looking for issues, you are inviting the release of a buggy product.
Originally Posted by :
It's my belif that SEGA forced CA to rush it out the door so fast they never got the polsihing even started on really.
Well, there is always going to be pressure of this sort. Sega or no-Sega. It would seem there is some level of balancing that is optimal pre-release. Some amount that gets the game good enough to not cost sales while getting it to the market on a timely basis.It's my belif that SEGA forced CA to rush it out the door so fast they never got the polsihing even started on really.
I think the game is a lot of fun as it is, but there's no question it got out the door with some big issues.
Originally Posted by :
The presence of the Elephant Artillery crash probably means the whole Timurid invasion received zero playtest as the player would play it. I can't imagine this is "best practice".
In the old days, playtesters would literally be charged with finding creative ways to get the software to produce weird outcomes.
If you aren't playing the game as a user will play it and actively looking for issues, you are inviting the release of a buggy product.
.
Thats my point Dismal though. They should have and probably DID intend to do that kind of playtesting before release, but never got that far before SEGA decided to publish the game. Had SEGA waited 3 months i'm sure this kind of thing would have been caught, the trouble was it got rushed out before they got more than the most basic testing done with real players.The presence of the Elephant Artillery crash probably means the whole Timurid invasion received zero playtest as the player would play it. I can't imagine this is "best practice".
In the old days, playtesters would literally be charged with finding creative ways to get the software to produce weird outcomes.
If you aren't playing the game as a user will play it and actively looking for issues, you are inviting the release of a buggy product.
.
Remeber, they obviouslly belived that the Auto-Calc engine was accuratlly representing what was going on in game. All they had left to do, (or so they thought), by the time they finish auto-calc testing is to check that their are no exploits like the Pike one, (where you order your cav to run through the pikes instead of charging them so that Cav now beat Pikes), Other than that i think they belived Auto-Calc and The player usage would produce nearly the same result.
Naive perhaps, but tottally understandable at the same time.
Originally Posted by Carl:
They should have and probably DID intend to do that kind of playtesting before release, but never got that far before SEGA decided to publish the game. Had SEGA waited 3 months i'm sure this kind of thing would have been caught, the trouble was it got rushed out before they got more than the most basic testing done with real players.
To me this sounds like BS. If CA wanted that level of playtesting and debugging they would have done so. CA knew the deadline SEGA set for them when they started on the project. That's when is decided how much time implementing, testing, debugging, playtesting, ... gets. If they needed 3 more months to get the playtesting done, they should hire a new projectmanager imho.They should have and probably DID intend to do that kind of playtesting before release, but never got that far before SEGA decided to publish the game. Had SEGA waited 3 months i'm sure this kind of thing would have been caught, the trouble was it got rushed out before they got more than the most basic testing done with real players.
Edit: to calrify: SEGA didn't bring the release date any closer at any point so that there was no time to (play)test, CA let the development stage run too long and thus didn't have enough time for testing.
The shield bug also suggests to me that CA's internal or Sega's QA didn't do sufficient "unit testing". By this i mean take a finished piece of code and input all sorts of regular and irregular numbers and see if they produce the right result. It's usually a basic part of (internal) testing. Things like the shield bug shouldn't be found in playtesting but while testing the code.
On the other hand, they must have so much code that it may be impossible to find all those little coding bugs, no matter how good their testing routines are. Only someone with enough experience in testing such a huge project could give a definite answer on that, and i have to admit that's not me.
Originally Posted by :
To me this sounds like BS. If CA wanted that level of playtesting and debugging they would have done so. CA knew the deadline SEGA set for them when they started on the project. That's when is decided how much time implementing, testing, debugging, playtesting, ... gets. If they needed 3 more months to get the playtesting done, they should hire a new projectmanager imho.
Dosen't matter, If SEGA didn't give them enough time to get the game done in, (and if they had they would have released a game with all those bugs taken out)., then it would have been done. SEGA didn't, it wasn't done and they still decided to release anyway. If SEGA actualy cared about it's custopmer they wouldn't have released it till testing was complete.To me this sounds like BS. If CA wanted that level of playtesting and debugging they would have done so. CA knew the deadline SEGA set for them when they started on the project. That's when is decided how much time implementing, testing, debugging, playtesting, ... gets. If they needed 3 more months to get the playtesting done, they should hire a new projectmanager imho.
As to managment, we don't know why they didn't hit the deadline, based on what i know about bussiness practises, it's my belief SEGA gave CA the absolute minimum time necessery, which means even 1 minor delay would mean they'd never make the target. And in my expiriance NOTHING ever goes without some delays.
And thats my point, regardles of what CA should or should not have done, SEGA shoul;d never have released the game when even CA belived their was somthing wrong with the balance and that their where bugs.
Originally Posted by :
On the other hand, they must have so much code that it may be impossible to find all those little coding bugs, no matter how good their testing routines are. Only someone with enough experience in testing such a huge project could give a definite answer on that, and i have to admit that's not me.
At least your being fair here (IMHO).On the other hand, they must have so much code that it may be impossible to find all those little coding bugs, no matter how good their testing routines are. Only someone with enough experience in testing such a huge project could give a definite answer on that, and i have to admit that's not me.
Intresting Data here for you guys, i did some sums to figure out the time and cost diffrances between Auto-Calc Testing and Human Beta testing to get a given amount of testing done.
To do a weeks worth of continus Auto-Calc testing by 1 PC, (8.6 million tests), would take 100 Beta Testers worjking 24 hours a day, 1 YEAR to complete. At the the British natinal minimum Wage it would cost CA approximetly 0.4 million pounds in beta tester wages too. Add in costs for electricity for the PC's, the PC's themselves, the cost of extra managment to recruit and manage Beta testers and the cost of employing the rest of the CA team for a year, and your probably looking at over 10 million pounds more in development costs. That would Jack up the price of games, (assuming they assumed 1 million copies would be sold), by some 10 pounds per copy.
They probably did a lot more, i'd bet at least 10 PC's doing a months worth of testing. hat would be impossibile with even 10,000 Beta Testers (42 million to employ that many too).
I stand by my concept of bringing the community of this game in to beta test by selling it early to them. It wouldn't cost much, it might even make a little money for itself. And nothing beats thousands of people who weren't part of the project banging away on it to find issues.
There is often a systemic flaw with in house testing of any kind, in any business, of the testers "knowing" how something "should" be used and thus not trying the things that would be out of bounds for the system. Yes, sometimes monkeys make the best testers. LOL.
I just beat the Timurids with the Portuguese, inflicting around 10-1 losses on their elephant army. The game crashed right after. Maybe I did something unexpected... or maybe it was a memory leak because I left the game on all night? Who knows. I'll see if I can post the screenshot later.
There is often a systemic flaw with in house testing of any kind, in any business, of the testers "knowing" how something "should" be used and thus not trying the things that would be out of bounds for the system. Yes, sometimes monkeys make the best testers. LOL.
I just beat the Timurids with the Portuguese, inflicting around 10-1 losses on their elephant army. The game crashed right after. Maybe I did something unexpected... or maybe it was a memory leak because I left the game on all night? Who knows. I'll see if I can post the screenshot later.
Originally Posted by Carl:
Dosen't matter, If SEGA didn't give them enough time to get the game done in, (and if they had they would have released a game with all those bugs taken out)., then it would have been done. SEGA didn't, it wasn't done and they still decided to release anyway. If SEGA actualy cared about it's custopmer they wouldn't have released it till testing was complete.
As to managment, we don't know why they didn't hit the deadline, based on what i know about bussiness practises, it's my belief SEGA gave CA the absolute minimum time necessery, which means even 1 minor delay would mean they'd never make the target. And in my expiriance NOTHING ever goes without some delays.
And thats my point, regardles of what CA should or should not have done, SEGA shoul;d never have released the game when even CA belived their was somthing wrong with the balance and that their where bugs.
At least your being fair here (IMHO).
Intresting Data here for you guys, i did some sums to figure out the time and cost diffrances between Auto-Calc Testing and Human Beta testing to get a given amount of testing done.
To do a weeks worth of continus Auto-Calc testing by 1 PC, (8.6 million tests), would take 100 Beta Testers worjking 24 hours a day, 1 YEAR to complete. At the the British natinal minimum Wage it would cost CA approximetly 0.4 million pounds in beta tester wages too. Add in costs for electricity for the PC's, the PC's themselves, the cost of extra managment to recruit and manage Beta testers and the cost of employing the rest of the CA team for a year, and your probably looking at over 10 million pounds more in development costs. That would Jack up the price of games, (assuming they assumed 1 million copies would be sold), by some 10 pounds per copy.
They probably did a lot more, i'd bet at least 10 PC's doing a months worth of testing. hat would be impossibile with even 10,000 Beta Testers (42 million to employ that many too).
Well lets just take a look at how TBC and M2tw were done (I'm assuming they are about similar complexitiy).Dosen't matter, If SEGA didn't give them enough time to get the game done in, (and if they had they would have released a game with all those bugs taken out)., then it would have been done. SEGA didn't, it wasn't done and they still decided to release anyway. If SEGA actualy cared about it's custopmer they wouldn't have released it till testing was complete.
As to managment, we don't know why they didn't hit the deadline, based on what i know about bussiness practises, it's my belief SEGA gave CA the absolute minimum time necessery, which means even 1 minor delay would mean they'd never make the target. And in my expiriance NOTHING ever goes without some delays.
And thats my point, regardles of what CA should or should not have done, SEGA shoul;d never have released the game when even CA belived their was somthing wrong with the balance and that their where bugs.
At least your being fair here (IMHO).
Intresting Data here for you guys, i did some sums to figure out the time and cost diffrances between Auto-Calc Testing and Human Beta testing to get a given amount of testing done.
To do a weeks worth of continus Auto-Calc testing by 1 PC, (8.6 million tests), would take 100 Beta Testers worjking 24 hours a day, 1 YEAR to complete. At the the British natinal minimum Wage it would cost CA approximetly 0.4 million pounds in beta tester wages too. Add in costs for electricity for the PC's, the PC's themselves, the cost of extra managment to recruit and manage Beta testers and the cost of employing the rest of the CA team for a year, and your probably looking at over 10 million pounds more in development costs. That would Jack up the price of games, (assuming they assumed 1 million copies would be sold), by some 10 pounds per copy.
They probably did a lot more, i'd bet at least 10 PC's doing a months worth of testing. hat would be impossibile with even 10,000 Beta Testers (42 million to employ that many too).
TBC come out with few bugs and at least no game killing bugs. It got its release date pushed back and was in beta for quite a while (I believe they handed out free keys).
Now just think about what might have happened if CA had given out say... 2k copies to various trusted individuals (maybe the reveiwing places or whatever) and M2tw was in beta for quite a few months (maybe 2-3) they probably could have gotten rid of the bugs right there and had plenty of feedback.
Yet again in every way CA loses in deployment and developement and blizzard wins. Learn from the best.....
Originally Posted by Brighdaasa:
SEGA didn't bring the release date any closer at any point so that there was no time to (play)test, CA let the development stage run too long and thus didn't have enough time for testing.
That's exactly the problem. It's easy to get carried away with adding features especially since there are plenty of customers who don't care if a game's features aren't working properly or if the gameplay is unbalanced. There are certainly plenty of people posting to this forum who don't care, so it would appear CA is right to give playtesting a low priority. This problem of inadequate playtesting extends into the patching phase, and I know that because I was on four Total War beta teams and it happened in all of them. You wouldn't believe how long we had to test the final build of RTW v1.2 if I told you, and that was going to be the final patch for it.SEGA didn't bring the release date any closer at any point so that there was no time to (play)test, CA let the development stage run too long and thus didn't have enough time for testing.
Originally Posted by Brighdaasa:
The shield bug also suggests to me that CA's internal or Sega's QA didn't do sufficient "unit testing". By this i mean take a finished piece of code and input all sorts of regular and irregular numbers and see if they produce the right result. It's usually a basic part of (internal) testing. Things like the shield bug shouldn't be found in playtesting but while testing the code.
Same thing happend in RTW with the phalanx not reversing the cav's charge bonus the way it was supposed to. To SEGA's credit they allowed two more patches to RTW after v1.2, and that charge reversal was fixed although now cav charge is reversed even when you charge into the rear of the phalanx which is still not quite right.The shield bug also suggests to me that CA's internal or Sega's QA didn't do sufficient "unit testing". By this i mean take a finished piece of code and input all sorts of regular and irregular numbers and see if they produce the right result. It's usually a basic part of (internal) testing. Things like the shield bug shouldn't be found in playtesting but while testing the code.
Originally Posted by Brighdaasa:
On the other hand, they must have so much code that it may be impossible to find all those little coding bugs, no matter how good their testing routines are.
That's why it becomes counterproductive to put too much pressure on the programmers since that increases the number of mistakes they make.On the other hand, they must have so much code that it may be impossible to find all those little coding bugs, no matter how good their testing routines are.
There is a lot of guessing here regarding the contract between CA and SEGA. Yes, publishers push out games early. Why? Because the developers give optimistic time frames and agree, contractwise, on certain dates where certain milestones need to be accomplished. As SEGA is paying for this project, it is obvious they want the revenue as they are in essence giving away money on a promise later on that it will be repaid cause the product will sell.
I'm guessing CA underestimated the time needed and the problems that would appear, while overestimating their abilities to program and fix them. Time flies, and suddenly they are sitting there with SEGA pointing out that the game is supposed to be released in a few weeks. I don't think they even tried to oppose it, remembe the whole "0 day patch" nonsense they pulled? They probably believed the major bugs would be easily fixed and a patch would be out in the days following the release.
M2TW like one of those IKEA furniture. You get the parts, but have to assemble em yourself and even drill new holes etc.
SEGA should have had an open beta with a few hundred participants. Even EA, the established evil empire of gaming, does it.
I'm guessing CA underestimated the time needed and the problems that would appear, while overestimating their abilities to program and fix them. Time flies, and suddenly they are sitting there with SEGA pointing out that the game is supposed to be released in a few weeks. I don't think they even tried to oppose it, remembe the whole "0 day patch" nonsense they pulled? They probably believed the major bugs would be easily fixed and a patch would be out in the days following the release.
M2TW like one of those IKEA furniture. You get the parts, but have to assemble em yourself and even drill new holes etc.
SEGA should have had an open beta with a few hundred participants. Even EA, the established evil empire of gaming, does it.
Originally Posted by :
There is a lot of guessing here regarding the contract between CA and SEGA.
Their isn't any contract I imagine, remeber, SEGA own CA so CA does exactly what SEGA tells them.There is a lot of guessing here regarding the contract between CA and SEGA.
p.s. I'm not trying to be a CA fanboy, just that I think CA are getting the blame for a lot of things that should really be laid at SEGA's door, yes maybe CA COULD have tried harder to get the game out in a good state, but at the end of the day the final blame (but not total blame), has to lie with the people who decided to publish it on time regardless of weather it was complete or not.
Single Sign On provided by vBSSO