I forgot to come back to that thread for a while, bunch of answers:
stainer wrote on Mar 12
th, 2013 at 4:30pm:
A lot of that stuff can be mimicked virtually, but I am not sure if everything can be. Regardless, what they have been doing for the last 7 years seems to be working well for them
yes, and it helps reduce the cost.
Asheras wrote on Mar 12
th, 2013 at 4:40pm:
I'd guess their IT budget is somewhere in the 2+ million per year range. More than enough to squeeze in a testing platform.
Sorry. I don't buy it. I've worked at companies with revenues below 50 million a year in revenue and they had proper testing environments.
Just the Symetrix in the 2 million range ( and the relevant SAN )... And that's one of the of the thing that cannot be really mimicked by something else sadly... you will still need a SAN.
( ok, there's cheaper SANs, but if we talk performance, you will have to have a full copy of the live system )
Terebinthia wrote on Mar 12
th, 2013 at 6:26pm:
Do you really need all of that stuff? Seriously.
Not trying to be an arsehole, just my business background of course is from the school of of course you have something to test live on. And there's still hilarity to telll. Am I missing something in the software industry here? (obviously capriciously losing your digital stuff in a videogame is a degree smaller than a FS company going tits up, but don't you build a test system in as a basic?
Seriously : no, but if you want to test thoroughly and have a test system that looks like live fairly way, there's no way around you have to pay the bill.
In my Industry ( Telco ), not long ago operators paid for a full fledged Public Exchange with 64K subscribers ( virtual )... Just because there were things that couldn't be reproduced any other way, and performance issues are different in a 16K system and a 64K system.
notajedi wrote on Mar 24
th, 2013 at 6:11pm:
Based on my experience, you are overkilling a test replicating environment. I GUARANTEE that is not what is being run for the developers at Turbine either. Their environment is MUCH MUCH smaller.
I talked about a live system when I described that...
But if you want a test server you will just need at least an Oracle Server ( with the cluster layer, even if it's only a single node ), a Clariion ( to replace the Symetrix ), The SAN ( way smaller, can probably be done straight from the Clariion interfaces, thus getting rid of the SAN Fiber Switches ), The back end network and the front end network can be the same, Game Server. Billing/Accounting can be simulated, as it doesn't matter.
Still that's two servers, 1 SAN Storage Array, 1 Network infrastructure... And with that you will definitely not be able to test any performance issue as it is not representative of the live system.
( on the other hand you'll be able to test all rest. )
The bitch in QA Test is the Performance tests... as most of the time they can only be done on the live system,
In Telco we do that on special test systems, when we can, ( my employer has a few like that, we used to have one in France ) that are basically a complete copy of a live site that are here only for the performance test purpose ( well there's a few other test cases that require these systems, but they are not relevant to the discussion )