Sorry to hear this, the publicity would do both of us some good – you don't think HP is sitting on their hands waiting to see their LoadRunner franchise evaporate do you?

In any case, referring to this as a "science experiment" is not fair and you know it – it's a contest between two companies who offer cloud testing to see which one can generate the most load the fastest. I suspect somebody at SOASTA read some of the material on my blog about the CapCal architecture and decided the risk was too high for SOASTA. If that's the case, I don't blame you at all – if I had a Windows load agent I would never go against someone with a Linux agent, for example. It's not a fair contest, even though the advantages are many.

So if you want to place a comment here that says "it happens all the time even though we haven't done it" and then refuse to show us the "beef" then it probably would have been better not to post at all. It looks both arrogant and chickensh*t, which I am sure is not your intention. Net result – CapCal challenged SOASTA to a load contest and SOASTA declined.

I love the name CloudTest, so much so that I decided to name CapCal that in 2005 and went so far as to snatch up the cloudtest.com domain. Because of a goof with an email address at the registrar, I ended up letting it elapse without knowing it and it's had two owners since then. Oh well, you don't need the domain to use it as a trademark and I like it. I also like Capacity Calibration because we do more than cloud testing. So I'm happy with where the names ended up!

Cheers

Randy

Sorry to hear this, the publicity would do both of us some good – you don’t think HP is sitting on their hands waiting to see their LoadRunner franchise evaporate do you?

In any case, referring to this as a “science experiment” is not fair and you know it – it’s a contest between two companies who offer cloud testing to see which one can generate the most load the fastest. I suspect somebody at SOASTA read some of the material on my blog about the CapCal architecture and decided the risk was too high for SOASTA. If that’s the case, I don’t blame you at all – if I had a Windows load agent I would never go against someone with a Linux agent, for example. It’s not a fair contest, even though the advantages are many.

So if you want to place a comment here that says “it happens all the time even though we haven’t done it” and then refuse to show us the “beef” then it probably would have been better not to post at all. It looks both arrogant and chickensh*t, which I am sure is not your intention. Net result – CapCal challenged SOASTA to a load contest and SOASTA declined.

I love the name CloudTest, so much so that I decided to name CapCal that in 2005 and went so far as to snatch up the cloudtest.com domain. Because of a goof with an email address at the registrar, I ended up letting it elapse without knowing it and it’s had two owners since then. Oh well, you don’t need the domain to use it as a trademark and I like it. I also like Capacity Calibration because we do more than cloud testing. So I’m happy with where the names ended up!

Cheers
Randy

Thanks for the compliments and the interesting proposal.

You’re absolutely right that we’re exceeding known load testing limits with customers all the time, whether 70M Users in 10 hours, 500,000+ concurrent users, or (the real challenge) 5.7 TB of real-time test data analyzed to pinpoint production issues in tests of this magnitude.

For our CloudTests, there’s little difference in provisioning 5, 500 or 5000+ test servers, so pressing “Run” on 1M concurrent users isn’t the issue. What matters are the customers’ environment, the methodology used to ramp the tests meaningfully, and the analytics’ capability to find issues. We’ll refrain from this science experiment and stick to meeting our customer’s requirements. Any records will fall naturally.

Brad

Thanks for the compliments and the interesting proposal.

You’re absolutely right that we’re exceeding known load testing limits with customers all the time, whether 70M Users in 10 hours, 500,000+ concurrent users, or (the real challenge) 5.7 TB of real-time test data analyzed to pinpoint production issues in tests of this magnitude.

For our CloudTests, there’s little difference in provisioning 5, 500 or 5000+ test servers, so pressing “Run” on 1M concurrent users isn’t the issue. What matters are the customers’ environment, the methodology used to ramp the tests meaningfully, and the analytics’ capability to find issues. We’ll refrain from this science experiment and stick to meeting our customer’s requirements. Any records will fall naturally.

Brad

Nice to meet you and thanks for the comment! 2007 was a good time to jump into the cloud, I have to admit we are a little late but we treated the Internet itself like a cloud for many years and I can tell you Amazon EC2 is MUCH easier!

The key, of course, is simultaneous virtual users and you have to admit that a million is a lot! You know that already because of the range of numbers you are testing in – before cloud computing, 200 – 500k simultaneous users was totally unheard of. So you guys are already breaking records compared to doing things the old way.

Anyway, I thought it might be fun to ask Geeks Are Sexy here if they want to host a competition – we know we can both spin circles around "legacy" tools like LoadRunner, but how about a million user race? It would be a neutral third party that would host the site and we would test the same pages in the same order with the same think times. How long it takes to reach 1 million simultaneous sessions from the first page hit would be the measuring stick.

What do you think?

In any case, thanks again for the comment and best of luck to you as well! You guys have an awesome UI, I like it!

Randy

Nice to meet you and thanks for the comment! 2007 was a good time to jump into the cloud, I have to admit we are a little late but we treated the Internet itself like a cloud for many years and I can tell you Amazon EC2 is MUCH easier!

The key, of course, is simultaneous virtual users and you have to admit that a million is a lot! You know that already because of the range of numbers you are testing in – before cloud computing, 200 – 500k simultaneous users was totally unheard of. So you guys are already breaking records compared to doing things the old way.

Anyway, I thought it might be fun to ask Geeks Are Sexy here if they want to host a competition – we know we can both spin circles around “legacy” tools like LoadRunner, but how about a million user race? It would be a neutral third party that would host the site and we would test the same pages in the same order with the same think times. How long it takes to reach 1 million simultaneous sessions from the first page hit would be the measuring stick.

What do you think?

In any case, thanks again for the comment and best of luck to you as well! You guys have an awesome UI, I like it!

Randy

The typical use-case we support (with major websites like Hallmark, TurboTax, Proctor&Gamble, BMW, etc) is a desire to hit volume goals – e.g. 10 Million users per day, 50 million downloads in 10 hours, etc. Customers' "simultaneous" objectives as part of these tests are often in the 10s of thousands (still large by traditional standards), but large sites like to see simultaneous users in the 200-500,000 user range. The only limit now to the size of test is the capacity of the system being tested – which, of course, is why we're testing (Pints of Guinness notwithstanding!)

The point here is that this CAN and IS begin done today, and with cloud testing, customers can now demand of their dev and ops teams that web apps be tested to full production scale. Thanks cloud computing, and good luck CapCal.

For musings on cloud testing.. http://www.soasta.com/blog/” rel=”nofollow”>.http://www.soasta.com/blog/

The typical use-case we support (with major websites like Hallmark, TurboTax, Proctor&Gamble, BMW, etc) is a desire to hit volume goals – e.g. 10 Million users per day, 50 million downloads in 10 hours, etc. Customers' "simultaneous" objectives as part of these tests are often in the 10s of thousands (still large by traditional standards), but large sites like to see simultaneous users in the 200-500,000 user range. The only limit now to the size of test is the capacity of the system being tested – which, of course, is why we're testing (Pints of Guinness notwithstanding!)

The point here is that this CAN and IS begin done today, and with cloud testing, customers can now demand of their dev and ops teams that web apps be tested to full production scale. Thanks cloud computing, and good luck CapCal.

For musings on cloud testing.. http://www.soasta.com/blog/” rel=”nofollow”>.http://www.soasta.com/blog/