If you are implementing a Cloud infrastructure you probably know that you need to perform a rigorous evaluation of all the offers you find on the market and follow a smart decision process in order to choose the service that fits most to your company.

If you find yourself in the middle of this process or are thinking about how to approach this process, you will find this interview interesting and helpful: we interviewed Scott Feuless, one of the world’s foremost experts on measurement and evaluation of Cloud services. His work with Cloud Computing began in 1999 when he helped transform a small software Value Added Reseller into a public startup Application Service Provider (ASP) as the firm’s Chief Technology Officer. He also joined the Cloud Services Measurement Initiative Consortium (CSMIC), where he was the primary author of the metrics.

He is the author of the book The Cloud Service Evaluation Handbook: How to Choose the Right Service, which presents a complete set of metrics for evaluating Cloud services and guidelines, leading practices and examples that can help you during the decision process.

– Scott, can you tell us something more about your book and why it was so important for you to write it?

Absolutely. When I looked around at the variety of Cloud books out there, what I saw was that they fell into two general categories. One was technical guidance, primarily for developers, on how to write applications for the Cloud and take advantage of the APIs and tools that you can get from Amazon, Microsoft and Google. The other was just a basic discussion of what Cloud was, what makes it different from traditional computing and why that was important.

I saw that there was a need for a second-generation Cloud book that went beyond the basics to give people practical knowledge that they would need to actually adopt Cloud services without making big mistakes that would cost them down the road. Too many companies were just signing up for a service with no rigorous decision process at all, and of course that put them at the mercy of the providers and ultimately disappointed them. Procurement departments in particular had been left completely out of the loop regarding changes to their process that would be required to successfully navigate the move to Cloud.

I was in a particularly good position to address these needs because I’d recently finished a standards exercise with a now-defunct industry consortium called CSMIC. As part of that I’d authored numerous standard metrics that could be used to evaluate Cloud services and feed into an informed procurement decision. Unfortunately, the consortium fell apart before the work was complete, and the standard was never published, but all of the knowledge and research that I’d put into authoring the metrics was still in my head.

So, I decided to write a book that could be used as a tool by procurement and IT professionals to measure, evaluate and select Cloud services using a comprehensive framework that would be completely consistent and reusable. That framework and all of its metrics are fully documented in the book. Anyone who wants to acquire in-depth knowledge on how acquiring Cloud services is different from acquiring traditional IT services, what factors should be considered when building the business case for Cloud, what metrics can be used and how a final selection should be made, is going to find a lot of value in the book.

– What are the new realities of the Cloud marketplace and how do they differ from the old ones?

Well, there are many new realities, but one that I touch on in the book is that the old market for IT services was very much built around doing custom deals that were specific to each customer. Each customer would build a very large, rigorous RFP document that laid out exactly how the service was to be delivered, what the provider’s responsibilities were, what kind of service level agreements were required, the volumes of service the customer expected, the level of expertise of the staff providing the service, the exact structure for submitting prices and so on. All each provider had to do was say “yes, we’ll do all these things you’re asking for, plus here’s some value-added things we’ll do to transform your operations to a more desirable state, and here’s what we’ll charge you in the format you asked for“. Since all the pricing was submitted in the same format it was easy to compare, and selecting a winner was straight forward once the RFP responses came in. Everything the selected provider agreed to in the RFP then went straight into the contract. You can still do the deal that way if you’re talking about a customized private Cloud from someone like IBM, CSC or HP, but public Cloud is an entirely different animal.

If you go to Amazon, Microsoft or Google they are not about doing custom deals. That’s not their business model, so if you want even a little customization you better be spending a truly large amount of money – something like the CIA did with Amazon a few years back.

Instead, customers are expected to settle for highly standardized services in return for low pricing and superior automation that delivers something that traditional IT never could: rapid elasticity. So, if the services are standardized, that means you’re basically choosing them from a menu instead of specifying something customized, and your telephone-book sized RFP is now irrelevant. That’s something that comes as a huge shock to most procurement people. The even-bigger problem is that the need for all that work that went into building RFP documents didn’t just disappear. It moved down the procurement cycle into the evaluation stage. Now, instead of having responses to your RFP that are all in the same format and simple to compare, you have to look at each provider’s service and all the unique features, service levels and pricing that go with each one. And let me tell you, the pricing of public Cloud providers may look easy to compare, but it definitely isn’t. Each provider includes different functionality, built on different technology, with different service levels, in each of their prices.

I remember back when Amazon was first announcing their Cloud Computing services, and everyone was scratching their heads and asking “why would a retailer like them be getting into the business of selling data center technology?” But they were looking at it the wrong way. Amazon had already had great success selling books, which they expanded into selling other commercial-off-the-shelf products like shrink-wrapped software. And they were essentially reinventing data center services so that they could be sold like software. By greatly simplifying and “packaging” a complex product they could sell it in bulk and at scale, the same way that they sold everything else.

Unfortunately, most customers focused only on the technology that Cloud Computing used to deliver services and the easy way you could buy it, so the market mostly glossed over the fact that you don’t buy a software package by just going out and comparing prices, and you shouldn’t buy Cloud Computing that way either. If I was going to buy, say, an HR system for my company, I wouldn’t just ask Workday and Peoplesoft for a price and choose the cheapest one. I’d do a rigorous evaluation first and make sure I chose the one that was going to best meet my needs, taking price into account as just one of many considerations. A lot of folks still aren’t doing that with Cloud today, but they should be.