As regular readers know, I tend to be somewhat skeptical about rankings of outsourcing vendors.  I probably should have shared some sort of disclaimer when I wrote about the “Black Book of Outsourcing” report.

Now, BusinessWeek has a story regarding some challenges to the validity of the report.

“… key players in the outsourcing industry question whether it merits such status. They say its methodology is not consistent with most accepted business research techniques. As a result, companies often rise or fall dramatically from one year to the next—an unusual pattern for a customer satisfaction ranking.”

The story was written by Steve Hamm, who also writes BusinessWeek’s well respected Globsespotting blog.  Mr. Hamm even found a professor to take a shot at the methodology behind the report.

“Claes G. Fornell, a professor at the University of Michigan Ross School of Business who specializes in customer satisfaction surveying, says Brown & Wilson’s methods aren’t sound. First, he says, the firm can’t be sure all the people who respond are qualified. Second, the results could be tilted in favor of companies that urge their customers to participate.”

Unfortunately, the story doesn’t mention that most of the industry rankings ignore clients completely, and simply rely on information submitted by the vendors.  Surely, this methodology is no more sound.  I guess that the take away is that you can’t rely on rankings.  You have to do your own evaluation.

Technorati Tags:  China, China Outsourcing