Despite claims by IBM Lotus that a recent Microsoft-funded white paper paints an inaccurate picture of small business messaging costs, the research firm responsible for the study stands by its work.
In July, Stamford, Conn.-based Meta Group issued "Messaging Total Cost of Ownership: Exchange Server 2003 and Lotus Domino in Small and Medium Organizations," a survey touted as an impartial comparison of the messaging-related hardware, software and personnel costs incurred by businesses with fewer than 10,000 workers.
The survey found Domino's average annual total cost of ownership to be $300.19 per user, while Exchange 2003's yearly TCO was $143.96, 52% less than Domino.
The report was discovered by the Lotus user community via a link on Microsoft's Web site. After searches revealed that the paper was not available via Meta's site, the research firm disclosed that the paper was in fact funded by Microsoft.
Like the recent Radicati Group report, IBM Lotus and users alike railed against the report, claiming that not only are its conclusions and methodologies inherently flawed, but it also ignored other factors critical to judging the cost of a messaging system.
Requires Free Membership to View
Register today to access targeted resources from our editorial writers and independent industry experts focused on Lotus Domino, Notes, Workplace and other related technologies.
 |
||||
|
|
|||
|
||||
Shafqat Azim, a senior vice president with Meta Group's consulting unit, said his group, not Meta's separate technology analyst group, is responsible for the study. He said the consulting team collaborates regularly with the analysts, and worked with them to establish the guidelines for this survey.
Azim said the survey pool began by targeting 50 of Meta's small business clients with less than 10,000 seats. From there, it narrowed its group to 13 that were similar in factors such as size, industry and business requirements.
He also said the research pool was further reduced because many small businesses didn't have enough data for the study; they don't keep detailed messaging cost information.
When asked why it chose to exclude certain TCO criteria that may have been less favorable to Microsoft, most notably security-related downtime, Azim said the survey was intended to be limited in its scope, though Meta hopes to conduct more comprehensive messaging TCO research in the near future.
"One thing we found out is because it was a small client footprint, a lot of the functionality that Notes offers was not being leveraged," Azim said, "and that could be part of the reason why the report ended up being more favorable to Microsoft."
Even though Microsoft funded the research and did provide "a couple of customers" for the report, Azim said Meta did not permit the software giant to have any input on the approach or methodology.
"Other than the funding -- sponsoring this comparison -- Microsoft had nothing to do with the outcome of the survey, so we stand by the fact that this [research] is very independent," Azim said.
In the weeks that followed the survey's release, Azim said IBM expressed concern that the report neither disclosed that it was funded by Microsoft nor that it was not written by technology analysts. He said those clarifications have been made in the most recent version of the report, though at press time Microsoft had not posted the updated copy on its Web site.
Ethically, these types of funded surveys fall under a gray area because they help research firms pay the bills, said Rob Enderle, president and principal analyst with the Enderle Group, a consulting firm in San Jose, Calif.
Enderle said that in the past, research firms would diligently work with several vendors, in this case Microsoft and IBM, to co-sponsor research, which would offer more value to vendors and users alike. However, vendors today are rarely willing to cooperate.
For users, the bottom line is that the findings of funded research papers need to be taken with a grain of salt. "You don't want to throw it out, but it's not something you can take at face value either," Enderle said.