News Stay informed about the latest enterprise technology news and product updates.

Meta Group defends Domino/Exchange TCO report

Meta Group says it stands by its recent small business messaging survey, which found Exchange's TCO to be 52% less than Domino's, even though it was funded by Microsoft.

Despite claims by IBM Lotus that a recent Microsoft-funded white paper paints an inaccurate picture of small business messaging costs, the research firm responsible for the study stands by its work.

In July, Stamford, Conn.-based Meta Group issued "Messaging Total Cost of Ownership: Exchange Server 2003 and Lotus Domino in Small and Medium Organizations," a survey touted as an impartial comparison of the messaging-related hardware, software and personnel costs incurred by businesses with fewer than 10,000 workers.

The survey found Domino's average annual total cost of ownership to be $300.19 per user, while Exchange 2003's yearly TCO was $143.96, 52% less than Domino.

The report was discovered by the Lotus user community via a link on Microsoft's Web site. After searches revealed that the paper was not available via Meta's site, the research firm disclosed that the paper was in fact funded by Microsoft.

Like the recent Radicati Group report, IBM Lotus and users alike railed against the report, claiming that not only are its conclusions and methodologies inherently flawed, but it also ignored other factors critical to judging the cost of a messaging system.

For more information

Learn about the Radicati report that stirred the Lotus community.


What's your take on the Meta report? Visit our Sound Off forum.

In a response on its Web site, IBM Lotus offered several objections to Meta's paper, noting that only nine companies were surveyed (four Lotus customers and five Microsoft shops), one Lotus customer in the survey was still using a version of R5 that is nearly five years old and costs related to downtime and performance were not included.

Shafqat Azim, a senior vice president with Meta Group's consulting unit, said his group, not Meta's separate technology analyst group, is responsible for the study. He said the consulting team collaborates regularly with the analysts, and worked with them to establish the guidelines for this survey.

Azim said the survey pool began by targeting 50 of Meta's small business clients with less than 10,000 seats. From there, it narrowed its group to 13 that were similar in factors such as size, industry and business requirements.

He also said the research pool was further reduced because many small businesses didn't have enough data for the study; they don't keep detailed messaging cost information.

When asked why it chose to exclude certain TCO criteria that may have been less favorable to Microsoft, most notably security-related downtime, Azim said the survey was intended to be limited in its scope, though Meta hopes to conduct more comprehensive messaging TCO research in the near future.

"One thing we found out is because it was a small client footprint, a lot of the functionality that Notes offers was not being leveraged," Azim said, "and that could be part of the reason why the report ended up being more favorable to Microsoft."

Even though Microsoft funded the research and did provide "a couple of customers" for the report, Azim said Meta did not permit the software giant to have any input on the approach or methodology.

"Other than the funding -- sponsoring this comparison -- Microsoft had nothing to do with the outcome of the survey, so we stand by the fact that this [research] is very independent," Azim said.

In the weeks that followed the survey's release, Azim said IBM expressed concern that the report neither disclosed that it was funded by Microsoft nor that it was not written by technology analysts. He said those clarifications have been made in the most recent version of the report, though at press time Microsoft had not posted the updated copy on its Web site.

Ethically, these types of funded surveys fall under a gray area because they help research firms pay the bills, said Rob Enderle, president and principal analyst with the Enderle Group, a consulting firm in San Jose, Calif.

Enderle said that in the past, research firms would diligently work with several vendors, in this case Microsoft and IBM, to co-sponsor research, which would offer more value to vendors and users alike. However, vendors today are rarely willing to cooperate.

For users, the bottom line is that the findings of funded research papers need to be taken with a grain of salt. "You don't want to throw it out, but it's not something you can take at face value either," Enderle said.

Dig Deeper on Lotus Notes Domino Administration Tools

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.




  • iSeries tutorials's tutorials provide in-depth information on the iSeries. Our iSeries tutorials address areas you need to know about...

  • V6R1 upgrade planning checklist

    When upgrading to V6R1, make sure your software will be supported, your programs will function and the correct PTFs have been ...

  • Connecting multiple iSeries systems through DDM

    Working with databases over multiple iSeries systems can be simple when remotely connecting logical partitions with distributed ...