View member feedback to this tip.
[IMAGE][IMAGE]
Chuck Connell
[IMAGE]
[IMAGE]
Lotus Domino can act as an HTTP Web server. As with all Web servers, Domino produces log files of user traffic. And like all Web log files, the results contained in the logs are nearly indecipherable to the human eye. This problem is even greater for Domino servers than for other Web servers, because many Domino URLs contain long hexadecimal strings rather than normal directory and file names. Therefore, it is almost a requirement to use an analysis tool to understand and interpret Domino Web logs.
At the request of one of my customers, I researched the availability of Web log analysis for Domino and found five primary options, although there are certainly other choices as well.
- WebTrends Log Analyzer -- This product, from NetIQ, is the dominant one among Web traffic analyzers. Participants of the Notes.net discussion groups report good results and satisfaction with its reports. Version 8.1 is the most recent release, and it is available for download (purchase or trial) from WebTrends.com. The purchase price is $495 for the base product on a single server, plus $89 per year for software maintenance. Other, more advanced, log analyzers from WebTrends provide additional customization and features, for higher prices.
WebTrends is certainly a high-quality product; however, it has its limits when used with Domino log files; it does not translate Domino hexadecimal URLs into their meaningful equivalent. Consider the following two Domino URLs, which are equivalent:
Products.nsf/f67f31f8faa31a2285256dd6007055dd!OpenView
Products.nsf/ProductName!OpenView
When reading a report about Domino Web traffic, you really want to see the second url listed above. But this requires the analysis tool to translate each raw Domino URL into its readable alternative. WedTrends (and many other log analyzers) does not do so for Domino log files.
- Workflo!Log Analyzer -- This product from Workflo Systems provides many of the standard aggregation and statistical reports as WebTrends, with the key difference that it translates Domino URLs to their readable form. The traffic reports will show that the view named ProductName was accessed, rather than the view with the ID f67f31f8faa31a2285256dd6007055dd.
The price of this product is $500, plus $250 per year for maintenance and upgrades beyond the first year. Workflo!Log Analyzer can also create various kinds of customized reports, which are only available with more expensive "enterprise" versions of WebTrends. The software and a demonstration movie are available at WFS.com.
- Webalizer -- Webalizer is a free open-source log analyzer that runs on the Linux operating system. It is not ...
To continue reading for free, register below or login
To read more you must become a member of SearchDomino.com
geared specifically to Domino Web traffic, but will work with Domino since Domino produces raw log files in an industry-standard format.
- Sandbox -- There is a Sandbox area of Notes.net, which contains free software submitted by participants of the Notes.net discussions boards. One of the Sandbox downloads is a Domino Web log analyzer named DomLog1.ntf. Several users of Notes.net reported they did not find this software to be satisfactory, however, and posted comments such as "Needs a lot of work to be useful" and "You can use the views as a place to start, but still requires more customization." So I do not recommend this solution, except as a point of departure if you want to create your own log views.
- Custom-developed analyzers -- As with any software project, it is always possible to create a new application from scratch to solve the problem. Even though I make part of my living doing this for Domino-based organizations, I do not recommend doing so in this case. There are at least three mature products, cited above, that provide good-quality analysis of Domino Web log files. All three of these products are available for reasonable prices and cost less than a custom solution would cost.
My recommendations:
- If price is important, I recommend Webalizer. It is virtually free to create a separate Linux computer to run this software. You can host it on an out-of-date PC, with the SUSE 9.0 Personal version of Linux ($40). The Webalizer application itself is free.
- If you don't mind spending $500 for this project, I suggest Workflo!Log Analyzer. This is the only product I could find that is specifically designed to analyze log files from a Domino Web server, and it appears to do so in an impressive manner. As with any product comparison, I expect there are some features where WebTrends wins against this product, but Workflo!Log Analyzer is worth a serious look if you are in the market for a Domino Web log viewer.
If anyone is aware of other Domino Web log analyzers that also perform automatic translation of Domino hex URLs (to their readable equivalent) please let me know.
Chuck Connell is president of CHC-3 Consulting, which helps organizations with all aspects of Domino and Notes.
MEMBER FEEDBACK TO THIS TIP
You asked about products that translated view IDs into names -- sorry, I don't know. To be honest, I'm more interested in databases than individual views/pages etc. But a standard product that I think you might include in your list is available from Analog.cx -- it works well with Domino text Weblogs and apparently works in 32 languages on lots of OSes.
—Mark M.
******************************************
Thanks for writing. You are right. That is actually the Web reporter that my own ISP uses. I should not have overlooked it!
—Chuck Connell, tip author
******************************************
We use WebTrends Reporting Centre...a big leap from the basic log analyzer tool (in cost as well).
It has a feature called "URL Search and Replace" (which we actually haven't used yet), but it looks like you could configure it to recognize a pattern via a regular expression back to something simpler. Sounds like it could work for this issue.
From the help file...
—Michael C.
******************************************
Thanks for writing. Yes, the more expensive products from WebTrends do include this "URL alias" feature. The WFS product does, too, in its base version. The problem with this as a Domino solution, though, is that you have to manually set up the mapping between the complex hex URLs and their meaningful name. Every time you add a new document to a Domino database, you have to add its URL/name mapping to the alias list. That's why you want a Domino Web log analyzer to do this automatically.
—Chuck Connell, tip author
******************************************
Here is another product that we use to analyze the Domino Weblogs. I'm not to sure about translating hex-URLs to readable URLs, because we use an older version of the product. Here's a quote from their Web site:
—Richard C.
******************************************
I read your article on the above with interest. I did develop a free utility, Beaver, that converts Domlog documents into W3C-compatible data. I prefer to use Domlog, as it's easier to manage the data from within Domino than it is to manage all those text files on servers. For more information click here.
Anyway, inspired by your article, I'm now working on getting Beaver to translate those horrible URLs to something more meaningful. It's not always straightforward as the title of a document may be in a field called "subject" or some even less obvious field, but I'll see how far I can get.
There is also something related on OpenNTF.org, but last time I looked it wasn't up to much.
—David C.
******************************************
I use AWStats for analyzing my Domino Web traffic. It is an open-source project written in Perl that runs as a CGI executable. Since it is Perl, it will run on any platform that supports the Perl runtime, and it has configuration examples for the various Domino server versions.
The only thing that I haven't been able to get to work correctly is a plug-in that assigns geo-locations to IP addresses. Everything else works like a champ. You can learn more about AWStats at AWStats.org.
—Andy B.
******************************************
I enjoy reading your tips and newsletters and expert advice. I just had one quick note on Webalizer. There is a Windows port of the product available so you can run it without Linux, which may be prohibitive for some shops. It's on the downloads page as "Win32 - Intel" pointing here.
—Adrian C.
******************************************
Chuck, some time ago we tested WebTrends and as I recall there was a feature where the tool would open each link to lookup the <title> tag to put some meaningful information in the Web statistics. I find it the only practical solution to "rename" the Domino unique IDs, because you would never know in which field in the Notes document the page title is. But in the HTML output, you should expect the title tag to be something meaningful.
—Markus K
******************************************
I have used WebTrends for a while and I have been annoyed by the same thing -- the unreadable URLs in the reports. You can use the Title tag but it doesn't give you much flexibility. In WebTrends you can make scripts that are supposed to change the actual log-file data before analyzing it. If anyone has examples of scripts like this, I would be interested.
My idea is to export a comma-separated textfile with the [universalID],[Subject?], each night and the have the script do the conversion in the actual log file. This could work.
In WebTrends there is already the function to use an external file for translation, but this only works on parameters so you can only do translations if the URL has a "?" and a "= " after the parameter. It is a nice function but I can't understand why you can't choose an external file in the URL replacement section.
—Fredric B.
Do you have comments of your own? Let us know.
