While it is possible to write unlimited amounts of data to RichText fields, it is not possible to retrieve more than 64k from a RichText field programmatically. The reading procedure simply will stop at 64k. Writing to multiple standard fields probably will stop you with a "32k limit" error message when you are trying to save the target document. The standard solution as it is used in log.nsf is writing to multiple documents instead of just a single doc.
But assume you need to write big amounts of data to a single document programmatically, with the option to retrieve that data later on (again programmatically). The solution is to write arrays of data to multiple standard fields. These fields are created on the fly through your script code with the ISSUMMARY flag set to FALSE; if a field's size passes a given threshold value, the next field is created (for an example see 1. example code below).
You also need a second field recording the amount of dynamically created fields. The value stored here is used when retrieving the data at a later time (for an example see 2. example code below). When I tested this solution, I also tried various threshold sizes for the target fields. I observed big differences in the time my script needed to write the data depending on the threshold size. The result was that it takes less time to write many small fields than writing few big fields. In fact, the time needed seems to rise exponentially, while the total size of the resulting document is hardly growing at all. See example measurements block below.
The goal for this in my case is a data analysis performed on a very "lively" application on a regular basis. Of course it take minutes just to open the resulting analysis document because currently there are some 10,000 documents analyzed and logged to a single log document. Since data are structured the way they are, I can compare the log documents created at various times (such as for finding out document movement within a workflow application, tracking document failures etc.).
1. example code: writing Set logdoc = targetdb.CreateDocument logdoc.Form = "log" Set logitm = New NotesItem (logdoc, "LogField" & Cstr (j), "") '--> j is initialized as 0 For i = 0 To 3000 If logitm.ValueLength >= 5000 Then j = j + 1 Set logitm = New NotesItem (logdoc, "LogField" & Cstr (j), "") '--> IsSummary is set to FALSE by default End If Call logitm.AppendToTextList (strText) Next logdoc.NumLogFields = j ' --> record number of fields created 2. example code: retrieving for i = 0 to logdoc.NumLogFields (0) set logitm = logdoc.GetFirstItem ("LogField" & Cstr (i)) Forall entry in logitm 'entry represents a line that previously was written to the item; do whatever you need to do with it End Forall Next 3. Here are some example measurements for 3000 lines of text (150 characters each) written to text fields: threshold size 5000 bytes: number of fields = 94 time needed / write = 2 s time needed / retrieve < 1 s doc size = 476.298 bytes threshold size 15000 bytes: number of fields = 32 time needed / write = 8 s time needed / retrieve < 1 s doc size = 476.174 bytes threshold size 35000 bytes: number of fields = 14 time needed / write = 36 s time needed / retrieve < 1 s doc size = 476.138 bytes threshold size 65000 bytes: number of fields = 8 time needed / write = 113 s time needed / retrieve < 1 s doc size = 476.126 bytes
This was first published in July 2001