We currently use LEI to populate several "control" or "lookup" databases. Each day LEI runs and refreshes these "lookup" databases with information from our DB2 warehouse tables. We have one "lookup" database for employee information, and another for operational data. There are about 16 in all, and they are loaded every morning.
All of these databases are clustered. So, once LEI has loaded in the data from DB2, it must be replicated to all of the other prod servers. Herein lies the problem. The replication task can handle this load under normal circumstances. However, if there is any extra server load, the added weight of replicating these "lookup" databases can bring the servers to their knees. Another problem is that we often need to "join" the information in these lookup databases. This is also a performance hit as the databases are large, and opening several views to do lookups takes time.
A proposed solution would have us change all of our lookups to these "lookup" databases to use agent.runonserver to look directly at the DB2 tables. That way we could avoid populating and replicating lots of data every day and the joins would be faster. We would use the RDBMS for what it was meant for and use Notes for what it was meant for. By using agent.runonserver to do the SQL queries, we won't have to set up ODBC connections on each client.
My question is this: Will 2,000 users spread over two production Domino app servers using agent.runonserver and LC LSX to do real-time lookups to DB2 cause as much or more of a performance problem as what we have now? If so, are there any other options?
That would probably not help. In my experience, keyword lists do not change every day, and when they do change, it's by the addition or removal of one or a few values, not total replacement of the entire list. Therefore, it seems wrong to me that your daily update of lookup values is causing a significant amount of replication overhead. I think that you need to add some intelligence to the process to update only those Notes documents containing values that have actually changed. Then the replication time should not be a problem.
You didn't say exactly how you're doing the copying of values from the DB2 database to Notes. I suspect you're using a LEI Direct Transfer, replacing every document in the Notes database. This is not a good idea for repeated use. If you use a Replication activity instead, I believe you will find that only the Notes documents that have actually changed will be updated.
If you're using a Replication activity and still have this problem, please post to the EI forum in LDD with more details about what you're doing.