So on day 2 of NewJob I’ve inherited a slight issue with SOLR. The DB team has a number of servers upon which they generate SOLR indexes. The servers aren’t beefy enough to run multiple indexes at the same time (mostly since the datasets are gigantic). They want, therefore, a way to determine when a SOLR run starts/ends.
Does anyone know what sequence shows up in the logs for this? I’ve had the DBs tell me when they kick off runs and when they end, but there don’t appear to be any unique log entries at those times that don’t appear during times when runs theoretically aren’t running (especially as runs must be manually triggered).
Any help would be greatly appreciated.
Solr’s indexes are modified by making API calls to the Solr web service.
Something outside of Solr should be making those calls to manage the members of the Solr index, and that’s where you’ll need to track the status of the work; Solr itself doesn’t have any concept of tracking the progress of a bulk indexing job. You could guess at it badly with some parsing of the access logs, but that’s not a good approach.
If you can, dig a little further into how the indexing work is being managed – that’ll be where you should focus your efforts.
Check more discussion of this question.