Improve indexing service resilience and make batch size configurable

[Copilot]

Description

Indexing operations in NamedIndexingService and synchronization in DefaultIndexProfileManager were non-resilient: any exception would terminate the entire process, leaving indexes partially updated and requiring manual intervention.

Changes

NamedIndexingService

• Changed BatchSize from private const to protected virtual property for derived class customization
• Added three-tier exception handling:
  • Per-task: logs failure, continues with remaining tasks in batch
  • Per-index: logs failure updating one index, continues with other indexes
  • Per-batch: logs failure, advances to next batch to prevent process termination
• Fixed incorrect loop condition (while (tasks.Count <= BatchSize)) that would exit prematurely after first batch
• Added batchProcessedSuccessfully flag to prevent double-incrementing lastTaskId on exceptions

DefaultIndexProfileManager

• Wrapped synchronization handler invocation in try-catch
• Errors logged but don't fail background job, allowing retry on next run

Example

Derived classes can now tune batch size:

public sealed class CustomIndexingService : NamedIndexingService
{
    protected override int BatchSize => 250; // Override for larger batches
}

Previously, any exception during indexing would halt the entire process:

// Old behavior: exception here stops all indexing
await _documentIndexHandlers.InvokeAsync(x => x.BuildIndexAsync(context), Logger);

Now indexes continue processing despite failures:

// New behavior: exception logged, remaining tasks continue
try
{
    await _documentIndexHandlers.InvokeAsync(x => x.BuildIndexAsync(context), Logger);
}
catch (Exception ex)
{
    Logger.LogError(ex, "Error processing task {TaskId}. Continuing.", task.Id);
}

Original prompt

Improve the resiliency of NamedIndexingService and the indexing synchronization process

The NamedIndexingService should be made more resilient and extensible to better handle failures during indexing operations.

1. Make BatchSize configurable

Change BatchSize to a protected virtual property.

This allows implementers to override and tune batch sizing based on workload, data source, or environment constraints.

2. Improve failure handling and retry behavior

Services such as ContentIndexingService are invoked both:

From request/handler-based execution

From background tasks (e.g., ContentIndexingBackgroundTask.cs)

Currently, if the indexing service throws an exception or encounters failures:

The process terminates immediately

There is no retry or recovery mechanism

This results in partial indexing and requires manual intervention.

Proposed improvement:

Introduce retry logic with clear failure boundaries:

Retry transient failures (e.g., timeouts, temporary data issues)

Capture and log failed records without terminating the entire process

Allow the indexing process to continue where possible

3. Make IIndexProfileManager.Synchronize more resilient

The default implementation of IIndexProfileManager runs Synchronize in a background process, which is a good design.

However, the synchronization process is not resilient:

Any exception in the indexing service can halt the entire synchronization

Failed records or batches stop further progress

Proposed improvement:

Enhance the synchronization loop to:

Handle per-batch or per-record failures gracefully

Apply retry policies (with backoff if needed)

Continue processing until completion whenever possible

Surface a final status (completed with warnings vs. failed)

Expected outcome

Indexing runs to completion even in the presence of transient or partial failures

Background indexing becomes self-healing and more production-safe

Implementers gain flexibility without breaking existing behavior

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.