Skip to content

BUG: Sync-service: Synchronous Network Calls in Watcher Loop Cause Lag #1034

@DeveloperAmrit

Description

@DeveloperAmrit

Is there an existing issue for this?

  • I have searched the existing issues

What happened?

Description

When file changes are detected, the watcher makes synchronous HTTP requests to the primary backend to trigger syncs. If multiple files change at once, these requests happen serially. If the backend is slow or timeouts (30s default), the watcher stops processing new file system events for the duration of the request.

Steps to reproduce

  1. Watch a folder.
  2. Rapidly create or modify 50 files in that folder.
  3. Observe that the "Successfully synced..." logs appear one by one, potentially taking seconds or minutes to complete.
  4. Changes happening during this "frozen" time might be batched inefficiently or delayed.

Root cause

watcher_util_call_sync_folder_api uses httpx.Client (blocking) instead of httpx.AsyncClient. It is called directly within the watcher_util_handle_file_changes loop.

Potential fix

  • Use queue.Queue to decouple file detection from API notification.
  • The watcher thread should push events to a queue.
  • A separate consumer thread (or asyncio task) should read from the queue and make API calls asynchronously or in parallel.

I would like to work on this issue

Record

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions