-
-
Notifications
You must be signed in to change notification settings - Fork 621
BUG: Sync-service: Synchronous Network Calls in Watcher Loop Cause Lag #1034
Copy link
Copy link
Open
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
Description
When file changes are detected, the watcher makes synchronous HTTP requests to the primary backend to trigger syncs. If multiple files change at once, these requests happen serially. If the backend is slow or timeouts (30s default), the watcher stops processing new file system events for the duration of the request.
Steps to reproduce
- Watch a folder.
- Rapidly create or modify 50 files in that folder.
- Observe that the "Successfully synced..." logs appear one by one, potentially taking seconds or minutes to complete.
- Changes happening during this "frozen" time might be batched inefficiently or delayed.
Root cause
watcher_util_call_sync_folder_api uses httpx.Client (blocking) instead of httpx.AsyncClient. It is called directly within the watcher_util_handle_file_changes loop.
Potential fix
- Use queue.Queue to decouple file detection from API notification.
- The watcher thread should push events to a queue.
- A separate consumer thread (or asyncio task) should read from the queue and make API calls asynchronously or in parallel.
I would like to work on this issue
Record
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels