Add progressive backoff to wait_for_available_thread()

[Copilot]

The wait_for_available_thread() function was a tight busy-wait loop that continuously yielded, burning significant CPU when tasks are long-lived or the system is oversubscribed.

Changes

• Implemented progressive backoff strategy:
  • Yields 10 times (low latency fast path)
  • Sleeps 1ms for next 20 attempts
  • Sleeps 5ms thereafter

void wait_for_available_thread()
{
   if (pool_)
   {
      static constexpr int MAX_YIELD_ATTEMPTS = 10;
      static constexpr int MAX_SHORT_SLEEP_ATTEMPTS = 20;
      static constexpr int MAX_BACKOFF_LEVEL = MAX_YIELD_ATTEMPTS + MAX_SHORT_SLEEP_ATTEMPTS;
      int attempts = 0;
      
      while (pool_->get_tasks_running() >= pool_->get_thread_count())
      {
         if (attempts < MAX_YIELD_ATTEMPTS)
         {
            // Fast path: yield to other threads
            sched_yield();  // or SwitchToThread() on Windows
         }
         else if (attempts < MAX_BACKOFF_LEVEL)
         {
            std::this_thread::sleep_for(std::chrono::milliseconds(1));
         }
         else
         {
            std::this_thread::sleep_for(std::chrono::milliseconds(5));
         }
         attempts++;
      }
   }
}

This preserves low latency for short waits while dramatically reducing CPU usage under sustained load.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.