I can't easily get Que to work under puma with multiple worker (single mode is fine). Que is getting instantiated in the root process and copied over into the child processes without reinitialization.
The docs say I can just set use Que.mode = :async to make it work, but I also have to reinitialize the connection and the logger.
The problem then remaining is that the workers are not woken up periodically, since the wrangler is in the wrong process. I need this for jobs added by cron-jobs.
Am I doing something wrong? My current work-around feels brittle.
The interesting lines of my current workaround:
Que.connection= ::ActiveRecord # else we get a /que.rb:40:in `adapter': Que connection not established! (RuntimeError)
Que.logger = Rails.logger # else we don't get any log entries
Que.worker_count = que_workers # enable the que again
# Que.wake_interval = 5.seconds # crashes with ThreadError
Thread.new do
loop do
sleep 1
Que.wake!
end
end
my full configuration is:
require 'que'
puma_workers = Integer(ENV['PUMA_WORKERS'] || 2)
min_threads = Integer(ENV['MIN_THREADS'] || 8)
max_threads = Integer(ENV['MAX_THREADS'] || 8)
que_workers = Integer(ENV['QUE_WORKERS'] || 1)
Que.mode = :off
workers puma_workers
threads min_threads, max_threads
preload_app!
port ENV['PORT'] || 3000
environment ENV['RACK_ENV'] || 'development'
on_worker_boot do
ActiveRecord::Base.connection_pool.disconnect!
ActiveSupport.on_load(:active_record) do
config = ActiveRecord::Base.configurations[Rails.env] ||
Rails.application.config.database_configuration[Rails.env]
config['pool'] = max_threads + que_workers
ActiveRecord::Base.establish_connection(config)
Que.connection= ::ActiveRecord # else we get a /que.rb:40:in `adapter': Que connection not established! (RuntimeError)
Que.logger = Rails.logger # else we don't get any log entries
Que.worker_count = que_workers # enable the que again
# Que.wake_interval = 5.seconds # crashes with ThreadError
Thread.new do
loop do
sleep 1
Que.wake!
end
end
end
end
Plus I have a Que.mode = :off in my application.rb to be safe.
I can't easily get Que to work under puma with multiple worker (single mode is fine). Que is getting instantiated in the root process and copied over into the child processes without reinitialization.
The docs say I can just set use
Que.mode = :asyncto make it work, but I also have to reinitialize the connection and the logger.The problem then remaining is that the workers are not woken up periodically, since the wrangler is in the wrong process. I need this for jobs added by cron-jobs.
Am I doing something wrong? My current work-around feels brittle.
The interesting lines of my current workaround:
my full configuration is:
Plus I have a Que.mode = :off in my application.rb to be safe.