github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-26 11:53:38 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

PERF: Run multiple threads for regular job schedules

Browse Source 
Under extreme load on large databases certain regular jobs can take quite
a while to run. We need to ensure we never starve a sidekiq from running
mini scheduler, cause without it we are unable to queue stuff such as
heartbeat jobs.
This commit is contained in:
Sam Saffron 2019-08-29 15:34:36 +10:00
parent 4fce6484fe
commit 098f9e8b5b
3 changed files with 10 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Gemfile.lock
View File

@ -186,7 +186,7 @@ GEM
mini_portile2 (2.4.0) mini_portile2 (2.4.0)
mini_racer (0.2.6) mini_racer (0.2.6)
libv8 (>= 6.9.411) libv8 (>= 6.9.411)
mini_scheduler (0.11.0) mini_scheduler (0.12.0)
sidekiq sidekiq
mini_sql (0.2.2) mini_sql (0.2.2)
mini_suffix (0.3.0) mini_suffix (0.3.0)
@ -274,7 +274,7 @@ GEM
rack-openid (1.3.1) rack-openid (1.3.1)
rack (>= 1.1.0) rack (>= 1.1.0)
ruby-openid (>= 2.1.8) ruby-openid (>= 2.1.8)
rack-protection (2.0.5) rack-protection (2.0.7)
rack rack
rack-test (1.1.0) rack-test (1.1.0)
rack (>= 1.0, < 3) rack (>= 1.0, < 3)

7
config/discourse_defaults.conf
View File

@ -259,3 +259,10 @@ enable_performance_http_headers = false
# gather JavaScript errors from clients (rate limited to 1 error per IP per minute) # gather JavaScript errors from clients (rate limited to 1 error per IP per minute)
enable_js_error_reporting = true enable_js_error_reporting = true
# This is probably not a number you want to touch, it controls the number of workers
# we allow mini scheduler to run. Prior to 2019 we ran a single worker.
# On extremely busy setups this could lead to situations where regular jobs would
# starve. Specifically jobs such as "run heartbeat" which keeps sidekiq running.
# Having a high number here is very low risk. Regular jobs are limited in scope and scale.
mini_scheduler_workers = 5

2
config/initializers/100-sidekiq.rb
View File

@ -62,7 +62,7 @@ if Sidekiq.server?
scheduler_hostname = ENV["UNICORN_SCHEDULER_HOSTNAME"] scheduler_hostname = ENV["UNICORN_SCHEDULER_HOSTNAME"]
if !scheduler_hostname || scheduler_hostname.split(',').include?(`hostname`.strip) if !scheduler_hostname || scheduler_hostname.split(',').include?(`hostname`.strip)
MiniScheduler.start MiniScheduler.start(workers: GlobalSetting.mini_scheduler_workers)
end end
end end
end end