mirror of
https://github.com/discourse/discourse.git
synced 2024-11-30 08:34:39 +08:00
f4d06f195d
We are investigating a memory leak in Sidekiq and saw the following line
when comparing heap dumps over time.
`Allocated IMEMO 14775 objects of size 591000/7389528 (in bytes) at:
/var/www/discourse/app/jobs/onceoff/onceoff.rb:36`
That line in question was doing a `.select { |klass| klass < self }` on
`ObjectSpace.each_object(Class)`. This for some reason is allocating a
whole bunch of `IMEMO` objects which are instruction sequence objects.
Instead of diving deeper into why this might be leaking, we can just
save our time by switching to an implementation that is more efficient
and does not require looping through a ton of objects.
49 lines
1.2 KiB
Ruby
49 lines
1.2 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
class Jobs::Onceoff < ::Jobs::Base
|
|
sidekiq_options retry: false
|
|
|
|
class << self
|
|
attr_reader :onceoff_job_klasses
|
|
|
|
def inherited(klass)
|
|
@onceoff_job_klasses ||= Set.new
|
|
@onceoff_job_klasses << klass
|
|
end
|
|
end
|
|
|
|
def self.name_for(klass)
|
|
klass.name.sub(/\AJobs\:\:/, "")
|
|
end
|
|
|
|
def running_key_name
|
|
"#{self.class.name}:running"
|
|
end
|
|
|
|
# Pass `force: true` to force it happen again
|
|
def execute(args)
|
|
job_name = self.class.name_for(self.class)
|
|
has_lock = Discourse.redis.setnx(running_key_name, Time.now.to_i)
|
|
|
|
# If we can't get a lock, just noop
|
|
if args[:force] || has_lock
|
|
begin
|
|
return if OnceoffLog.where(job_name: job_name).exists? && !args[:force]
|
|
execute_onceoff(args)
|
|
OnceoffLog.create!(job_name: job_name)
|
|
ensure
|
|
Discourse.redis.del(running_key_name) if has_lock
|
|
end
|
|
end
|
|
end
|
|
|
|
def self.enqueue_all
|
|
previously_ran = OnceoffLog.pluck(:job_name).uniq
|
|
|
|
self.onceoff_job_klasses.each do |klass|
|
|
job_name = name_for(klass)
|
|
Jobs.enqueue(job_name.underscore.to_sym) if previously_ran.exclude?(job_name)
|
|
end
|
|
end
|
|
end
|