DEV: Log error encountered when reopening sidekiq logs (#27411)

We are seeing the following error in our logs when Sidekiq is sent a
`USR1` signal in production when logrotate happens:

```
log writing failed. stream closed in another thread
Error encountered while starting Sidekiq: can't be called from trap context\n/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/unicorn-6.1.0/lib/unicorn/util.rb:71:in `reopen'
```

I'm not quite sure where the error is triggered from so I'm improving
the way we log errors.
This commit is contained in:
Alan Guo Xiang Tan 2024-06-11 12:29:48 +08:00 committed by GitHub
parent 10b9a32abb
commit 9fdcdcf58d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -35,9 +35,18 @@ class Demon::Sidekiq < ::Demon::Base
# Unicorn uses USR1 to indicate that log files have been rotated
Signal.trap("USR1") do
log_in_trap("Sidekiq reopening logs...")
Unicorn::Util.reopen_logs
log_in_trap("Sidekiq done reopening logs...")
begin
log_in_trap("Sidekiq reopening logs...")
Unicorn::Util.reopen_logs
log_in_trap("Sidekiq done reopening logs...")
rescue => error
log_in_trap(
"Error encountered while reopening logs: [#{error.class}] #{error.message}\n#{error.backtrace.join("\n")}",
level: :error,
)
exit 1
end
end
options = ["-c", GlobalSetting.sidekiq_workers.to_s]
@ -58,8 +67,12 @@ class Demon::Sidekiq < ::Demon::Base
cli.parse(options)
load Rails.root + "config/initializers/100-sidekiq.rb"
cli.run
rescue => e
log("Error encountered while starting Sidekiq: #{e.message}\n#{e.backtrace.join("\n")}")
rescue => error
log(
"Error encountered while starting Sidekiq: [#{error.class}] #{error.message}\n#{error.backtrace.join("\n")}",
level: :error,
)
exit 1
end
end