discourse/lib/signal_trap_logger.rb
Alan Guo Xiang Tan 23c38cbf11
DEV: Log Unicorn worker timeout backtraces to Rails.logger (#27257)
This commit introduces the following changes:

1. Introduce the `SignalTrapLogger` singleton which starts a single
   thread that polls a queue to log messages with the specified logger.
   This thread is necessary becasue most loggers cannot be used inside
   the `Signal.trap` context as they rely on mutexes which are not
   allowed within the context.

2. Moves the monkey patch in `freedom_patches/unicorn_http_server_patch.rb` to
   `config/unicorn.config.rb` which is already monkey patching
   `Unicorn::HttpServer`.

3. `Unicorn::HttpServer` will now automatically send a `USR2` signal to
   a unicorn worker 2 seconds before the worker is timed out by the
   Unicorn master.

4. When a Unicorn worker receives a `USR2` signal, it will now log only
   the main thread's backtraces to `Rails.logger`. Previously, it was
   `put`ing the backtraces to `STDOUT` which most people wouldn't read.
   Logging it via `Rails.logger` will make the backtraces easily
   accessible via `/logs`.
2024-06-03 12:51:12 +08:00

50 lines
1.3 KiB
Ruby

# frozen_string_literal: true
# This class is used to log messages to a specified logger from within a `Signal.trap` context. Most loggers rely on
# methods that are prohibited within a `Signal.trap` context, so this class is used to queue up log messages and then
# log them from a separate thread outside of the `Signal.trap` context.
#
# Example:
# Signal.trap("USR1") do
# SignalTrapLogger.instance.log(Rails.logger, "Received USR1 signal")
# end
#
# Do note that you need to call `SignalTrapLogger.instance.after_fork` after forking a new process to ensure that the
# logging thread is running in the new process.
class SignalTrapLogger
include Singleton
def initialize
@queue = Queue.new
ensure_logging_thread_running
end
def log(logger, message, level: :info)
@queue << { logger:, message:, level: }
end
def after_fork
ensure_logging_thread_running
end
private
def ensure_logging_thread_running
return if @thread&.alive?
@thread =
Thread.new do
loop do
begin
log_entry = @queue.pop
log_entry[:logger].public_send(log_entry[:level], log_entry[:message])
rescue => error
Rails.logger.error(
"Error in SignalTrapLogger thread: #{error.message}\n#{error.backtrace.join("\n")}",
)
end
end
end
end
end