28 Commits

Author SHA1 Message Date
Daniel Waterworth
30922855f2
PERF: Don't allow a single user to monopolize the defer queue () 2024-02-07 13:47:50 -06:00
Daniel Waterworth
5c92d7da22
FIX: Increase defer queue length ()
It's important that there is a limit, but the current limit is too
restrictive.
2023-11-01 14:02:53 -05:00
Daniel Waterworth
26e267478d
SECURITY: Don't allow a particular site to monopolize the defer queue 2023-07-28 12:53:51 +01:00
David Taylor
29f7ec7090
DEV: Prevent defer stats exception when thread aborted ()
When the thread is aborted, an exception is raised before the `start` of a job is set, and therefore raises an exception in the `ensure` block. This commit checks that `start` exists, and also adds `abort_on_exception=true` so that this issue would have caused test failures.
2023-01-16 09:08:44 +11:00
Sam
7b63c42304
FEATURE: add basic instrumentation to defer queue ()
This will give us some aggregate stats on the defer queue performance.

It is limited to 100 entries (for safety) which is stored in an LRU cache.

Scheduler::Defer.stats can then be used to get an array that denotes:

- number of runs and completions (queued, finished)
- error count (errors)
- total duration (duration)

We can look later at exposing these metrics to gain visibility on the reason
the defer queue is clogged.
2023-01-12 12:29:50 +11:00
David Taylor
6417173082
DEV: Apply syntax_tree formatting to lib/* 2023-01-09 12:10:19 +00:00
Sam
057087e0e8 FEATURE: log long running jobs in the defer queue
If a job in the defer queue takes longer than 90 seconds log an error
2018-10-12 17:03:47 +11:00
Guo Xiang Tan
7eff64773c Revert "FIX: Don't clear active connections in defer queue."
This reverts commit c9feadf9ecf9ee2850074c616a3c83f46bc2a014.
2018-06-19 17:58:21 +08:00
Guo Xiang Tan
df24c51c6f Revert "FIX: Don't try to dequeue an empty queue."
This reverts commit 1af7d4a894ad809de6424d043600e32e7c01bda9.
2018-06-19 15:49:45 +08:00
Guo Xiang Tan
1af7d4a894 FIX: Don't try to dequeue an empty queue. 2018-06-19 15:25:44 +08:00
Guo Xiang Tan
c9feadf9ec FIX: Don't clear active connections in defer queue. 2018-06-19 12:45:16 +08:00
Sam
6974b7d6a8 FIX: run deferred jobs inline in sidekiq 2018-05-23 12:05:37 +10:00
Régis Hanol
93ed8d2522
PERF: defer user notifications () 2018-05-15 09:51:32 +02:00
Sam
2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Sam
7ca08216bd FIX: ensure we have no dangling db connections on threads
This correct 10 second timeouts in dev mode, when reloader kicks in
2017-10-30 14:24:15 +11:00
Guo Xiang Tan
9dcb11f553 Fix the build. 2017-10-11 17:45:19 +08:00
Guo Xiang Tan
36f8697a59 FIX: Exception has to be wrapped in the connection as well. 2017-10-11 17:19:26 +08:00
Guo Xiang Tan
09721090a3 FIX: Ensure that we revert back to default connection after running jobs. 2017-10-11 17:17:03 +08:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. () 2017-07-28 10:20:09 +09:00
Sam
6960639c58 Merge pull request from riking/thrown_logging
Delete old ErrorLog, use Logster for 500 errors
2015-02-23 14:19:16 +11:00
Sam
cdef67667a PERF: allow background jobs to flush between requests in same thread 2015-02-17 09:58:43 +11:00
riking
5657006aca Rename handle_exception to handle_job_exception 2015-02-09 12:47:46 -08:00
Sam
08b790b3c2 improve metrics gathered using in our traffic section
this also pulls out the middleware into its own home and inserts in front
2015-02-05 16:08:52 +11:00
Akshay
b4e38e5646 updated checks of environment with Rails.env by Rails.env methods 2014-08-18 15:36:47 +05:30
riking
d90404e830 Change 'code' to 'message' 2014-07-17 15:19:58 -07:00
riking
12cb682548 Start passing more context to Discourse.handle_exception 2014-07-17 14:11:56 -07:00
Sam
798b8444cf BUGFIX: work correctly if process forks 2014-03-17 15:22:11 +11:00
Sam
2c8ae22b87 FEATURE: add a simple queue Scheduler::Defer.later {}
For quick jobs that do not need to be sent to sidekiq,
runs inline in a single thread but does not block
2014-03-17 12:16:19 +11:00