Sidekiq was failing to boot in dev due to the following error. It seems
like constantizing stuff before the autoloader has kicked in caused
stuff to go weird. Root cause has not been identified but there is no
reason for us to have to warm up the cache during the initialization of
Rails.
```
2021-09-06T04:28:43.338Z pid=112028 tid=26vc WARN: NameError: uninitialized constant #<Class:0x0000564b365040d8>::RateLimiter
2021-09-06T04:28:43.338Z pid=112028 tid=26vc WARN: /discourse/app/models/post.rb:9:in `<class:Post>'
/discourse/app/models/post.rb:6:in `<top (required)>'
/ruby/2.7.4/lib/ruby/gems/2.7.0/gems/zeitwerk-2.4.2/lib/zeitwerk/kernel.rb:26:in `require'
/ruby/2.7.4/lib/ruby/gems/2.7.0/gems/zeitwerk-2.4.2/lib/zeitwerk/kernel.rb:26:in `require'
```
Take 2 of https://github.com/discourse/discourse/pull/13466.
Fixes a few issues with the original PR:
- color definition stylesheet target now includes the theme id, to avoid themes set to use the default color scheme loading the same stylesheet
- changes the internal cache key for color definition stylesheet to reset the pre-existing cache
When editing the first post for the topic we do two AJAX requests
to two separate controllers in this order:
PUT /t/topic-name
PUT /posts/2489523
This causes two post revisor calls, which end up triggering the
:post_edited DiscourseEvent twice. This is then picked up and sent
as a WebHook event twice. However we do not need to send a :post_edited
webhook event if the first post is being edited and topic_changed is
true from the :post_edited DiscourseEvent, because a second event will
shortly come through for just the post.
See https://meta.discourse.org/t/post-webhook-fires-two-times-on-post-edited-for-first-post-in-a-topic/162408
Continued on from https://github.com/discourse/discourse/pull/10590
Over the years we accrued many spelling mistakes in the code base.
This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change
- comments
- test descriptions
- other low risk areas
The auto restart logic was sending a USR2 to the parent process without checking what the parent process actually was. In some situations, it might not be the `bin/unicorn` supervisor.
This commit switches to use a global variable for the supervisor PID. This will be much less prone to unexpected behavior.
This commit adds a listener on (almost) all `.rb` files in the repository. When a change occurs, it checks whether Zeitwerk is responsible for autoloading it. If not, a warning will be printed to the console and the server will be automatically restarted. Optionally, you can pass the `AUTO_RESTART=0` environment variable to prevent auto-restart.
* FEATURE: add support for like webhooks
Add support for like webhooks. Webhook events only send on user membership
in the defined webhook group filters.
This also fixes group webhook events, as before this was never used, and
the logic was not correct.
Adds a webhook to notify when a reviewable score is updated.
This is different from created or status changed as additional flags can
roll in and update the score without updating status. Useful for applications
looking to integrate in with Discourse's scores
This filter hides reviewables with a score lower than the "reviewable_low_priority_threshold" setting. We only use reviewables that already met this threshold to calculate the Medium and High priority filters.
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))
I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
The logster initializer tries to adds RailsMultisite::Formatter to the STDOUT logger. In production, the lograge initializer then removes the RailsMultisite:Formatter because the JSON log will include the database.
e10a74694a used `Rails.application.reloader.to_prepare` to defer running the 100-logster initializer, which meant it ran **after** 101-lograge. This meant that we were writing JSON logs with a non-json text prefix.
The `to_prepare` was added because our freedom-patches are now deferred using `to_prepare`, and some initializers were relying on the freedom patches. However, following 1533cbb38b, we decided to load the RailsMultisite freedom patch without `to_prepare`. Therefore, `005-site_settings` and `100-logster` no longer need to use `to_prepare`. Removing it means that these initializers are back to running in sequential order, and the logging issue will be resolved.
The only remaining initializer which depends on freedom patches is `100-i18n`. I've added a comment to explain why.
Get rid of deprecation related to Zeitwerk autoloader.
Original PR was reverted because of multisite bug #12381 - thank you @davidtaylorhq for fixing it.
I added the last commit to fix that multisite problem.
- Bump rails_failover for new per-backend callback feature
- If the master backend fails over, make all sites readonly. And vice-versa for fallback
- If a single backend fails over, make that individual site readonly. And vice-versa for fallback
- When a single backend fails, also check connection to the master backend
This reverts commit e3de45359f.
We need to improve out strategy by adding a cache breaker with this change ... some assets on CDNs and clients may have incorrect CORS headers which can cause stuff to break.
Allows site administrators to pick different fonts for headings in the wizard and in their site settings. Also correctly displays the header logos in wizard previews.
Previously, I thought it was better to drop the site into reading mode
when Redis has failed over to the replica but it created more errors
while Redis is in readonly mode since ActiveRecord would prevent us from
writing to PG even though PG is up.
The risk here is that the database for one site goes down in the multisite setup and we drop everything to readonly mode. However, I discussed this with Sam and we agree that one database having problem is very rare. Most of the time, it is the entire DB cluster that goes down.
This reverts commit 20780a1eee.
* SECURITY: re-adds accidentally reverted commit:
03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
instead of the 03d26cd6 parent (which contains security fixes)
Upgrades Rails to latest, this version has better compatibility
with Ruby 2.7
During the upgrade we needed a new cleaner mechanism for configuring
message bus.
All tests are green.
If anything weird pops up please revert.
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.
For files with `# frozen_string_literal: true`
```
puts %w{a b}[0].frozen?
=> true
puts "hi".frozen?
=> true
puts "a #{1} b".frozen?
=> true
puts ("a " + "b").frozen?
=> false
puts (-("a " + "b")).frozen?
=> true
```
For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
Previously we would consider a user "present" and "last seen" if the
browser window was visible.
This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.
Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility
This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.
This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.
This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
* Remove some `.es6` from comments where it does not matter
* Use a post processor for transpilation
This will allow us to eventually use the directory structure to
transpile rather than the extension.
* FIX: Some errors and clean up in confirm-new-email
It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.
* Remove `no-module`
Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.
* DEV: Enable babel transpilation based on directory
If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.
* REFACTOR: Remove Tilt/ES6ModuleTranspiler
* FIX: We need to skip users with associated reviewables when auto-approving them
* Update spec/initializers/track_setting_changes_spec.rb
* Update spec/initializers/track_setting_changes_spec.rb
Co-authored-by: Robin Ward <robin.ward@gmail.com>
Tracking down concurrency issues from backtraces and manual repros is a fraught process.
Sometimes you've just got to get your hands dirty and do a live debug.
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways
1. We cache the result in `Discourse.os_hostname` so it is only ever called once
2. We prefer to use Socket.gethostname which avoids making a shell command
This improves performance as we are not spawning hostname processes throughout
the app lifetime
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.
Context at https://meta.discourse.org/t/-/113249
Out-of-the-box Oj uses :object mode, this shifts us to use :compat mode
by default which is safer.
It means any de-serialization going forward will default to this mode.
If we wish to serialize or deserialize arbitrary objects going forward with
no json interfaces we will have to opt in.
- Refactor source_url to avoid using eval in development
- Precompile handlebars in development
- Include template compilers when running qunit
- Remove unsafe-eval in development CSP
- Include unsafe-eval only for qunit routes in development
According to the [Rails
Source](https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L20)
the `ActiveRecord::Migrator.migrations_paths` are overwritten with the
value of `ActiveRecord::Tasks::DatabaseTasks.migrations_paths` every
time the config is loaded.
This caused a bug for Discourse development where if you ran:
`rake db:drop db:create db:migrate` in one line, you would not get our
post migrations, as those had a custom value for `migrations_paths`.
The fix is to use `ActiveRecord::Tasks::DatabaseTasks.migrations_paths`
to set up all our custom paths. Everything seems to work as expected.
Post-zeitwerk, rails has deprecated autoloading modules during
initialization and forces all autoloaded modules to be reloaded after
initialization.
Requiring the file explicitly prevents autoloading and therefore
prevents the state on SiteSetting being trashed which was causing the
problem here.
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains.
We no longer need to use Rails "require_dependency" anywhere and instead can just use standard
Ruby patterns to require files.
This is a far reaching change and we expect some followups here.
Under extreme load on large databases certain regular jobs can take quite
a while to run. We need to ensure we never starve a sidekiq from running
mini scheduler, cause without it we are unable to queue stuff such as
heartbeat jobs.
This adds a 1 minute rate limit to all JS error reporting per IP. Previously
we would only use the global rate limit.
This also introduces DISCOURSE_ENABLE_JS_ERROR_REPORTING, if it is set to
false then no JS error reporting will be allowed on the site.
Previously the default stack suppressor in rack-mini-profiler was excluding
the plugin directory.
This made islolating issues more complicated cause you needed to defer to
pp=full-backtrace which is both slow and noisy
If enabled, this will fire a webhook whenever a user's notification has
been created. This could potentially be a lot of data depending on your
forum, and should be used carefully since it includes everything all users
will see in their feeds.
