Commit Graph

10358 Commits

Author SHA1 Message Date
Sam
4437aced91
FIX: use relations for new_in_category (#30313)
`new_in_category` was using `first` instead of `limit`

This meant it gets an array and that means that you can not operate on it easily in a modifier.

This ensures we always give the modifier a relation, with the notable exception of suggested topics.
2024-12-17 16:39:07 +11:00
David Taylor
ea9cdf7d47
DEV: Compile theme raw-hbr to modules (#30299)
Previously, theme hbr files were compiled to an IIFE, which would be executed before the app is booted. That is causing silenced deprecations to be printed, because the deprecation-workflow isn't set up when the IIFE is run.

This commit updates the theme compiler so that it matches the ember-cli-based raw-hbs compiler. Templates are output to normal modules, which will then be loaded by the existing `eager-load-raw-templates` initializer. This runs after the app has started booting.
2024-12-16 17:31:49 +00:00
Gerhard Schlager
6b3e28216c
FEATURE: Allow pausing of restore before DB migration and uploads are restored (#30269)
This can be helpful if you need to fix problems in the DB before the DB gets migrated as well as before uploads are restored.
2024-12-16 12:50:08 +01:00
David Taylor
ce8c2ef6d9
Revert "DEV: prioritize new email styles over existing, to make customization easier (#30244)" (#30297)
This reverts commit 9694dc6cb0.

Some of our previous email styling depended on this 'incorrect' ordering, so the change caused some text to become illegible. Reverting while we work out a better solution
2024-12-16 11:16:17 +00:00
Loïc Guitaut
9e9abe0a82 DEV: Unify params access in services
Currently, there are two ways (kind of) for accessing `params` inside a
service:
- when there is no contract or it hasn’t been reached yet, `params` is
  just the hash that was provided to the service. To access a key, you
  have to use the bracket notation `params[:my_key]`.
- when there is a contract and it has been executed successfully,
  `params` now references the contract and the attributes are accessible
  using methods (`params.my_key`).

This patch unifies how `params` exposes its attributes. Now, even if
there is no contract at all in a service, `params` will expose its
attributes through methods, that way things are more consistent.

This patch also makes sure there is always a `params` object available
even when no `params` key is provided to the service (this allows a
contract to fail because its attributes are blank instead of having the
service raising an error because it doesn’t find `params` in its context).
2024-12-13 11:13:18 +01:00
Alan Guo Xiang Tan
ebfc33b556
DEV: Remove line of code that does not work (#30258)
We can't delete the file from disk as some of the assets are still
served by the app instead of going through the S3 bucket. It is a bug we
need to fix but it also means this ENV is unsafe now. Just drop the env
until we ensure all assets requested by the app are requested from the
S3 bucket directly.
2024-12-13 09:36:51 +08:00
Kris
9694dc6cb0
DEV: prioritize new email styles over existing, to make customization easier (#30244) 2024-12-12 11:42:50 -05:00
Loïc Guitaut
a589b48f9a DEV: Display better output when inspecting service steps
This patch aims to improve the steps inspector output:
- The service class name is displayed at the top.
- Next to each step is displayed the time it took to run said step.
- Steps that didn’t run are hidden.
- `#inspect` automatically outputs the error when it is present.
2024-12-12 15:21:10 +01:00
Régis Hanol
44cabc3569
FIX: proper details / summary excerpt (#30229)
It doesn't make much sense to have the content of a `<details>` in an excerpt so I replaced them with "▶ summary" instead.

That way, they can't be (ab)used in user cards for example.

Reference - https://meta.discourse.org/t/335094
2024-12-12 09:09:49 +01:00
Bianca Nenciu
a835fd99bd FIX: Truncate bookmarks.name when remapping
The new name may be too long for the bookmarks.name column and raise an
exception. This changes allows the remapper to truncate the new value to
fit (truncates to 100 characters).
2024-12-11 18:53:17 -05:00
Alan Guo Xiang Tan
c97d1d7c59
DEV: Remove max compression level for brotli in assets.rake (#30220)
The `max_compress?` logic is totally broken at least when used for
brotli compression because we are only seeing 4 assets subjected to the
max compression level in production. Instead of fixing the broken logic,
we should just drop this unnecessary complexity cause things are easier
to reason about when we only have one compression level to deal with
across all assets.
2024-12-11 14:01:33 +08:00
Alan Guo Xiang Tan
19321a0b86
DEV: Fix s3:upload_assets not logging newlines (#30219)
Follow-up to 9a2e31b9af
2024-12-11 12:59:17 +08:00
Alan Guo Xiang Tan
9a2e31b9af
DEV: Use a Logger for s3:upload_assets (#30218)
Now that we run the `upload` method in different threads, we need to
synchronize writes to `STDOUT` which we can do so by using a `Logger`.

Follow-up to 49e8353959
2024-12-11 11:48:06 +08:00
Alan Guo Xiang Tan
49e8353959
FIX: s3:upload_assets was uploaded some source maps twice (#30216)
This is because Sprocket's manifest already contains the source maps.
The easy and safe fix here is to just use a `Set` to prevent
duplications.
2024-12-11 11:19:38 +08:00
Bianca Nenciu
b9f8a77d9b
DEV: Upload assets to S3 in parallel (#30210)
In my local setup (with Minio), this uploads the assets to S3 ~40% faster.
2024-12-11 10:51:05 +08:00
Alan Guo Xiang Tan
864b7b6bc8
DEV: Fix flaky test (#30215)
The test was flaky and failing with the following errors:

```
Failure/Error:
  klass
    .connection
    .select_raw(relation.arel) do |result, _|
      result.type_map = DB.type_map
      result.nfields == 1 ? result.column_values(0) : result.values
    end

NoMethodError:
  undefined method `select_raw' for nil

./lib/freedom_patches/fast_pluck.rb:60:in `pluck'
./vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/relation/calculations.rb:354:in `pick'
./app/models/web_crawler_request.rb:27:in `request_id'
./app/models/web_crawler_request.rb:31:in `rescue in request_id'
./app/models/web_crawler_request.rb:26:in `request_id'
./app/models/web_crawler_request.rb:19:in `write_cache!'
./app/models/concerns/cached_counting.rb:135:in `block (3 levels) in flush_to_db'
./vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
./vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `with_connection'
./app/models/concerns/cached_counting.rb:134:in `block (2 levels) in flush_to_db'
./app/models/concerns/cached_counting.rb:124:in `each'
./app/models/concerns/cached_counting.rb:124:in `block in flush_to_db'
./lib/distributed_mutex.rb:53:in `block in synchronize'
./lib/distributed_mutex.rb:49:in `synchronize'
./lib/distributed_mutex.rb:49:in `synchronize'
./lib/distributed_mutex.rb:34:in `synchronize'
./app/models/concerns/cached_counting.rb:120:in `flush_to_db'
./app/models/concerns/cached_counting.rb:187:in `perform_increment!'
./app/models/web_crawler_request.rb:15:in `increment!'
./lib/middleware/request_tracker.rb:74:in `log_request'
./lib/middleware/request_tracker.rb:409:in `block in log_later'
./lib/scheduler/defer.rb:125:in `block in do_work'
./vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
./vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `with_connection'
./lib/scheduler/defer.rb:119:in `do_work'
./lib/scheduler/defer.rb:105:in `block (2 levels) in start_thread'
```

This was due to running the defer thread in an async manner which is
actually no representative of the production environment. It also
revealed a spot in our code base where writes are happening in a GET
request which can cause requests to fail if ActiveRecord is in readonly
mode.
2024-12-11 10:12:58 +08:00
Alan Guo Xiang Tan
eeb01ea0de
DEV: Remove unnecessary thread in Jobs::Base::JobInstrumenter take 2 (#30195)
This reverts commit 766ff723f8.

Ensure that we create the sidekiq log file first before opening it for
logging. This avoids any issue of the log file not being present when we
initialize an instance of the `Logger`.
2024-12-10 12:44:56 +08:00
Michael Brown
c546111703 DEV: add the notion of a 'crawler identifier' in anonymous_cache
We identify and deny blocked crawlers here in anonymous_cache.

Separating the notion of the crawler identifier here lets plugins perform an
override if they perform more advanced detection.
2024-12-09 13:40:22 -05:00
Osama Sayegh
976aca68f6
FEATURE: Restrict profile visibility of low-trust users (#29981)
We've seen in some communities abuse of user profile where bios and other fields are used in malicious ways, such as malware distribution. A common pattern between all the abuse cases we've seen is that the malicious actors tend to have 0 posts and have a low trust level.

To eliminate this abuse vector, or at least make it much less effective, we're making the following changes to user profiles:

1. Anonymous, TL0 and TL1 users cannot see any user profiles for users with 0 posts except for staff users
2. Anonymous and TL0 users can only see profiles of TL1 users and above

Users can always see their own profile, and they can still hide their profiles via the "Hide my public profile" preference. Staff can always see any user's profile.

Internal topic: t/142853.
2024-12-09 13:07:59 +03:00
Alan Guo Xiang Tan
25ce1f3399
PERF: Don't execute a git command each time we log a log line (#30177)
We already have a `GIT_VERSION` constant in `DiscourseLogstashLogger` so
we can just use that.
2024-12-09 11:11:03 +08:00
Martin Brennan
8a89a77248
FIX: Discard empty bundles for reviewables (#30121)
Followup c7e471d35a

It is currently possible to add a bundle (which is a collection
of actions used for a dropdown on the client) for a reviewable
via actions.add_bundle and then never add any actions to it.

This causes the client to explode, as seen in the referenced
commit, because of the way our store expects to resolve objects
referenced by ID that are passed down by the serializer, which
then causes Ember to have an unrecoverable render error.

Fixing this on the serializer level is not really possible because
of all the ActiveModel::Serializer magic that serializes
objects by ID reference when doing things like has_many.
`Reviewable#actions_for` is a better place to do this anyway,
because this is the main location where the bundles and actions
are built for every action via the serializer.
2024-12-05 15:41:13 +10:00
Krzysztof Kotlarek
28b4ff6cb6
FIX: update flag reason message with default value (#30026)
Currently only system flags are translated. When we send message to the user that their post was deleted because of custom flag, we should default to custom flag name.
2024-12-04 14:46:52 +11:00
Kris
60826162b5
A11Y: remove redundant alt text from github oneboxes (#30083) 2024-12-04 12:25:03 +11:00
Gary Pendergast
2513339955
FEATURE: Show when a badge has been granted for a post (#29696)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-12-03 13:43:27 +11:00
Kelv
435fbb7408
DEV: unsilence deprecation warning for old Font Awesome icons (#30028)
* DEV: unsilence deprecation warnings for old Font Awesome icon names

* update fa-user to user font awesome icon name
* update pencil-alt to pencil font awesome 6 icon name
2024-12-03 10:28:39 +08:00
David Taylor
b47ae6d437
UX: Strip multiline comments in github oneboxes (#30040)
We were already stripping comments from GitHub issue/PR oneboxes, but the regex was not correctly matching multiline comments.
2024-12-02 18:08:55 +00:00
Alan Guo Xiang Tan
44a81069ac
DEV: Avoid creating system message when system user initiates restore (#30027)
There is no point creating a message for the system user since it is a
non-human user.
2024-12-02 16:13:38 +08:00
Régis Hanol
7d58793759
DEV: deduplicate inline styles in emails (#30015)
In order to limit issues with duplicate inline CSS definitions, this will now deduplicate inline CSS styles with the "last-to-be-defined-wins" strategy.

Also removes unecessary whitespaces in inline styles.

Context - https://meta.discourse.org/t/resolve-final-styles-in-email-notifications/310219

Co-authored-by: Thomas Kalka <thomas.kalka@gmail.com>
2024-11-30 16:38:45 +01:00
Régis Hanol
20d46c9583
PERF: only diff HTML / Markdown when needed (#30014)
When serializing the `body_changes` in the `PostRevisionSerializer`, we create two diffs: one for the `cooked` and another one for the `raw` version of the post.

Inside `DiscourseDiff`, we generate both `html` and `markdown` diffs when we only need the `html` diffs for the `cooked` version of the post and the `markdown` diff for the `raw` version of the post.

This solves the issue repored in https://meta.discourse.org/t/server-error-accessing-topic-revisions-on-a-specific-topic/339185 where some revisions would return 500 because of a `ArgumentError : Attributes per element limit exceeded` exception when trying to generate the `html` diff on a very large `raw`.
2024-11-30 16:30:30 +01:00
Jarek Radosz
85ead5ac7a
Revert "FIX: deduplicate css in mails (#30003)" (#30013)
This reverts commit 6e726d436f.

The specs were failing in the original PR but the CI didn't run.
2024-11-30 15:32:32 +01:00
Thomas Kalka
6e726d436f
FIX: deduplicate css in mails (#30003)
Feature: Resolve final styles in email notifications

Context - https://meta.discourse.org/t/resolve-final-styles-in-email-notifications/310219
2024-11-30 14:51:02 +01:00
Hoa Nguyen
607dd2cbd8
DEV: improve the plugin:spec rake task (#29050)
Allow the plugin:spec to receive the test file path, rather than always run all tests of the plugin.
2024-11-29 06:33:14 +11:00
Bianca Nenciu
5abee8ac6b
DEV: Log number of live slots used by requests (#29884) 2024-11-28 18:25:48 +02:00
Loïc Guitaut
88f1b3b195 DEV: Try fixing flaky spec related to Scheduler::Defer
Checking if a connection is available is probably not enough, when the
connection is available, we should still verify it’s not stale.
2024-11-28 15:30:13 +01:00
Loïc Guitaut
f69f0211df DEV: Fix flaky spec related to Scheduler::Defer
In some cases in CI env, it seems the AR connection isn’t available and
the `ensure` block is executed. It’s calling `#verify!` on the
connection, so it can fail sometimes. This is probably why
`#clear_active_connections!` was failing too sometimes.

Here, we just check the connection is present before clearing the
connections.
2024-11-28 11:46:52 +01:00
Sam
07813ba83c
DEV: fix hanging spec (#29974) 2024-11-28 11:06:19 +08:00
Sam
72132c35fb
DEV: fix flaky spec (#29972)
Spec was flaky cause work could still be in pipeline after the defer
length is 0. Our length denotes the backlog, not the in progress
count.

This adds a mechanism for gracefully stopping the queue and avoids
wait_for callse
2024-11-28 11:21:35 +11:00
Angus McLeod
6acf673f8d
FIX: topic post counts for webhook post_destroyed event (#29853)
* FIX: topic post counts for webhook post_destroyed event

- Generate webhook data after posts are destroyed
- Don't count user_deleted posts

* Remove unnecessary conditional
2024-11-27 11:36:51 -08:00
Loïc Guitaut
fac6147039 DEV: Verify DB connection before trying to clear active connections 2024-11-27 18:12:11 +01:00
Loïc Guitaut
d6bec460a8 DEV: Upgrade Rails to version 7.2 2024-11-27 10:48:47 +01:00
Ted Johansson
f4d0a77d5f
DEV: Add "delete user" options to illegal flag review (#29956)
We already add the "delete user" and "delete and block user" options to the drop-down for potential spam, but we should do this for potentially illegal posts as well.

This is entirely based on the implementation for the potential spam one, including caching the status on the reviewable record.

Also note that just as for potential spam, the user must be "deletable" for the option to appear.

I also took the liberty to move the options in the drop-down to what I think is a more intuitive place. (Between delete post and suspend/silence user.)
2024-11-27 17:23:57 +08:00
Martin Brennan
2ef9d6ac47
FEATURE: Allow admins to force refresh "What's new?" (#29911)
Sometimes changes to "What's new?" feed items are made or the feed items are
removed altogether, and the polling interval to check for new features is 1 day.

This is quite long, so this commit introduces a "Check for updates"
button for admins to click on the "What's new?" page which will bust
the cache for the feed and check again at the new features endpoint.
This is limited to 5 times per minute to avoid rapid sending of
requests.
2024-11-27 09:40:55 +10:00
Martin Brennan
b8a5f95eb6
FIX: Handle multiple In-Reply-To Message-ID in group inbox (#29912)
This fix handles the case where an In-Reply-To mail header
can contain multiple Message-IDs. We use this header to
try look up an EmailLog record to find the post to reply
to in the group email inbox flow.

Since the case where multiple In-Reply-To Message-IDs is
rare (we've only seen a couple of instances of this causing
errors in the wild), we are just going to use the first one
in the array.

Also, Discourse does not support replying to multiple posts
at once, so it doesn't really make sense to use multiple
In-Reply-To Message-IDs anyway.
2024-11-26 11:12:40 +10:00
Selase Krakani
a20b7fa83f
DEV: Gracefully handle regex_replace max column length violations (#29787)
* DEV: Gracefully handle `regex_replace` violations of column length constraints

This is a follow-up to the `remap` [refactor](9b0cfa99c5).
Similar to `remap`, the entire `regex_replace` operation fails if the new content exceeds the column’s max length.

This change introduces an optional mode, controlled by the new `skip_max_length_violations` param
to skip records eligible for `regex_replace`  where the new content violates the max column length constraint.

It also includes updates to the exception message raised when `regex_replace` fails to include more details

* DEV: Remove string escapes in heredoc text
2024-11-25 11:39:53 +00:00
Osama Sayegh
eaa3f813c1
FIX: Don't secure the about banner image (#29889)
Uploads that are linked to site settings shouldn't be flagged as secure in login-required sites that enable secure uploads. However, in order for site setting uploads to not be marked secured, the frontend uploader has to include 2 params in the upload request: `for_site_setting: true` and `type: "site_setting"`.

Since these 2 params are semantically identical, we want the `type: "site_setting"` param alone to make the upload correctly treated as a site setting upload. To achieve that, we need to include the `site_setting` type in the public types list because the `for_site_setting` param has the same effect — it marks the upload as a public type.

b138eaf9e5/lib/upload_security.rb (L128-L131)
2024-11-25 11:12:00 +03:00
Ted Johansson
88af23e1ca
DEV: Modernize admin user fields (#29843)
This PR modernizes the user fields area of the admin UI. It is largely based on the work on the emoji section.
2024-11-25 11:54:43 +08:00
Mark VanLandingham
d880db3b7b
DEV: Apply modifier for topic_view link_counts (#29883) 2024-11-22 14:49:39 -06:00
Jarek Radosz
2589545623
DEV: Detect hbr topic list customizations (#29793) 2024-11-21 16:00:49 +01:00
Loïc Guitaut
581fb97bfa DEV: Fix benchmark script
Following a recent commit (cb4b8146a3),
the benchmark script wasn’t working anymore (and the related rake task).

This patch fixes it. It also adds some information about Ruby YJIT being
enabled or not.
2024-11-20 14:36:44 +01:00
David Taylor
32665cf9dd
DEV: Consolidate i18n import paths (#29804)
Enables our new eslint rules which enforce consistent i18n imports. For more info, see 0d58b40cd7
2024-11-19 20:45:18 +00:00