Commit Graph

1357 Commits

Author SHA1 Message Date
Angus McLeod
427ea5301b
Apply the same url normalisation to embed_urls inserted in the PostCreator (#30641) 2025-01-09 14:59:08 -05:00
Gary Pendergast
ec30b6f6c6
FIX: Inline oneboxes should obey the locale. (#30664)
Following on from f369db5ae9, we need to apply a similar fix to inline oneboxes, since they use a different code path to retrieve the onebox provider data.

This change ensures the Accept-Language header is sent by inline onebox requests, too.
2025-01-09 17:22:22 +11:00
Gary Pendergast
f53c734ba6
FEATURE: Add a onebox_locale site setting. (#30655)
Following on from f369db5ae9, this change adds the ability to choose a custom locale to send to onebox providers.

If this setting is left blank, it will fall back to using default_locale.
2025-01-09 14:11:37 +11:00
Gary Pendergast
f369db5ae9
FIX: Ensure Onebox requests ask for the correct language. (#30637)
Onebox embeds currently default to accepting any language response from the destination, which can have some surprising behaviour. For example the `curl` equivalent of what Onebox does:

```
% curl -si -H "Accept-Language: *" 'https://developer.android.com/studio' | grep location:
location: /studio?hl=hi
```

This PR uses the value of `SiteSetting.default_locale` to populate the `Accept-Language` header, falling back to English if that isn't available, then finally accepting whatever language the destination makes available.
2025-01-09 09:08:27 +11:00
Martin Brennan
c1a46995a7
FIX: Wizard improvements post-merge part 1 (#30612)
* FIX: Wizard improvements post-merge part 1

Followup 3135f472e2

Fixes the following:

* On mobile, the Styling step was very narrow
* When clicking Next on the Styling step after previously
  selecting Hot, we got an error

Also makes the following UX improvements for the preview:

* Use different topic titles for Latest and Hot
* Also make Hot view and reply numbers higher

This helps differentiate the two previews.

* DEV: Review fixes
2025-01-07 17:01:05 +10:00
Natalie Tay
0f0b3a21e6
FIX: Allow attachments to be opened in a new tab instead of downloading them (#30535)
Back then in 31e31ef, we added the Content-Disposition headers so that SVGs get downloaded instead of get run in the browser. Inadvertently, this also causes other attachments like pdfs and videos to be downloaded instead of heeding the "Open in new tab" option that users choose in the browser.

When the header is removed, the default value is "inline", this allows the browser to perform as requested. This also applies to other file types like pdfs, allowing users to "Open in new tab" and view them in the browser instead of always downloading them.

Existing tests (#10205) already do check that SVGs remain downloaded. Some existing tests written for PDFs have been modified to cater for SVGs instead, when there was a bug in defining the filenames per #10108
2025-01-07 10:32:32 +08:00
Martin Brennan
725e146dca
FIX: Calculate experiment_enabled on server for "What's new?" (#30599)
Experimental "What's new?" feature feed items previously calculated
a boolean for experimentEnabled on the client based on the siteSettings
service, and this would control the initial state of the experiment
toggle.

However this requires the person who creates the site setting for the
experiment to remember to set it to `client: true`. This commit removes
that manual step by calculating whether the experiment is enabled
server-side, where we have access to all the site settings.
2025-01-07 11:27:24 +10:00
Jarek Radosz
5ce33991f4
DEV: Fix flaky deprecated setting specs (#30550)
…and remove obsolete (and already-disabled) TL-migration related specs
2025-01-04 12:55:22 +01:00
Jarek Radosz
a33e9bc2f9
DEV: Remove deprecated search_tokenize_chinese_japanese_korean setting (#30549)
It was supposed to be removed ~3 years ago (there was [a migration](b4f0a8748d/db/migrate/20220126052157_change_segment_cjk_site_setting.rb) already)
2025-01-03 23:21:18 +01:00
Martin Brennan
6b36b0b68d
FIX: Reports did not respect user locale (#30524)
Our bulk report endpoint uses `hijack`, which does not
use the current user's locale via the `with_resolved_locale`
method in `ApplicationController`. This is happening because
we are doing `around_action` to set the locale, then calling
the code in the block inside the action directly when we use
`hijack`.

We can fix this by capturing `I18n.locale` when starting the
hijack then using `I18n.with_locale` when evaluating the
block inside `hijack`, this way the translations will always
use the correct locale based on the current user.
2025-01-02 13:05:53 +10:00
Martin Brennan
3135f472e2
FEATURE: Improve wizard quality and rearrange steps (#30055)
This commit contains various quality improvements to
our site setup wizard, along with some rearrangement of
steps to improve the admin setup experience and encourage
admins to customize the site early to avoid "all sites look the
same" sentiment.

#### Step rearrangement

* “Your site is ready” from 3 → 4
* “Logos” from 4 → 5
* “Look and feel” from 5 → 3

#### Font selector improvements

Changes the wizard font selector dropdown to show
a preview of all fonts with a CSS class so you don't
have to choose the font to get a preview.

Also makes the fonts appear in alphabetical order.

#### Preview improvements

Placeholder text changed from lorem ipsum to actual topic titles,
category names, and post content. This makes it feel more "real".

Fixes "undefined" categories. Added a date to the topic timeline.

Fixes button rectangles and other UI elements not changing in
size when the font changed, leading to cut off text which looked super
messy. Also fixed some font color issues.

Fixed table header alignment for Latest topic list.

#### Homepage style selector improvements

Limited the big list of homepage styles to Latest, Hot, Categories with latest topics,
and Category boxes based on research into the most common options.

#### Preview header

Changed the preview header to move the hamburger to the left
and add a chat icon

#### And more!

Changed the background of the wizard to use our branded blob style.
2025-01-02 09:28:23 +10:00
Régis Hanol
d523c37057
FIX: correctly extract body and/or reply from exchange emails (#30512)
When receiving emails sent with Exchange, we look for some markers to identify the body of the mail and the reply (aka. previous email).

For some reasons, those markers aren't 100% reliable and sometimes, only one of them is present.

The commit 20ba54d536 introduced the bug because the `HTML_EXTRACTERS` regex for exchange looks for either `messageBodySection` or `messageReplySection` but we were only using the `reply` section. So if an email had only the `body` section, it would not be correctly extracted.

This commit handle the cases where either one of them is missing and use the other one as the actual "reply". When both are present, it correctly elides the "reply" section.
2024-12-31 15:29:36 +01:00
zogstrip
4d889f2ef8 FEATURE: add support for One-Click unsubscribe (RFC 8058)
We were missing the "List-Unsubscribe-Post" header in emails we sent to allow Yahoo / GMail and others to automagically show a link to unsubscribe.

Internal ref - t/144713
2024-12-31 15:28:59 +01:00
Sam
9b9babdd72
FEATURE: allow disabling of anti spam profile hiding feature (#30508)
The profile hiding feature is particularly problematic on sites that are
private (invite only or must approve users) so it is unconditionally disabled.

Also certain sites may prefer to disable the anti spam feature, they can
opt out using `hide_new_user_profiles`


Co-authored-by: Martin Brennan <martin@discourse.org>
2024-12-31 15:04:53 +11:00
Osama Sayegh
3187606d34
FEATURE: Add option to hide full name input at signup (#30471)
This commit replaces the `full_name_required` setting with a new `full_name_requirement` setting to allow more flexibility with the name field in the signup form. The new setting has 2 options, "Required at signup" and "Optional at signup", which are equivalent to the true/false possibilities of the old setting, and a third option "Hidden at signup" that hides the name field from the signup form, making it effectively optional too.

New sites will have the "Hidden at signup" option as the default option, and existing site will continue to use the option that maps to their current configuration.

Internal topic: t/136746.
2024-12-30 22:26:20 +03:00
Gary Pendergast
92793c5b73
DEV: Ensure all site settings have a keyword entry (#30485) 2024-12-30 12:22:55 +11:00
Alan Guo Xiang Tan
859d61003e
DEV: API to register custom request rate limiting conditions (#30239)
This commit adds the `add_request_rate_limiter` plugin API which allows plugins to add custom rate limiters on top of the default rate limiters which requests by a user's id or the request's IP address.

Example to add a rate limiter that rate limits all requests from Googlebot under the same rate limit bucket:

```
add_request_rate_limiter(
  identifier: :country,
  key: ->(request) { "country/#{DiscourseIpInfo.get(request.ip)[:country]}" },
  activate_when: ->(request) { DiscourseIpInfo.get(request.ip)[:country].present? },
)
```
2024-12-23 09:57:18 +08:00
Sam
c315e26485
FIX: handle more thread pool edge cases (#30392)
* Split `shutdown` into two separate methods for better control:
  - `shutdown` - signals threads to stop accepting new work
  - `wait_for_termination` - waits for threads to finish (with optional timeout)

* Add tracking of busy threads via `@busy_threads` Set
* Make idle_time parameter optional with 30-second default
* Improve thread spawning logic:
  - Spawn initial thread immediately when work is posted
  - Spawn additional threads when all threads are busy and work is queued
* Fix race condition in work distribution
* Add busy thread count to stats output
* Add test coverage for zero min_threads configuration

This commit makes the ThreadPool more reliable, easier to use, and adds 
better visibility into its internal state.

---------

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-12-20 11:50:00 +11:00
Sam
efa50a4da2
FEATURE: ThreadPool implementation (#30364)
This commit introduces a new ThreadPool class that provides efficient worker
thread management for background tasks. Key features include:

- Dynamic scaling from min to max threads based on workload
- Proper database connection management in multisite setup
- Graceful shutdown with task completion
- Robust error handling and logging
- FIFO task processing with a managed queue
- Configurable idle timeout for worker threads

The implementation is thoroughly tested, including stress tests, error
scenarios, and multisite compatibility.
2024-12-20 07:37:12 +11:00
Alan Guo Xiang Tan
e4e5db57f0
DEV: Fix undefined method check_email_sync_heartbeat in unicorn conf (#30360)
This is a follow-up to 9812407f76
2024-12-19 10:10:11 +08:00
Loïc Guitaut
133a648d9b DEV: Fix policy classes delegating their #call method in services
There’s currently a bug when using a dedicated class as a policy in
services: if that class delegates its `#call` method (to an underlying
strategy object for example), then an error will be raised saying steps
aren’t allowed to provide default parameters.

This should not happen, and this patch fixes that issue.
2024-12-18 09:59:40 +01:00
Alan Guo Xiang Tan
9812407f76
FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198)
This commit reimplements how we monitor Sidekiq processes that are
forked from the Unicorn master process. Prior to this change, we rely on
`Jobs::Heartbeat` to enqueue a `Jobs::RunHeartbeat` job every 3 minutes.
The `Jobs::RunHeartbeat` job then sets a Redis key with a timestamp. In
the Unicorn master process, we then fetch the timestamp that has been set
by the job from Redis every 30 minutes. If the timestamp has not been
updated for more than 30 minutes, we restart the Sidekiq process. The
fundamental flaw with this approach is that it fails to consider
deployments with multiple hosts and multiple Sidekiq processes. A
sidekiq process on a host may be in a bad state but the heartbeat check
will not restart the process because the `Jobs::RunHeartbeat` job is
still being executed by the working Sidekiq processes on other hosts.

In order to properly ensure that stuck Sidekiq processs are restarted,
we now rely on the [Sidekiq::ProcessSet](https://github.com/sidekiq/sidekiq/wiki/API#processes)
API that is supported by Sidekiq. The API provides us with "near real-time (updated every 5 sec)
info about the current set of Sidekiq processes running". The API
provides useful information like the hostname, pid and also when Sidekiq
last did its own heartbeat check. With that information, we can easily
determine if a Sidekiq process needs to be restarted from the Unicorn
master process.
2024-12-18 12:48:50 +08:00
Sam
4437aced91
FIX: use relations for new_in_category (#30313)
`new_in_category` was using `first` instead of `limit`

This meant it gets an array and that means that you can not operate on it easily in a modifier.

This ensures we always give the modifier a relation, with the notable exception of suggested topics.
2024-12-17 16:39:07 +11:00
Ted Johansson
c1c7ea8959
DEV: Change hide_email_address_taken default to true (#30293)
We're changing the default of hide_email_address_taken to true. This is a trade-off we want to make, as it prevents account enumeration with minimal impact on legitimate users. If you forget you have an account and try to sign up again with the same e-mail you'll receive an e-mail letting you know.
2024-12-17 10:46:04 +08:00
David Taylor
ea9cdf7d47
DEV: Compile theme raw-hbr to modules (#30299)
Previously, theme hbr files were compiled to an IIFE, which would be executed before the app is booted. That is causing silenced deprecations to be printed, because the deprecation-workflow isn't set up when the IIFE is run.

This commit updates the theme compiler so that it matches the ember-cli-based raw-hbs compiler. Templates are output to normal modules, which will then be loaded by the existing `eager-load-raw-templates` initializer. This runs after the app has started booting.
2024-12-16 17:31:49 +00:00
Loïc Guitaut
9e9abe0a82 DEV: Unify params access in services
Currently, there are two ways (kind of) for accessing `params` inside a
service:
- when there is no contract or it hasn’t been reached yet, `params` is
  just the hash that was provided to the service. To access a key, you
  have to use the bracket notation `params[:my_key]`.
- when there is a contract and it has been executed successfully,
  `params` now references the contract and the attributes are accessible
  using methods (`params.my_key`).

This patch unifies how `params` exposes its attributes. Now, even if
there is no contract at all in a service, `params` will expose its
attributes through methods, that way things are more consistent.

This patch also makes sure there is always a `params` object available
even when no `params` key is provided to the service (this allows a
contract to fail because its attributes are blank instead of having the
service raising an error because it doesn’t find `params` in its context).
2024-12-13 11:13:18 +01:00
Loïc Guitaut
a589b48f9a DEV: Display better output when inspecting service steps
This patch aims to improve the steps inspector output:
- The service class name is displayed at the top.
- Next to each step is displayed the time it took to run said step.
- Steps that didn’t run are hidden.
- `#inspect` automatically outputs the error when it is present.
2024-12-12 15:21:10 +01:00
Régis Hanol
44cabc3569
FIX: proper details / summary excerpt (#30229)
It doesn't make much sense to have the content of a `<details>` in an excerpt so I replaced them with "▶ summary" instead.

That way, they can't be (ab)used in user cards for example.

Reference - https://meta.discourse.org/t/335094
2024-12-12 09:09:49 +01:00
Bianca Nenciu
a835fd99bd FIX: Truncate bookmarks.name when remapping
The new name may be too long for the bookmarks.name column and raise an
exception. This changes allows the remapper to truncate the new value to
fit (truncates to 100 characters).
2024-12-11 18:53:17 -05:00
Sérgio Saquetim
1505978586
DEV: Upgrade dependencies to Ember 5.12 (#30131) 2024-12-11 11:09:25 -03:00
Osama Sayegh
976aca68f6
FEATURE: Restrict profile visibility of low-trust users (#29981)
We've seen in some communities abuse of user profile where bios and other fields are used in malicious ways, such as malware distribution. A common pattern between all the abuse cases we've seen is that the malicious actors tend to have 0 posts and have a low trust level.

To eliminate this abuse vector, or at least make it much less effective, we're making the following changes to user profiles:

1. Anonymous, TL0 and TL1 users cannot see any user profiles for users with 0 posts except for staff users
2. Anonymous and TL0 users can only see profiles of TL1 users and above

Users can always see their own profile, and they can still hide their profiles via the "Hide my public profile" preference. Staff can always see any user's profile.

Internal topic: t/142853.
2024-12-09 13:07:59 +03:00
Kelv
1ca90c3070
DEV: update more deprecated font awesome icon names (#30123)
* update more deprecated font awesome icon names

* add migration to remove deprecated default for badges.icon
2024-12-05 15:20:02 +08:00
Krzysztof Kotlarek
28b4ff6cb6
FIX: update flag reason message with default value (#30026)
Currently only system flags are translated. When we send message to the user that their post was deleted because of custom flag, we should default to custom flag name.
2024-12-04 14:46:52 +11:00
Juan David Martínez Cubillos
7b70905326
FIX: Sanitization issue when replacing default emoji with custom emoji that contains or ~/Discourse/discourse symbols (#30053) 2024-12-03 19:27:12 -05:00
Loïc Guitaut
cf2b4d9934 DEV: Apply new Rubocop linting on services 2024-12-02 17:31:36 +01:00
Jarek Radosz
b3423c40b0
DEV: Fix random typos (#30023)
(late) Nov 2024 edition
2024-12-02 07:34:05 +08:00
Régis Hanol
7d58793759
DEV: deduplicate inline styles in emails (#30015)
In order to limit issues with duplicate inline CSS definitions, this will now deduplicate inline CSS styles with the "last-to-be-defined-wins" strategy.

Also removes unecessary whitespaces in inline styles.

Context - https://meta.discourse.org/t/resolve-final-styles-in-email-notifications/310219

Co-authored-by: Thomas Kalka <thomas.kalka@gmail.com>
2024-11-30 16:38:45 +01:00
Jarek Radosz
85ead5ac7a
Revert "FIX: deduplicate css in mails (#30003)" (#30013)
This reverts commit 6e726d436f.

The specs were failing in the original PR but the CI didn't run.
2024-11-30 15:32:32 +01:00
Thomas Kalka
6e726d436f
FIX: deduplicate css in mails (#30003)
Feature: Resolve final styles in email notifications

Context - https://meta.discourse.org/t/resolve-final-styles-in-email-notifications/310219
2024-11-30 14:51:02 +01:00
Sam
07813ba83c
DEV: fix hanging spec (#29974) 2024-11-28 11:06:19 +08:00
Ted Johansson
2d945e2373
DEV: Enable the normalize_emails site setting by default (#29952) 2024-11-28 10:23:00 +08:00
Sam
72132c35fb
DEV: fix flaky spec (#29972)
Spec was flaky cause work could still be in pipeline after the defer
length is 0. Our length denotes the backlog, not the in progress
count.

This adds a mechanism for gracefully stopping the queue and avoids
wait_for callse
2024-11-28 11:21:35 +11:00
Loïc Guitaut
d6bec460a8 DEV: Upgrade Rails to version 7.2 2024-11-27 10:48:47 +01:00
Martin Brennan
2ef9d6ac47
FEATURE: Allow admins to force refresh "What's new?" (#29911)
Sometimes changes to "What's new?" feed items are made or the feed items are
removed altogether, and the polling interval to check for new features is 1 day.

This is quite long, so this commit introduces a "Check for updates"
button for admins to click on the "What's new?" page which will bust
the cache for the feed and check again at the new features endpoint.
This is limited to 5 times per minute to avoid rapid sending of
requests.
2024-11-27 09:40:55 +10:00
Martin Brennan
b8a5f95eb6
FIX: Handle multiple In-Reply-To Message-ID in group inbox (#29912)
This fix handles the case where an In-Reply-To mail header
can contain multiple Message-IDs. We use this header to
try look up an EmailLog record to find the post to reply
to in the group email inbox flow.

Since the case where multiple In-Reply-To Message-IDs is
rare (we've only seen a couple of instances of this causing
errors in the wild), we are just going to use the first one
in the array.

Also, Discourse does not support replying to multiple posts
at once, so it doesn't really make sense to use multiple
In-Reply-To Message-IDs anyway.
2024-11-26 11:12:40 +10:00
Selase Krakani
a20b7fa83f
DEV: Gracefully handle regex_replace max column length violations (#29787)
* DEV: Gracefully handle `regex_replace` violations of column length constraints

This is a follow-up to the `remap` [refactor](9b0cfa99c5).
Similar to `remap`, the entire `regex_replace` operation fails if the new content exceeds the column’s max length.

This change introduces an optional mode, controlled by the new `skip_max_length_violations` param
to skip records eligible for `regex_replace`  where the new content violates the max column length constraint.

It also includes updates to the exception message raised when `regex_replace` fails to include more details

* DEV: Remove string escapes in heredoc text
2024-11-25 11:39:53 +00:00
Mark VanLandingham
d880db3b7b
DEV: Apply modifier for topic_view link_counts (#29883) 2024-11-22 14:49:39 -06:00
Loïc Guitaut
719457e430 DEV: Add a try step to services
This patch adds a new step to services named `try`.

It’s useful to rescue exceptions that some steps could raise. That way,
if an exception is caught, the service will stop its execution and can
be inspected like with any other steps.

Just wrap the steps that can raise with a `try` block:
```ruby
try do
  step :step_that_can_raise
  step :another_step_that_can_raise
end
```
By default, `try` will catch any exception inheriting from
`StandardError`, but we can specify what exceptions to catch:
```ruby
try(ArgumentError, RuntimeError) do
  step :will_raise
end
```

An outcome matcher has been added: `on_exceptions`. By default it will
be executed for any exception caught by the `try` step.
Here also, we can specify what exceptions to catch:
```ruby
on_exceptions(ArgumentError, RuntimeError) do |exception|
  …
end
```

Finally, an RSpec matcher has been added:
```ruby
  it { is_expected.to fail_with_exception }
  # or
  it { is_expected.to fail_with_exception(ArgumentError) }
```
2024-11-19 12:01:07 +01:00
Selase Krakani
9b0cfa99c5
DEV: Gracefully handle remaps which violate DB column constraints (#29501)
* DEV: Gracefully handle remaps which violate DB column constraints

This change implements length constraint enforcement to skip remaps
which exceed column max lengths

* DEV: Only perform skipped column stats lookup when verbose is true

* DEV: Tidy up specs

* DEV: Make skipping violating remap behaviour opt-in

This change introduces a new `skip_max_length_violations` param for
`remap`, set to `false` by default to ensure we still continue to fail
hard when max lenth constraints are violated.

To aid in quick resolution when remaps fail, this change also
adds more context to the exception message to include the offending table
and column information

* Apply suggestions from code review

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>

* FIX: Various fixes

- Linter errors
- Remap status "logger" early return condition

---------

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2024-11-15 10:42:25 +00:00
Jan Cernik
234133bd3b
UX: Split hide_profile_and_presence user option (#29632)
It splits the hide_profile_and_presence user option and the default_hide_profile_and_presence site setting for more granular control. It keeps the option to hide the profile under /u/username/preferences/interface and adds the presence toggle in the quick user menu.

Co-authored-by: Régis Hanol <regis@hanol.fr>
2024-11-12 22:22:58 -03:00