Commit Graph

11962 Commits

Author SHA1 Message Date
Loïc Guitaut
5ec227334a FIX: Don’t list values from disabled plugins
Currently, when a plugin registers a new reviewable type or extends a
list method (through `register_reviewble_type` and `extend_list_method`
respectively), the new array is statically computed and always returns
the same value. It will continue to return the same value even if the
plugin is disabled (it can be a problem in a multisite env too).

To address this issue, this patch changes how `extend_list_method`
works. It’s now using `DiscoursePluginRegistry.define_filtered_register`
to create a register on the fly and store the extra values from various
plugins. It then combines the original values with the ones from the
registry. The registry is already aware of disabled plugins, so when a
plugin is disabled, its registered values won’t be returned.
2024-07-11 10:51:48 +02:00
Alan Guo Xiang Tan
66878a9e80
DEV: Improve logging of Sidekiq errors when logstash logger is enabled (#27855)
This commit improves the logging of Sidekiq errors when
`ENABLE_LOGSTASH_LOGGER` is set to 1. Prior to this change, we would
only log the message and the backtrace. After this change, useful
information like `job.class`, `job.opts`, `job.problem_db`,
`exception.class` and `exception.message` are included in the log line
as well.
2024-07-11 14:17:18 +08:00
Martin Brennan
7b627dc14b
FIX: Office365/Outlook auth method for group SMTP (#27854)
Both office365 and outlook SMTP servers need LOGIN
SMTP authentication instead of PLAIN (which is what
we are using by default). This commit uses that
unconditionally for these servers, and also makes
sure to use STARTTLS for them too.
2024-07-11 16:16:54 +10:00
Kris
0e3ed7ea2a
A11Y: improve topic list table markup for screenreaders (#27808)
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-10 13:14:36 -04:00
Loïc Guitaut
ab99f31760 DEV: Fix the I18n integrity spec
Before Rails 7.1, the `config.i18n.raise_on_missing_translations` option
was raising only in controllers and views, now it’s anywhere in the app.
It means it raises each time `#description` is called for a setting that
is missing a proper description (and we have a ton of them). Most of the
time it’s fine, as those are usually settings that aren’t shown to the
user.

We can’t just let the code blow up every time there’s a setting with a
missing description, that’s why it’s currently returning an empty
string when the translation is missing.

However, this silently broke our I18n integrity spec that was relying on
the old “Translation missing” message to detect missing translations.

This patch addresses this issue by checking the description isn’t an
empty string. It caught a missing translation by the way.
2024-07-10 11:39:13 +02:00
Régis Hanol
758b9dd0ba
FEATURE: email attachments in a details (#27804)
This change how we present attachments from incoming emails to now be "hidden" in a "[details]" so they don't "hang" at the end of the post.

This is especially useful when using Discourse as a support tool where email is the main communication channel. For various reasons, images are often duplicated by email user agents, and hiding them behind the details block help keep the conversation focused on the isssue at hand.

Internal ref t/122333
2024-07-10 09:59:27 +02:00
Loïc Guitaut
301713ef96 DEV: Upgrade the MessageFormat library (JS)
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.

Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.

We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.

The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.

Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.

A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.

The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
2024-07-10 09:51:25 +02:00
Bianca Nenciu
6591a0654b
FIX: Destroy Drafts when increasing sequences (#27739)
Drafts used to be deleted instead of being destroyed. The callbacks that
clean up the upload references were not being called. As a result, the
upload references were not cleaned up and uploads were not deleted
either. This has been partially fixed in 9655bf3e.
2024-07-10 10:43:11 +03:00
Alan Guo Xiang Tan
c9775d5f72
DEV: Apply Logster.store.ignore to DiscourseLogstashLogger as well (#27819)
This commit updates `DiscourseLogstashLogger#add_with_opts` to avoid
logging messages that matches regexp patterns configured in
`Logster.store.ignore`. Those error logs are mostly triggered by clients
and do not serve any useful purpose.
2024-07-10 13:51:42 +08:00
Alan Guo Xiang Tan
b4b7fa17af
DEV: Add exception class/message to DiscourseLogstashLogger take 2 (#27815)
This is the second take of af2bd4cc50 to
account for messages which contains newlines.
2024-07-10 11:04:17 +08:00
Alan Guo Xiang Tan
0b64cb9d8c
Revert "DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)" (#27814)
This reverts commit bb0daa33cd.

This commit was not causing the problems we thought it was.
2024-07-10 10:36:22 +08:00
Alan Guo Xiang Tan
af2bd4cc50
DEV: Add exception class and message fields to DiscourseLogstashLogger (#27787)
This commit updates `DiscourseLogstashlogger` to add the
`exception_class` and `exception_message` field to the log line when the
`progname` of the log message is `web-exception` which is Logster's
logging of exceptions during a web request.

The `exception_class` and `exception_message` fields allows consumers of
the logs to easily group logs together.
2024-07-10 08:54:39 +08:00
Martin Brennan
560e8aff75
FEATURE: Allow oneboxing private GitHub URLs (#27705)
This commit adds the ability to onebox private GitHub
commits, pull requests, issues, blobs, and actions using
a new `github_onebox_access_token` site setting. The token
must be set up in correctly to have access to the repos needed.

To do this successfully with the Oneboxer, we need to skip
redirects on the github.com host, otherwise we get a 404
on the URL before it is translated into a GitHub API URL
and has the appropriate headers added.
2024-07-10 09:39:31 +10:00
Sérgio Saquetim
bbd67eff08
DEV: Improve the sidebar section expansion handling (#27805)
Handles the cases where the sections titles are Unicode only strings, allowing them to be expanded separately if the Unicode string contains letters.

Also prevents a sidebar section with the header hidden to be displayed collapsed.
2024-07-09 18:32:29 -03:00
Martin Brennan
7a7bdc9be5
FEATURE: Use group based setting for unsafe-none COOP (#27783)
Followup 3ff7ce78e7

Basing this setting on referrer was too brittle --
the referrer header can easily be ommitted or changed.
Instead, for the small amount of use cases that this
site setting serves, we can use a group-based setting
instead, changing it to `cross_origin_opener_unsafe_none_groups`
instead.
2024-07-09 11:25:49 -05:00
Guhyoun Nam
a01be4150a
DEV: Specs for redeliver_web_hook_events job (#27779)
It is a PR to add a spec for checking redeliver_web_hook_events job not to delete webhook event in process.
2024-07-09 10:35:10 -05:00
Régis Hanol
0846862cb5
FIX: deleted topic author in crawler view (#27788)
When a crawler visits a topic that has a deleted author, it would error because the `show.html.erb` view was expecting a user to be always present.

This ensure we don't render the "author" meta data when the author of the topic has been deleted.

Internal ref t/132508
2024-07-09 10:44:03 +02:00
David Battersby
f44ec18fd2
DEV: update base url links to respect subfolder installs (#27740)
This change eliminates a couple of instances where subfolder urls are badly formatted, in most cases we can use Discourse.base_url_no_prefix to prevent adding the subfolder to the base url.
2024-07-09 12:42:38 +04:00
Martin Brennan
e58cf24fcc
FEATURE: Topic view stats report (#27760)
Adds a report to show the top 100 most viewed topics in a date range,
combining logged in and anonymous views. Can be filtered by category.

This is a followup to 527f02e99f
and d1191b7f5f. We are also going to
be able to see this data in a new topic map, but this admin report
helps to see an overview across the forum for a date range.
2024-07-09 15:39:10 +10:00
Alan Guo Xiang Tan
86e5f46175
DEV: Add hidden s3_inventory_bucket_region site setting (#27786)
This commit adds a hidden `s3_inventory_bucket_region` site setting to
specify the region of the `s3_inventory_bucket` when the `S3Inventory`
class initializes an instance of the `S3Helper`. By default, the
`S3Helper` class uses the value of the `s3_region` site setting but the
region of the `s3_inventory_bucket` is not always the same as the
`s3_region` configured.
2024-07-09 12:03:43 +08:00
Martin Brennan
7111d5e4bf
DEV: Fix flaky admin confirmation spec (#27784)
Waiting for the dialog to close was not enough,
need to wait for the overridden indicator to
show on the site setting.
2024-07-09 12:05:31 +10:00
Guhyoun Nam
784c04ea81
FEATURE: Add Mechanism to redeliver all failed webhook events (#27609)
Background:
In order to redrive failed webhook events, an operator has to go through and click on each. This PR is adding a mechanism to retry all failed events to help resolve issues quickly once the underlying failure has been resolved.

What is the change?:
Previously, we had to redeliver each webhook event. This merge is adding a 'Redeliver Failed' button next to the webhook event filter to redeliver all failed events. If there is no failed webhook events to redeliver, 'Redeliver Failed' gets disabled. If you click it, a window pops up to confirm the operator. Failed webhook events will be added to the queue and webhook event list will show the redelivering progress. Every minute, a job will be ran to go through 20 events to redeliver. Every hour, a job will cleanup the redelivering events which have been stored more than 8 hours.
2024-07-08 15:43:16 -05:00
Kelv
17aa831337
DEV: replace imagemagick convert commands with magick (#27767) 2024-07-08 16:55:59 +08:00
Alan Guo Xiang Tan
28f5550886
DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27759)
This reverts commit 92d7d24d0f.
2024-07-08 14:03:11 +08:00
Alan Guo Xiang Tan
c3598847fe
DEV: Truncate user agent string when it is too long instead of null (#27758)
This is a follow up to 005f623c42 where
we want to truncate the user agent string instead of nulling out the
column when the user agent string is too low. By truncating, we still
get to retain information that can still be useful.
2024-07-08 13:58:20 +08:00
Martin Brennan
df6f950200
DEV: Hide admin Moderation Flags UI behind feature flag for now (#27756)
Adds experimental_flags_admin_page_enabled_groups (default "")
to remove the Moderation Flags link from the admin sidebar for now,
there are still a few bugfixes that need to be done before we
are comfortable with turning this on more widely. This is
a _temporary_ flag, we will be removing this once the feature
is more stable.
2024-07-08 11:09:30 +10:00
Sérgio Saquetim
6022cc2af8
DEV: Escape the sidebar filter and admin sidebar no results description (#27746) 2024-07-05 17:54:22 -03:00
Isaac Janzen
005f623c42
DEV: Add user_agent column to search_logs (#27742)
Add a new column - `user_agent` - to the `SearchLog` table. 

This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
2024-07-05 14:05:00 -05:00
Sérgio Saquetim
b36cbc7d21
DEV: Untangle the admin sidebar from the sidebar code (#27640) 2024-07-05 13:11:15 -03:00
David Battersby
640dccd224
FIX: show primary user group options to members in account prefs (#27664)
The user serializer groups method previously relied on the members_visible_groups to determine groups that the user should be able to see, however this setting was intended for visibility of group members (which is entirely different).

The result of this could be seen when choosing a primary group from user preferences -> account, due to the serializer the group name was not visible when members_visible_groups was set to owners.
2024-07-05 19:43:50 +04:00
David Taylor
1bc18a5bf4
DEV: Remove spec following revert in 92d7d24d0f (#27736)
This spec was introduced as part of the original commit (8e10878e1a), but was missed from the revert.
2024-07-05 12:28:16 +01:00
Régis Hanol
1045d22b41
FIX: show cooked bio in user profile (#27734)
When editing your bio in your user profile and then saving it, it would be removed from the profile header section.

This was broken for over 7 years since ae68795d82 was merged 😅

Context - https://meta.discourse.org/t/after-saving-changes-to-profile-expanding-full-profile-only-shows-half/314118
2024-07-05 12:31:57 +02:00
Alan Guo Xiang Tan
bb0daa33cd
DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)
In our official Docker image, running git commands results in the
following error:

```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:

	git config --global --add safe.directory /var/www/discourse
```
2024-07-05 15:22:03 +08:00
Martin Brennan
72fd509fd4
DEV: Comment flaky specs (#27729)
Followup 527f02e99f,
don't have time to investigate now, will look next week
2024-07-05 14:45:54 +10:00
Jan Cernik
33c68b28b6
DEV: Use serializers for user_notification_schedule and featured_topic (#27719)
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-07-05 00:00:24 -03:00
Alan Guo Xiang Tan
8e10878e1a
DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.

Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
2024-07-05 09:41:52 +08:00
Régis Hanol
0dbcc54d4b
DEV: use new 'ignore allowed groups' site settings (#27670)
Instead of the deprecated 'min trust level to allow ignore' in order to reduce the number of deprecation notices in the logs.

This tweaks a few serializers so that the 'can_ignore_users?` property is always coming from the server and properly used on the client-side.
2024-07-04 19:27:26 +02:00
Jan Cernik
0d608ceff8
UX: Registration design and validation improvements (#27666) 2024-07-04 09:17:07 -03:00
Natalie Tay
8bbb4c5cca
FIX: Always noindex /u routes (#27712)
SiteSetting.hide_user_profiles_from_public raises a Forbidden, which disallows our after_action: add no index header from triggering.

This fix makes sure that the no index header gets added via before_action instead
2024-07-04 19:48:38 +08:00
Loïc Guitaut
8d249457e8 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-04 10:58:21 +02:00
Vinoth Kannan
d1ea598fa2
FIX: should not raise error when both group & site tag preferences are same. (#27704)
When tag preference in group and site settings are both used with same default notification level it will break new users signups because it tries to create duplicate records in the tag_users table which can’t happen because we have a unique index set.
2024-07-04 11:53:28 +05:30
Natalie Tay
55bf0e21fb
FIX: Don't show that an existing user is invited_by another user (#27665)
If an existing user (John) accepts an invite created by Kenny to a group, John may be seen as invited by Kenny, despite already having an account on the site.

This fix removes the bug by excluding invites that determine the invited_by after the user's creation date. The delay buffer in the query accounts for invites that also create the user at the same time.
2024-07-04 10:27:37 +08:00
Ted Johansson
6aa8d97f32
FIX: Don't error out when loading a badge with a deleted image (#27688)
Badges can have their associated image uploads deleted. When this happens, any user who has that badge will have their profile page error out.

After this fix, when deleting an upload that's associated with a badge, we nullify the foreign key ID on the badge. This makes the existing safeguard work correctly.
2024-07-04 10:03:09 +08:00
Arpit Jalan
947249719a
FEATURE: add option to delete user associated account on password reset (#27696) 2024-07-04 06:58:00 +05:30
Osama Sayegh
7c12b75a5c
DEV: Fix flaky spec for the /about admin config area (#27697)
There's currently a race condition in the following spec:

65be7a7880/spec/system/admin_about_config_area_spec.rb (L70-L95)

where the form can be saved before the image uploader field has finished uploading the selected image and causing the assertion at line 94 to fail with the following error:

```
Failure/Error: expect(SiteSetting.about_banner_image.sha1).to eq(Upload.generate_digest(image_file))

NoMethodError:
  undefined method `sha1' for nil

[Screenshot Image]: /__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_admin_about_config_area_page_the_general_settings_card_can_saves_its_fields_to_their_corresponding_site_settings_312.png

~~~~~~~ JS LOGS ~~~~~~~
http://localhost:31338/assets/vendor.js 15902:14 "WARNING: uppy needs a unique id, pass one in to the component implementing this mixin"
~~~~~ END JS LOGS ~~~~~

./spec/system/admin_about_config_area_spec.rb:94:in `block (3 levels) in <main>'
./spec/rails_helper.rb:552:in `block (3 levels) in <top (required)>'
./spec/rails_helper.rb:552:in `block (2 levels) in <top (required)>'
./spec/rails_helper.rb:513:in `block (3 levels) in <top (required)>'
./spec/rails_helper.rb:503:in `block (2 levels) in <top (required)>'
./spec/rails_helper.rb:460:in `block (2 levels) in <top (required)>'
./vendor/bundle/ruby/3.3.0/gems/webmock-3.23.1/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'
```

This PR fixes the problem by making the system test wait for the image to finish uploading (with 10 seconds timeout) before carrying out the rest of the system test.
2024-07-04 10:39:22 +10:00
Martin Brennan
5655447aca
FIX: Discrepency between admin page view reports (#27685)
Followup 2f2da72747

When the "Consolidated Pageviews with Browser Detection (Experimental)"
report was introduced, we started counting the original
"page_view_logged_in" and "page_view_anon" ApplicationRequest
data as "Other Pageviews", subtracting
"page_view_anon_browser" and "page_view_logged_in_browser" from
this number.

However we unknowingly automatically started counting these
browser-based page views, which are a subset of the total
"page_view_logged_in" and "page_view_anon" counts, in the
original "Pageviews" report, leading to double counting
which meant that when you looked at the data for each
report side-by-side the data didn't add up.

This commit fixes the issue by not counting the "browser"
pageviews in the Pageviews report, and making the code where
we were only counting certain types of requests for this
report more plain, explicitly stating which types of requests
we want.
2024-07-04 10:16:09 +10:00
Jeff Wong
70fc39211b
FIX: topic embed blank tags or passed with nil do not blank out existing topic tags (#27699)
When a topic embed is run with either no tags argument or a nil tag argument
this should not affect any existing tags.

Only update topic tags when tags argument is explicitly empty.
2024-07-03 14:50:59 -07:00
OsamaSayegh
9c4a5f39d3
SECURITY: Don't allow suspending staff users via other_user_ids param 2024-07-03 20:49:29 +08:00
Jan Cernik
311b737c91
SECURITY: Fix Stored-dom XSS via Facebook Oneboxes 2024-07-03 20:49:22 +08:00
Blake Erickson
e2a7265dba
SECURITY: Update reviewable user serializer payload
Exclude email from reviewable user serializer based on user scope.
2024-07-03 20:49:19 +08:00