Commit Graph

10124 Commits

Author SHA1 Message Date
Régis Hanol
8fe73ab404
DEV: better log around nil client_settings_json (#27868)
Internal ref t/132507
2024-07-11 11:30:53 +02:00
Loïc Guitaut
5ec227334a FIX: Don’t list values from disabled plugins
Currently, when a plugin registers a new reviewable type or extends a
list method (through `register_reviewble_type` and `extend_list_method`
respectively), the new array is statically computed and always returns
the same value. It will continue to return the same value even if the
plugin is disabled (it can be a problem in a multisite env too).

To address this issue, this patch changes how `extend_list_method`
works. It’s now using `DiscoursePluginRegistry.define_filtered_register`
to create a register on the fly and store the extra values from various
plugins. It then combines the original values with the ones from the
registry. The registry is already aware of disabled plugins, so when a
plugin is disabled, its registered values won’t be returned.
2024-07-11 10:51:48 +02:00
Alan Guo Xiang Tan
66878a9e80
DEV: Improve logging of Sidekiq errors when logstash logger is enabled (#27855)
This commit improves the logging of Sidekiq errors when
`ENABLE_LOGSTASH_LOGGER` is set to 1. Prior to this change, we would
only log the message and the backtrace. After this change, useful
information like `job.class`, `job.opts`, `job.problem_db`,
`exception.class` and `exception.message` are included in the log line
as well.
2024-07-11 14:17:18 +08:00
David Taylor
ce3d91f422
DEV: Load @json-editor/json-editor using webpack async import (#27773) 2024-07-10 11:15:42 +01:00
Régis Hanol
758b9dd0ba
FEATURE: email attachments in a details (#27804)
This change how we present attachments from incoming emails to now be "hidden" in a "[details]" so they don't "hang" at the end of the post.

This is especially useful when using Discourse as a support tool where email is the main communication channel. For various reasons, images are often duplicated by email user agents, and hiding them behind the details block help keep the conversation focused on the isssue at hand.

Internal ref t/122333
2024-07-10 09:59:27 +02:00
Loïc Guitaut
301713ef96 DEV: Upgrade the MessageFormat library (JS)
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.

Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.

We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.

The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.

Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.

A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.

The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
2024-07-10 09:51:25 +02:00
Alan Guo Xiang Tan
c9775d5f72
DEV: Apply Logster.store.ignore to DiscourseLogstashLogger as well (#27819)
This commit updates `DiscourseLogstashLogger#add_with_opts` to avoid
logging messages that matches regexp patterns configured in
`Logster.store.ignore`. Those error logs are mostly triggered by clients
and do not serve any useful purpose.
2024-07-10 13:51:42 +08:00
Alan Guo Xiang Tan
3fee52eae3
PERF: Update docker:test to run QUnit with more cores if available (#27816)
This commit updates the `docker:test` rake task to run core and plugin
QUnit tests in parallel using half the number of available CPU
processors to speed up time it takes to run the tests on hardware with
more CPU cores.

Before this commit, core QUnit tests ran by the `docker:test` rake task was capped at 3 parallel
processes while plugin QUnit tests was not ran in parallel.
2024-07-10 11:27:20 +08:00
Alan Guo Xiang Tan
b4b7fa17af
DEV: Add exception class/message to DiscourseLogstashLogger take 2 (#27815)
This is the second take of af2bd4cc50 to
account for messages which contains newlines.
2024-07-10 11:04:17 +08:00
Alan Guo Xiang Tan
0b64cb9d8c
Revert "DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)" (#27814)
This reverts commit bb0daa33cd.

This commit was not causing the problems we thought it was.
2024-07-10 10:36:22 +08:00
Alan Guo Xiang Tan
af2bd4cc50
DEV: Add exception class and message fields to DiscourseLogstashLogger (#27787)
This commit updates `DiscourseLogstashlogger` to add the
`exception_class` and `exception_message` field to the log line when the
`progname` of the log message is `web-exception` which is Logster's
logging of exceptions during a web request.

The `exception_class` and `exception_message` fields allows consumers of
the logs to easily group logs together.
2024-07-10 08:54:39 +08:00
Martin Brennan
560e8aff75
FEATURE: Allow oneboxing private GitHub URLs (#27705)
This commit adds the ability to onebox private GitHub
commits, pull requests, issues, blobs, and actions using
a new `github_onebox_access_token` site setting. The token
must be set up in correctly to have access to the repos needed.

To do this successfully with the Oneboxer, we need to skip
redirects on the github.com host, otherwise we get a 404
on the URL before it is translated into a GitHub API URL
and has the appropriate headers added.
2024-07-10 09:39:31 +10:00
Natalie Tay
27eea3250c
DEV: Fix deprecation typo (#27789) 2024-07-09 17:46:10 +08:00
David Taylor
a461eaddc3
DEV: Refactor ace-editor loading and bump to latest version (#27772)
- Delete vendored copy
- Create a JS entrypoint under `static/` which imports all the modes/themes/extensions we need
- Create an async `load-ace-editor` entrypoint
- Update `<AceEditor` component to use the new entrypoint
- De-jquery-ify `<AceEditor`
- Bump `v1.4.13` -> `v1.35.2`
2024-07-09 10:09:30 +01:00
David Battersby
f44ec18fd2
DEV: update base url links to respect subfolder installs (#27740)
This change eliminates a couple of instances where subfolder urls are badly formatted, in most cases we can use Discourse.base_url_no_prefix to prevent adding the subfolder to the base url.
2024-07-09 12:42:38 +04:00
Alan Guo Xiang Tan
86e5f46175
DEV: Add hidden s3_inventory_bucket_region site setting (#27786)
This commit adds a hidden `s3_inventory_bucket_region` site setting to
specify the region of the `s3_inventory_bucket` when the `S3Inventory`
class initializes an instance of the `S3Helper`. By default, the
`S3Helper` class uses the value of the `s3_region` site setting but the
region of the `s3_inventory_bucket` is not always the same as the
`s3_region` configured.
2024-07-09 12:03:43 +08:00
Kelv
17aa831337
DEV: replace imagemagick convert commands with magick (#27767) 2024-07-08 16:55:59 +08:00
Alan Guo Xiang Tan
28f5550886
DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27759)
This reverts commit 92d7d24d0f.
2024-07-08 14:03:11 +08:00
Isaac Janzen
005f623c42
DEV: Add user_agent column to search_logs (#27742)
Add a new column - `user_agent` - to the `SearchLog` table. 

This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
2024-07-05 14:05:00 -05:00
Alan Guo Xiang Tan
92d7d24d0f
Revert "DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)" (#27733)
This reverts commit 8e10878e1a.

Something is broken on a friday so reverting first before I pick this up
again next Monday.
2024-07-05 17:26:58 +08:00
Alan Guo Xiang Tan
bb0daa33cd
DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)
In our official Docker image, running git commands results in the
following error:

```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:

	git config --global --add safe.directory /var/www/discourse
```
2024-07-05 15:22:03 +08:00
Martin Brennan
964945a11f
DEV: Add case-converter common lib and use with AdminPlugin (#27725)
The AdminPlugin JS model uses a similar pattern to chat models,
where it is a plain JS class manually converting provided
snake_case attributes from the serializer to JS camelCase.

However this doesn't work when it comes to using `add_to_serializer`
in plugins since core does not know about these new attributes.

Instead, we can use a JS function to convert snake_case to camelCase
and use that when initializing AdminPlugin. This commit also moves
similar functions to a new case-converter.js file in
discourse-common/lib.
2024-07-05 13:22:48 +10:00
Alan Guo Xiang Tan
37413c0ecd
DEV: Fix unicorn not booting due to 8e10878 (#27727)
`String#present?` is a method defined by Rails on the `String` class but
this class is used before Rails has been loaded.
2024-07-05 10:36:13 +08:00
Alan Guo Xiang Tan
8e10878e1a
DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.

Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
2024-07-05 09:41:52 +08:00
Alan Guo Xiang Tan
d8e7fc4f5d
DEV: check_pending -> check_all_pending! (#27724)
Fixes the following deprecation warning:

> DEPRECATION WARNING: The `check_pending!` method is deprecated in favor of `check_all_pending!`. The new implementation will loop through all available database configurations and find pending migrations. The prior implementation did not permit this.
2024-07-05 09:29:32 +08:00
Régis Hanol
0dbcc54d4b
DEV: use new 'ignore allowed groups' site settings (#27670)
Instead of the deprecated 'min trust level to allow ignore' in order to reduce the number of deprecation notices in the logs.

This tweaks a few serializers so that the 'can_ignore_users?` property is always coming from the server and properly used on the client-side.
2024-07-04 19:27:26 +02:00
Loïc Guitaut
8d249457e8 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-04 10:58:21 +02:00
David Taylor
eadda77edf
DEV: Drop diffhtml dependency (#27695)
We switched to morphlex via:

- c5f1fc3a08

- bc561eb332
2024-07-04 12:24:51 +10:00
Nat
65be7a7880
Bump version to v3.3.0.beta4-dev 2024-07-03 20:56:17 +08:00
Nat
a29ec30985
Bump version to v3.3.0.beta3 2024-07-03 20:56:17 +08:00
Alan Guo Xiang Tan
10afe5fcf1
SECURITY: Avoid the use of Object#send in Onebox::Engine::StandardEmbed
Use `Object#public_send` instead which is much safer
2024-07-03 20:49:25 +08:00
Jan Cernik
311b737c91
SECURITY: Fix Stored-dom XSS via Facebook Oneboxes 2024-07-03 20:49:22 +08:00
Ted Johansson
67e7808603
SECURITY: Add FastImage SSRF safe freedom patch 2024-07-03 20:49:15 +08:00
Natalie Tay
751750c7f8
FIX: Do not schedule avatar download if image is '' (#27687) 2024-07-03 19:50:31 +08:00
Martin Brennan
527f02e99f
FEATURE: Only count topic views for explicit/deferred tracked views (#27533)
Followup 2f2da72747

This commit moves topic view tracking from happening
every time a Topic is requested, which is susceptible
to inflating numbers of views from web crawlers, to
our request tracker middleware.

In this new location, topic views are only tracked when
the following headers are sent:

* HTTP_DISCOURSE_TRACK_VIEW - This is sent on every page navigation when
  clicking around the ember app. We count these as browser page views
  because we know it comes from the AJAX call in our app. The topic ID
  is extracted from HTTP_DISCOURSE_TRACK_VIEW_TOPIC_ID
* HTTP_DISCOURSE_DEFERRED_TRACK_VIEW - Sent when MessageBus initializes
  after first loading the page to count the initial page load view. The
  topic ID is extracted from HTTP_DISCOURSE_DEFERRED_TRACK_VIEW.

This will bring topic views more in line with the change we
made to page views in the referenced commit and result in
more realistic topic view counts.
2024-07-03 10:38:49 +10:00
Krzysztof Kotlarek
c3fadc7330
FEATURE: created edit and delete flags (#27484)
Allow admins to create edit and delete flags.
2024-07-03 08:45:37 +10:00
Leonardo Mosquera
a86590ffd6
UX: Add a rake task to monitor progress for long rebakes (#27517)
* UX: Add a rake task to monitor progress for long rebakes

When doing mass rebaking, this task will print progress, speed and
expected time to completion (ETC) on a loop. It is meant for rebakes
that take several hours or days.

It will calculate a 10m moving average over the past 6 hours, and print
ETC accordingly.

It also shows Sidekiq stats in case Sidekiq jobs for rebaking were
enqueued separately, instead of using the rebake_posts tasks in this
file (which are 100% synchronous and do not touch Sidekiq at all).

NOTE: only the currently unbaked count at task start time is considered;
this is useful in live communities with lots of traffic, where new posts
might otherwise change the goal posts continuously.

* Satisfy stree
2024-07-03 07:52:47 +10:00
Keegan George
ea58140032
DEV: Remove summarization code (#27373) 2024-07-02 08:51:47 -07:00
Jan Cernik
6599b85a75
DEV: Block accidental serialization of entire AR models (#27668) 2024-07-01 17:08:48 -03:00
Martin Brennan
ffc99253fa
DEV: Resolve TODO comments for martin-brennan
I am changing many of these to notes or resolving them as is,
most of these I have not actively worked on in years so someone
else can work on them when we get to these areas again.
2024-07-01 15:32:30 +10:00
Osama Sayegh
4c1c25027d
DEV: Wiring for the admin about config page (#27492)
This commit continues work laid out by ffec8163b0 for the admin config page for the /about page. The last commit set up the user interface, and this one sets up all the wiring needed to make the input fields and save buttons actually work.

Internal topic: t/128544.
2024-07-01 05:40:37 +03:00
Jay Pfaffman
6a5ee0693d
FIX: rescue mmdb download failure (#27556)
* rescue mmdb download failure

If mmdb database fails to download a bootstrap fails. This is a trivial fix for that problem. A more elegant solution might check whether the dataabase was downloaded and provide a helpful error message.

* linting

---------

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2024-06-28 05:10:14 +08:00
Régis Hanol
a56321efb5 FIX: topic search order
When using the full page search and filtering down to a specific topic, the sort order was overwritten to by by "post_number".

This was confusing because we allow different type of sort order in the full search page.

This fixes it by only sorting by post_number when there's no "global" sort order defined.

Since the "new topic map" uses the search endpoint behind the scene, this also fixes the "most likes" popup.

Context - https://meta.discourse.org/t/searching-order-seems-to-be-broken-when-searching-in-topic/312303
2024-06-27 18:13:26 +02:00
Martin Brennan
cada172981
FIX: Do not capture OAuth user on 2FA page (#27617)
If the `enforce_second_factor_on_external_auth` setting
is disabled and a user logs in with an OAuth method,
they don't automatically get redirected to /preferences/second-factor
on login. However, they can get there manually, and once there
they cannot leave.

This commit fixes the issue and allows them to leave
and also does some refactors to indicate to the client
what login method is used as a followup to
0e1102b332
2024-06-27 10:27:49 +10:00
Loïc Guitaut
f58b844f45
Revert "DEV: Upgrade Rails to version 7.1" (#27625)
This reverts commit ce00f83173.
2024-06-26 18:55:05 +02:00
Régis Hanol
54a59be617 FEATURE: new 'should_add_email_attachments' plugin modifier
That can be used by plugins to control whether email attachments should be sent.

Internal ref - t/132149
2024-06-26 12:36:35 +02:00
Martin Brennan
a128ce5c4c
FIX: Missing multipart/mixed boundary on emails (#27599)
Followup 96a0781bc1

When sending emails where secure uploads is enabled
and secure_uploads_allow_embed_images_in_emails is
true, we attach the images to the email, and we
do some munging with the final email so the structure
of the MIME parts looks like this:

```
multipart/mixed
  multipart/alternative
    text/plain
    text/html
  image/jpeg
  image/jpeg
  image/png
```

However, we were not specifying the `boundary` of the
`multipart/mixed` main content-type of the email, so
sometimes the email would come through appearing to
have an empty body with the entire thing attached as
one attachment, and some mail parsers considered the
entire email as the "epilogue" and/or "preamble".

This commit fixes the issue by specifying the boundary
in the content-type header per https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
2024-06-25 13:43:10 +10:00
Renato Atilio
55da8a7701
FEATURE: add Untranslated filter to admin text customization (#27555)
Adds a checkbox to filter untranslated text strings in the admin UI, behind a hidden and default `false` site setting `admin_allow_filter_untranslated_text`.
2024-06-24 06:24:06 -03:00
Loïc Guitaut
ce00f83173 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-06-24 11:16:14 +02:00
Sam
cb3456e18f
FEATURE: add JavaScript mode to ace (#27586)
This allows plugins to use Ace Editor in JavaScript mode. This will be used in Discourse AI when defining custom tools.
2024-06-24 13:38:24 +10:00