Commit Graph

9141 Commits

Author SHA1 Message Date
David Taylor
29f7ec7090
DEV: Prevent defer stats exception when thread aborted (#19863)
When the thread is aborted, an exception is raised before the `start` of a job is set, and therefore raises an exception in the `ensure` block. This commit checks that `start` exists, and also adds `abort_on_exception=true` so that this issue would have caused test failures.
2023-01-16 09:08:44 +11:00
Alan Guo Xiang Tan
f72875c729
DEV: Introduce enable_new_notifications_menu site setting (#19860)
The `enable_new_notifications_menu` site setting allows sites that have
`navigation_menu` set to `legacy` to use the redesigned notifications
menu before switching to the new sidebar navigation menu.
2023-01-16 06:04:53 +08:00
Selase Krakani
73ec80893d
FEATURE: Extend topic update API scope to allow status updates (#19654)
Allow an API key created with topic:update API scope to make updates to
topic status. This change also introduces an optional category_id scope
param.
2023-01-13 01:21:04 +00:00
Ted Johansson
5dcb245eac
FIX: Ruby 2 backward compatible plugin logout redirect (#19845)
This is a very subtle one. Setting the redirect URL is done by passing
a hash through a Discourse event. This is broken on Ruby 2 since the
support for keyword arguments in events was added.

In Ruby 2 the last argument is cast to keyword arguments if it is a
hash. The key point here is that creates a new copy of the hash, so
what the plugin is modifying is not the hash that was passed.
2023-01-12 19:12:20 +08:00
Loïc Guitaut
4093fc6074 Revert "DEV: Migrate existing cookies to Rails 7 format"
This reverts commit 66e8fe9cc6 as it
unexpectedly caused some users to be logged out. We are investigating
the problem.
2023-01-12 12:07:49 +01:00
Loïc Guitaut
66e8fe9cc6 DEV: Migrate existing cookies to Rails 7 format
This patch introduces a cookies rotator as indicated in the Rails
upgrade guide. This allows to migrate from the old SHA1 digest to the
new SHA256 digest.
2023-01-12 11:09:07 +01:00
Martin Brennan
1f59a8299d
DEV: Address TODOs for email Message-ID generation (#19842)
Remove some old deprecated methods and update docs. Will leave
the backwards-compatible Message-ID formats a little while longer
just to be sure.
2023-01-12 13:54:15 +10:00
Sam
7b63c42304
FEATURE: add basic instrumentation to defer queue (#19824)
This will give us some aggregate stats on the defer queue performance.

It is limited to 100 entries (for safety) which is stored in an LRU cache.

Scheduler::Defer.stats can then be used to get an array that denotes:

- number of runs and completions (queued, finished)
- error count (errors)
- total duration (duration)

We can look later at exposing these metrics to gain visibility on the reason
the defer queue is clogged.
2023-01-12 12:29:50 +11:00
Martin Brennan
387693e889
FIX: Improve error reporting and failure modes for channel archiving (#19791)
There was an issue with channel archiving, where at times the topic
creation could fail which left the archive in a bad state, as read-only
instead of archived. This commit does several things:

* Changes the ChatChannelArchiveService to validate the topic being
  created first and if it is not valid report the topic creation errors
  in the PM we send to the user
* Changes the UI message in the channel with the archive status to reflect
  that topic creation failed
* Validate the new topic when starting the archive process from the UI,
  and show the validation errors to the user straight away instead of
  creating the archive record and starting the process

This also fixes another issue in the discourse_dev config which was
failing because YAML parsing does not enable all classes by default now,
which was making the seeding rake task for chat fail.
2023-01-12 10:04:46 +10:00
Martin Brennan
21a95b000e
DEV: Remove defunct TODOs (#19825)
* Firefox now finally returns PerformanceMeasure from performance.measure
* Some TODOs were really more NOTE or FIXME material or no longer relevant
* retain_hours is not needed in ExternalUploadsManager,  it doesn't seem like anywhere in the UI sends this as a param for uploads
* https://github.com/discourse/discourse/pull/18413 was merged so we can remove JS test workaround for settings
2023-01-12 09:41:39 +10:00
Loïc Guitaut
14d97f9cf1 FEATURE: Show more context in Discourse topic oneboxes
Currently when generating a onebox for Discourse topics, some important
context is missing such as categories and tags.

This patch addresses this issue by introducing a new onebox engine
dedicated to display this information when available. Indeed to get this
new information, categories and tags are exposed in the topic metadata
as opengraph tags.
2023-01-11 14:22:53 +01:00
Osama Sayegh
17daf077e2
Version bump to v3.1.0.beta1 (#19822) 2023-01-11 08:39:15 +03:00
Justin René Back
aad7a14b36
FIX: Make DBHelper.find build valid query (#19408) 2023-01-10 22:38:52 +02:00
Daniel Waterworth
dfc449a32e
FIX: Continue if checking themes if updating one fails (#19665) 2023-01-09 11:56:43 -06:00
Bianca Nenciu
fb780c50fd
FIX: Replace all quote-like unicodes with quotes (#19714)
If unaccent is called with quote-like Unicode characters then it can
generate invalid queries because some of the transformed quotes by
unaccent are not escaped and to_tsquery fails because of bad input.

This commits replaces more quote-like Unicode characters before
unaccent is called.
2023-01-09 19:19:51 +02:00
David Taylor
6417173082
DEV: Apply syntax_tree formatting to lib/* 2023-01-09 12:10:19 +00:00
Jarek Radosz
ff508d1ae5
FIX: Correctly support DiscourseEvent kwargs (#19788)
Fixes the support for kwargs in `DiscourseEvent.trigger()` on Ruby 3, e.g.

```rb
DiscourseEvent.trigger(:before_system_message_sent, message_type: type, recipient: @recipient, post_creator_args: post_creator_args, params: method_params)
```

Fixes https://github.com/discourse/discourse-local-site-contacts
2023-01-09 11:26:39 +01:00
Martin Brennan
56eaf91589
FIX: Do not error when anon user looks at secure upload for deleted post (#19792)
If a secure upload's access_control_post was trashed, and an anon user
tried to look at that upload, they would get a 500 error rather than
the correct 403 because of an error inside the PostGuardian logic.
2023-01-09 16:12:10 +10:00
David Taylor
66e8a35b4d
DEV: Include message-bus request type in HTTP request data (#19762) 2023-01-06 11:26:18 +00:00
Martin Brennan
c4ea158656
FIX: Improve tags in email subjects and add filter headers (#19760)
This commit does a couple of things:

1. Changes the limit of tags to include a subject for a
   notification email to the `max_tags_per_topic` setting
   instead of the arbitrary 3 limit
2. Adds both an X-Discourse-Tags and X-Discourse-Category
   custom header to outbound emails containing the tags
   and category from the subject, so people on mail clients
   that allow advanced filtering (i.e. not Gmail) can filter
   mail by tags and category, which is useful for mailing
   list mode users

c.f. https://meta.discourse.org/t/headers-for-email-notifications-so-that-gmail-users-can-filter-on-tags/249982/17
2023-01-06 10:03:02 +10:00
Alan Guo Xiang Tan
6543dec7cb
Version bump to v3.0.0.beta16 (#19751) 2023-01-05 09:45:40 +08:00
Martin Brennan
16b9165630
FIX: Bookmark auto delete preference usage and default value (#19707)
This commit fixes an issue where the chat message bookmarks
did not respect the user's `bookmark_auto_delete_preference`
which they select in their user preference page.

Also, it changes the default for that value to "keep bookmark and clear reminder"
rather than "never", which ends up leaving a lot of expired bookmark
reminders around which are a pain to clean up.
2023-01-05 08:43:58 +10:00
Alan Guo Xiang Tan
918dd4d635
SECURITY: use rstrip instead of regex gsub to prevent ReDOS (#19737)
`rstrip` implementation is much more performant than regex

Co-authored-by: Krzysztof Kotlarek <kotlarek.krzysztof@gmail.com>
2023-01-05 06:09:17 +08:00
Alan Guo Xiang Tan
bf6b08670a
SECURITY: Check the length of raw post body (#19733)
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2023-01-05 06:08:43 +08:00
Alan Guo Xiang Tan
ab3a032b4b
SECURITY: BCC active user emails from group SMTP (#19725)
When sending emails out via group SMTP, if we
are sending them to non-staged users we want
to mask those emails with BCC, just so we don't
expose them to anyone we shouldn't. Staged users
are ones that have likely only interacted with
support via email, and will likely include other
people who were CC'd on the original email to the
group.

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-01-05 06:07:50 +08:00
Osama Sayegh
bbcdf74c58
DEV: Flip primary_email_verified? default to false (#19703)
This commit changes the default return value of `Auth::ManagedAuthenticator#primary_email_verified?` to false. We're changing the default to force developers to think about email verification when building a new authentication method. All existing authenticators (in core and official plugins) have been updated to explicitly define the `primary_email_verified?` method in their subclass of `Auth::ManagedAuthenticator` (example commit 65f57a4d05).

Internal topic: t/82084.
2023-01-04 10:51:10 +03:00
Roman Rizzi
c2e18c41a3
FIX: Check that the node has a src attr when getting size (#19696) 2023-01-03 15:27:05 -03:00
Alan Guo Xiang Tan
f04e454628
Version bump to v3.0.0.beta15 (#19685) 2023-01-03 14:29:26 +08:00
Alan Guo Xiang Tan
af76f291e6
PERF: Avoid running redundant bookmarks query for anon viewing topic (#19659)
The `TopicView#bookmarks` method is called by `TopicViewSerializer` and `PostSerializer`
so we want to avoid running a meaningless query when user is not
present.
2023-01-03 10:00:36 +08:00
Alan Guo Xiang Tan
24db6fbb73
PERF: Memoize topic level checks in PostGuardian (#19647)
When loading posts in a topic, the topic level guardian
checks are run multiple times even though all the posts belong to the
same topic. Profiling in production revealed that this accounted for a
significant amount of request time for a user that is not staff or anon.
Therefore, we're optimizing this by adding memoizing the topic level
calls in `PostGuardian`. Speficifally, the result of
`TopicGuardian#can_see_topic?` and `PostGuardian#can_create_post?`
method calls are memoized per topic.

Locally profiling shows a significant improvement for normal users
loading a topic with 100 posts.

Benchmark script command: `ruby script/bench.rb --unicorn --skip-bundle-assets --iterations 100`

Before:

```
topic user:
  50: 114
  75: 117
  90: 122
  99: 209
topic.json user:
  50: 67
  75: 69
  90: 72
  99: 162
```

After:

```
topic user:
  50: 101
  75: 104
  90: 107
  99: 184
topic.json user:
  50: 53
  75: 53
  90: 56
  99: 138
```
2023-01-03 09:00:42 +08:00
Alan Guo Xiang Tan
be1ae9411b
PERF: Remove redundant DB queries from TopicView (#19658)
This commit removes 3 redundant DB queries when loading posts.

1. `@posts` will eventually have to be loaded so we can avoid two
additional queries.

2. No need to preload topic association of posts as we're already
dealing with a fixed topic in `TopicView`.
2023-01-03 08:17:52 +08:00
David Taylor
eebe2f256c
DEV: Skip s3 asset deletion when in readonly mode (#19611)
In some situations (e.g. disaster recovery), it may make sense to spin up a temporary readonly version of a cluster. In that situation, the s3 `expire_missing_assets` job would delete assets which are still in use by the canonical read-write version of the cluster.

To avoid that, this commit will skip deletion if the site is currently in readonly mode.
2022-12-30 11:35:40 +00:00
Alan Guo Xiang Tan
0da79561c3
DEV: Improve/Fix script/bench.rb (#19646)
1. Fix bug where we were not waiting for all unicorn workers to start up
before running benchmarks.

2. Fix a bug where headers were not used when benchmarking. Admin
benchmarks were basically running as anon user.

3. Disable rate limits when in profile env. We're pretty much going to
hit the rate limit every time as a normal user.

4. Benchmark against topic with a fixed posts count of 100. Previously profiling script was just randomly creating posts
and we would benchmark against a topic with a fixed posts count of 30.
Sometimes, the script fails because no topics with a posts count of 30
exists.

5. Benchmarks are not run against a normal user on top of anon and
admin.

6. Add script option to select tests that should be run.
2022-12-30 07:25:11 +08:00
Isaac Janzen
e5349e43af
DEV: Update group moderator behavior to better mimic staff (#19618)
# Context
When a topic is reviewable by a group we give those group moderators some admin abilities including the ability to delete a topic.

# Problem
There are two main problems:

1. Currently when a group moderator deletes a topic they are redirected to root (not the same for staff)
2. Viewing the categories deleted topics (`c/foo/1/?status=deleted`) does not display the deleted topic to the group moderator (not the same for staff).

# Fix
If the `deleted_by` user is part a group that matches the `reviewable_by_group` on a topic then don't redirect. This is the default interaction for staff to give them the ability to do things like restore the topic in case it was accidentally deleted.

To render the deleted topics as expected for the group moderator I am utilizing [the guardian scope of `guardian.can_see_deleted_topics?` for said category](https://github.com/discourse/discourse/pull/19618/files#diff-288e61b8bacdb29d9c2e05b42da6837b0036dcf1867332d977ca7c5e74a44297R802-R803)
2022-12-29 10:07:03 -06:00
David Taylor
104a16610a DEV: Configure prettier for hbs templates 2022-12-28 13:11:12 +00:00
David Taylor
d4d9d60a5f
DEV: Print system test logs with other test metadata (#19637)
Previously, browser logs would be printed to STDOUT halfway through the test run. This commit changes the behaviour so that the logs are included in the failure summary along with other rspec failure information.
2022-12-28 10:47:57 +00:00
Ted Johansson
06db264f24
FIX: Gracefully handle DNS issued from SSRF lookup when inline oneboxing (#19631)
There is an issue where chat message processing breaks due to
unhandles `SocketError` exceptions originating in the SSRF check,
specifically in `FinalDestination::Resolver`.

This change gives `FinalDestination::SSRFDetector` a new error class
to wrap the `SocketError` in, and haves the `RetrieveTitle` class
handle that error gracefully.
2022-12-28 10:30:20 +08:00
Alan Guo Xiang Tan
1fd0e82da7
PERF: Fix N+1 queries problem on topic view page (#19629)
`User#flair_group` was not preloaded leading to the N+1 queries problem
when multiple users have flair groups.
2022-12-27 09:05:37 +08:00
Osama Sayegh
d8b39810d2
DEV: Stop leaking state in dashboard controller specs (#19608)
A few specs in `dashboard_controller_spec.rb` set some state in redis but don't clean it up afterwards which causes other specs to fail when they're ran after `dashboard_controller_spec.rb`.

Related commit: 18467d4.
2022-12-23 15:41:30 +03:00
Alan Guo Xiang Tan
a8ee56ebee
PERF: Remove unnecessary query from TopicQuery.remove_muted_tags (#19586)
When `default_tags_muted` site settings has not been set, there is no
need to execute the query.
2022-12-23 04:29:17 +08:00
Penar Musaraj
8546c2084a
DEV: Load SVG sprites during system spec runs (#19497)
Co-authored-by: David Taylor <david@taylorhq.com>
2022-12-22 08:13:43 -05:00
Alan Guo Xiang Tan
4eee1320b0
PERF: Fix N+1 queries when serializing topic posters (#19545)
At the time of writing, this is how the `TopicPosterSerializer` looks
like:

```
class TopicPosterSerializer < ApplicationSerializer
  attributes :extras, :description

  has_one :user, serializer: PosterSerializer
  has_one :primary_group, serializer: PrimaryGroupSerializer
  has_one :flair_group, serializer: FlairGroupSerializer
end
```

Within `PosterSerializer`, the `primary_group` and `flair_group`
association is requested on the `user` object. However, the associations
have not been loaded on the `user` object at this point leading to the
N+1 queries problem. One may wonder
why the associations have not been loaded when the `TopicPosterSerializer`
has `has_one :primary_group` and `has_one :flair_group`. It turns out that `TopicPoster`
is just a struct containing the  `user`, `primary_group` and
`flair_group` objects. The `primary_group` and `flair_group`
ActiveRecord objects are loaded seperately in `UserLookup` and not preloaded when querying for
the users. This is done for performance reason so that we are able to
load the `primary_group` and `flair_group` records in a single query
without duplication.
2022-12-22 05:30:29 +08:00
Bianca Nenciu
58479fe10b
DEV: Pass kwargs to defined method (#19552) 2022-12-21 19:02:40 +02:00
David Taylor
f1ec8c869a
DEV: Fix FinalDestination::Resolver race condition (#19558)
We were adding to the resolver's work queue before setting up the `@lookup` and `@parent` information. That could lead to the lookup being performed on the wrong (or `nil`) hostname. This also lead to some flakiness in specs.
2022-12-21 16:02:24 +00:00
David Taylor
a6af981e1b
DEV: Add retry logic to plugin:install_all_official (#19551)
This task sometimes fails in CI due to temporary network issues. Retrying twice should help resolve those situations without needing to manually restart the job.
2022-12-21 15:31:03 +00:00
Jarek Radosz
dc13e8ecfd
DEV: Fix TurboTests::Runner fail_fast condition (#19540)
It fast-failed after the first failure regardless of the param…
2022-12-21 02:33:52 +01:00
Martin Brennan
94b258deda
FIX: TL0 could not message group with everyone messageable_level (#19525)
The commits e62e93f83a and
d6bd4ad7ee caused a regression
to the behaviour added for
https://meta.discourse.org/t/allow-tl0-to-write-messages-to-staff-group-not-to-other-members-or-non-staff/124335,
which allowed a user to message a group with the messageable_level
set to Everyone even if they were TL0 (or otherwise did not reach
the appropriate trust level). This commit fixes the issue and adjusts
the spec to reflect the real scenario.

c.f. https://meta.discourse.org/t/tl0-cant-message-groups-with-messageable-level-everyone-recession/249205
2022-12-20 13:11:14 +10:00
Blake Erickson
ae2153b330
UX: Wizard Step Enhancements (#19487)
* UX: Wizard Step Enhancements

- Remove illustrations
- Add Emoji graphic to top of steps
- Add description below step title
- Move point of contact to last step

* Move step count to header, plus some button navigation tweaks

* add remaining emoji to step headers

* fix button logic on steps

* Update Point of Contact

* remove automated messages field

* adjust styling for counter, title, and emoji

* Update wording for logos

* Fix tests

* fix prettier

* fix specs

* set same with for steps except for styling screen

* use sentence case; remove duplicate copy under your organization fields

* fix missing buttons on small screens

* add spacing to buttons; adjust font weight to labels

* adjust styling for community logo step; use sentence case for button

* update copy for point of contact text helper

* use sentence case for field labels

* fix ui tests

* use btn-back class to fix ui tests

* reduce bottom margin for toggle fields

* clean up

Co-authored-by: Ella <ella.estigoy@gmail.com>
2022-12-19 17:24:09 -07:00
Martin Brennan
6b9c0ee554
DEV: Change HashtagAutocompleteService to use DiscoursePluginRegistry (#19491)
Follow up to a review in #18937, this commit changes the HashtagAutocompleteService to no longer use class variables to register hashtag data sources or types in context priority order. This is to address multisite concerns, where one site could e.g. have chat disabled and another might not. The filtered plugin registers I added will not be included if the plugin is disabled.
2022-12-19 13:46:17 +10:00
Martin Brennan
baf78d3d91
FIX: Add missing user_id args for ChatMessage.cook (#19508)
In both ChatMessage#rebake! and in ChatMessageProcessor
when we were calling ChatMessage.cook we were missing the
user_id to cook with, which causes missed hashtag cooks
because of missing permissions.
2022-12-19 11:05:37 +10:00