Commit Graph

47723 Commits

Author SHA1 Message Date
Bianca Nenciu
0fea826f42
FIX: Validate tags parameter of TopicQuery (#19830)
Recently, we have seen some errors related to invalid tags value being
passed to TopicQuery.
2023-01-16 19:20:19 +02:00
Bianca Nenciu
c3070288ea
FEATURE: Verify email webhook signatures (#19690)
* FEATURE: Verify Sendgrid webhook signature

* FEATURE: Verify more webhook signatures

* DEV: Add test for AWS webhook

* FEATURE: Implement algorithm for Mandrill

* FEATURE: Add warning if webhooks are unsafe
2023-01-16 19:16:17 +02:00
Loïc Guitaut
1ce9582a6c FIX: Display Discourse onebox tag icon properly in chat 2023-01-16 14:53:49 +01:00
Régis Hanol
b704e338ef
DEV: extract anniversary badge query (#19716)
So it can easily be overwritten in a plugin for example.

### Added more tests to provide better coverage

We previously only had `u.silenced_till IS NULL` but I made it consistent with pretty much every other places where we check for "active" users.

These two new lines do change the query a tiny bit though. 

**Before** 

- You could not get the badge if you were currently silenced (no matter what period is being checked)
- You could get the badge if you were suspended 😬 

**After**

- You can't get the badge if you were silenced during the past year
- You can't get the badge if you were suspended during the past year


### Improved the performance of the query by using `NOT EXISTS` instead of `LEFT JOIN / COUNT() = 0`

There is no difference in behaviour between 

```sql
LEFT JOIN user_badges AS ub ON ub.user_id = u.id AND ...
[...]
HAVING COUNT(ub.*) = 0
```

and

```sql
NOT EXISTS (SELECT 1 FROM user_badges AS ub WHERE ub.user_id = u.id AND ...)
```

The only difference is performance-wise. The `NOT EXISTS` is 10-30% faster on very large databases (aka. posts and users in X millions). I checked on 3 of the largest datasets I could find.
2023-01-16 11:55:00 +01:00
Martin Brennan
553b4888ba
DEV: Revert syntax-tree line change in unicorn.conf.rb listen (#19874)
Some internal tooling expects this to be one line, see /t/90198/13
2023-01-16 13:17:23 +10:00
Martin Brennan
7c97548159
FIX: Fix incorrect hashtag setting migration (#19857)
Added in c2013865d7,
this migration was supposed to only turn off the hashtag
setting for existing sites (since that was the old default)
but its doing it for new ones too because we run all migrations
on new sites.

Instead, we should only run this if the first migration was
only just created, meaning its a new site.
2023-01-16 10:53:00 +10:00
Martin Brennan
2eb0a300b6
FIX: Use hashtags in channel archive PMs if available (#19859)
If the enable_experimental_hashtag_autocomplete setting is
enabled, then we should autolink hashtag references to the
archived channels (e.g. #blah::channel) for a nicer UX, and
just show the channel name if not (since doing #channelName
can lead to weird inconsistent results).
2023-01-16 10:20:37 +10:00
chapoi
d59ed1cbfe
UX: fix alignment issues with autocomplete (#19828) 2023-01-16 09:09:23 +11:00
David Taylor
29f7ec7090
DEV: Prevent defer stats exception when thread aborted (#19863)
When the thread is aborted, an exception is raised before the `start` of a job is set, and therefore raises an exception in the `ensure` block. This commit checks that `start` exists, and also adds `abort_on_exception=true` so that this issue would have caused test failures.
2023-01-16 09:08:44 +11:00
Alan Guo Xiang Tan
f72875c729
DEV: Introduce enable_new_notifications_menu site setting (#19860)
The `enable_new_notifications_menu` site setting allows sites that have
`navigation_menu` set to `legacy` to use the redesigned notifications
menu before switching to the new sidebar navigation menu.
2023-01-16 06:04:53 +08:00
dependabot[bot]
9ed4550b86
Build(deps): Bump eslint in /app/assets/javascripts (#19873)
Bumps [eslint](https://github.com/eslint/eslint) from 8.31.0 to 8.32.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.31.0...v8.32.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-15 22:32:00 +01:00
dependabot[bot]
b27415d8a8
Build(deps-dev): Bump parallel_tests from 4.0.0 to 4.1.0 (#19872)
Bumps [parallel_tests](https://github.com/grosser/parallel_tests) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/grosser/parallel_tests/releases)
- [Changelog](https://github.com/grosser/parallel_tests/blob/master/CHANGELOG.md)
- [Commits](https://github.com/grosser/parallel_tests/compare/v4.0.0...v4.1.0)

---
updated-dependencies:
- dependency-name: parallel_tests
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-15 22:31:06 +01:00
dependabot[bot]
0dc938ef01
Build(deps): Bump rubocop-rspec from 2.16.0 to 2.17.0 (#19866)
Bumps [rubocop-rspec](https://github.com/rubocop/rubocop-rspec) from 2.16.0 to 2.17.0.
- [Release notes](https://github.com/rubocop/rubocop-rspec/releases)
- [Changelog](https://github.com/rubocop/rubocop-rspec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop-rspec/compare/v2.16.0...v2.17.0)

---
updated-dependencies:
- dependency-name: rubocop-rspec
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-15 19:18:52 +01:00
Rafael dos Santos Silva
b8bbdcf012
DEV: Update PostgreSQL and Redis versions (#19869)
* DEV: Update PostgreSQL and Redis versions

* DEV: Update versions in main readme
2023-01-13 13:45:07 -03:00
Joffrey JAFFEUX
a444023113
DEV: adds row index support (#19871)
This commits adds a data-index attribute on each `select-kit-row` DOM node and also makes available `this.index` in each `select-kit-row` template.
2023-01-13 16:39:21 +01:00
Joffrey JAFFEUX
f525f722ea
DEV: adds expandedOnInsert option to sk (#19870)
Allows to display a select-kit component expanded by default.

Usage:

```
<SingleSelect
  @value={{this.value}}
  @content={{this.content}}
  @options={{hash expandedOnInsert=true}}
/>
```
2023-01-13 16:13:13 +01:00
Rafael dos Santos Silva
076b3a6514
DEV: Key bundler CI cache on Ruby version (#19868) 2023-01-13 11:39:49 -03:00
Rafael dos Santos Silva
8e7e6e14c7
DEV: Add Ruby 3.2 to test matrix (#19862)
* DEV: Add Ruby 3.2 to test matrix

* DEV: Update test name
2023-01-13 09:22:33 -03:00
dependabot[bot]
e21c79ae23
Build(deps): Bump message_bus from 4.3.1 to 4.3.2 (#19865)
Bumps [message_bus](https://github.com/discourse/message_bus) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/discourse/message_bus/releases)
- [Changelog](https://github.com/discourse/message_bus/blob/main/CHANGELOG)
- [Commits](https://github.com/discourse/message_bus/compare/v4.3.1...v4.3.2)

---
updated-dependencies:
- dependency-name: message_bus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-13 12:22:21 +00:00
dependabot[bot]
5cd136510a
Build(deps): Bump message-bus-client in /app/assets/javascripts (#19864)
Bumps [message-bus-client](https://github.com/discourse/message_bus) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/discourse/message_bus/releases)
- [Changelog](https://github.com/discourse/message_bus/blob/main/CHANGELOG)
- [Commits](https://github.com/discourse/message_bus/compare/v4.3.1...v4.3.2)

---
updated-dependencies:
- dependency-name: message-bus-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-13 12:22:00 +00:00
David Taylor
ce6335693a
DEV: Bump sprockets to include ERB kwargs fix (#19850)
This should resolve these warnings under Ruby 3.1

```
warning: Passing safe_level with the 2nd argument of ERB.new is deprecated
```

Unfortunately Sprockets 3.x has not seen a rubygems release since 2018, so we need to fetch these improvements via git.
2023-01-13 10:52:05 +00:00
Martin Brennan
b4b8b03461
DEV: Add option to disable rspec diff truncation ENV var (#19861)
Sometimes you may have a large string or object that you are comparing
with some expectation, and you want to see the full diff between actual
and expected without rspec truncating 90% of the diff. Setting the
max_formatted_output_length to nil disables this truncation completely.

c.f. https://www.rubydoc.info/gems/rspec-expectations/RSpec/Expectations/Configuration#max_formatted_output_length=-instance_method

Use `RSPEC_DISABLE_DIFF_TRUNCATION=1` to disable this.
2023-01-13 13:31:28 +10:00
Alan Guo Xiang Tan
8ee71d439b
FIX: Add migration to reindex invalid indexes (#19858)
In Discourse, there are many migration files where we CREATE INDEX CONCURRENTLY which requires us to set disable_ddl_transaction!. Setting disable_ddl_transaction! in a migration file runs the SQL statements outside of a transaction. The implication of this is that there is no ROLLBACK should any of the SQL statements fail.

We have seen lock timeouts occuring when running CREATE INDEX CONCURRENTLY. When that happens, the index would still have been created but marked as invalid by Postgres.

Per the postgres documentation:

> If a problem arises while scanning the table, such as a deadlock or a uniqueness violation in a unique index, the CREATE INDEX command will fail but leave behind an “invalid” index. This index will be ignored for querying purposes because it might be incomplete; however it will still consume update overhead.
> The recommended recovery method in such cases is to drop the index and try again to perform CREATE INDEX CONCURRENTLY . (Another possibility is to rebuild the index with REINDEX INDEX CONCURRENTLY ).

When such scenarios happen, we are supposed to either drop and create the index again or run a REINDEX operation. However, I noticed today that we have not been doing so in Discourse. Instead, we’ve been incorrectly working around the problem by checking for the index existence before creating the index in order to make the migration idempotent. What this potentially mean is that we might have invalid indexes which are lying around in the database which PG will ignore for querying purposes.

This commits adds a migration which queries for all the
invalid indexes in the `public` namespace and reindexes them.
2023-01-13 11:04:26 +08:00
dependabot[bot]
690e2f15ab
Build(deps): Bump nokogiri from 1.13.10 to 1.14.0 (#19856)
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.13.10 to 1.14.0.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.13.10...v1.14.0)

---
updated-dependencies:
- dependency-name: nokogiri
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-12 23:25:14 -03:00
Selase Krakani
55bdab2b3b
FIX: Ensure poll extraction is not attempted if post body is absent (#19718)
Since the poll post handler runs very early in the post creation
process, it's possible to run the handler on an obiviously invalid post.

This change ensures the post's `raw` value is present  before
proceeding.
2023-01-13 01:47:44 +00:00
Selase Krakani
73ec80893d
FEATURE: Extend topic update API scope to allow status updates (#19654)
Allow an API key created with topic:update API scope to make updates to
topic status. This change also introduces an optional category_id scope
param.
2023-01-13 01:21:04 +00:00
Alan Guo Xiang Tan
8a1b50f62d
DEV: Update README to reflect that at least Ruby 3.1 is required (#19855)
Minimum Ruby version required was bumped in ab9ea50917
2023-01-13 07:03:48 +08:00
Alan Guo Xiang Tan
5db72f8daf
FIX: Preload user sidebar attrs when ?enable_sidebar=1 (#19843)
This allows users to preview the sidebar even when
`SiteSetting.naviation_menu` is set to `false`.
2023-01-13 06:47:58 +08:00
Daniel Waterworth
3030a53819
FIX: Prevent concurrent updates to top_topics (#19854)
to prevent lock timeouts
2023-01-12 14:03:26 -06:00
Isaac Janzen
28078d78e2
DEV: Make 'username' optional for bookmark notifications (#19851)
Data Explorer queries have a `user_id` assigned to each query created. DE Reports can be bookmarked for later reference. 

When creating the bookmark notification there was the possibility of a notification error being thrown (that made the notification menu inaccessible) due to a DE Query not having a owner (associated user_id). This can happen in a couple ways: 
- having a query created by a user that was then later deleted leaving the query without ownership
- having a TA create a query for a customer using a temporary account, that would then later be deleted leaving the query without ownership

Since there is a case that `bookmark.user` is not valid the PR makes the `bookmark.user.username` optional for a bookmark notification. As [tested](https://github.com/discourse/discourse/pull/19851/files#diff-5b5154de37f96988d551feff6f1dfe5ba804fbcbc1c33b5478dde02a447a634f) in the case a username is not present, we will still render the `content` of the notification minus the username. This creates a safe fallback when looking up non-valid users.
2023-01-12 12:22:11 -06:00
Sérgio Saquetim
8fd9a93a1a
DEV: Added notification type for 'discourse-circles' (#19834)
Reserved an ID to be used by notifications generated on the `discourse-circles` plugin.
2023-01-12 14:07:42 -03:00
Rafael dos Santos Silva
192d8c25e6
DEV: Move back to web-push gem (#19849)
Our fork was needed for OpenSSL 3 and Ruby 2.X compatibility.

The OpenSSL 3 part was merged into the gem for version 3.

Discourse dropped support for Ruby 2.X.

That means we don't need our fork anymore.
2023-01-12 12:12:19 -03:00
David Taylor
ab9ea50917
Bump minimum Ruby version to 3.1 (#19848) 2023-01-12 13:52:50 +00:00
Ted Johansson
5dcb245eac
FIX: Ruby 2 backward compatible plugin logout redirect (#19845)
This is a very subtle one. Setting the redirect URL is done by passing
a hash through a Discourse event. This is broken on Ruby 2 since the
support for keyword arguments in events was added.

In Ruby 2 the last argument is cast to keyword arguments if it is a
hash. The key point here is that creates a new copy of the hash, so
what the plugin is modifying is not the hash that was passed.
2023-01-12 19:12:20 +08:00
Loïc Guitaut
4093fc6074 Revert "DEV: Migrate existing cookies to Rails 7 format"
This reverts commit 66e8fe9cc6 as it
unexpectedly caused some users to be logged out. We are investigating
the problem.
2023-01-12 12:07:49 +01:00
Loïc Guitaut
66e8fe9cc6 DEV: Migrate existing cookies to Rails 7 format
This patch introduces a cookies rotator as indicated in the Rails
upgrade guide. This allows to migrate from the old SHA1 digest to the
new SHA256 digest.
2023-01-12 11:09:07 +01:00
dependabot[bot]
1a759fd75f
Build(deps): Bump @ember/render-modifiers in /app/assets/javascripts (#19832)
Bumps [@ember/render-modifiers](https://github.com/emberjs/ember-render-modifiers) from 2.0.4 to 2.0.5.
- [Release notes](https://github.com/emberjs/ember-render-modifiers/releases)
- [Changelog](https://github.com/emberjs/ember-render-modifiers/blob/master/CHANGELOG.md)
- [Commits](https://github.com/emberjs/ember-render-modifiers/compare/v2.0.4...v2.0.5)

---
updated-dependencies:
- dependency-name: "@ember/render-modifiers"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-12 10:47:23 +01:00
Alan Guo Xiang Tan
9e55a1ca88
DEV: Fix typo in chat spec (#19836) 2023-01-12 14:46:48 +10:00
Martin Brennan
9a6eefaafc
DEV: Fix chat sidebar system spec flaky (#19844)
The spec was flaky because it was dependent on order,
when usernames got high enough sequence numbers in them
we would get this error:

> expected to find text "bruce99, bruce100" in "bruce100, bruce99"

Also move selectors into page object and use them in the
spec instead.
2023-01-12 14:12:49 +10:00
Martin Brennan
2ed75dbaf6
DEV: DRY up PageObject::Topic and PageObject::Components::Composer (#19841)
The latter can be called directly from the Topic page object,
so we can remove some duplication between the two. There are
levels of page objects (e.g. entire page, component, complete flow)
and its perfectly valid to call one from another.
2023-01-12 13:54:26 +10:00
Martin Brennan
1f59a8299d
DEV: Address TODOs for email Message-ID generation (#19842)
Remove some old deprecated methods and update docs. Will leave
the backwards-compatible Message-ID formats a little while longer
just to be sure.
2023-01-12 13:54:15 +10:00
Ted Johansson
421fbfd1c7
FIX: Fix flaky test resulting from PostAlerter keyword arguments (#19826)
We've been doing some work to support new keyword argument semantics in Ruby 3. As part of that we made some changes to `DiscourseEvent::TestHelper`. The backwards compatibility fix doesn't work if the method is called with an empty hash as the final argument. This fix adds a valid option to the final hash in the particular test.
2023-01-12 09:46:50 +08:00
Martin Brennan
98d5a0e63c
DEV: Remove old TODO for chat webhooks (#19839)
I inspected the JSON payload from OpsGenie and added a
note to show what it can look like.
2023-01-12 11:45:53 +10:00
Martin Brennan
779b9add24
DEV: Remove bookmark column ignores (#19838)
These columns were deleted in f8f55cef67
2023-01-12 11:45:37 +10:00
Sam
29ef2cb550
FEATURE: raise redirect avatar cache to 1 day (#19840)
In "GlobalSetting.redirect_avatar_requests" mode, when the application gets
an avatar request it returns a "redirect" to the S3 CDN.

This shields the application from caching avatars and downloading from S3.
However clients will make 2 requests per avatar. (one to get redirect,
second to get avatar)

A one hour cache on a redirect means there may be an increase in CDN
traffic, given more clients will ask for the redirect every hour.

This may also lead to an increase in origin requests to the application.

To mitigate lets cache the CDN URL for 1 day.

The downside is that any changes to S3 CDN need extra care to allow for
the extra 1 day delay. (leave data around for 1 extra day)
2023-01-12 12:40:42 +11:00
Sam
7b63c42304
FEATURE: add basic instrumentation to defer queue (#19824)
This will give us some aggregate stats on the defer queue performance.

It is limited to 100 entries (for safety) which is stored in an LRU cache.

Scheduler::Defer.stats can then be used to get an array that denotes:

- number of runs and completions (queued, finished)
- error count (errors)
- total duration (duration)

We can look later at exposing these metrics to gain visibility on the reason
the defer queue is clogged.
2023-01-12 12:29:50 +11:00
Alan Guo Xiang Tan
9fcd8336e4
FIX: Regression in TopicTrackingState MessageBus message scope. (#19835)
0403cda1d1 introduced a regression where
topics in non read-restricted categories have its TopicTrackingState
MessageBus messages published with the `group_ids: [nil]` option. This
essentially means that no one would be able to view the message.
2023-01-12 08:22:28 +08:00
Martin Brennan
387693e889
FIX: Improve error reporting and failure modes for channel archiving (#19791)
There was an issue with channel archiving, where at times the topic
creation could fail which left the archive in a bad state, as read-only
instead of archived. This commit does several things:

* Changes the ChatChannelArchiveService to validate the topic being
  created first and if it is not valid report the topic creation errors
  in the PM we send to the user
* Changes the UI message in the channel with the archive status to reflect
  that topic creation failed
* Validate the new topic when starting the archive process from the UI,
  and show the validation errors to the user straight away instead of
  creating the archive record and starting the process

This also fixes another issue in the discourse_dev config which was
failing because YAML parsing does not enable all classes by default now,
which was making the seeding rake task for chat fail.
2023-01-12 10:04:46 +10:00
Martin Brennan
21a95b000e
DEV: Remove defunct TODOs (#19825)
* Firefox now finally returns PerformanceMeasure from performance.measure
* Some TODOs were really more NOTE or FIXME material or no longer relevant
* retain_hours is not needed in ExternalUploadsManager,  it doesn't seem like anywhere in the UI sends this as a param for uploads
* https://github.com/discourse/discourse/pull/18413 was merged so we can remove JS test workaround for settings
2023-01-12 09:41:39 +10:00
Isaac Janzen
92bb728fe5
DEV: Add search suggestions for tag-intersections (#19777)
Added `tagIntersection` search context for handling search suggestions on tag intersection and tag+category routes.

# Tag & Category Route Search Suggestions
eg. /tags/c/general/5/updates
### Before
<img width="422" alt="Screenshot 2023-01-06 at 2 58 50 PM" src="https://user-images.githubusercontent.com/50783505/211098933-ade438c6-5008-49ce-9a90-c8200ec5fe00.png">

### After
<img width="359" alt="Screenshot 2023-01-06 at 3 00 35 PM" src="https://user-images.githubusercontent.com/50783505/211099183-c3feaeac-8661-47ed-843c-da9d9fb78e9e.png">

# Tag Intersection Route Search Suggestions
eg. /tags/intersection/updates/foo
### Before
<img width="421" alt="Screenshot 2023-01-06 at 3 02 23 PM" src="https://user-images.githubusercontent.com/50783505/211099435-e8fc6d87-2772-45b5-b455-1831f80eab3a.png">

### After
<img width="362" alt="Screenshot 2023-01-09 at 2 02 09 PM" src="https://user-images.githubusercontent.com/50783505/211397349-acb350f7-8e6a-4d9f-a749-8292e49400d9.png">

I defaulted to using `+` as a separator for tag intersections. The reasoning behind this is that we don't make the tag intersection routes easily accessible, so if you are going out of your way to view multiple tags, you are most likely going to be searching by **both** of those tags as well.

# General Search
Introducing flex wrap removes whitespace causing a [test](https://github.com/discourse/discourse/pull/19777/files#diff-5d3d13fabc1a511635eb7471ffe74f4d455d77f6984543c2be6ad136dfaa6d3aR813) to fail, but to remedy this I added spacing to the `.search-item-prefix` and `.search-item-slug` which achieves the same thing.

### After
<img width="359" alt="Screenshot 2023-01-09 at 2 04 54 PM" src="https://user-images.githubusercontent.com/50783505/211397900-60220394-5596-4e13-afd0-b6130afa0de2.png">
2023-01-11 13:02:22 -06:00