Commit Graph

1683 Commits

Author SHA1 Message Date
Angus McLeod
cb4b8146a3
Add dedicated user_api_key_clients table to allow for 1:many use cases (#28119) 2024-11-08 12:05:03 -05:00
Krzysztof Kotlarek
d57b7abe67
FIX: setup with db:create db:migrate (#29609)
After data seed, we should reset Redis cache to ensure that the correct flags are cached.

However, `db:create` is skipping Redis 

https://github.com/discourse/discourse/blob/main/lib/tasks/db.rake#L39

And uses `ActiveSupport::Cache::MemoryStore.new`

e2292d4c59/lib/discourse.rb (L523)

Therefore, the reset flags cache was moved to initializers and evaluated only when the cache is Redis and Flags table already exists.

Meta: https://meta.discourse.org/t/development-install-fails-when-running-bin-rails-db-migrate/332754
2024-11-06 15:06:17 +11:00
Alan Guo Xiang Tan
57f4176b57
DEV: Bump rubocop_discourse (#29608) 2024-11-06 06:27:49 +08:00
Bianca Nenciu
723dc1fa55
Dev fix some types (#29547)
The primary key is usually a bigint column, but the foreign key columns
are usually of integer type. This can lead to issues when joining these
columns due to mismatched types and different value ranges.

This was using a temporary plugin / test API to make tests pass. After
more careful consideration, we concluded that it is safe to alter the
tables directly.
2024-11-01 19:19:25 +02:00
Osama Sayegh
397d41008e
FIX: Account for sidebars with no primary links when adding invite link (#29535)
Some checks are pending
Licenses / run (push) Waiting to run
Linting / run (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (annotations, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, themes) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, chat) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, themes) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Chrome) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox ESR) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox Evergreen) (push) Waiting to run
Meta topic: https://meta.discourse.org/t/stumped-about-launcher-rebuild-app-error-process-pid-2096/333876?u=osama.

Follow up to 19672faba6.

The migration that adds the invite link to the sidebar determines the position of the link by looking up the max position that a primary link has and inserts the invite link at the max position plus 1. This approach works fine for most sites, however, sites that have deleted all primary links from the sidebar will fail because the max position will be `nil` which blows up the migration.

This commit addresses this edge case by falling back to looking up the min position of secondary links,    or to zero if there're also no secondary links, and then inserts the invite link at the min position minus 1.
2024-11-01 10:31:21 +03:00
Osama Sayegh
19672faba6
FEATURE: Add invite link to the sidebar (#29448)
Some checks are pending
Licenses / run (push) Waiting to run
Linting / run (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, themes) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (annotations, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, themes) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, chat) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, core) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Chrome) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox ESR) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox Evergreen) (push) Waiting to run
This commit adds a new "Invite" link to the sidebar for all users who can invite to the site. Clicking the link opens the invite modal without changing the current route the user is on. Admins can customize the new link or remove it entirely if they wish by editing the sidebar section.

Internal topic: t/129752.
2024-10-30 05:31:14 +03:00
Osama Sayegh
2ffe413b0b
FEATURE: Enable the new /about page for everyone (#29390)
This commit removes the feature flag for the new /about page, enabling it for all sites, and removes the code for old the /about page.

Internal topic: t/140413.
2024-10-29 18:40:11 +03:00
Martin Brennan
27bdfb6437
FEATURE: Add user preference to disable smart lists (#29434)
Some checks are pending
Licenses / run (push) Waiting to run
Linting / run (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (annotations, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (backend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (frontend, themes) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, chat) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, core) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, plugins) (push) Waiting to run
Tests / ${{ matrix.target }} ${{ matrix.build_type }} (system, themes) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Chrome) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox ESR) (push) Waiting to run
Tests / core frontend (${{ matrix.browser }}) (Firefox Evergreen) (push) Waiting to run
Followup 30fdd7738e

Adds a new site setting and corresponding user preference
to disable smart lists. By default they are enabled, because
this is a better experience for most users. A small number of
users would prefer to not have this enabled.

Smart lists automatically append new items to each
list started in the composer when enter is pressed. If
enter is pressed on an empty list item, it is cleared.

This setting will be removed when the new composer is complete.
2024-10-28 14:18:15 +10:00
Kelv
a89767913d
resolve Rails/ReversibleMigrationMethodDefinition errors (#29412) 2024-10-25 15:13:06 +08:00
Guhyoun Nam
82ad7f9d17
DEV: Delete plugin specific webhook event types (#29374)
Background
When creating webhooks on a site without the Discourse Category Experts plugin installed, the category_experts_unapproved_event and category_experts_approved_event webhook events are getting automatically added to webhooks without a way to disable them.

The category_experts_unapproved_event and category_experts_approved_event webhook events are associated with the Discourse Category Experts plugin so I am moving these webhook events into the Category Experts plugin.

Changes
This PR deletes Category Experts plugin specific webhook event types added into core.
2024-10-24 11:02:58 -05:00
Kelv
8f9b827d15
DEV: drop password-related columns from users table (#29187) 2024-10-23 13:42:35 +08:00
Ted Johansson
ca9549b04f
DEV: Clear duplicate admin problem notices (#29358)
In #29272 we added a backwards-compatible way to prevent duplicate problem check trackers. However, there was a mistake in the PR that would instead create duplicate admin notices. As a result, a number of admins now have multiple copies of the same admin notice in their dashboard.

The root cause was fixed in #29329, preventing new duplicates.

This PR is here to clean up notices that were already created.

Admin notices are meant to be ephemeral. Instead of going through hoops to delete duplicates and update the remaining notices' unstructured field with the correct target, it is a lot less error-prone to delete all notices and let the problem check system re-create them.

The real-time checks run every time the dashboard is loaded, so they will appear to never have been deleted. Any notices related to scheduled checks will be added back on the next run. This will happen within at most one hour, and isn't time sensitive.
2024-10-23 13:02:02 +08:00
Ted Johansson
63ed1b5bd7
DEV: Remove NULLS NOT DISTINCT from problem check trackers (#29327)
We added NULLS NOT DISTINCT to a unique index on problem_check_trackers.

This option is only available in PG15+. It does not in itself break PG13, but restoring a PG15+ backup to PG13 currently errors out. It seems this is an operation that's more common than we first thought.

This commit fixes that by removing the NULLS NOT DISTINCT.

We already have another, backwards-compatible approach to do the same thing in place, so this shouldn't change existing behaviour.
2024-10-22 10:47:24 +08:00
Ted Johansson
93625ef07c
FIX: Make problem check tracker unique constraint work on PG13 (#29272)
In #29169 we added a NULLS NOT DISTINCT option to the unique index on problem_check_trackers. This is to enforce uniqueness even when the target is NULL. (Postgres considers all NULLs to be distinct by default.)

However, this only works in PG15. In PG13 it does nothing.

This commit adds a default dummy string value __NULL__ to target. Since it's a string, PG13 will be able to correctly identify duplicate records.
2024-10-21 10:45:09 +08:00
Sam
c08c40dc23
FEATURE: theme_modifiers can depend on theme settings (plus serialize_post_user_badges) (#29227)
Theme modifiers can now be defined as theme settings, this allows for
site operators to override behavior of theme modifiers.

New syntax is:

```
{
    ...
   "modifiers": {
      "modifier_name": {
         "type": "setting",
         "value": "setting_name"
      }
   }
}
```

This also introduces a new theme modifier for serialize_post_user_badges. Name of badge must match the name of the badge in the badges table. The client-side is updated to load this new data from the post-stream serializer.

Co-authored-by: David Taylor <david@taylorhq.com>
2024-10-17 15:16:16 +01:00
Alan Guo Xiang Tan
322a3be2db
DEV: Remove logical OR assignment of constants (#29201)
Constants should always be only assigned once. The logical OR assignment
of a constant is a relic of the past before we used zeitwerk for
autoloading and had bugs where a file could be loaded twice resulting in
constant redefinition warnings.
2024-10-16 10:09:07 +08:00
Bianca Nenciu
8016fcab33
DEV: Drop old notification id columns (#28550)
The `id` column of `notifications` table and `notification_id` columns
of the other tables have been migrated to bigint in previous commits
(for example, 799a45a).

In order to run the migrations with zero downtime, the data had to be
copied to new columns and swapped, but the old columns have been kept
to allow for rollback. They are no longer needed now.
2024-10-15 11:58:57 +03:00
Ted Johansson
45c9316d7d
DEV: Fix problem check tracker unique index not respecting NULLs (#29169)
By default, when checking uniqueness on a tuple for the purposes of enforcing a unique index, PostgreSQL considers NULLs to be distinct values. Because of this we could incorrectly have multiple entries with { identifier: "rails_env", target: nil } created due to race conditions. This would then cause errors at runtime.
2024-10-14 13:55:35 +08:00
Alan Guo Xiang Tan
d90203f27f
DEV: Break up PG query in MakePasswordColumnsFromUsersReadOnly (#29167)
We are getting a lock timeout but I can't tell which statement so break
it up.
2024-10-11 11:25:55 +08:00
Kelv
32e261ef73
DEV: Migrate user passwords data to UserPassword table (#28746)
* Add migrations to ensure password hash is synced across users & user_passwords

* Persist password-related data in user_passwords instead of users

* Merge User#expire_old_email_tokens with User#expire_tokens_if_password_changed

* Add post deploy migration to mark password-related columns from users table as read-only

* Refactored UserPassword#confirm_password? and changes required to accommodate hashing the password after validations
2024-10-10 09:23:06 +08:00
Alan Guo Xiang Tan
02380af75c
PERF: Update migration to drop index concurrently. (#28955)
This avoids locking the index's table
2024-09-18 10:20:24 +08:00
Loïc Guitaut
bee8214399 DEV: Drop unused column translation_overrides.compiled_js 2024-09-17 09:58:10 +02:00
Kris
a914d3230b
DEV: remap all core icons for fontawesome 6 upgrade (#28715)
Followup to 7d8974d02f

Co-authored-by: David Taylor <david@taylorhq.com>
2024-09-13 16:50:52 +01:00
Osama Sayegh
3baf6233aa
FIX: Remove anonymous users from users directory (#28892)
Anonymous users are "shadow" users created when an existing real user desires to post anonymously. This feature is off by default, but it can be enabled via the `allow_anonymous_posting` site setting. Those shadow users shouldn't be included in the users directory (`/u`).
2024-09-13 15:12:57 +03:00
Alan Guo Xiang Tan
97143efc52
PERF: Drop user_search_similar_results site setting (#28874)
In 14cf8eacf1, we added the
`user_search_similar_results` site setting which when enabled will use
trigram matching for similarity search in `UserSearch`. However, we
noted that adding the `index_users_on_username_lower_trgm` index is
causing the PG planner to not use the `index_users_on_username_lower`
index when the `=` operator is used against the `username_lower` column.

Based on the PG mailing list discussion where support for the `=`
operator in gist_trgm_ops was being considered, it stated that "I also have checked that btree_gist is preferred over pg_trgm gist
index for equality search." This is however quite different from reality
on our own PG clusters where the btree index is not preferred leading to
significantly slower queries when the `=` operator is used.

Since the pg_trgm gist index is only used for queries when the `user_search_similar_results` site setting
is enabled, we decided to drop the feature instead as it is hidden and
disabled by default. As such, we can consider it experiemental and drop
it without deprecation.

PG mailing list discussiong: https://www.postgresql.org/message-id/CAPpHfducQ0U8noyb2L3VChsyBMsc5V2Ej2whmEuxmAgHa2jVXg%40mail.gmail.com
2024-09-13 09:04:02 +08:00
Joffrey JAFFEUX
ad90b9cfd1
DEV: Increases external avatar url limit (#28893)
We recently increased it to 1500 but that was not enough: 113e6fd274
2024-09-13 01:08:01 +02:00
Keegan George
f2059bf15f
FIX: Form template limit validation (#28791) 2024-09-10 08:11:44 -07:00
Guhyoun Nam
aacd354de5
FEATURE: Added Category Experts unapproved post WebHook Event (#28802)
This PR is adding a Category Experts unapproved WebHook event type.
2024-09-10 07:56:35 +09:00
David Taylor
7d8974d02f
UX: Upgrade to fontawesome 6.6.0 (#28778)
This upgrade is designed to be fully backwards-compatible. Any icon names which have changed will be automatically remapped to the new name. For now, this will happen silently. In future, once core & official themes/plugins have been updated, we will start raising deprecation errors to help theme/plugin authors update their code.

Extracted from https://github.com/discourse/discourse/pull/28715

Announcement at https://meta.discourse.org/t/were-upgrading-our-icons-to-font-awesome-6/325349

Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2024-09-09 14:40:56 +01:00
Krzysztof Kotlarek
2bb740cb16
FIX: reset cache after flags are seed (#28801)
There was a bug report on meta that after deployment, some topics were not loaded properly for 24 hours - https://meta.discourse.org/t/endless-spinner-on-some-posts/323031/17

In this PR we moved the `like` post action type to the database - https://github.com/discourse/discourse/pull/28362

However, flags are cached for performance reasons https://github.com/discourse/discourse/blob/main/app/serializers/site_serializer.rb#L113

After seed, we should ensure that the cache is reset so the site has access to all post-action types.
2024-09-09 10:40:43 +10:00
Sérgio Saquetim
7c3ad27de6
DEV: Remove the old header widgets code (#28390)
Remove the header widget code.

More info can be found in https://meta.discourse.org/t/upcoming-header-changes-preparing-themes-and-plugins/296544
2024-09-04 14:50:53 -03:00
Guhyoun Nam
93564bfa7d
DEV: Approved Post Webhook Event Category Update (#28725)
'Post approved by category experts' is moved to the post category
2024-09-04 13:58:17 +09:00
Osama Sayegh
280adda09c
FEATURE: Support designating multiple groups as mods on category (#28655)
Currently, categories support designating only 1 group as a moderation group on the category. This commit removes the one group limitation and makes it possible to designate multiple groups as mods on a category.

Internal topic: t/124648.
2024-09-04 04:38:46 +03:00
Natalie Tay
cc873977ec
DEV: Ensure unique notification level per tag user (#28638)
TagUser.rb is used to set user notification levels for a tag, we don't have a unique index on the notification level itself. This means that there might be some weird case where a user may have multiple of the same notification level on a tag.

This PR adds a migration which de-duplicates this based on defaults, where we keep the earliest record in the event there is multiple notification level per-user-per-tag.
2024-09-03 15:43:02 +08:00
Kelv
a455567f9e
DEV: make UserPassword 1:1 to User (#28528)
* add data migration to keep only unexpired or most recently expired user password
* refactor to 1:1 relationship between User and UserPassword
* add migration to remove redundant indexes on user passwords
2024-09-03 11:09:33 +08:00
Guhyoun Nam
050c665160
FEATURE: Added Category Experts approved WebHook Event (#28525)
This PR is adding a Category Experts WebHook event type.
2024-09-03 06:59:15 +09:00
Alan Guo Xiang Tan
4a6fc45429
DEV: Migrate User#seen_notification_id to bigint (#28572)
`Notification#id` was migrated to `bigint` in 799a45a291
2024-08-27 14:32:55 +03:00
Bianca Nenciu
b11d901e12
DEV: Migrate user_badges#notification_id to bigint (#28546)
The `notifications.id` has been migrated to bigint in previous commit
799a45a291. This commit migrates one of
the related columns, `user_badges.notification_id`, to `bigint`.
2024-08-27 11:46:48 +08:00
Bianca Nenciu
ec8ba5a0b9
DEV: Migrate shelved_notifications#notification_id to bigint (#28549)
DEV: Migrate shelved_notifications#notification_id to bigint

The `notifications.id` has been migrated to `bigint` in previous commit
799a45a291.
2024-08-27 10:56:00 +08:00
Alan Guo Xiang Tan
6832c7b7e7
DEV: Fix bigint notifications id migration to deal with public schema (#28538)
Follow up to 799a45a291
2024-08-26 10:25:13 +08:00
Bianca Nenciu
799a45a291
DEV: Migrate notifications#id to bigint (#28444)
The `notifications.id` column is the most probable column to run out of
values. This is because it is an `int` column that has only 2147483647
values and many notifications are generated on a regular basis in an
active community. This commit migrates the column to `bigint`.

These migrations do not use `ALTER TABLE ... COLUMN ... TYPE` in order
to avoid the `ACCESS EXCLUSIVE` lock on the entire table. Instead, they
create a new `bigint` column, copy the values to the new column and
then sets the new column as primary key.

Related columns (see `user_badges`, `shelved_notifications`) will
be migrated in a follow-up commit.
2024-08-26 09:35:12 +08:00
Krzysztof Kotlarek
e82e255531
FIX: serialize Flags instead of PostActionType (#28362)
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems. 

### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons. 

At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.

To test backward compatibility you can add this code to any plugin
```ruby
  replace_flags do |flag_settings|
    flag_settings.add(
      4,
      :inappropriate,
      topic_type: true,
      notify_type: true,
      auto_action_type: true,
    )
    flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
  end
```
2024-08-14 12:13:46 +10:00
Krzysztof Kotlarek
559c9dfe0a
REVERT: FIX: serialize Flags instead of PostActionType (#28334) 2024-08-13 18:32:11 +10:00
Krzysztof Kotlarek
d2c09e5ce1
PERF: do not expire cache when PostActionType is saved (#28337)
Reload is too expensive in a multisite environment. A proper way to expire cache is coming with the next PR.
2024-08-13 18:01:31 +10:00
Krzysztof Kotlarek
a6748a7c37
PERF: no redis calls when running migrations (#28336)
Avoid reaching Redis cache while running migrations
2024-08-13 14:20:31 +08:00
Krzysztof Kotlarek
c1f6ec3019
PERF: skip reset flag callbacks on seed (#28333)
When flags are seeded, we don't need to reset the cache for each record.

We should instead create all flags and reset the cache at the very end.
2024-08-13 13:22:57 +10:00
Krzysztof Kotlarek
09262ac9b9
FIX: not hit Redis cache in seeds (#28329)
Hardcode IDs because seed in multisite environment is run concurrently which can cause performance problems.
2024-08-13 12:42:54 +10:00
Krzysztof Kotlarek
094052c1ff
FIX: serialize Flags instead of PostActionType (#28259)
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems. 

### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons. 

At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.

To test backward compatibility you can add this code to any plugin
```ruby
  replace_flags do |flag_settings|
    flag_settings.add(
      4,
      :inappropriate,
      topic_type: true,
      notify_type: true,
      auto_action_type: true,
    )
    flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
  end
```
2024-08-13 11:22:37 +10:00
carson chang
854b8b7093
FEATURE: Add option to immediately delete stub topics upon merge (#28228)
Currently to handle stub topics after merging, there are only options to (1) never delete a stub topic and (2) delete a stub topic after X amount of days. This adds the option to immediately delete a stub topic upon merge.

---------

Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
Co-authored-by: Renato Atilio <renato@discourse.org>
2024-08-07 10:05:40 -03:00
Blake Erickson
6ee6b1f1d1
DEV: Add validation for allowed iframes setting (#28178)
- Adds a validator for the allowed iframes site setting
- Adds a migration to update any values that don't pass the validator

Follow up to: 188cb58daa
2024-08-01 06:51:02 -06:00