Commit Graph

1695 Commits

Author SHA1 Message Date
Ted Johansson
36057638ca
DEV: Convert min_trust_to_edit_post to groups (#24840)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_to_edit_post site setting to edit_post_allowed_groups.

The old implementation will co-exist for a short period while I update any references in plugins and themes.
2023-12-13 13:25:13 +08:00
Krzysztof Kotlarek
702d0620d7
DEV: Convert min_trust_to_create_topic to groups (#24740)
This change converts the min_trust_to_create_topic site setting to
create_topic_allowed_groups.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change
- After a couple of months, we will remove the min_trust_to_create_topicsetting entirely.

Internal ref: /t/117248
2023-12-13 14:50:13 +11:00
Krzysztof Kotlarek
1017820012
DEV: Convert allow_uploaded_avatars to groups (#24810)
This change converts the allow_uploaded_avatars site setting to uploaded_avatars_allowed_groups.

See: https://meta.discourse.org/t/283408

Hides the old setting
Adds the new site setting
Adds a deprecation warning
Updates to use the new setting
Adds a migration to fill in the new setting if the old setting was changed
Adds an entry to the site_setting.keywords section
Updates tests to account for the new change
After a couple of months, we will remove the allow_uploaded_avatars setting entirely.

Internal ref: /t/117248
2023-12-13 10:53:19 +11:00
Krzysztof Kotlarek
7dd150bc95
DEV: Convert min_trust_to_edit_wiki_post to groups (#24766)
This change converts the min_trust_to_edit_wiki_post site setting to edit_wiki_post_allowed_groups.

See: https://meta.discourse.org/t/283408

Hides the old setting
Adds the new site setting
Add a deprecation warning
Updates to use the new setting
Adds a migration to fill in the new setting if the old setting was changed
Adds an entry to the site_setting.keywords section
Updates tests to account for the new change
After a couple of months, we will remove the email_in_min_trust setting entirely.

Internal ref: /t/117248
2023-12-12 15:20:37 +11:00
Martin Brennan
7afb5fc481
DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743)
I couldn't find where we created the system user and
this is why -- everywhere else in the app we reference
SYSTEM_USER_ID but here.
2023-12-07 09:04:45 +10:00
Jarek Radosz
6a66dc1cfb
DEV: Fix Lint/BooleanSymbol (#24747) 2023-12-06 13:19:09 +01:00
Krzysztof Kotlarek
5551a71c55
FEATURE: increase tag description limit to 1000 (#24561)
Admin can add tag description up to 1000 characters.

Full description is displayed on tag page, however on topic list it is truncated to 80 characters.
2023-11-28 08:45:40 +11:00
Blake Erickson
52617e7425
DEV: Auto link video placeholder posts (#24576)
* DEV: Auto link video placeholder posts

Followup to 91232847e3

Adds a migration to auto-link any video placeholder posts so that upload
references will be created.

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2023-11-27 13:22:42 -07:00
Sam
b3920e05e7
PERF: avoid table scan while performing a very large update (#24525)
We were seeing lots of deadlocks deploying this migration. This improves
the situation in 2 ways.

1. ddl transaction is avoided, so we hold locks for far shorter times
2. we operate in chunks of a maximum of 100_000 posts (though it is heavily filtered down)

* improve code so it is clearer
2023-11-23 18:15:40 +11:00
Blake Erickson
c89edd9e86
DEV: Convert email_in_min_trust to groups (#24515)
This change converts the `email_in_min_trust` site setting to
`email_in_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`email_in_min_trust` setting entirely.

Internal ref: /t/115696
2023-11-22 18:03:28 -07:00
Blake Erickson
8a45f84277
DEV: Convert approve_new_topics_unless_trust_level to groups (#24504)
* DEV: Convert approve_new_topics_unless_trust_level to groups

This change converts the `approve_new_topics_unless_trust_level` site
setting to `approve_new_topics_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`approve_new_topics_unless_trust_level` setting entirely.

Internal ref: /t/115696

* add missing translation

* Add keyword entry

* Add migration
2023-11-22 10:44:59 -07:00
Blake Erickson
447d9b2105
DEV: Convert approve_unless_trust_level to groups (#24357)
This change converts the `approve_unless_trust_level` site setting to
`approve_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Adds the new site setting
- Adds a deprecation warning
- Updates core to use the new settings.
- Adds a migration to fill in the new setting of the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates many tests to account for the new change

After a couple of months we will remove the `approve_unless_trust_level`
setting entirely.

Internal ref: /t/115696
2023-11-21 11:31:42 -07:00
Blake Erickson
ef7e5ed9b4
DEV: Rebake posts w/ quotes to update category badge style (#24469)
Posts with quotes need to be rebaked to apply the new category badge
style css.

See: https://meta.discourse.org/t/285705
2023-11-20 14:17:44 -07:00
Penar Musaraj
a814348176
DEV: Rename experimental_passkeys to enable_passkeys (#24349)
Also includes a migration.
2023-11-13 15:04:15 -05:00
Martin Brennan
67ac4c5616
DEV: Move min_trust_level_for_here_mention to group setting (#24263)
c.f. https://meta.discourse.org/t/-/283408
2023-11-09 12:07:01 +10:00
Martin Brennan
b90b7ac705
DEV: Move shared_drafts_min_trust_level to group setting (#24257)
No plugins or themes rely on shared_drafts_min_trust_level so we
can just switch straight over to shared_drafts_allowed_groups

c.f. https://meta.discourse.org/t/changes-coming-to-settings-for-giving-access-to-features-from-trust-levels-to-groups/283408
2023-11-07 14:03:25 +10:00
Martin Brennan
a86833fe91
FIX: Deleting/recovering a post in topics caused bookmark side effects (#24226)
This commit fixes an issue where when some actions were done
(deleting/recovering post, moving posts) we updated the
topic_users.bookmarked column to the wrong value. This was happening
because the SyncTopicUserBookmarked job was not taking into account
Topic level bookmarks, so if there was a Topic bookmark and no
Post bookmarks for a user in the topic, they would have
topic_users.bookmarked set to false, which meant the bookmark would
no longer show in the /bookmarks list.

To reproduce before the fix:

* Bookmark a topic and don’t bookmark any posts within
* Delete or recover any post in the topic

c.f. https://meta.discourse.org/t/disappearing-bookmarks-and-expected-behavior-of-bookmarks/264670/36
2023-11-07 12:54:05 +10:00
Osama Sayegh
3cadd6769e
FEATURE: Theme settings migrations (#24071)
This commit introduces a new feature that allows theme developers to manage the transformation of theme settings over time. Similar to Rails migrations, the theme settings migration system enables developers to write and execute migrations for theme settings, ensuring a smooth transition when changes are required in the format or structure of setting values.

Example use cases for the theme settings migration system:

1. Renaming a theme setting.

2. Changing the data type of a theme setting (e.g., transforming a string setting containing comma-separated values into a proper list setting).

3. Altering the format of data stored in a theme setting.

All of these use cases and more are now possible while preserving theme setting values for sites that have already modified their theme settings.

Usage:

1. Create a top-level directory called `migrations` in your theme/component, and then within the `migrations` directory create another directory called `settings`.

2. Inside the `migrations/settings` directory, create a JavaScript file using the format `XXXX-some-name.js`, where `XXXX` is a unique 4-digit number, and `some-name` is a descriptor of your choice that describes the migration.

3. Within the JavaScript file, define and export (as the default) a function called `migrate`. This function will receive a `Map` object and must also return a `Map` object (it's acceptable to return the same `Map` object that the function received).

4. The `Map` object received by the `migrate` function will include settings that have been overridden or changed by site administrators. Settings that have never been changed from the default will not be included.

5. The keys and values contained in the `Map` object that the `migrate` function returns will replace all the currently changed settings of the theme.

6. Migrations are executed in numerical order based on the XXXX segment in the migration filenames. For instance, `0001-some-migration.js` will be executed before `0002-another-migration.js`.

Here's a complete example migration script that renames a setting from `setting_with_old_name` to `setting_with_new_name`:

```js
// File name: 0001-rename-setting.js

export default function migrate(settings) {
  if (settings.has("setting_with_old_name")) {
    settings.set("setting_with_new_name", settings.get("setting_with_old_name"));
  }
  return settings;
}
```

Internal topic: t/109980
2023-11-02 08:10:15 +03:00
Martin Brennan
24e087e2ba
FIX: Typo in migration (#24089)
Should be DB.exec not just exec, silly typo fix for
9db4eaa870
2023-10-25 12:54:25 +10:00
Martin Brennan
9db4eaa870
DEV: Change anonymous_posting_min_trust_level to a group-based setting (#24072)
No plugins or themes rely on anonymous_posting_min_trust_level so we
can just switch straight over to anonymous_posting_allowed_groups

This also adds an AUTO_GROUPS const which can be imported in JS
tests which is analogous to the one defined in group.rb. This can be used
to set the current user's groups where JS tests call for checking these groups
against site settings.

Finally a AtLeastOneGroupValidator validator is added for group_list site
settings which ensures that at least one group is always selected, since if
you want to allow all users to use a feature in this way you can just use
the everyone group.
2023-10-25 11:45:10 +10:00
Sérgio Saquetim
0cfc42e0e6
FEATURE: Add dark mode option for category backgrounds (#24003)
Adds a new upload field for a dark mode category background that will be used as an alternative when Discourse is using a dark mode theme.
2023-10-20 12:48:06 +00:00
Martin Brennan
3cde7f125d
DEV: Remove hashtag autocomplete experimental setting (#23954)
Everything relying on this was removed in
09223e5ae7
2023-10-17 15:05:58 +10:00
Krzysztof Kotlarek
c468110929
FEATURE: granular webhooks (#23070)
Before this change, webhooks could be only configured for specific groups like for example, all topic events.

We would like to have more granular control like for example topic_created or topic_destroyed.

Test are failing because plugins changed has to be merged as well:
discourse/discourse-assign#498
discourse/discourse-solved#248
discourse/discourse-topic-voting#159
2023-10-09 03:35:31 +00:00
Alan Guo Xiang Tan
832b3b9e60
FEATURE: Remove support for legacy navigation menu (#23752)
Why this change?

Back in May 17 2023 along with the release of Discourse 3.1, we announced
on meta that the legacy hamburger dropdown navigation menu is
deprecated and will be dropped in Discourse 3.2. This is the link to the announcement
on meta: https://meta.discourse.org/t/removing-the-legacy-hamburger-navigation-menu-option/265274

## What does this change do?

This change removes the `legacy` option from the `navigation_menu` site
setting and migrates existing sites on the `legacy` option to the
`header dropdown` option.

All references to the `legacy` option in code and tests have been
removed as well.
2023-10-09 07:24:10 +08:00
David Taylor
fc2e92d423
DEV: Drop post_uploads table (#23673)
The post_uploads table has not been used since 9db8f00b3d
2023-09-27 12:43:19 +01:00
Renato Atilio
d93c2cb3d2
FEATURE: site settings to revoke api keys older than a number of days (#23595)
* FEATURE: site settings to revoke api keys older than a number of days
2023-09-15 16:31:29 -03:00
Sérgio Saquetim
e03dd76dc6
FEATURE: add outgoing web hooks for Chat messages 2023-09-13 17:31:42 -03:00
OsamaSayegh
c1b5faa5fd
SECURITY: Limit name field length of TOTP authenticators and security keys 2023-09-12 15:31:17 -03:00
Ted Johansson
f08c6d2756
DEV: Switch over category settings to new table - Part 3 (#20657)
In #20135 we prevented invalid inputs from being accepted in category setting form fields on the front-end. We didn't do anything on the back-end at that time, because we were still discussing which path we wanted to take. Eventually we decided we want to move this to a new CategorySetting model.

This PR moves the require_topic_approval and require_reply_approval from custom fields to the new CategorySetting model.

This PR is nearly identical to #20580, which migrated num_auto_bump_daily, but since these are slightly more sensitive, they are moved after the previous one is verified.
2023-09-12 09:51:49 +08:00
Alan Guo Xiang Tan
26dfb8a489
DEV: Remove chat related migration (#23449)
What motivated this change?

A core migration contains chat related code and this should not be the
case since chat related migration code should live in the chat plugin.

What does this change do?

This change removes the migration which was introduced to keep existing
sites on the legacy navigation menu as well as keep chat disabled when
the defaults for the `navigation_menu` and `chat_enabled` site settings
were flipped. Since this migration doesn't apply to new sites and
the migration has already been introduced for 9 months, it is safe for
us to remove it.
2023-09-08 12:41:56 +08:00
Ted Johansson
ede73f923f
DEV: Change fast typer trust level setting to enum type (#23429)
We have one site setting, `auto_silence_fast_typers_max_trust_level`, which expects a trust level. However, the type is set to integer, which makes it very hard for a layman to enter the correct thing.

This PR changes the type of the site setting to the `TrustLevelSetting` enum.

The use of these are interchangeable in the back-end, since `SiteSetting.auto_silence_fast_typers_max_trust_level` still returns the integer value with the enum.
2023-09-06 17:35:11 +08:00
Blake Erickson
be32f580ad
DEV: Delete unused site settings (#23353)
In a previous commit these site settings were removed from the codebase
because they were identified as unused settings. This commit removes
these settings from the db in case they existed in the site settings
table.

Follow up to: da389d7844
2023-08-31 18:19:55 -06:00
Daniel Waterworth
80a0d88e4c
FIX: Ensure javascript caches are unique per theme/theme_field (#23126)
We were assuming that this was the case before, but not enforcing it.
2023-08-17 13:57:04 -05:00
Krzysztof Kotlarek
f8cd1da92a
FIX: increase sidebar URL limit to 1000 (#23120)
Before this change, sidebar URL had a limit of 200 characters. In some cases it is not enough, therefore it was increased to 1000.
2023-08-17 14:46:24 +10:00
Renato Atilio
840bea3c51
FEATURE: add topic voting webhook event type (#23072)
* FEATURE: add topic upvote webhook event type

* DEV: use a generic event type name for other actions in the same plugin
2023-08-11 13:42:28 -03:00
Ted Johansson
207d983809
FIX: Account for empty string setting values when migrating category settings (#22979)
This migration that populates category setting data from custom fields did not account for the fact that custom fields that expect an integer can still be an empty string. An empty string can't be inserted into the new integer type column.

If the setting is an empty string, cast it to NULL.
2023-08-04 20:57:48 +08:00
Ted Johansson
1f7e5e8e75
DEV: Switch over category settings to new table - Part 2 (#20580)
In #20135 we prevented invalid inputs from being accepted in category setting form fields on the front-end. We didn't do anything on the back-end at that time, because we were still discussing which path we wanted to take. Eventually we decided we want to move this to a new CategorySetting model.

This PR moves the num_auto_bump_daily from custom fields to the new CategorySetting model.

In addition it sets the default value to 0, which exhibits the same behaviour as when the value is NULL.
2023-08-04 10:53:22 +08:00
Ted Johansson
c4d0bbce62
DEV: Delete upload references upon deleting draft (#22851)
We currently are accumulating orphaned upload references whenever drafts are deleted.

This change deals with future cases by adding a dependent strategy of delete_all on the Draft#upload_references association. (We don't really need destroy strategy here, since UploadReference is a simple data bag and there are no validations or callbacks on the model.)

It deals with existing cases through a migration that deletes all existing, orphaned draft upload references.
2023-07-31 10:16:23 +08:00
Alan Guo Xiang Tan
fe5cd479eb
PERF: Add index on topic_id and created_at to posts table (#22818)
Why this change?

In `PostDestroyer#make_previous_post_the_last_one` and
`Topic.reset_highest`, we have a query that looks something like this:

```
SELECT user_id FROM posts
WHERE topic_id = :topic_id AND
      deleted_at IS NULL AND
      post_type <> 4
      #{post_type}
ORDER BY created_at desc
LIMIT 1
```

However, we currently don't have an index that caters directly to this
query. As a result, we have seen this query performing poorly on large
sites if the PG planner ends up using an index that is suboptimal for
the query.

This commit adds an index to the `posts` table on `topic_id` and then
`created_at`. For the query above, PG will be able to do a backwards
index scan efficiently.
2023-07-27 10:55:10 +08:00
Loïc Guitaut
25138db433 DEV: Promote historic post_deploy migrations
This commit promotes all post_deploy migrations which existed in
Discourse v3.0.0 (timestamp <= 20221212234948)
2023-07-26 10:36:37 +02:00
Ted Johansson
46f21104f4
FIX: Inline deprecated settings in migration file (#22737)
After deleting SiteSetting::ALLOWLIST_DEPRECATED_SITE_SETTINGS, we found out this was referenced in a migration file. This PR inlines the constant into the migration file to prevent it from erroring out.
2023-07-21 12:22:00 +08:00
Selase Krakani
3d554aa10e
FIX: Keep ReviewableQueuedPosts even with user delete reviewable actions (#22501)
Performing a `Delete User`/`Delete and Block User` reviewable actions for a
queued post reviewable from the `review.show` route results in an error
popup even if the action completes successfully.

This happens because unlike other reviewable types, a user delete action
on a queued post reviewable results in the deletion of the reviewable
itself. A subsequent attempt to reload the reviewable record results in
404. The deletion happens as part of the call to `UserDestroyer` which
includes a step for destroying reviewables created by the user being
destroyed. At the root of this is the creator of the queued post
being set as the creator of the reviewable as instead of the system
user.

This change assigns the creator of the reviewable to the system user and
uses the more approapriate `target_created_by` column for the creator of the
post being queued.
2023-07-18 11:50:31 +00:00
Krzysztof Kotlarek
bdecd697b9
FIX: more performance improvement for PostAlert job (#22487)
Simplified query based on SiteSettings to join only relevant user_options rows.
In addition, index was added to 'watched_precedence_over_muted` column in `user_options` table to speed up query
2023-07-13 09:02:23 +10:00
Ted Johansson
9915236e42
FEATURE: Warn about outdated translation overrides in admin dashboard (#22384)
This PR adds a feature to help admins stay up-to-date with their translations. We already have protections preventing admins from problems when they update their overrides. This change adds some protection in the other direction (where translations change in core due to an upgrade) by creating a notice for admins when defaults have changed.

Terms:

- In the case where Discourse core changes the default translation, the translation override is considered "outdated".
- In the case above where interpolation keys were changed from the ones the override is using, it is considered "invalid".
- If none of the above applies, the override is considered "up to date".

How does it work?

There are a few pieces that makes this work:

- When an admin creates or updates a translation override, we store the original translation at the time of write. (This is used to detect changes later on.)
- There is a background job that runs once every day and checks for outdated and invalid overrides, and marks them as such.
- When there are any outdated or invalid overrides, a notice is shown in admin dashboard with a link to the text customization page.

Known limitations

The link from the dashboard links to the default locale text customization page. Given there might be invalid overrides in multiple languages, I'm not sure what we could do here. Consideration for future improvement.
2023-07-10 10:06:40 +08:00
Guhyoun Nam
b70bd4366b
FEATURE: Separated 'trusted users can edit others' setting for trust level 3 & 4 (#21493) 2023-07-07 10:48:14 -05:00
Krzysztof Kotlarek
134dcdd63a
FEATURE: allow user to override watched_precedence_over_muted setting (#22340)
Recently, site setting watched_precedence_over_muted was introduced - https://github.com/discourse/discourse/pull/22252

In this PR, we are allowing users to override it. The option is only displayed when the user has watched categories and muted tags, or vice versa.
2023-07-04 15:08:29 +10:00
Alan Guo Xiang Tan
aef7c2fe8f
UX: Use modals to edit categories and tags that appear in sidebar (#22295)
Why this change?

We are currently not fully satisfied with the current way to edit the
categories and tags that appears in the sidebar where the user is
redirected to the tracking preferences tab in the user's profile causing
the user to lose context of the current page. In addition, the dropdown
to select categories or tags limits the amount of information we can
display.

Since editing or adding a custom categories section is already using a
modal, we have decided to switch editing the categories and tags that
appear in the sidebar to use a modal as well.

This commit removes the `new_edit_sidebar_categories_tags_interface_groups` site setting and
make the modals the default for all users.
2023-06-28 07:20:31 +08:00
Jarek Radosz
1027a8a2d7
DEV: Move migration to the correct directory (#22309)
db/migrate/db/migrate ain't it :P
2023-06-27 23:01:39 +02:00
Roman Rizzi
f4e7a80600
DEV: Cache summarization strategy results. (#22230)
Updates the interface for implementing summarization strategies and adds a cache layer to summarize topics once.

The cache stores the final summary and each chunk used to build it, which will be useful when we have to extend or rebuild it.
2023-06-27 11:44:34 -03:00
Osama Sayegh
b27e12445d
FEATURE: Split navigation preference for count and behavior of sidebar links (#22203)
This PR splits up the preference that controls the count vs dot and destination of sidebar links, which is really hard to understand, into 2 simpler checkboxes:

The new preferences/checkboxes are off by default, but there are database migrations to switch the old preference to the new ones so that existing users don't have to update their preferences to keep their preferred behavior of sidebar links when this changed is rolled out.

Internal topic: t/103529.
2023-06-22 19:04:13 +03:00
Krzysztof Kotlarek
959c50001d
FIX: rename everything link to topics (#22076)
Rename everything link in community sidebar section to topics, which is
a bit more descriptive.
2023-06-15 11:36:38 +10:00
Krzysztof Kotlarek
2effcaa0f9
FIX: Update sidebar to be navigation menu (#22101)
Communities can use sidebar or header dropdown, therefore navigation menu is a better name settings in 2 places:

- Old user sidebar preferences;
- Site setting about default tags and categories.
2023-06-15 09:31:28 +10:00
Martin Brennan
44446afe58
FEATURE: Use new hashtag autocomplete system on all sites (#21788)
Followup to:

* 7c97548159
* 0b3cf83e3c
* eae47d82e2

This commit enables the no-longer-experimental hashtag autocomplete changes
for all sites. It is now at a point where it is ready to be promoted
to the only way, and following this commit all old hashtag code
will be deleted

c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866
2023-06-07 11:11:39 +10:00
Martin Brennan
54db01d156
DEV: Delete old personal message settings (#21381)
Followup to e62e93f

Both enable_personal_messages and min_trust_to_send_messages
have been deprecated for a long time now, they can be deleted.
2023-05-23 09:58:58 +02:00
Bianca Nenciu
899eb96798
FEATURE: Enable user tips by default (#21341) 2023-05-08 20:33:08 +03:00
Krzysztof Kotlarek
709fa24558
DEV: move sidebar community section to database (#21166)
* DEV: move sidebar community section to database

Before, community section was hard-coded. In the future, we are planning to allow admins to edit it. Therefore, it has to be moved to database to `custom_sections` table.

Few steps and simplifications has to be made:
- custom section was hidden behind `enable_custom_sidebar_sections` feature flag. It has to be deleted so all forums, see community section;
- migration to add `section_type` column to sidebar section to show it is a special type;
- migration to add `segment` column to sidebar links to determine if link should be displayed in primary section or in more section;
- simplify more section to have one level only (secondary section links are merged);
- ensure that links like `everything` are correctly tracking state;
- make user an anonymous links position consistence. For example, from now on `faq` link for user and anonymous is visible in more tab;
- delete old community-section template.
2023-05-04 12:14:09 +10:00
Krzysztof Kotlarek
a8e28060d1
FIX: rename notify_about_flags_after to notify_about_reviewable_item_after (#21320)
Change name and description for SiteSetting to make it easier to understand.
2023-05-02 08:08:22 +10:00
Arpit Jalan
8405ae7733
FEATURE: add a setting to allowlist DiscourseConnect return path domains (#21110)
* FEATURE: add a setting to allowlist DiscourseConnect return path domains

This commit adds a site setting to allowlist DiscourseConnect return
path domains. The setting needs supports exact domain or wildcard
character (*) to allow for any domain as return path.

* Add more specs to clarify what is allowed in site setting

* Update setting description to explain what is allowed
2023-04-17 22:53:50 +05:30
David Taylor
9238767f7e
FEATURE: Persist password hashing algorithm/params in database (#20980)
Previously, Discourse's password hashing was hard-coded to a specific algorithm and parameters. Any changes to the algorithm or parameters would essentially invalidate all existing user passwords.

This commit introduces a new `password_algorithm` column on the `users` table. This persists the algorithm/parameters which were use to generate the hash for a given user. All existing rows in the users table are assumed to be using Discourse's current algorithm/parameters. With this data stored per-user in the database, we'll be able to keep existing passwords working while adjusting the algorithm/parameters for newly hashed passwords.

Passwords which were hashed with an old algorithm will be automatically re-hashed with the new algorithm when the user next logs in.

Values in the `password_algorithm` column are based on the PHC string format (https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md). Discourse's existing algorithm is described by the string `$pbkdf2-sha256$i=64000,l=32$`

To introduce a new algorithm and start using it, make sure it's implemented in the `PasswordHasher` library, then update `User::TARGET_PASSWORD_ALGORITHM`.
2023-04-11 10:16:28 +01:00
Krzysztof Kotlarek
b72282123b
FIX: public sidebar sections belong to system user (#20972)
Before, public sidebar sections were belonging to admin. However, a better choice is system user.
2023-04-05 10:52:18 +10:00
Martin Brennan
6b9dd22ba7
DEV: Use Migration::Helpers.existing_site? for other migrations (#20949)
Followup to f3402be262, adds
the new helper to two other core migrations that need it.
2023-04-05 07:28:56 +08:00
David Taylor
2386ad12f2
Update default ga_version to v4 and add warning message for v3 (#20936)
Sites which are already using ga3 will stay on that version, and will be shown a warning in the admin panel until they update.

https://meta.discourse.org/t/upgrade-to-google-analytics-4-before-july-2023/260498
2023-04-04 13:14:20 +01:00
Alan Guo Xiang Tan
e54359c135
DEV: Drop plugin related index from core (#20930)
The migration of the index has since been moved into the
discourse-akismet plugin where it should have always been.
2023-04-04 07:31:09 +08:00
David Taylor
f3402be262
DEV: Introduce Migration::Helpers for new-site detection (#20934)
We use schema_migration_details to determine the age of a site in multiple migrations. This commit moves the logic into a dedicated `Migration::Helpers` module so that it doesn't need to be re-implemented every time.
2023-04-03 11:46:39 +01:00
Sam
7038540af6
PERF: ensure sidebar section link index is correctly ordered (#20854)
We perform lookups on sidebar section links based on sidebar_section_id
totally ignoring user. This ensures we have an index to work with.

This removes the previous index `links_user_id_section_id_position` which
partially doubled up `idx_unique_sidebar_section_links`
2023-03-28 15:13:44 +11:00
Krzysztof Kotlarek
db74e9484b
FEATURE: ability to reorder links in custom sidebar sections (#20626)
Drag and drop to reorder custom sidebar sections
2023-03-21 12:23:28 +11:00
Daniel Waterworth
84f590ab83
DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
Ted Johansson
87ec058b8b
FEATURE: Configurable auto-bump cooldown (#20507)
Currently the auto-bump cooldown is hard-coded to 24 hours.

This change makes the highlighted 24 hours part configurable (defaulting to 24 hours), and the rest of the process remains the same.

This uses the new CategorySetting model associated with Category. We decided to add this because we want to move away from custom fields due to the lack of type casting and validations, but we want to keep the loading of these optional as they are not needed for almost all of the flows.

Category settings will be back-filled to all categories as part of this change, and creating a new category will now also create a category setting.
2023-03-10 13:45:01 +08:00
Gerhard Schlager
12436d054d
DEV: Remove badge_granted_title column from user_profiles (#20476)
That column is obsolete since we added the `granted_title_badge_id` column in 2019 (56d3e29a69). Having both columns can lead to inconsistencies (mostly due to old data from before 2019).

For example, `BadgeGranter.revoke_ungranted_titles!` doesn't work correctly if `badge_granted_title` is `false` while `granted_title_badge_id` points to the badge that is used as title.
2023-03-08 13:37:20 +01:00
Krzysztof Kotlarek
a16ea24461
FEATURE: allow external links in custom sidebar sections (#20503)
Originally, only Discourse site links were available. After feedback, it was decided to extend this feature to external URLs.

/t/93491
2023-03-07 11:47:18 +11:00
Natalie Tay
4855a2879c
UX: Update badge icons (#20462) 2023-02-28 10:42:05 +08:00
Keegan George
6108eee31d
DEV: Apply form template to categories (#20337) 2023-02-23 11:18:14 -08:00
Krzysztof Kotlarek
b9d037770c
DEV: configurable public sidebar sections (#20303)
Extension of https://github.com/discourse/discourse/pull/20057

Admin can create a public session visible to everyone. An additional checkbox is displayed for staff members.
2023-02-22 08:55:44 +11:00
Daniel Waterworth
b96a23978a
PERF: Add index to speed up ReindexSearch job (#20151) 2023-02-21 13:16:43 -05:00
Loïc Guitaut
f7c57fbc19 DEV: Enable unless cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Alan Guo Xiang Tan
359dc1c532
UX: Release new user profile navigation for sidebar compatibility (#20134)
With the introduction of the sidebar navigation menu, the design team at
Discourse redesigned the user profile navigation to better coexist with
the sidebar.
2023-02-21 10:16:16 +08:00
Martin Brennan
e502175f62
FIX: Category settings migration failing on '' integer (#20261)
Fixes migration introduced in a90ad52dff,
some category custom fields like `num_auto_bump_daily` which should be
an integer are actually empty string ''.
2023-02-13 18:07:46 +10:00
Ted Johansson
25a226279a
DEV: Replace #pluck_first freedom patch with AR #pick in core (#19893)
The #pluck_first freedom patch, first introduced by @danielwaterworth has served us well, and is used widely throughout both core and plugins. It seems to have been a common enough use case that Rails 6 introduced it's own method #pick with the exact same implementation. This allows us to retire the freedom patch and switch over to the built-in ActiveRecord method.

There is no replacement for #pluck_first!, but a quick search shows we are using this in a very limited capacity, and in some cases incorrectly (by assuming a nil return rather than an exception), which can quite easily be replaced with #pick plus some extra handling.
2023-02-13 12:39:45 +08:00
Ted Johansson
a90ad52dff
DEV: Add dedicated category settings model - Part 1 (#20211)
This is the first in a multi-part change to move the custom fields to a new table. It includes:

- Adding a new CategorySetting model and corresponding table.
- Populating it with data from the category_custom_fields table.
2023-02-13 12:37:59 +08:00
Krzysztof Kotlarek
cbd021db15
FIX: add index to sidebar_section_link (#20234)
Index on linkable_type and linkable_id should increase performance of this subquery https://github.com/discourse/discourse/blob/main/app/services/sidebar_site_settings_backfiller.rb#L86

Also, distinct is removing duplicates which are unnecessary.
2023-02-10 11:14:22 +11:00
Keegan George
871607a420
DEV: Create form templates (#20189) 2023-02-08 11:21:39 -08:00
Krzysztof Kotlarek
7332873051
FIX: hide add section button (#20208)
Custom section feature is currently hidden behind feature flag - https://github.com/discourse/discourse/blob/main/app/assets/javascripts/discourse/app/components/sidebar/user/sections.hbs#L3

In last PR, add section button was moved to footer. It should be hidden as well.
2023-02-08 13:08:05 +11:00
Krzysztof Kotlarek
6e1f3e0023
FIX: improvements for user custom sections (#20190)
Improvements for this PR: https://github.com/discourse/discourse/pull/20057

What was fixed:
- [x] Use ember transitions instead of full reload
- [x] Link was inaccurately kept active
- [x] "+ save" renamed to just "save"
- [x] Render emojis in link name
- [x] UI to set icon
- [x] Delete link is trash icon instead of "x"
- [x] Add another link to on the left and rewording
- [x] Raname "link name" -> "name", "points to" ->  link
- [x] Add limits to fields
- [x] Move add section button to the bottom
2023-02-08 11:45:34 +11:00
Krzysztof Kotlarek
84a87a703c
DEV: configurable custom sidebar sections (#20057)
Allows users to configure their own custom sidebar sections with links withing Discourse instance. Links can be passed as relative path, for example "/tags" or full URL.

Only path is saved in DB, so when Discourse domain is changed, links will be still valid.

Feature is hidden behind SiteSetting.enable_custom_sidebar_sections. This hidden setting determines the group which members have access to this new feature.
2023-02-03 14:44:40 +11:00
Rafael dos Santos Silva
14cf8eacf1
FEATURE: Use similarity in user search (#20112)
Currently, when doing `@mention` for users we have 0 tolerance for typos and misspellings.

With this patch, if a user search doesn't return enough results we go and use `pg_trgm` features to try and find more matches based on trigrams of usernames and names.

It also introduces GiST indexes on those fields in order to improve performance of this search, going from 130ms down to 15ms in my tests.

This is all gated in a feature flag and can be enabled by running  `SiteSetting.user_search_similar_results = true` in the rails console.
2023-02-02 13:35:04 -03:00
Bianca Nenciu
23a74ecf8f
FIX: Truncate existing user status to 100 chars (#20044)
This commits adds a database migration to limit the user status to 100
characters, limits the user status in the UI and makes sure that the
emoji is valid.

Follow up to commit b6f75e231c.
2023-01-30 10:49:08 +02:00
Sam
2c8dfc3dbc
FEATURE: rate limit anon searches per second (#19708) 2023-01-27 10:05:27 -08:00
Bianca Nenciu
c186a46910
SECURITY: Prevent XSS in local oneboxes (#20008)
Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
2023-01-25 19:17:21 +02:00
Natalie Tay
f91ac52a22
SECURITY: Limit the length of drafts (#19989)
Co-authored-by: Loïc Guitaut <loic@discourse.org>
2023-01-25 13:50:21 +02:00
Loïc Guitaut
ec2ed5b7f6 FIX: Delete reviewables associated to posts automatically
Currently we don’t have an association between reviewables and posts.
This sometimes leads to inconsistencies in the DB as a post can have
been deleted but an associated reviewable is still present.

This patch addresses this issue simply by adding a new association to
the `Post` model and by using the `dependent: :destroy` option.
2023-01-25 09:45:36 +01:00
Krzysztof Kotlarek
f409e977a9
FIX: deleted misconfigured embeddable hosts (#19833)
When EmbeddableHost is configured for a specific category and that category is deleted, then EmbeddableHost should be deleted as well.

In addition, migration was added to fix existing data.
2023-01-20 13:29:49 +11:00
Alan Guo Xiang Tan
f122f24b35
SECURITY: Default tags to show count of topics in unrestricted categories (#19916)
Currently, `Tag#topic_count` is a count of all regular topics regardless of whether the topic is in a read restricted category or not. As a result, any users can technically poll a sensitive tag to determine if a new topic is created in a category which the user has not excess to. We classify this as a minor leak in sensitive information.

The following changes are introduced in this commit:

1. Introduce `Tag#public_topic_count` which only count topics which have been tagged with a given tag in public categories.
2. Rename `Tag#topic_count` to `Tag#staff_topic_count` which counts the same way as `Tag#topic_count`. In other words, it counts all topics tagged with a given tag regardless of the category the topic is in. The rename is also done so that we indicate that this column contains sensitive information. 
3. Change all previous spots which relied on `Topic#topic_count` to rely on `Tag.topic_column_count(guardian)` which will return the right "topic count" column to use based on the current scope. 
4. Introduce `SiteSetting.include_secure_categories_in_tag_counts` site setting to allow site administrators to always display the tag topics count using `Tag#staff_topic_count` instead.
2023-01-20 09:50:24 +08:00
Selase Krakani
cc39effe0e
FIX: Switch email domain site settings type to host_list (#19922)
Specifying wildcard characters which also happen to be regex
meta characters for `auto_approve_email_domains`, `allowed_email_domains`
and `blocked_email_domains` site settings currently breaks email
validation.

This change prevents these characters from being specified for these
site settings. It does this by switching the site setting type
from `list` to `host_list`. The `host_list` validator checks for these
characters.

In addition, this change also improves the site setting descriptions and
introduces a migration to  fix existing records.
2023-01-19 16:07:59 +00:00
Martin Brennan
7c97548159
FIX: Fix incorrect hashtag setting migration (#19857)
Added in c2013865d7,
this migration was supposed to only turn off the hashtag
setting for existing sites (since that was the old default)
but its doing it for new ones too because we run all migrations
on new sites.

Instead, we should only run this if the first migration was
only just created, meaning its a new site.
2023-01-16 10:53:00 +10:00
Alan Guo Xiang Tan
8ee71d439b
FIX: Add migration to reindex invalid indexes (#19858)
In Discourse, there are many migration files where we CREATE INDEX CONCURRENTLY which requires us to set disable_ddl_transaction!. Setting disable_ddl_transaction! in a migration file runs the SQL statements outside of a transaction. The implication of this is that there is no ROLLBACK should any of the SQL statements fail.

We have seen lock timeouts occuring when running CREATE INDEX CONCURRENTLY. When that happens, the index would still have been created but marked as invalid by Postgres.

Per the postgres documentation:

> If a problem arises while scanning the table, such as a deadlock or a uniqueness violation in a unique index, the CREATE INDEX command will fail but leave behind an “invalid” index. This index will be ignored for querying purposes because it might be incomplete; however it will still consume update overhead.
> The recommended recovery method in such cases is to drop the index and try again to perform CREATE INDEX CONCURRENTLY . (Another possibility is to rebuild the index with REINDEX INDEX CONCURRENTLY ).

When such scenarios happen, we are supposed to either drop and create the index again or run a REINDEX operation. However, I noticed today that we have not been doing so in Discourse. Instead, we’ve been incorrectly working around the problem by checking for the index existence before creating the index in order to make the migration idempotent. What this potentially mean is that we might have invalid indexes which are lying around in the database which PG will ignore for querying purposes.

This commits adds a migration which queries for all the
invalid indexes in the `public` namespace and reindexes them.
2023-01-13 11:04:26 +08:00
David Taylor
b0fda61a8e
DEV: Apply syntax_tree formatting to db/* 2023-01-09 11:59:41 +00:00
Martin Brennan
c2013865d7
FEATURE: Make experimental hashtag autocomplete default for new sites (#19681)
This feature is stable enough now to make it the default going forward
for new sites. Existing sites that have not yet set enable_experimental_hashtag_autocomplete
to `true` will have it set to `false` for their site settings, which was the old default.

c.f https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866
2023-01-05 08:44:58 +10:00
Alan Guo Xiang Tan
ab3a032b4b
SECURITY: BCC active user emails from group SMTP (#19725)
When sending emails out via group SMTP, if we
are sending them to non-staged users we want
to mask those emails with BCC, just so we don't
expose them to anyone we shouldn't. Staged users
are ones that have likely only interacted with
support via email, and will likely include other
people who were CC'd on the original email to the
group.

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-01-05 06:07:50 +08:00
Bianca Nenciu
b80765f1f4
DEV: Remove enable_whispers site setting (#19196)
* DEV: Remove enable_whispers site setting

Whispers are enabled as long as there is at least one group allowed to
whisper, see whispers_allowed_groups site setting.

* DEV: Always enable whispers for admins if at least one group is allowed.
2022-12-16 18:42:51 +02:00
Bianca Nenciu
c358151a6c
DEV: Promote historic post_deploy migrations (#19492)
This commit promotes all post_deploy migrations which existed in
Discourse v2.8.0 (timestamp <= 20220107014925).

This commit includes a fix to the promote_migrations script to promote
all migrations of the first version of the previous stable version. For
example, if the current stable version is v2.8.13, the version used as
a cutoff for promoting migrations is v2.8.0.
2022-12-16 13:36:30 +02:00
Blake Erickson
5c925f2db3
FEATURE: Chat and Sidebar are now on by default (#19406)
FEATURE: Chat and Sidebar are now on by default

- Set the sidebar site setting to be enabled by default
- Set the chat site setting to be enabled by default
- Updated existing specs that assumed the original default
- Use a migration to keep old defaults for existing sites
2022-12-13 17:25:19 -07:00
Martin Brennan
4001e6f174
DEV: Delete tmp table from migration for invites (#19431)
Follow up a414520742
2022-12-13 10:17:25 +10:00
Gerhard Schlager
3eafa39aeb REFACTOR: Remove updated strings from locale files and translation overrides 2022-12-12 17:01:04 +01:00
Alan Guo Xiang Tan
fde9e6bc25
DEV: Migrate sidebar site settings (#19336)
This new site setting replaces the
`enable_experimental_sidebar_hamburger` and `enable_sidebar` site
settings as the sidebar feature exits the experimental phase.

Note that we're replacing this without depreciation since the previous
site setting was considered experimental.

Internal Ref: /t/86563
2022-12-08 09:44:29 +08:00
Bianca Nenciu
ac272c041e
FEATURE: Add user tips for post and topic features (#18964)
* DEV: Add utility to hide all user tips

* DEV: Add UserTip Glimmer component

* DEV: Add tests for existing user tips

* FEATURE: Add user tip for post menu

* FEATURE: Add user tip for topic notification level

* FEATURE: Add user tip for suggested topics

* FEATURE: Hide new popups for existing users
2022-11-15 17:36:08 +02:00
Blake Erickson
7be53b1588
FEATURE: Default Composer Category Site Setting (#18967)
* FEATURE: Default Composer Category Site Setting

- Create the default_composer_category site setting
- Replace general_category_id logic for auto selecting the composer
  category
- Prevent Uncategorized from being selected if not allowed
- Add default_composer_category option to seeded categories
- Create a migration to populate the default_composer_category site
  setting if there is a general_category_id populated
- Added some tests

* Add missing translation for the new site setting

* fix some js tests

* Just check that the header value is null
2022-11-14 11:09:57 -07:00
Martin Brennan
a414520742
SECURITY: Prevent email from being nil in InviteRedeemer (#19004)
This commit adds some protections in InviteRedeemer to ensure that email
can never be nil, which could cause issues with inviting the invited
person to private topics since there was an incorrect inner join.

If the email is nil and the invite is scoped to an email, we just use
that invite.email unconditionally.  If a redeeming_user (an existing
user) is passed in when redeeming an email, we use their email to
override the passed in email.  Otherwise we just use the passed in
email.  We now raise an error after all this if the email is still nil.
This commit also adds some tests to catch the private topic fix, and
some general improvements and comments around the invite code.

This commit also includes a migration to delete TopicAllowedUser records
for users who were mistakenly added to topics as part of the invite
redemption process.
2022-11-14 12:02:06 +10:00
Bianca Nenciu
4dad7816b2
FEATURE: Rename onboarding popups to user tips (#18826)
This commit also hides the new user tips for existing users.
2022-11-09 20:20:34 +02:00
Martin Brennan
f8f55cef67
DEV: TODO followups (#18936)
* Remove old bookmark column ignores to follow up b22450c7a8
* Change some group site setting checks to use the _map helper
* Remove old secure_media helper stub for chat
* Change attr_accessor to attr_reader for preloaded_custom_fields to follow up 70af45055a
2022-11-09 07:48:05 +10:00
David Taylor
07ef1a80a1
SECURITY: Fix invite link email validation (#18817)
See https://github.com/discourse/discourse/security/advisories/GHSA-x8w7-rwmr-w278

Co-authored-by: Martin Brennan <martin@discourse.org>
2022-11-01 16:33:32 +00:00
Alan Guo Xiang Tan
305b7c8fae
DEV: Update rubocop (#18754) 2022-10-26 09:05:15 +08:00
David Taylor
be3d6a56ce
DEV: Introduce minification and source maps for Theme JS (#18646)
Theme javascript is now minified using Terser, just like our core/plugin JS bundles. This reduces the amount of data sent over the network.

This commit also introduces sourcemaps for theme JS. Browser developer tools will now be able show each source file separately when browsing, and also in backtraces.

For theme test JS, the sourcemap is inlined for simplicity. Network load is not a concern for tests.
2022-10-18 18:20:10 +01:00
Krzysztof Kotlarek
243efa8931
FEATURE: allow user to set preferred sidebar list destination (#18594)
User can choose between latest or new/unread and that preference will affect behavior of sidebar links.
2022-10-18 13:21:52 +11:00
Blake Erickson
570bb68579
FIX: Reset general_category_id if the general category was deleted (#18634)
On sites that were seeded with a general category id and that category
was deleted prior to the fix in efb116d2bd
this migration will reset the SiteSetting.general_category_id back to
the default because there is no longer a corresponding category for it.

This is to fix a composer bug that occurs if there is a
SiteSetting.general_category_id value present that doesn't match an
existing category.
2022-10-17 18:07:53 -06:00
Bianca Nenciu
7611fec0da
FEATURE: Implement new onboarding popups (#18362)
This commit introduces a new framework for building user tutorials as
popups using the Tippy JS library. Currently, the new framework is used
to replace the old notification spotlight and tips and show a new one
related to the topic timeline.

All popups follow the same structure and have a title, a description and
two buttons for either dismissing just the current tip or all of them
at once.

The state of all seen popups is stored in a user option. Updating
skip_new_user_tips will automatically update the list of seen popups
accordingly.
2022-10-12 18:38:45 +03:00
Jan Cernik
08476f17ff
FEATURE: Add dark mode option for category logos (#18460)
Adds a new upload field for a second dark mode category logo. 
This alternative will be used when the browser is in dark mode (similar to the global site setting for a dark logo).
2022-10-07 11:00:44 -04:00
Blake Erickson
3b86974367
FEATURE: Make General the default category (#18383)
* FEATURE: Make General the default category

* Set general as the default category in the composer model instead

* use semicolon

* Enable allow_uncategorized_topics in create_post spec helper for now

* Check if general_category_id is set

* Enable allow_uncategorized_topics for test env

* Provide an option to the create_post helper to not set allow_uncategorized_topics

* Add tests to check that category… is not present and that General is selected automatically
2022-09-30 12:20:21 -06:00
Martin Brennan
87a6cab772
FIX: Migration typo for secure_uploads (#18408)
Fixes typo from 8ebd5edd1e causing
deploy issues.
2022-09-29 11:30:14 +10:00
Martin Brennan
8ebd5edd1e
DEV: Rename secure_media to secure_uploads (#18376)
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.

This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.

Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.

This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.

Deprecated settings:

* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 09:24:33 +10:00
Martin Brennan
e62e93f83a
FEATURE: Introduce personal_message_enabled_groups setting (#18042)
This will replace `enable_personal_messages` and
`min_trust_to_send_messages`, this commit introduces
the setting `personal_message_enabled_groups`
and uses it in all places that `enable_personal_messages`
and `min_trust_to_send_messages` currently apply.

A migration is included to set `personal_message_enabled_groups`
based on the following rules:

* If `enable_personal_messages` was false, then set
  `personal_message_enabled_groups` to `3`, which is
  the staff auto group
* If `min_trust_to_send_messages` is not default (1)
  and the above condition is false, then set the
  `personal_message_enabled_groups` setting to
  the appropriate auto group based on the trust level
* Otherwise just set `personal_message_enabled_groups` to
  11 which is the TL1 auto group

After follow-up PRs to plugins using these old settings, we will be
able to drop the old settings from core, in the meantime I've added
 DEPRECATED notices to their descriptions and added them
to the deprecated site settings list.

This commit also introduces a `_map` shortcut method definition
for all `group_list` site settings, e.g. `SiteSetting.personal_message_enabled_groups`
also has `SiteSetting.personal_message_enabled_groups_map` available,
which automatically splits the setting by `|` and converts it into
an array of integers.
2022-09-26 13:58:40 +10:00
Martin Brennan
e3d495850d
FEATURE: Overhaul email threading (#17996)
See https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499
for thorough reasoning.

This commit changes how we generate Message-IDs and do email
threading for emails sent from Discourse. The main changes are
as follows:

* Introduce an outbound_message_id column on Post that
  is either a) filled with a Discourse-generated Message-ID
  the first time that post is used for an outbound email
  or b) filled with an original Message-ID from an external
  mail client or service if the post was created from an
  incoming email.
* Change Discourse-generated Message-IDs to be more consistent
  and static, in the format `discourse/post/:post_id@:host`
* Do not send References or In-Reply-To headers for emails sent
  for the OP of topics.
* Make sure that In-Reply-To is filled with either a) the OP's
  Message-ID if the post is not a direct reply or b) the parent
  post's Message-ID
* Make sure that In-Reply-To has all referenced post's Message-IDs
* Make sure that References is filled with a chain of Message-IDs
  from the OP down to the parent post of the new post.

We also are keeping X-Discourse-Post-Id and X-Discourse-Topic-Id,
headers that we previously removed, for easier visual debugging
of outbound emails.

Finally, we backfill the `outbound_message_id` for posts that have
a linked `IncomingEmail` record, using the `message_id` of that record.
We do not need to do that for posts that don't have an incoming email
since they are backfilled at runtime if `outbound_message_id` is missing.
2022-09-26 09:14:24 +10:00
Daniel Waterworth
e3d7253199
DEV: Only lock rows that need to change in migration (#18319) 2022-09-21 14:14:45 -05:00
Gerhard Schlager
b3831362a1
FIX: DB migration fails if old site setting has empty value (#18296) 2022-09-21 14:06:14 +02:00
David Taylor
0f5a8cc526
DEV: Enforce dominant_color length in validation (#18309)
The `add_column` `limit` parameter has no effect on a postgres `text` column. Instead we can perform the check in ActiveRecord.

We never expect this condition to be hit - users cannot control this value. It's just a safety net.
2022-09-21 11:01:21 +01:00
Martin Brennan
e69f7d2fd9
SECURITY: Limit user profile field length (#18302)
Adds limits to location and website fields at model and DB level
to match the bio_raw field limits. A limit cannot be added at the
DB level for bio_raw because it is a postgres text field.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-09-21 12:07:06 +10:00
David Taylor
d0243f741e
UX: Use dominant color as image loading placeholder (#18248)
We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images.

This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly.

Dominant color will be calculated:
1. When a new upload is created
2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored
3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job)

Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 10:28:17 +01:00
Bianca Nenciu
0d8ecab362
FIX: Restore trust level when leaving group (#17954)
If a user was granted a trust level, joined a group that granted a trust
level and left the group, the trust level was reset. This commit tries
to restore the last known trust level before joining the group by
looking into staff logs.

This commit also migrates old :change_trust_level user history records
to use previous_value and new_value fields.
2022-08-29 13:00:48 +03:00
Bianca Nenciu
4b70594173
FIX: Reset flair group if user is removed from group (#17862)
The flair used to stay set even if the user was removed from the group.
2022-08-12 15:45:09 +03:00
Selase Krakani
862007fb18
FEATURE: Add support for case-sensitive Watched Words (#17445)
* FEATURE: Add case-sensitivity flag to watched_words

Currently, all watched words are matched case-insensitively. This flag
allows a watched word to be flagged for case-sensitive matching.
To allow allow for backwards compatibility the flag is set to false by
default.

* FEATURE: Support case-sensitive creation of Watched Words via API

Extend admin creation and upload of Watched Words to support case
sensitive flag. This lays the ground work for supporting
case-insensitive matching of Watched Words.

Support for an extra column has also been introduced for the Watched
Words upload CSV file. The new column structure is as follows:

 word,replacement,case_sentive

* FEATURE: Enable case-sensitive matching of Watched Words

WordWatcher's word_matcher_regexp now returns a list of regular
expressions instead of one case-insensitive regular expression.

With the ability to flag a Watched Word as case-sensitive, an action
can have words of both sensitivities.This makes the use of the global
Regexp::IGNORECASE flag added to all words problematic.

To get around platform limitations around the use of subexpression level
switches/flags, a list of regular expressions is returned instead, one for each
case sensitivity.

Word matching has also been updated to use this list of regular expressions
instead of one.

* FEATURE: Use case-sensitive regular expressions for Watched Words

Update Watched Words regular expressions matching and processing to handle
the extra metadata which comes along with the introduction of
case-sensitive Watched Words.

This allows case-sensitive Watched Words to matched as such.

* DEV: Simplify type casting of case-sensitive flag from uploads

Use builtin semantics instead of a custom method for converting
string case flags in uploaded Watched Words to boolean.

* UX: Add case-sensitivity details to Admin Watched Words UI

Update Watched Word form to include a toggle for case-sensitivity.
This also adds support for, case-sensitive testing and matching of  Watched Word
in the admin UI.

* DEV: Code improvements from review feedback

 - Extract watched word regex creation out to a utility function
 - Make JS array presence check more explicit and readable

* DEV: Extract Watched Word regex creation to utility function

Clean-up work from review feedback. Reduce code duplication.

* DEV: Rename word_matcher_regexp to word_matcher_regexp_list

Since a list is returned now instead of a single regular expression,
change `word_matcher_regexp` to `word_matcher_regexp_list` to better communicate
this change.

* DEV:  Incorporate WordWatcher updates from upstream

Resolve conflicts and ensure apply_to_text does not remove non-word characters in matches
that aren't at the beginning of the line.
2022-08-02 10:06:03 +02:00
Osama Sayegh
988a175e94
DEV: Add reviewables tab to the new user menu (#17630)
This commit is a subset of the changes proposed in https://github.com/discourse/discourse/pull/17379.
2022-07-28 11:16:33 +03:00
Alan Guo Xiang Tan
dcf84fce7b
PERF: Add index for TopicTimer#topic_id (#17680)
When viewing a topic, we execute two queries to fetch the topic's
public topic timer and slow mode timer. The former query happens to be
able to use a unique index but the latter has to do a seq scan which is
slow. The query itself is not expensive but since viewing a topic is a
hot path, the little cuts add up overtime and the query itself
contributes significantly to the load of the database.
2022-07-27 16:21:11 +08:00
Blake Erickson
8b08b9a763
FIX: Rejected emails should not be cleaned up before their logs (#17648)
* FIX: Rejected emails should not be cleaned up before their logs

If we delete the rejected emails before we delete their associated logs
we will receive 404 errors trying to inspect an email message for that
log.

* don't add a blank line

* test for max value as well

* pr cleanup and add migration

* Fix failing test
2022-07-27 07:28:44 +01:00
Jordan Vidrine
fec3df60cb
DEV: Add more color schemes (#17398) 2022-07-08 16:46:32 -05:00
Alan Guo Xiang Tan
3266350e80
FEATURE: Decouple category/tag presence in sidebar from notifi level (#17273) 2022-06-30 14:54:20 +08:00
Andrei Prigorshnev
033f72c65f
FEATURE: user status emoji (#17025) 2022-06-22 18:15:33 +04:00
Vinoth Kannan
deee3c6f02
DEV: drop unused column flair_url from groups table. (#17179)
It's already included in the `ignored_columns` list in the group model. 03ffb0bf27/app/models/group.rb (L9)

Also, removed the `MigrateGroupFlairImages` onceoff job and spec.
2022-06-22 00:15:05 +05:30
Osama Sayegh
5176c689e9
UX: Change wording for 'regular' categories to 'normal' (#17134)
At some point in the past we decided to rename the 'regular' notification state of topics/categories to 'normal'. However, some UI copy was missed when the initial renaming was done so this commit changes the spots that were missed to the new name.
2022-06-20 06:49:33 +03:00
David Taylor
410ab5d587
FIX: Skip invalid values in site setting upload references migration (#17138) 2022-06-19 19:02:55 +01:00
Martin Brennan
4d3c1ceb44
FEATURE: Log the SMTP response in EmailLog (#17056)
When sending emails with delivery_method_options -> return_response
set to true, the SMTP sending code inside Mail will return the SMTP
response when calling deliver! for mail within the app. This commit
ensures that Email::Sender captures this response if it is returned
and stores it against the EmailLog created for the sent email.

A follow up PR will make this visible within the admin email UI.
2022-06-15 10:28:30 +10:00
David Taylor
002d62b847
FIX: Cleanup invalid historic site setting data (#17049)
`selectable_avatars_urls` contains invalid data (it's a backup from 20200810194943_change_selectable_avatars_site_setting.rb)

This migration is deliberately backdated so that it runs before `20220330160747_copy_site_settings_uploads_to_upload_references`
2022-06-09 09:51:29 +01:00
Martin Brennan
20f118c4c9
FIX: Handle empty string in theme_settings for upload_references (#17047)
Follow up to 9db8f00b3d,
the theme_settings.value field is not an integer and so
can be '', we need to account for this in the migration
otherwise we get this error:

> PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type integer: ""
2022-06-09 10:19:44 +02:00
Bianca Nenciu
9db8f00b3d
FEATURE: Create upload_references table (#16146)
This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use.

* DEV: Create upload_references
* DEV: Use UploadReference instead of PostUpload
* DEV: Use UploadReference for SiteSetting
* DEV: Use UploadReference for Badge
* DEV: Use UploadReference for Category
* DEV: Use UploadReference for CustomEmoji
* DEV: Use UploadReference for Group
* DEV: Use UploadReference for ThemeField
* DEV: Use UploadReference for ThemeSetting
* DEV: Use UploadReference for User
* DEV: Use UploadReference for UserAvatar
* DEV: Use UploadReference for UserExport
* DEV: Use UploadReference for UserProfile
* DEV: Add method to extract uploads from raw text
* DEV: Use UploadReference for Draft
* DEV: Use UploadReference for ReviewableQueuedPost
* DEV: Use UploadReference for UserProfile's bio_raw
* DEV: Do not copy user uploads to upload references
* DEV: Copy post uploads again after deploy
* DEV: Use created_at and updated_at from uploads table
* FIX: Check if upload site setting is empty
* DEV: Copy user uploads to upload references
* DEV: Make upload extraction less strict
2022-06-09 09:24:30 +10:00
Andrei Prigorshnev
5c596273a0
FEATURE: user status (#16875) 2022-05-27 13:15:14 +04:00
Martin Brennan
5ebad3be0a
FIX: Further refine duplicate bookmark delete query (#16931)
In f00e282067 we added this
DELETE query to delete duplicate for_topic bookmarks, we
just need this further refinement to the WHERE clause to
avoid deleting post bookmarks.
2022-05-27 08:22:23 +10:00
Daniel Waterworth
f00e282067
FIX: Delete extraneous topic bookmarks (#16926)
Otherwise the migration can fail

Co-authored-by: David Taylor <david@taylorhq.com>
2022-05-26 11:58:15 -05:00
Martin Brennan
fcc2e7ebbf
FEATURE: Promote polymorphic bookmarks to default and migrate (#16729)
This commit migrates all bookmarks to be polymorphic (using the
bookmarkable_id and bookmarkable_type) columns. It also deletes
all the old code guarded behind the use_polymorphic_bookmarks setting
and changes that setting to true for all sites and by default for
the sake of plugins.

No data is deleted in the migrations, the old post_id and for_topic
columns for bookmarks will be dropped later on.
2022-05-23 10:07:15 +10:00
David Taylor
991b62b6f1
DEV: Drop old hotlinked image data from post_custom_fields (#16594)
`20220428094026_create_post_hotlinked_media` moved this data into a dedicated table
2022-05-12 15:34:35 +01:00
David Taylor
d90065e0ef
Revert "DEV: Update add_column migration to remove transaction (#16715)" (#16716)
This reverts commit e599b5b08f.

This didn't solve the problem we were trying to fix - reverting back to the standard migration pattern
2022-05-11 12:36:11 +01:00
David Taylor
e599b5b08f
DEV: Update add_column migration to remove transaction (#16715)
This migration is failing to acquire a lock under some production conditions. We're only performing one action, so removing the transaction is safe and may help to resolve the issue.
2022-05-11 11:28:34 +01:00
Isaac Janzen
1a12e4cfc8
FEATURE: Introduce site setting to allow for non staff pm tagging (#16671)
Currently the only way to allow tagging on pms is to use the `allow_staff_to_tag_pms` site setting.  We are removing that site setting and replacing it with `pm_tags_allowed_for_groups` which will allow for non staff tagging. It will be group based permissions instead of requiring the user to be staff.

If the existing value of `allow_staff_to_tag_pms` is `true` then we include the `staff` groups as a default for `pm_tags_allowed_for_groups`.
2022-05-10 10:02:28 -05:00
Martin Brennan
fbcc35b417
DEV: Remove PostAction/UserAction bookmark refs (#16681)
We have not used anything related to bookmarks for PostAction
or UserAction records since 2020, bookmarks are their own thing
now. Deleting all this is just cleaning up old cruft.
2022-05-10 10:42:18 +10:00
Roman Rizzi
6bdcd7afb2
FEATURE: Promote the "delete group" staff action log. (#16656)
We used to log group deletion as custom, which means we can't later search for them. Include group ID in the details.
2022-05-09 12:12:52 -03:00
Osama Sayegh
7e026e8ddf
DEV: Add last_seen_reviewable_id to the users table (#16666)
We're adding this column now in preparation for a future commit(s) that will
redesign the avatar/notifications menu. The reason the column is added in a
separate commit is because the redesign changes are going to be complex with a
high risk of getting (temporarily) reverted and if they included a database
migration, they wouldn't revert cleanly/easily.

Internal ticket: t65045.
2022-05-06 10:11:16 +03:00
David Taylor
db9ae32e41
FIX: Skip invalid custom_field json in hotlinked_media migration (#16630)
Truly testing for JSON validity would require defining a new postgres function. Checking just the first character should take care of all the cases of invalid historic data that we've seen.
2022-05-04 15:05:18 +01:00
David Taylor
1c9d8b4999
FIX: Handle invalid JSON from downloaded_images custom fields (#16613)
custom_field data on some sites has duplicate values for a given url key in the JSON value. This is invalid, so we should drop the extra data.
2022-05-03 18:51:25 +01:00
David Taylor
bebb4fd111
FIX: Handle duplicates in hotlinked_media migration (#16611)
In the old custom_field-based system, it was possible for a url to be both 'downloaded' and 'broken'. The new table enforces uniqueness, so we need to drop invalid data.
2022-05-03 17:17:27 +01:00
David Taylor
19d2d55011
FIX: Ensure post_hotlinked_media index does not exceed size limit (#16609)
On some installations, this would fail with 'index row size exceeds btree version 4 maximum'. This commit replaces the (post_id, url)` index with a `(post_id, md5(url))` index, which is much more space efficient.
2022-05-03 15:47:58 +01:00
David Taylor
c1db968740
DEV: Move hotlinked image information into a dedicated table (#16585)
This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change.

For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 13:53:32 +01:00
Loïc Guitaut
008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Alan Guo Xiang Tan
98c49acad5
DEV: Setup experimental sidebar skeleton (#16575)
* hidden siteSetting to enable experimental sidebar
* user preference to enable experimental sidebar
* `experimental_sidebar_enabled` attribute for current user
* Empty glimmer component for Sidebar
2022-04-28 15:27:06 +08:00
Martin Brennan
c841e34b62
FIX: Check if bookmarkable column exists before adding (#16497)
Because in 8040b95e8c we removed
a previous post migrate file, some people may not have had those
original polymorphic bookmark columns removed, and the migration
from this PR running will cause duplicate column errors.

cf. https://meta.discourse.org/t/duplicatecolumn-and-multisite-migrate-failed/224480
2022-04-19 12:01:18 +10:00
Roman Rizzi
6f76a12e0a
FEATURE: Let sites add a sitemap.xml file. (#16357)
* FEATURE: Let sites add a sitemap.xml file.

This PR adds the same features discourse-sitemap provides to core. Sitemaps are only added to the robots.txt file if the `enable_sitemap` setting is enabled and `login_required` disabled.

After merging discourse/discourse-sitemap#34, this change will take priority over the sitemap plugin because it will disable itself. We're also using the same sitemaps table, so our migration won't try to create it
again using `if_not_exists: true`.
2022-04-12 10:33:59 -03:00
David Taylor
1e436f242e
DEV: Move tag_group cleanup from post_migrate -> migrate (#16434)
There is no need to wait until after the deploy for this cleanup. In fact, running it later will mean there could be a window of a few minutes during which the site is broken.

The only requirement is that it runs after the broken `20220401130745_create_category_required_tag_groups` migration.

Followup to 39a6de3d73
2022-04-11 13:28:24 +01:00
Roman Rizzi
39a6de3d73
FIX: Clean required category tag groups with invalid tag_group ids (#16414)
The category table's required_tag_group_id contained references to deleted tag groups, which we copied to the new table. The new serializer tries to get the associated tag group name but fails because the tag group is nil.

This PR adds an inner join in the original migration to make sure tag groups still exist and adds a new post-migration to fix already migrated sites.
2022-04-07 17:02:11 -04:00
David Taylor
68c74e9b93
FEATURE: Allow multiple required tag groups for a category (#16381)
Previously we only supported a single 'required tag group' for a category. This commit allows admins to specify multiple required tag groups, each with their own minimum tag count.

A new category_required_tag_groups database table replaces the existing columns on the categories table. Data is automatically migrated.
2022-04-06 14:08:06 +01:00
Alan Guo Xiang Tan
d0c2eb3359
DEV: Drop enable_long_polling and long_polling_interval settings (#16323)
Scheduled to drop in 2.9.

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2022-03-30 16:32:40 +08:00
Martin Brennan
230e82e948
DEV: Make Bookmark#post_id column nullable (#16287)
As we are gradually moving to having a polymorphic
bookmarkable relationship on the Bookmark table,
we need to make the post_id column nullable to be
able to develop and test the new columns, and
for cutover/migration purposes later as well.
2022-03-28 13:09:13 +10:00
David Taylor
f3aab19829
DEV: Promote historic post_deploy migrations (#16288)
This commit promotes all post_deploy migrations which existed in Discourse v2.7.13 (timestamp <= 20210328233843)

This reduces the likelihood of issues relating to migration run order

Also fixes a couple of typos in `script/promote_migrations`
2022-03-25 15:48:20 +00:00
Martin Brennan
8040b95e8c
DEV: Re-add polymorphic bookmark columns (#16261)
This commit is a redo of2f1ddadff7dd47f824070c8a3f633f00a27aacde
which we reverted because it blew up an internal CI check. I looked
into it, and it happened because the old migration to add the bookmark
columns still existed, and those columns were dropped in a post migrate,
so the two migrations to add the columns were conflicting before
the post migrate was run.

------

This commit only includes the creation of the new columns and index,
and does not add any triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

Add an experimental site setting for using polymorphic bookmarks,
and make sure in the places where bookmarks are created or updated
we fill in the columns. This setting will be used in subsequent
PRs as well.
Listing and searching bookmarks based on polymorphic associations
Creating post and topic bookmarks using polymorphic associations,
and changing special for_topic logic to just rely on the Topic
bookmarkable_type
Querying bookmark reminders based on polymorphic associations
Make sure various other areas like importers, bookmark guardian,
and others all rely on the associations
Prepare plugins that rely on the Bookmark model to use polymorphic
associations
The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-23 12:43:08 +10:00
Alan Guo Xiang Tan
4e6f7a1a6e
Revert "DEV: Re-add polymorphic bookmark columns (#16245)" (#16246)
This reverts commit 2f1ddadff7.
2022-03-22 13:13:42 +08:00
Martin Brennan
2f1ddadff7
DEV: Re-add polymorphic bookmark columns (#16245)
This commit is a redo of e21c640a3c
which we reverted to not include half-done work in a release.

This commit is slightly different though, in that it only includes
the creation of the new columns and index, and does not add any
triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

1. Add an experimental site setting for using polymorphic bookmarks,
   and make sure in the places where bookmarks are created or updated
   we fill in the columns. This setting will be used in subsequent
   PRs as well.
2. Listing and searching bookmarks based on polymorphic associations
3. Creating post and topic bookmarks using polymorphic associations,
   and changing special for_topic logic to just rely on the Topic
   bookmarkable_type
4. Querying bookmark reminders based on polymorphic associations
5. Make sure various other areas like importers, bookmark guardian,
   and others all rely on the associations
6. Prepare plugins that rely on the Bookmark model to use polymorphic
   associations

The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-22 14:26:13 +10:00
Bianca Nenciu
30f3e78834
FIX: Reset last sent for existent bookmarks (#16202)
The meaning of reminder_at and reminder_last_sent_at changed after
commit 6d422a8033. A bookmark reminder
will fire only if reminder_last_sent_at is null, but before that it
fired everytime reminder_at was set. This is no longer true because
sometimes reminder_at continues to exist even after a reminder fired.
2022-03-18 16:31:35 +02:00
Bianca Nenciu
6d422a8033
FEATURE: Highlight expired bookmark reminders (#15317)
The user can select what happens with a bookamrk after it expires. New
option allow bookmark's reminder to be kept even after it has expired.
After a bookmark's reminder notification is created, the reminder date
will be highlighted in red until the user resets the reminder date.
User can do that using the new Clear Reminder button from the dropdown.
2022-03-08 19:44:18 +02:00
Bianca Nenciu
34b4b53bac
FEATURE: Use Postgres unaccent to ignore accents (#16100)
The search_ignore_accents site setting can be used to make the search
indexer remove the accents before indexing the content. The unaccent
function from PostgreSQL is better than Ruby's unicode_normalize(:nfkd).
2022-03-07 23:03:10 +02:00
David Taylor
241e34e6cf
FIX: Update and rebake uses of the old centralized avatar service (#16086)
This URL was originally updated in 89cb537fae. However, some sites are not using the proxy, and have configured their forum to hotlink images directly to avatars.discourse.org.

We intend to shut down this domain in favor of `avatars.discourse-cdn.com`, so this migration will re-write any matching site setting values and queue affected posts for rebaking.
2022-03-07 09:45:20 +00:00
Jarek Radosz
2fc70c5572
DEV: Correctly tag heredocs (#16061)
This allows text editors to use correct syntax coloring for the heredoc sections.

Heredoc tag names we use:

languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS
other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 20:50:55 +01:00
Alan Guo Xiang Tan
e613aef419
DEV: Remove use of Discourse application code in migration. (#16068)
This makes the migration less brittle to changes in the future.

Follow-up to 21a356e3af
2022-02-28 10:48:46 +08:00
Jeff Wong
d1bdb6c65d
FEATURE: upload an avatar option for uploading avatars with selectable avatars (#15878)
* FEATURE: upload an avatar option for uploading avatars with selectable avatars

Allow staff or users at or above a trust level to upload avatars even when the site
has selectable avatars enabled.

Everyone can still pick from the list of avatars. The option to upload is shown
below the selectable avatar list.

refactored boolean site setting into an enum with the following values:

disabled: No selectable avatars enabled (default)
everyone: Show selectable avatars, and allow everyone to upload custom avatars
tl1: Show selectable avatars, but require tl1+ and staff to upload custom avatars
tl2: Show selectable avatars, but require tl2+ and staff to upload custom avatars
tl3: Show selectable avatars, but require tl3+ and staff to upload custom avatars
tl4: Show selectable avatars, but require tl4 and staff to upload custom avatars
staff: Show selectable avatars, but only allow staff to upload custom avatars
no_one: Show selectable avatars. No users can upload custom avatars

Co-authored-by: Régis Hanol <regis@hanol.fr>
2022-02-24 12:57:39 -08:00
Joffrey JAFFEUX
f5ec32bc8c
FEATURE: adds the user_promoted event to webhooks (#15996) 2022-02-22 10:57:18 +01:00
Martin Brennan
fa0c796baf
DEV: Fix SMTP bounce regexp (#16019)
Never trust me with regexp. Follow up to
01ef1d08fc,
which did not take into account codes in
the format X.X.XX (with the 2 digits on the end)
2022-02-22 08:54:01 +10:00
Martin Brennan
01ef1d08fc
FIX: Conform EmailLog#bounce_error_code to RFC (#16010)
This commit makes sure that the email log's bounce_error_code
conforms to the SMTP error code RFC on save, so that
it is always in the format X.X.X or XXX without any
additional string details. Also included is a migration
to fix this issue for past records.
2022-02-21 11:26:39 +10:00
Martin Brennan
f9ec2b90a0
DEV: Drop user_stats count column constraints (#15949)
We added this constraint in 5bd55acf83
but it is causing problems in hosted sites and is catching the
issue too far down the line. This commit removes the constraint
for now, and also fixes an issue found with PostDestroyer
which wasn't using the UserStatCountUpdater when updating post_count
and thus was causing negative numbers to occur.
2022-02-16 12:49:11 +11:00
Martin Brennan
4086ee551e
DEV: Add bounce_error_code to EmailLog (#15948)
Whenever we got a bounced email in the Email::Receiver we
previously would just set bounced: true on the EmailLog and
discard the status/diagnostic code. This commit changes this
flow to store the bounce error code (defined in the RFC at
https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml)
not just in the Email::Receiver, but also via webhook events
from other mail services and from SNS.

This commit does not surface the bounce error in the UI,
we can do that later if necessary.
2022-02-15 14:17:26 +10:00
Martin Brennan
ad9e791ba7
FIX: Post migration to fix custom emojis with secure URL (#15944)
Follow up to 88a8584348. Sets
the baked version of all posts with custom emoji and a secure
media URL in the cooked content to 0. Then our periodic rebake
posts job will rebake them to apply the fix in the linked
commit. This only matters on sites with secure media enabled.
2022-02-15 10:37:16 +10:00
Blake Erickson
71f7f7ed49
FEATURE: Add external_id to topics (#15825)
* FEATURE: Add external_id to topics

This commit allows for topics to be created and fetched by an
external_id. These changes are API only for now as there aren't any
front changes.

* add annotations

* add external_id to this spec

* Several PR feedback changes

- Add guardian to find topic
- 403 is returned for not found as well now
- add `include_external_id?`
- external_id is now case insensitive
- added test for posts_controller
- added test for topic creator
- created constant for max length
- check that it redirects to the correct path
- restrain external id in routes file

* remove puts

* fix tests

* only check for external_id in webhook if exists

* Update index to exclude external_id if null

* annotate

* Update app/controllers/topics_controller.rb

We need to check whether the topic is present first before passing it to the guardian.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-02-08 20:55:32 -07:00
Martin Brennan
0a738bd5bc
FEATURE: Allow sending group SMTP emails with from alias (#15687)
This commit allows group SMTP emails to be sent with a
different from email address that has been set up as an
alias in the email provider. Emails from the alias will
be grouped correctly using Message-IDs in the mail client,
and replies to the alias go into the correct group inbox.
2022-02-07 13:52:01 +10:00
Alan Guo Xiang Tan
5bd55acf83
FIX: Add DB constraints for post & topic counter cache for UserStat (#15626)
Ensures that `UserStat#post_count` and `UserStat#topic_count` does not
go below 0. When it does like it did now, we tend to have bugs in our
code since we're usually coding with the assumption that the count isn't
negative.

In order to support the constraints, our post and topic fabricators in
tests will now automatically increment the count for the respective
user's `UserStat` as well. We have to do this because our fabricators
bypasss `PostCreator` which holds the responsibility of updating `UserStat#post_count` and
`UserStat#topic_count`.
2022-02-07 11:23:34 +08:00
Alan Guo Xiang Tan
930f51e175 FEATURE: Split up text segmentation for Chinese and Japanese.
* Chinese segmenetation will continue to rely on cppjieba
* Japanese segmentation will use our port of TinySegmenter
* Korean currently does not rely on segmentation which was dropped in c677877e4f
* SiteSetting.search_tokenize_chinese_japanese_korean has been split
into SiteSetting.search_tokenize_chinese and
SiteSetting.search_tokenize_japanese respectively
2022-02-07 09:21:14 +08:00
Alan Guo Xiang Tan
4aa9a813ec
FIX: Liking whispers should not contribute to Topic#like_count. (#15703)
Non-staff users are not allowed to see whisper so this change prevents
non-staff user from seeing a like count that does not make sense to
them. In the future, we might consider adding another like count column
for staff user.

Follow-up to 4492718864
2022-02-03 16:24:33 +11:00
Andrei Prigorshnev
dad2e5e513
FEATURE: make the use_email_for_username_and_name_suggestions setting visible and on by default on existing sites (#15751) 2022-02-01 14:55:17 +04:00
Martin Brennan
6b6c0fef4c
DEV: Drop bookmark trigger correctly (#15486)
Apparently `DROP FUNCTION X CASCADE` is the better way to
do this, we were running into cross-schema errors on try.
2022-01-07 15:20:07 +10:00
Martin Brennan
04c7776650
DEV: Rolling back bookmarkable column changes (#15482)
It is too close to release of 2.8 for incomplete
feature shenanigans. Ignores and drops the columns and drops
the trigger/function introduced in
e21c640a3c.
Will pick this feature back up post-release.
2022-01-07 12:16:43 +10:00
Martin Brennan
e21c640a3c
DEV: Add polymorphic bookmarkable columns (#15454)
We are planning on attaching bookmarks to more and
more other models, so it makes sense to make a polymorphic
relationship to handle this. This commit adds the new
columns and backfills them in the bookmark table, and
makes sure that any new bookmark changes fill in the columns
via DB triggers.

This way we can gradually change the frontend and backend
to use these new columns, and eventually delete the
old post_id and for_topic columns in `bookmarks`.
2022-01-06 08:56:05 +10:00
Martin Brennan
b22450c7a8
DEV: Drop old bookmark columns (#15405)
As per 22208836c5
these are no longer needed, a follow up commit will remove the ignored
columns.
2022-01-04 11:19:27 +10:00
Osama Sayegh
8e6988163f
FIX: Don't allow NULL values for notification_level in category_users (#15407) 2021-12-29 09:19:39 +11:00
Martin Brennan
a263743268
DEV: Remove old enable_experimental_composer_uploader site setting (#15406)
No longer used, the uppy method is now the default for
composer uploads and the old code is deleted.
2021-12-24 15:27:25 +11:00
Osama Sayegh
0192efb37b
FIX: Remove length constraint of fancy_title in topics (#15381)
The `fancy_title` column in the `topics` table currently has a constraint that limits the column to 400 characters. We need to remove that constraint because it causes some automatic topics/PMs from the system to fail when using Discourse in locales that need more than 400 characters to the translate the content of those automatic messages.

Internal ticket: t58030.
2021-12-21 21:24:23 +03:00
Martin Brennan
d5e380e5c1
DEV: Promote uppy backup uploader to primary uploader (#15363)
This commit removes the enable_experimental_backup_uploader site
setting and the flags in backups-index.hbs to make the uppy
backup uploader the main one from now on.

A follow-up commit will delete the old backup uploader code and
also remove resumable.js from the project.
2021-12-20 13:39:35 +10:00
Penar Musaraj
60851037d8
DEV: Remove unused composer editor preview sync (#15322) 2021-12-16 17:18:54 -05:00
David Taylor
7d72041fff
FIX: Correct email_tokens_token_to_nullable migration (#15277)
We were checking for the existence of the column in any schema, including the `backup` schema. This can cause 'column does not exist' errors. In fact, we should only be checking in the `public` schema.
2021-12-13 16:19:09 +00:00