Commit Graph

6421 Commits

Author SHA1 Message Date
Krzysztof Kotlarek
63a0466548
FIX: improve performance of UserStat.ensure_consistency ()
Optimize `UserStatpost_read_count` calculation.

In addition, tests were updated to fail when code is not evaluated. Creation of PostTiming was updating `post_read_count`. Count it has to be reset to ensure that ensure_consitency correctly calculates result.

Extracting users seen in the last hour to separate Common Table Expression reduces the amount of processed rows.

Before
```
Update on user_stats  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12606.121..12606.127 rows=0 loops=1)
  ->  Hash Join  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12561.814..12603.689 rows=10 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 67
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.014..39.173 rows=75534 loops=1)
        ->  Hash  (cost=267455.80..267455.80 rows=2901 width=48) (actual time=12558.613..12558.617 rows=77 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 39kB
              ->  Subquery Scan on x  (cost=267376.03..267455.80 rows=2901 width=48) (actual time=12168.601..12558.572 rows=77 loops=1)
                    ->  GroupAggregate  (cost=267376.03..267426.79 rows=2901 width=12) (actual time=12168.595..12558.525 rows=77 loops=1)
                          Group Key: pt.user_id
                          ->  Sort  (cost=267376.03..267383.28 rows=2901 width=4) (actual time=12100.490..12352.106 rows=2072830 loops=1)
                                Sort Key: pt.user_id
                                Sort Method: external merge  Disk: 28488kB
                                ->  Nested Loop  (cost=1.28..267209.18 rows=2901 width=4) (actual time=0.040..11528.680 rows=2072830 loops=1)
                                      ->  Nested Loop  (cost=0.86..261390.02 rows=13159 width=8) (actual time=0.030..3492.887 rows=3581648 loops=1)
                                            ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..89.71 rows=28 width=4) (actual time=0.010..0.201 rows=78 loops=1)
                                                  Index Cond: (last_seen_at > '2023-04-11 00:22:49.555537'::timestamp without time zone)
                                            ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..9287.60 rows=4455 width=8) (actual time=0.081..38.542 rows=45919 loops=78)
                                                  Index Cond: (user_id = u.id)
                                      ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=3581648)
                                            Index Cond: (id = pt.topic_id)
                                            Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                            Rows Removed by Filter: 0
Planning Time: 0.692 ms
Execution Time: 12612.587 ms
```
After
```
Update on user_stats  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=677.724..677.729 rows=0 loops=1)
  ->  Hash Join  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=672.536..677.706 rows=1 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 54
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.012..23.977 rows=75534 loops=1)
        ->  Hash  (cost=9438.24..9438.24 rows=2829 width=48) (actual time=647.818..647.822 rows=55 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 37kB
              ->  Subquery Scan on x  (cost=9381.66..9438.24 rows=2829 width=48) (actual time=647.409..647.805 rows=55 loops=1)
                    ->  HashAggregate  (cost=9381.66..9409.95 rows=2829 width=12) (actual time=647.403..647.786 rows=55 loops=1)
                          Group Key: pt.user_id
                          Batches: 1  Memory Usage: 121kB
                          ->  Nested Loop  (cost=1.86..9367.51 rows=2829 width=4) (actual time=0.056..625.245 rows=120022 loops=1)
                                ->  Nested Loop  (cost=1.44..3692.96 rows=12832 width=8) (actual time=0.047..171.754 rows=217440 loops=1)
                                      ->  Nested Loop  (cost=1.00..254.63 rows=25 width=12) (actual time=0.030..1.407 rows=56 loops=1)
                                            Join Filter: (u.id = user_stats_1.user_id)
                                            ->  Nested Loop  (cost=0.71..243.08 rows=25 width=8) (actual time=0.018..1.207 rows=87 loops=1)
                                                  ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..86.71 rows=27 width=4) (actual time=0.009..0.156 rows=87 loops=1)
                                                        Index Cond: (last_seen_at > '2023-04-11 00:47:07.437568'::timestamp without time zone)
                                                  ->  Index Only Scan using user_stats_pkey on user_stats us  (cost=0.29..5.79 rows=1 width=4) (actual time=0.011..0.011 rows=1 loops=87)
                                                        Index Cond: (user_id = u.id)
                                                        Heap Fetches: 87
                                            ->  Index Scan using user_stats_pkey on user_stats user_stats_1  (cost=0.29..0.45 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=87)
                                                  Index Cond: (user_id = us.user_id)
                                                  Filter: (posts_read_count < 10000)
                                                  Rows Removed by Filter: 0
                                      ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..92.98 rows=4455 width=8) (actual time=0.036..2.492 rows=3883 loops=56)
                                            Index Cond: (user_id = user_stats_1.user_id)
                                ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=217440)
                                      Index Cond: (id = pt.topic_id)
                                      Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                      Rows Removed by Filter: 0
Planning Time: 1.406 ms
Execution Time: 677.817 ms
```
2023-04-11 12:28:08 +10:00
Daniel Waterworth
fcc73b441d
FIX: Make emoji cache store marshalled objects ()
We only marshal arrays, hashes and sets, which meant that the emojis
here were just getting `to_s`ed.

This is a hack.
2023-04-10 10:44:11 -05:00
Krzysztof Kotlarek
e586f6052f
FEATURE: public custom sidebar sections visible to anonymous ()
Previously, public custom sections were only visible to logged-in users. In this PR, we are making them visible to anonymous as well.

The reason is that Community Section will be moved into custom section model to be easily editable by admins.
2023-04-06 08:55:47 +10:00
Krzysztof Kotlarek
b72282123b
FIX: public sidebar sections belong to system user ()
Before, public sidebar sections were belonging to admin. However, a better choice is system user.
2023-04-05 10:52:18 +10:00
David Taylor
2386ad12f2
Update default ga_version to v4 and add warning message for v3 ()
Sites which are already using ga3 will stay on that version, and will be shown a warning in the admin panel until they update.

https://meta.discourse.org/t/upgrade-to-google-analytics-4-before-july-2023/260498
2023-04-04 13:14:20 +01:00
Alan Guo Xiang Tan
e54359c135
DEV: Drop plugin related index from core ()
The migration of the index has since been moved into the
discourse-akismet plugin where it should have always been.
2023-04-04 07:31:09 +08:00
Alan Guo Xiang Tan
fd34032db2
DEV: Support filter for topics in specific subcategories on /filter ()
This commit adds support for filtering for topics in specific
subcategories via the categories filter query language.

For example: `category:documentation:admins` will filter for topics and
subcategory topics in
the category with slug "admins" whose parent category has the slug
"documentation".

The `=` prefix can also be used such that
`=category:documentation:admins` will exclude subcategory topics of the
category with slug "admins" whose parent category has the slug
"documentation".
2023-04-03 18:36:59 +08:00
Krzysztof Kotlarek
c86d772277
FIX: Drop internal URL validation for paths in sidebar ()
`Rails.application.routes.recognize_path(value)` was not working for /admin paths because StaffConstraint.new requires user to check permission.

This validation is not bringing much value, and the easiest way is to drop it. In the worse case scenario, a user will have an incorrect link in their sidebar.

Bug reported: https://meta.discourse.org/t/custom-sidebar-sections-being-tested-on-meta/255303/66
2023-03-31 13:26:56 +08:00
Mark VanLandingham
9518e47204
FEATURE: ability to bulk_remove users from a group () 2023-03-30 08:06:36 -05:00
Martin Brennan
84ff96bd07
FIX: Do not validate email in TL promotion ()
There is no need to validate the user's emails when
promoting/demoting their trust level, this can cause
issues in things like Jobs::Tl3Promotions, we don't
need to fail in that case when all we are doing is changing
trust level.
2023-03-30 13:52:10 +10:00
Blake Erickson
aa09a78d14
DEV: Remove diffhtml_preview for video thumbnails ()
Video thumbnail generation is no longer dependent on
`enable_diffhtml_preview` being enabled.
2023-03-28 12:38:50 -06:00
Vinoth Kannan
08ff6eebad
FIX: skip category preference update if already set by group. ()
`default_categories_*` site settings will update the category preferences on user creation. But it shouldn't update the user's category preference if a group's setting already updated it for that user.
2023-03-28 19:43:01 +05:30
Sam
7038540af6
PERF: ensure sidebar section link index is correctly ordered ()
We perform lookups on sidebar section links based on sidebar_section_id
totally ignoring user. This ensures we have an index to work with.

This removes the previous index `links_user_id_section_id_position` which
partially doubled up `idx_unique_sidebar_section_links`
2023-03-28 15:13:44 +11:00
Daniel Waterworth
d563b73202
DEV: Make category topic_ids cache work like other caches () 2023-03-27 09:23:29 -05:00
Daniel Waterworth
3e80ab3143
DEV: Make category subcategory_ids cache work like other caches () 2023-03-27 09:23:23 -05:00
Daniel Waterworth
b2554e1490
DEV: Make hex cache work like other caches () 2023-03-27 09:22:53 -05:00
Daniel Waterworth
0f4bb19524
DEV: make developer cache work like other caches () 2023-03-27 09:22:33 -05:00
Krzysztof Kotlarek
4047073292
FIX: display validation under custom sidebar fields ()
Before, incorrectly filled fields were marked with red border. Now, additional information under the field is displayed to notify the user what is incorrect.

/t/93696
2023-03-27 13:03:16 +11:00
Daniel Waterworth
0728fa055b
DEV: Make theme cache work like other caches () 2023-03-24 12:37:16 -05:00
Mark VanLandingham
ebada4a6b0
DEV: More specific API to including extra associations in CategoryList () 2023-03-23 12:39:38 -05:00
Mark VanLandingham
32aa821f12
DEV: Add preload API to CategoryList () 2023-03-22 15:12:08 -05:00
Martin Brennan
520d4f504b
FEATURE: Auto-remove users without permission from channel ()
There are many situations that may cause users to lose permission to
send messages in a chat channel. Until now we have relied on security
checks in `Chat::ChatChannelFetcher` to remove channels which the
user may have a `UserChatChannelMembership` record for but which
they do not have access to.

This commit takes a more proactive approach. Now any of these following
`DiscourseEvent` triggers may cause `UserChatChannelMembership`
records to be deleted:

* `category_updated` - Permissions of the category changed
   (i.e. CategoryGroup records changed)
* `user_removed_from_group` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`
* `site_setting_changed` - The `chat_allowed_groups` was updated, some
   users may no longer be in groups that can access chat.
* `group_destroyed` - Means the user may not be able to access the
   channel based on `GroupUser` or also `chat_allowed_groups`

All of these are handled in a distinct service run in a background
job. Users removed are logged via `StaffActionLog` and then we
publish messages on a per-channel basis to users who had their
memberships deleted.

When the user has a channel they are kicked from open, we show
a dialog saying "You no longer have access to this channel".

When they click OK we redirect them either:

* To their first other public channel, if they have any followed
* The chat browse page if they don't

This is to save on tons of requests from kicked out users getting messages
from other channels.

When the user does not have the kicked channel open, we can just
silently yoink it out of their sidebar and turn off subscriptions.
2023-03-22 10:19:59 +10:00
Daniel Waterworth
7b01576c8d
DEV: Remove emoji cache dead code ()
The cache is already not shared between app servers that have different
app_versions, so this check was redundant.
2023-03-21 12:33:12 -05:00
Daniel Waterworth
a0a6f6d71b
FIX: Fix the emoji toned regexes () 2023-03-21 11:48:55 -05:00
Régis Hanol
37609897e8
FEATURE: log manual bounce reset ()
DEV: rename the route "/admin/users/:id/reset_bounce_score" to use dashes instead of underscores
2023-03-21 15:26:26 +01:00
Krzysztof Kotlarek
1859025228
FIX: my links in sidebar section ()
Links like `/my/preferences` were invalid in custom section. The reason is that `/my` links are just redirects from backend, and they are not recognized as valid Ember paths.

https://github.com/discourse/discourse/blob/main/config/routes.rb#L433

Therefore, regex match allowlist was added - similar to backend check:

https://github.com/discourse/discourse/blob/main/app/controllers/users_controller.rb#L471

/safe-mode is same case
2023-03-21 15:58:42 +11:00
Krzysztof Kotlarek
db74e9484b
FEATURE: ability to reorder links in custom sidebar sections ()
Drag and drop to reorder custom sidebar sections
2023-03-21 12:23:28 +11:00
Martin Brennan
2f6987a241
DEV: Minor refactor for bookmark polymorphic changes ()
Followup to 184ce647ea,
this just implements Bianca's suggestion on the original
PR and catches the NameError, which was not necessary
before as we were not actually resolving any class from
bookmarkable_type.
2023-03-20 10:19:28 +01:00
Joffrey JAFFEUX
184ce647ea
FIX: correctly infer polymorphic class from bookmarkable type ()
Prior to this change `registered_bookmarkable` would return `nil` as  `type` in `Bookmark.registered_bookmarkable_from_type(type)` would be `ChatMessage` and we registered a `Chat::Message` class.

This commit will now properly rely on each model `polymorphic_class_for(name)` to help us infer the proper type from a a `bookmarkable_type`.

Tests have also been added to ensure that creating/destroying chat message bookmarks is working correctly.

---

Longer explanation

Currently when you save a bookmark in the database, it's associated to another object through a polymorphic relationship, which will is represented by two columns: `bookmarkable_id` and `bookmarkable_type`. The `bookmarkable_id` contains the id of the relationship (a post ID for example) and the `bookmarkable_type` contains the type of the object as a string by default, (`"Post"` for example).

Chat plugin just started namespacing objects, as a result a model named `ChatMessage` is now named `Chat::Message`, to avoid complex and risky migrations we rely on methods provided by rails to alter the `bookmarkable_type` when we save it: we want to still save it as `"ChatMessage"` and not `"Chat::Message"`. And, to retrieve the correct model when we load the bookmark from the database: we want `"ChatMessage"` to load the `Chat::Message` model and not the `ChatMessage`model which doesn't exist anymore.

On top of this the bookmark codepath is allowing plugins to register types and will check against these types, so we alter this code path to be able to do a similar ChatMessage <-> Chat::Message dance and allow to check the type is valid. In the specific case of this commit, we were retrieving a `"ChatMessage"` bookmarkable_type from the DB and looking for it in the registered bookmarkable types which contain `Chat::Message` and not `ChatMessage`.
2023-03-17 17:20:24 +01:00
Joffrey JAFFEUX
12a18d4d55
DEV: properly namespace chat ()
This commit main goal was to comply with Zeitwerk and properly rely on autoloading. To achieve this, most resources have been namespaced under the `Chat` module.

- Given all models are now namespaced with `Chat::` and would change the stored types in DB when using polymorphism or STI (single table inheritance), this commit uses various Rails methods to ensure proper class is loaded and the stored name in DB is unchanged, eg: `Chat::Message` model will be stored as `"ChatMessage"`, and `"ChatMessage"` will correctly load `Chat::Message` model.
- Jobs are now using constants only, eg: `Jobs::Chat::Foo` and should only be enqueued this way

Notes:
- This commit also used this opportunity to limit the number of registered css files in plugin.rb
- `discourse_dev` support has been removed within this commit and will be reintroduced later

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
2023-03-17 14:24:38 +01:00
David Taylor
303f97ce89
PERF: Use native postgres upsert for ApplicationRequest ()
Using `create_or_find_by!`, followed by `update_all!` requires two or three queries (two when the row doesn't already exist, three when it does). Instead, we can use postgres's native `INSERT ... ON CONFLICT ... DO UPDATE SET` feature to do the logic in a single atomic call.
2023-03-17 09:35:29 +00:00
Daniel Waterworth
84f590ab83
DEV: Store theme sprites in the DB ()
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
Blake Erickson
943068a634
FIX: Welcome topic banner showing after general category is deleted ()
If you happen to delete the general category before editing the welcome
topic, the banner will still display. This fix adds a after destroy hook
that will clear the entries for the welcome topic banner in the redis
cache.
2023-03-10 12:33:12 -07:00
Ted Johansson
87ec058b8b
FEATURE: Configurable auto-bump cooldown ()
Currently the auto-bump cooldown is hard-coded to 24 hours.

This change makes the highlighted 24 hours part configurable (defaulting to 24 hours), and the rest of the process remains the same.

This uses the new CategorySetting model associated with Category. We decided to add this because we want to move away from custom fields due to the lack of type casting and validations, but we want to keep the loading of these optional as they are not needed for almost all of the flows.

Category settings will be back-filled to all categories as part of this change, and creating a new category will now also create a category setting.
2023-03-10 13:45:01 +08:00
Blake Erickson
f144c64e13
Generate thumbnail images for video uploads ()
* FEATURE: Generate thumbnail images for uploaded videos

Topics in Discourse have a topic thumbnail feature which allows themes
to show a preview image before viewing the actual Topic.

This PR allows for the ability to generate a thumbnail image from an
uploaded video that can be use for the topic preview.
2023-03-09 09:26:47 -07:00
Krzysztof Kotlarek
22bccef8f4
FIX: set external flag before validation ()
Previously, `before_save` callback was used but `before_validation` has to be used to set external flag.
2023-03-09 10:44:54 +11:00
Roman Rizzi
910bf74c2e
FIX: Display a proper error when user already exists and email addresses are hidden. ()
Follow-up to . Returning an empty response leads to a bad UX since the user
has no feedback about what happened.
2023-03-08 12:38:58 -03:00
Loïc Guitaut
27f7cf18b1 FIX: Don’t email suspended users from group PM
Currently, when a suspended user belongs to a group PM (private message
with more than two people in it) and a staff member sends a message to
this group PM, then the suspended user will receive an email.
This happens because a suspended user can only receive emails from staff
members. But in this case, this can be seen as a bug as the expected
behavior would be instead to not send any email to the suspended user. A
staff member can participate in active discussions like any other
member and so their messages in this context shouldn’t be treated
differently than the ones from regular users.

This patch addresses this issue by checking if a suspended user receives
a message from a group PM or not. If that’s the case then an email won’t
be sent no matter if the post originated from a staff member or not.
2023-03-08 15:53:53 +01:00
Gerhard Schlager
12436d054d
DEV: Remove badge_granted_title column from user_profiles ()
That column is obsolete since we added the `granted_title_badge_id` column in 2019 (56d3e29a69). Having both columns can lead to inconsistencies (mostly due to old data from before 2019).

For example, `BadgeGranter.revoke_ungranted_titles!` doesn't work correctly if `badge_granted_title` is `false` while `granted_title_badge_id` points to the badge that is used as title.
2023-03-08 13:37:20 +01:00
Martin Brennan
360d0dde65
DEV: Change Bookmarkable registration to DiscoursePluginRegistry ()
Similar spirit to e195e6f614,
this moves the Bookmarkable registration to DiscoursePluginRegistry
so plugins which are not enabled do not register additional
bookmarkable classes.
2023-03-08 10:39:12 +10:00
Krzysztof Kotlarek
a16ea24461
FEATURE: allow external links in custom sidebar sections ()
Originally, only Discourse site links were available. After feedback, it was decided to extend this feature to external URLs.

/t/93491
2023-03-07 11:47:18 +11:00
Alan Guo Xiang Tan
e3977f84a3
FIX: Incorrect topic tracking state count when a new category is created ()
What is the problem?

We have a hidden site setting `show_category_definitions_in_topic_lists`
which is set to false by default. What this means is that category
definition topics are not shown in the topic list by default. Only the
category definition topic for the category being viewed will be shown.
However, we have a bug where we would show that a category has new
topics when a new child category along with its category definition
topic is created even though the topic list does not list the child
category's category definition topic.

What is the fix here?

This commit fixes the problem by shipping down an additional
`is_category_topic` attribute in `TopicTrackingStateItemSerializer` when
the `show_category_definitions_in_topic_lists` site setting has been set
to false. With the new attribute, we can then exclude counting child
categories' category definition topics when counting new and unread
counts for a category.
2023-03-06 10:13:10 +08:00
chapoi
e52bbc1230
UX/DEV: Review queue redesign fixes ()
* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* DEV: switch to selectKit

* UX: color approve/reject buttons in RQ

* DEV: regroup actions

* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* Join questions for flagged post with "or" with new I18n function
* Move ReviewableScores component out of context
* Add CSS classes to reviewable-item based on human type

* UX: add table header for scoring

* UX: don't display % score

* UX: prefix modifier class with dash

* UX: reviewQ flag table styling

* UX: consistent use of ignore icon

* DEV: only show context question on pending status

* UX: only show table headers on pending status

* DEV: reviewQ regroup actions for hidden posts

* UX: reviewQ > approve/reject buttons

* UX: reviewQ add fadeout

* UX: reviewQ styling

* DEV: move scores back into component

* UX: reviewQ mobile styling

* UX: score table on mobile

* UX: reviewQ > move meta info outside table

* UX: reviewQ > score layout fixes

* DEV: readd `agree_and_keep` and fix the spec tests.

* Fix the spec tests

* fix the quint test

* DEV: readd deleting replies

* UX: reviewQ copy tweaks

* DEV: readd test for ignore + delete replies

* Remove old

* FIX: Add perform_ignore back in for backwards compat

* DEV: add an action alias `ignore` for `ignore_and_do_nothing`.

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Vinoth Kannan <svkn.87@gmail.com>
2023-03-02 16:40:53 +01:00
David Battersby
96d03ea9c0
FIX: No small action created when a non-author removes itself from a PM ()
Fixes a small issue where allowed user removes themselves from a private message before the post activity (small action) is created.

I also added some test coverage to prevent regression.

/t/92811
2023-03-02 13:47:54 +08:00
Martin Brennan
e195e6f614
DEV: Move about_stat_groups to DiscoursePluginRegistry ()
Follow up to 098ab29d41. Since
we just used a `cattr_reader` on `About` this was not safe
for multisite, since some sites could have the chat plugin
enabled and some may not. Using `DiscoursePluginRegistry` gets
around this issue, and makes it so the chat stats only show
for a site if `chat_enabled` is true.
2023-03-02 08:10:16 +10:00
David Battersby
8a2995f719
FIX: only show approved users in search_user results when site setting enabled ()
Returns only approved users when using @ in composer (if must_approve_users site setting enabled).
2023-03-01 12:23:29 +08:00
Natalie Tay
44b7706a2b
UX: Skip applying link-type watched words to user custom fields ()
We currently apply type: :link watched words to custom user fields. This makes the user card pretty ugly because we don't allow html / links there. Additionally, the admin UI also does not say that we apply this to custom user fields, but only words in posts.

So this PR is to remove the replacement of link-type watch words for custom user fields.
2023-03-01 10:43:34 +08:00
Martin Brennan
d3a1b09361
FEATURE: Chat header icon indicator preference ()
This commit allows the user to set their preference vis-a-vis
the chat icon in the header of the page. There are three options:

- All New (default) - This maintains the existing behaviour where
  all new messages in the channel show a blue dot on the icon
- Direct Messages and Mentions - Only show the green dot on the
  icon when you are directly messaged or mentioned, the blue dot
  is never shown
- Never - Never show any dot on the chat icon, for those who
  want tractor-beam-laser-focus
2023-03-01 11:01:44 +10:00
Keegan George
def4133d59
DEV: Move form_template_ids serializer location () 2023-02-28 09:09:20 -08:00
Bianca Nenciu
ccb345bd88
FEATURE: Update topic/comment embedding parameters ()
This commit implements many changes to topic and comments embedding. It
deprecates the class_name field from EmbeddableHost and suggests using
the className parameter. discourse_username parameter has been
deprecated and it will fetch it from embedded site from the author or
discourse-username meta.

See the updated code sample from Admin > Customize > Embedding page.

* FEATURE: Add className parameter for Discourse embed

* DEV: Hide class_name from EmbeddableHost

* DEV: Deprecate class_name field of EmbeddableHost

* FEATURE: Use either author or discourse-username meta tag

* DEV: Deprecate discourse_username parameter

* DEV: Improve embed code sample
2023-02-28 14:31:59 +02:00
Blake Erickson
451ee71930
FEATURE: Add API scope for listing topics in a category ()
Adding an api scope for `/c/{slug}/{id}.json`. Without this you had to
append `/l/latest` to the url in order for it to work with existing
scopes.
2023-02-27 16:21:39 -07:00
Osama Sayegh
a509441148
DEV: Include unread topics in New topic lists and link to it in sidebar ()
This commit introduces a few experimental changes to the New topics list and "Everything" link in the sidebar:

1. Make the New topics list include unread topics
2. Make the Everything section in the sidebar link to the New topics list (`/new`)
3. Remove "unread" or "new" text next to the count and keep the count
4. The count is a sum of new and unread topics counts

All of these of changes are behind an off-by-default feature flag. I've not written extensive tests for these changes because they're highly experimental.

Internal topic: t/77234.
2023-02-27 15:11:01 +03:00
Krzysztof Kotlarek
d92fd30d23
FIX: do not escape slash for category text description ()
Original solution to use `description` instead of `text_description` was wrong: https://github.com/discourse/discourse/pull/20436

Problem is that we have to escape HTML tags.

However, we would like to use escape method which is keep `/` intact.   Expected behavior is given by  ERB::Util.html_escape instead of Rack::Utils.escape_html

/t/92015
2023-02-27 12:48:48 +11:00
Keegan George
6108eee31d
DEV: Apply form template to categories () 2023-02-23 11:18:14 -08:00
Loïc Guitaut
a9f2c6db64 SECURITY: Show only visible tags in metadata
Currently, the topic metadata show both public and private
tags whereas only visible ones should be exposed.
2023-02-23 17:22:20 +01:00
Blake Erickson
5dbdcb3f23
FEATURE: Adding some more api scopes ()
Adds api scopes for

- deleting a topic
- deleting a post
- listing tags
2023-02-23 08:33:29 -07:00
Blake Erickson
51a7cd899e
FEATURE: Add API scopes for group endpoints ()
Added two new api scopes for managing and administering groups.

See https://meta.discourse.org/t/249710
2023-02-22 09:06:49 -07:00
Krzysztof Kotlarek
b9d037770c
DEV: configurable public sidebar sections ()
Extension of https://github.com/discourse/discourse/pull/20057

Admin can create a public session visible to everyone. An additional checkbox is displayed for staff members.
2023-02-22 08:55:44 +11:00
Daniel Waterworth
b96a23978a
PERF: Add index to speed up ReindexSearch job () 2023-02-21 13:16:43 -05:00
Loïc Guitaut
f7c57fbc19 DEV: Enable unless cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Gerhard Schlager
7ef482a292
REFACTOR: Fix pluralized strings in chat plugin ()
* FIX: Use pluralized string

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Fix misuse of pluralized string

* DEV: Remove linting of `one` key in MessageFormat string, it doesn't work

* REFACTOR: Fix misuse of pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff. The string is quite complicated, so the best option was to switch to MessageFormat.

* REFACTOR: Fix misuse of pluralized string

* FIX: Use pluralized string

This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff.

* REFACTOR: Correctly pluralize reaction tooltips in chat

This also ensures that maximum 5 usernames are shown and fixes the number of "others" which was off by 1 if the current user reacted on a message.

* REFACTOR: Use translatable string as comma separator

* DEV: Add comment to translation to clarify the meaning of `%{identifier}`

* REFACTOR: Use translatable comma separator and use explicit interpolation keys

* REFACTOR: Don't interpolate lowercase channel status

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Don't interpolate channel status

* REFACTOR: Use %{count} interpolation key

* REFACTOR: Fix misuse of pluralized string

* REFACTOR: Correctly pluralize DM chat channel titles
2023-02-20 10:31:02 +01:00
Osama Sayegh
f91631b625
DEV: Update javascript:update_constants rake task following template colocation ()
As of ba3f62f576, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-20 06:20:47 +03:00
Martin Brennan
7ad92b7ab9
FIX: Do not enqueue update_post_uploads_secure_status unnecessarily ()
We call `post.update_uploads_secure_status` in both
`PostCreator` and `PostRevisor`. Only the former was checking
if `SiteSetting.secure_uploads?` was enabled, but the latter
was not. There is no need to enqueue the job
`UpdatePostUploadsSecureStatus` if secure_uploads is not
enabled for the site.
2023-02-20 11:21:54 +10:00
Leonardo Mosquera
6fa9a8990d
Add username template variables to Welcome User email template ()
* Add username and name_or_username variables to SystemMessage defaults

* Allow username and name variables on welcome_user email template overrides

* Satisfy linting

* Add test
2023-02-17 18:38:21 -03:00
Sam
f96ef33856
FIX: dominant color not working for 16bit images ()
16 bit images were not returning the correct dominant color due truncation

The routine expected an 8bit color eg: #FFAA00, but ended up getting a 16bit one eg: #FFFAAA000. This caused a truncation, which leads to wildly off colors.
2023-02-15 12:41:04 +11:00
Ted Johansson
25a226279a
DEV: Replace #pluck_first freedom patch with AR #pick in core ()
The #pluck_first freedom patch, first introduced by @danielwaterworth has served us well, and is used widely throughout both core and plugins. It seems to have been a common enough use case that Rails 6 introduced it's own method #pick with the exact same implementation. This allows us to retire the freedom patch and switch over to the built-in ActiveRecord method.

There is no replacement for #pluck_first!, but a quick search shows we are using this in a very limited capacity, and in some cases incorrectly (by assuming a nil return rather than an exception), which can quite easily be replaced with #pick plus some extra handling.
2023-02-13 12:39:45 +08:00
Ted Johansson
a90ad52dff
DEV: Add dedicated category settings model - Part 1 ()
This is the first in a multi-part change to move the custom fields to a new table. It includes:

- Adding a new CategorySetting model and corresponding table.
- Populating it with data from the category_custom_fields table.
2023-02-13 12:37:59 +08:00
Krzysztof Kotlarek
cbd021db15
FIX: add index to sidebar_section_link ()
Index on linkable_type and linkable_id should increase performance of this subquery https://github.com/discourse/discourse/blob/main/app/services/sidebar_site_settings_backfiller.rb#L86

Also, distinct is removing duplicates which are unnecessary.
2023-02-10 11:14:22 +11:00
Keegan George
871607a420
DEV: Create form templates () 2023-02-08 11:21:39 -08:00
Krzysztof Kotlarek
6e1f3e0023
FIX: improvements for user custom sections ()
Improvements for this PR: https://github.com/discourse/discourse/pull/20057

What was fixed:
- [x] Use ember transitions instead of full reload
- [x] Link was inaccurately kept active
- [x] "+ save" renamed to just "save"
- [x] Render emojis in link name
- [x] UI to set icon
- [x] Delete link is trash icon instead of "x"
- [x] Add another link to on the left and rewording
- [x] Raname "link name" -> "name", "points to" ->  link
- [x] Add limits to fields
- [x] Move add section button to the bottom
2023-02-08 11:45:34 +11:00
Sérgio Saquetim
5d32db76dd
DEV: Added .only_deleted scope in the Trashable module () 2023-02-07 15:28:59 -03:00
Rafael dos Santos Silva
6e522e4aad
DEV: Move to Sass compilation to dart-sass ()
This PR is a major change to Sass compilation in Discourse.

The new version of sass-ruby moves to dart-sass putting we back on the supported version of Sass. It does so while keeping compatibility with the existing method signatures, so minimal change is needed in Discourse for this change.

This moves us

From:
  - sassc 2.0.1 (Feb 2019)
  - libsass 3.5.2 (May 2018)

To:
  - dart-sass 1.58

This update applies the following breaking changes:

> 
> These breaking changes are coming soon or have recently been released:
> 
>  [Functions are stricter about which units they allow](https://sass-lang.com/documentation/breaking-changes/function-units) beginning in Dart Sass 1.32.0.
> 
>  [Selectors with invalid combinators are invalid](https://sass-lang.com/documentation/breaking-changes/bogus-combinators) beginning in Dart Sass 1.54.0.
> 
>  [/ is changing from a division operation to a list separator](https://sass-lang.com/documentation/breaking-changes/slash-div) beginning in Dart Sass 1.33.0.
> 
>  [Parsing the special syntax of @-moz-document will be invalid](https://sass-lang.com/documentation/breaking-changes/moz-document) beginning in Dart Sass 1.7.2.
> 
>  [Compound selectors could not be extended](https://sass-lang.com/documentation/breaking-changes/extend-compound) in Dart Sass 1.0.0 and Ruby Sass 4.0.0.


SCSS files have been migrated automatically using `sass-migrator division app/assets/stylesheets/**/*.scss`
2023-02-07 12:24:57 -03:00
Ted Johansson
676d5fadab
DEV: Limit and validate category settings inputs ()
We recently had a bug which caused auto-bumping to "not work". The problem was that the value had been set to 0.5, which when coerced to an integer turned into 0. So the feature is "working as intended", but there's a possibility of misconfiguration.

When looking into this, I noticed that the inputs on the category settings page doesn't have any particular sanitisation in the front-end, and also one or two validations missing in the back-end.

This change:

- Takes an existing component, NumberField and enhances that by only allowing numeric input, essentially turning it into a managed input using the same approach as our PasswordField.
- Changes the numeric inputs on category settings page to use this component.
- Adds appropriate min constraints to the fields to disallow out-of-range values.
- Adds missing back-end validations to relevant fields.
2023-02-07 12:27:38 +08:00
Blake Erickson
c540167982
FIX: Remove action buttons if post has already been reviewed ()
* FIX: Remove action buttons if post has already been reviewed

* Change the approve to reject test to expect an error

* Adds a controller spec to ensure you can't edit a non-pending review item

* Remove unnessary conditional
2023-02-06 11:55:52 -07:00
David Taylor
754d1b71aa
FIX: Ensure ColorScheme#resolve falls back to base for missing color ()
When a CUSTOM_SCHEME is missing a color (e.g. 'Dracula' is missing a 'highlight' color), we need to fallback to `ColorScheme.base_colors`. This regressed in 66256c15bd
2023-02-06 18:24:12 +00:00
Penar Musaraj
a86112fc25
FEATURE: Allow embedded view to include a header ()
This commits adds the ability to add a header to the embedded comments
view. One use case for this is to allow `postMessage` communication
between the comments iframe and the parent frame, for example, when
toggling the theme of the parent webpage.
2023-02-06 11:10:50 -05:00
Andrei Prigorshnev
84e13e9b1c
FIX: avoid race condition when setting user status ()
We caught it in logs, race condition led to this error:

    ActiveRecord::RecordNotUnique 
    (PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "user_statuses_pkey"
    DETAIL:  Key (user_id)=(15) already exists.)


The reason the problem happened was that we were checking if a user has status and if not inserting status:

    if user_status
      ...
    else
      self.user_status = UserStatus.create!(status)
    end

The problem is that it's possible that another request will insert status just after we check if status exists and just before our request call `UserStatus.create!(status)`. Using `upsert` fixes the problem because under the hood `upsert` generates the only SQL request that uses "INSERT ... ON CONFLICT DO UPDATE". So we do everything in one SQL query, and that query takes care of resolving possible conflicts.
2023-02-06 18:56:28 +04:00
Gerhard Schlager
e17c145e8d
FIX: Category hashtags weren't always found for sub-sub-categories ()
The algorithm failed to find the correct category by slug when there are multiple sub-sub-categories with the same child-category name and the first child doesn't  have the correct grandchild.

So, searching for "child / grandchild" worked in the following case, it found (3):

- (1) parent 1
  - (2) child
    - (3) grandchild
- (4) parent 2
  - (5) child
    - (6) grandchild

But it failed to find the grandchild in the following case:

- (1) parent 1
  - (2) child
- (4) parent 2
  - (5) child
    - (6) grandchild

And this also fixes a flaky spec by forcing categories to always order by by `parent_category_id` and `id`.
This makes it possible to partly revert 60990aab55
2023-02-03 12:17:52 +01:00
Krzysztof Kotlarek
84a87a703c
DEV: configurable custom sidebar sections ()
Allows users to configure their own custom sidebar sections with links withing Discourse instance. Links can be passed as relative path, for example "/tags" or full URL.

Only path is saved in DB, so when Discourse domain is changed, links will be still valid.

Feature is hidden behind SiteSetting.enable_custom_sidebar_sections. This hidden setting determines the group which members have access to this new feature.
2023-02-03 14:44:40 +11:00
Rafael dos Santos Silva
e4fd3d9850
FIX: Better ordering of similar user search suggestions ()
* FIX: Better ordering of similar user search suggestions
2023-02-02 14:39:44 -03:00
Rafael dos Santos Silva
14cf8eacf1
FEATURE: Use similarity in user search ()
Currently, when doing `@mention` for users we have 0 tolerance for typos and misspellings.

With this patch, if a user search doesn't return enough results we go and use `pg_trgm` features to try and find more matches based on trigrams of usernames and names.

It also introduces GiST indexes on those fields in order to improve performance of this search, going from 130ms down to 15ms in my tests.

This is all gated in a feature flag and can be enabled by running  `SiteSetting.user_search_similar_results = true` in the rails console.
2023-02-02 13:35:04 -03:00
David Taylor
66256c15bd
UX: Calculate missing hover/selected colors from existing colors ()
`--d-hover` is calculated to be equivalent to primary-100 in light mode, or primary-low in dark mode

`--d-selected` is calculated to be equivalent to primary-low in light mode, or primary-100 in dark mode

`lib/color_math` is introduced to provide some utilities for making these calculations.
2023-02-01 09:55:21 +00:00
Alan Guo Xiang Tan
07ef828db9
DEV: Improve MessageBus subscriptions for TopicTrackingState ()
## Why do we need this change? 

When loading the ember app, [MessageBus does not start polling immediately](f31f0b70f8/app/assets/javascripts/discourse/app/initializers/message-bus.js (L71-L81)) and instead waits for `document.readyState` to be `complete`. What this means is that if there are new messages being created while we have yet to start polling, those messages will not be received by the client.

With sidebar being the default navigation menu, the counts derived from `topic-tracking-state.js` on the client side is prominently displayed on every page. Therefore, we want to ensure that we are not dropping any messages on the channels that `topic-tracking-state.js` subscribes to.  

## What does this change do? 

This includes the `MessageBus.last_id`s for the MessageBus channels which `topic-tracking-state.js` subscribes to as part of the preloaded data when loading a page. The last ids are then used when we subscribe the MessageBus channels so that messages which are published before MessageBus starts polling will not be missed.

## Review Notes

1. See https://github.com/discourse/message_bus#client-support for documentation about subscribing from a given message id.
2023-02-01 07:18:45 +08:00
Bianca Nenciu
23a74ecf8f
FIX: Truncate existing user status to 100 chars ()
This commits adds a database migration to limit the user status to 100
characters, limits the user status in the UI and makes sure that the
emoji is valid.

Follow up to commit b6f75e231c.
2023-01-30 10:49:08 +02:00
Alan Guo Xiang Tan
7ec6e6b3d0
PERF: N+1 queries on /tags with multiple categories tags ()
When the `tags_listed_by_group` site setting is disable, we were seeing
the N+1 queries problem when multiple `CategoryTag` records are listed.
This commit fixes that by ensuring that we are not filtering through the
category `tags` association after the association has been eager loaded.
2023-01-30 08:53:17 +08:00
Blake Erickson
4ecfac39a6
FEATURE: Add separate api scope for topic status ()
This will allow us more granular control over changing a topic status.
For example you can now force the scope to only allow closing topics in
a specific category. This means that the same scope can't be used to
re-open topics, or close topics in a different category.
2023-01-27 08:05:29 -07:00
chapoi
e03f6057ec
UX: Highlight var refactor ()
* Add new color vars

* Select-kit > use new color vars

* update all color schemes with values for new hover/select vars

* Add variable yml names
2023-01-27 15:50:36 +01:00
Bianca Nenciu
b6f75e231c
FIX: Limit user status to 100 characters ()
* FIX: Limit user status to 100 characters

* FIX: Make sure the emoji is valid
2023-01-27 16:32:27 +02:00
Martin Brennan
079e1108f1
DEV: Remove stale TODO ()
This TODO is irrelevant -- in reality this has not been a
perf issue, and there is not actually an N1 here. Furthermore,
this is only used in a single plugin, not in core.
2023-01-27 16:32:11 +10:00
Martin Brennan
48eb8d5f5a
Revert "DEV: Delete dead Topic#incoming_email_addresses code ()" ()
This reverts commit 88a972c61b.

It's actually used in some plugins.
2023-01-27 11:27:15 +10:00
Jan Cernik
665065bbc7
DEV: Fix ruby syntax tree linting error () 2023-01-26 17:17:00 -03:00
Roman Rizzi
d96b4bbb46
DEV: Include upload_id on image optimization errors to have more context () 2023-01-26 14:57:40 -03:00
Bianca Nenciu
cd7c8861ae
SECURITY: Remove bypass for base_url ()
The check used to be necessary because we validated the referrer too and
this bypass was a workaround a bug that is present in some browsers that
do not send the correct referrer.
2023-01-25 13:50:45 +02:00
Natalie Tay
d5745d34c2
SECURITY: Limit the character count of group membership requests ()
When creating a group membership request, there is no character
limit on the 'reason' field. This can be potentially be used by
an attacker to create enormous amount of data in the database.

Co-authored-by: Ted Johansson <ted@discourse.org>
2023-01-25 13:50:33 +02:00
Loïc Guitaut
ec2ed5b7f6 FIX: Delete reviewables associated to posts automatically
Currently we don’t have an association between reviewables and posts.
This sometimes leads to inconsistencies in the DB as a post can have
been deleted but an associated reviewable is still present.

This patch addresses this issue simply by adding a new association to
the `Post` model and by using the `dependent: :destroy` option.
2023-01-25 09:45:36 +01:00
Martin Brennan
82182ec0c7
DEV: Add hashtag controller specs ()
This is just cleaning up a TODO I had to add more specs
to this controller -- there are more thorough tests on the
actual HashtagService class and the type-specific hashtag
classes.
2023-01-25 17:13:32 +10:00
Martin Brennan
88a972c61b
DEV: Delete dead Topic#incoming_email_addresses code ()
This code has been dead since b463a80cbf,
we can delete it now.
2023-01-25 09:34:41 +10:00
Vinoth Kannan
799202d50b
FIX: skip email if blank while syncing SSO attributes. ()
Also, return email blank error in `EmailValidator`  when the email is blank.
2023-01-24 09:10:24 +05:30
Martin Brennan
0924f874bd
DEV: Use UploadReference instead of ChatUpload in chat ()
We've had the UploadReference table for some time now in core,
but it was added after ChatUpload was and chat was just never
moved over to this new system.

This commit changes all chat code dealing with uploads to create/
update/delete/query UploadReference records instead of ChatUpload
records for consistency. At a later date we will drop the ChatUpload
table, but for now keeping it for data backup.

The migration + post migration are the same, we need both in case
any chat uploads are added/removed during deploy.
2023-01-24 13:28:21 +10:00
Blake Erickson
a6291cd854
FEATURE: Add api scope for suspending users ()
See: https://meta.discourse.org/t/request-separate-api-granular-api-scope-for-suspend-user/249928/5
2023-01-23 16:20:49 -07:00
Blake Erickson
774feb6614
FEATURE: Add api scope for create invite endpoint ()
Adds an api scope for the POST /invite endpoint.
2023-01-23 16:20:22 -07:00
Blake Erickson
09f5235538
FEATURE: Add api scope for search endpoint ()
Adds two new api scopes for the /search endpoints:

- `/search.json?q=term`
- `/search/query.json?term=term`

see: https://meta.discourse.org/t/search-api-key-permissions/227244
2023-01-23 14:06:57 -07:00
Daniel Waterworth
666536cbd1
DEV: Prefer \A and \z over ^ and $ in regexes () 2023-01-20 12:52:49 -06:00
Krzysztof Kotlarek
f409e977a9
FIX: deleted misconfigured embeddable hosts ()
When EmbeddableHost is configured for a specific category and that category is deleted, then EmbeddableHost should be deleted as well.

In addition, migration was added to fix existing data.
2023-01-20 13:29:49 +11:00
Alan Guo Xiang Tan
f122f24b35
SECURITY: Default tags to show count of topics in unrestricted categories ()
Currently, `Tag#topic_count` is a count of all regular topics regardless of whether the topic is in a read restricted category or not. As a result, any users can technically poll a sensitive tag to determine if a new topic is created in a category which the user has not excess to. We classify this as a minor leak in sensitive information.

The following changes are introduced in this commit:

1. Introduce `Tag#public_topic_count` which only count topics which have been tagged with a given tag in public categories.
2. Rename `Tag#topic_count` to `Tag#staff_topic_count` which counts the same way as `Tag#topic_count`. In other words, it counts all topics tagged with a given tag regardless of the category the topic is in. The rename is also done so that we indicate that this column contains sensitive information. 
3. Change all previous spots which relied on `Topic#topic_count` to rely on `Tag.topic_column_count(guardian)` which will return the right "topic count" column to use based on the current scope. 
4. Introduce `SiteSetting.include_secure_categories_in_tag_counts` site setting to allow site administrators to always display the tag topics count using `Tag#staff_topic_count` instead.
2023-01-20 09:50:24 +08:00
Isaac Janzen
292d3677e9
FEATURE: Allow admins to permanently delete revisions ()
# Context
This PR introduces the ability to permanently delete revisions from a post while maintaining the changes implemented by the revisions.
Additional Context: /t/90301

# Functionality
In the case a staff member wants to _remove the visual cue_ that a post has been edited eg.

<img width="86" alt="Screenshot 2023-01-18 at 2 59 12 PM" src="https://user-images.githubusercontent.com/50783505/213293333-9c881229-ab18-4591-b39b-e3419a67907d.png">

while maintaining the changes made in the edits, they can enable the (hidden) site setting of `can_permanently_delete`.
When this is enabled, after _hiding_ the revisions

<img width="149" alt="Screenshot 2023-01-19 at 1 53 35 PM" src="https://user-images.githubusercontent.com/50783505/213546080-2a9e9c55-b3ef-428e-a93d-1b6ba287dfae.png">

there will be an additional button in the history modal to <kbd>Delete revisions</kbd> on a post.

<img width="997" alt="Screenshot 2023-01-19 at 1 49 51 PM" src="https://user-images.githubusercontent.com/50783505/213546333-49042558-50ab-4724-9da7-08bacc68d38d.png">

Since this action is permanent, we display a confirmation dialog prior to triggering the destroy call

<img width="722" alt="Screenshot 2023-01-19 at 1 55 59 PM" src="https://user-images.githubusercontent.com/50783505/213546487-96ea6e89-ac49-4892-b4b0-28996e3c867f.png">

Once confirmed the history modal will close and the post will `rebake` to display an _unedited_ post.

<img width="868" alt="Screenshot 2023-01-19 at 1 56 35 PM" src="https://user-images.githubusercontent.com/50783505/213546608-d6436717-8484-4132-a1a8-b7a348d92728.png">
 
see that there is not a visual que for _revision have been made on this post_ for a post that **HAS** been edited. In addition to this, a user history log for `purge_post_revisions` will be added for each action completed.

# Limits
- Admins are rate limited to 20 posts per minute
2023-01-19 15:09:01 -06:00
Alan Guo Xiang Tan
1e8a1a0d24
PERF: N+1 queries when viewing tags ()
When the `tags_listed_by_group` site setting is enabled, we were seeing
the N+1 queries problem when multiple `TagGroup` records are listed.
This commit fixes that by ensuring that we are not filtering through the
`tags` association after the association has been eager loaded.
2023-01-17 15:50:21 +08:00
David Taylor
41f3bb8b50
FIX: Restore class-property babel transform for themes ()
This seems to be required for decorators to work on class properties. Followup to 624f4a7de9
2023-01-16 18:06:46 +00:00
David Taylor
624f4a7de9
Drop support for iOS < 15.7 ()
https://meta.discourse.org/t/224747
2023-01-16 17:28:59 +00:00
Alan Guo Xiang Tan
f72875c729
DEV: Introduce enable_new_notifications_menu site setting ()
The `enable_new_notifications_menu` site setting allows sites that have
`navigation_menu` set to `legacy` to use the redesigned notifications
menu before switching to the new sidebar navigation menu.
2023-01-16 06:04:53 +08:00
Selase Krakani
73ec80893d
FEATURE: Extend topic update API scope to allow status updates ()
Allow an API key created with topic:update API scope to make updates to
topic status. This change also introduces an optional category_id scope
param.
2023-01-13 01:21:04 +00:00
Daniel Waterworth
3030a53819
FIX: Prevent concurrent updates to top_topics ()
to prevent lock timeouts
2023-01-12 14:03:26 -06:00
Sérgio Saquetim
8fd9a93a1a
DEV: Added notification type for 'discourse-circles' ()
Reserved an ID to be used by notifications generated on the `discourse-circles` plugin.
2023-01-12 14:07:42 -03:00
Martin Brennan
779b9add24
DEV: Remove bookmark column ignores ()
These columns were deleted in f8f55cef67
2023-01-12 11:45:37 +10:00
Alan Guo Xiang Tan
9fcd8336e4
FIX: Regression in TopicTrackingState MessageBus message scope. ()
0403cda1d1 introduced a regression where
topics in non read-restricted categories have its TopicTrackingState
MessageBus messages published with the `group_ids: [nil]` option. This
essentially means that no one would be able to view the message.
2023-01-12 08:22:28 +08:00
Martin Brennan
21a95b000e
DEV: Remove defunct TODOs ()
* Firefox now finally returns PerformanceMeasure from performance.measure
* Some TODOs were really more NOTE or FIXME material or no longer relevant
* retain_hours is not needed in ExternalUploadsManager,  it doesn't seem like anywhere in the UI sends this as a param for uploads
* https://github.com/discourse/discourse/pull/18413 was merged so we can remove JS test workaround for settings
2023-01-12 09:41:39 +10:00
Alan Guo Xiang Tan
0403cda1d1
FIX: Error when publishing TopicTrackingState updates for certain topics ()
When a topic belongs to category that is read restricted but permission
has not been granted to any groups, publishing ceratin topic tracking state
updates for the topic will result in the `MessageBus::InvalidMessageTarget` error being raised
because we're passing `nil` to `group_ids` which is not support by
MessageBus.

This commit ensures that for said category above, we will publish the
updates to the admin groups.
2023-01-11 06:15:52 +08:00
David Taylor
5a003715d3
DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
Roman Rizzi
7b5f7b4484
FIX: Don't change the default allowed_attribute when calling #sanitize_field () 2023-01-06 11:47:15 -03:00
Matt Marjanović
aa4ff47208
FEATURE: Allow target attribute in links in user_field descriptions ()
This change adds `target` to the set of attributes allowed by the
HTML sanitizer which is applied to the description of a user_field.

The rationale for this change:

 * If one puts a link (<a>...</a>) in the description of a user_field
   that is present and/or required at sign-up, the expectation is that
   a prospective new user will click on that link during sign-up.
 * Without an appropriate `target` attribute on the link, the new page
   will be loaded in the same window/tab as the sign-up form, but this
   will obliterate any fields that the user had already filled-out on
   the form.  (E.g., hitting the back-button will return to an
   empty form.)
 * Such UX behavior is incredibly aggravating to new users.

This change allows an admin to add a `target` attribute to links, to
instruct the browser to open them in a different window/tab, leaving
a sign-up form intact.
2023-01-06 10:18:35 -03:00
Vinoth Kannan
7ecf4d12a9
FIX: use reviewer's guardian permissions to create post/topic while approve. ()
We previously used post creator's guardian permissions which will raise an error if the reviewer added a staff-only (restricted) tag.

Co-authored-by: Natalie Tay <natalie.tay@discourse.org>
2023-01-05 19:31:37 +05:30
Alan Guo Xiang Tan
83944213b2
SECURITY: Delete email tokens when a user's email is changed or deleted ()
Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
2023-01-05 06:08:55 +08:00
Alan Guo Xiang Tan
ab3a032b4b
SECURITY: BCC active user emails from group SMTP ()
When sending emails out via group SMTP, if we
are sending them to non-staged users we want
to mask those emails with BCC, just so we don't
expose them to anyone we shouldn't. Staged users
are ones that have likely only interacted with
support via email, and will likely include other
people who were CC'd on the original email to the
group.

Co-authored-by: Martin Brennan <martin@discourse.org>
2023-01-05 06:07:50 +08:00
David Taylor
45435cbbd5
PERF: Use user-specific channel for message-bus logout ()
Using a shared channel means that every user receives an update to the 'last_id' when *any* other user is logged out. If many users are being programmatically logged out at the same time, this can cause a very large number of message-bus polls.

This commit switches to use a user-specific channel, which means that each user has its own 'last id' which will only increment when they are logged out
2023-01-04 19:55:52 +00:00
Alan Guo Xiang Tan
24db6fbb73
PERF: Memoize topic level checks in PostGuardian ()
When loading posts in a topic, the topic level guardian
checks are run multiple times even though all the posts belong to the
same topic. Profiling in production revealed that this accounted for a
significant amount of request time for a user that is not staff or anon.
Therefore, we're optimizing this by adding memoizing the topic level
calls in `PostGuardian`. Speficifally, the result of
`TopicGuardian#can_see_topic?` and `PostGuardian#can_create_post?`
method calls are memoized per topic.

Locally profiling shows a significant improvement for normal users
loading a topic with 100 posts.

Benchmark script command: `ruby script/bench.rb --unicorn --skip-bundle-assets --iterations 100`

Before:

```
topic user:
  50: 114
  75: 117
  90: 122
  99: 209
topic.json user:
  50: 67
  75: 69
  90: 72
  99: 162
```

After:

```
topic user:
  50: 101
  75: 104
  90: 107
  99: 184
topic.json user:
  50: 53
  75: 53
  90: 56
  99: 138
```
2023-01-03 09:00:42 +08:00
Alan Guo Xiang Tan
070eac16a8
PERF: Optimize the performance of TopicLink.topic_map query ()
Instead of relying on the `ILIKE` operator to filter out image links, we
can instead rely on the `TopicLink#extension` column which allows us to
more efficiently filter out image links.

This optimization mainly affects topics that are link heavy which is
common in topics with alot of replies. When profiling a production
instance for a topic with 10K replies and 2.5K `topic_links`, this
optimization reduces the query time from ~18ms to around ~4ms.
2022-12-28 08:10:36 +08:00
Alan Guo Xiang Tan
a7f30cfcf7
PERF: Fix N+1 queries on /categories route ()
Featured topics are eventually serialized by `ListableTopicSerializer`
which calls `Topic#image_url` which requires us to preload
`Topic#topic_thumbnails`.
2022-12-23 04:29:49 +08:00
Krzysztof Kotlarek
a6edd6ea61
FIX: discourse-follow notification types ()
Bring information about discourse-follow notification types to Core

https://github.com/discourse/discourse-follow/blob/main/plugin.rb#L32
2022-12-19 15:57:35 +11:00
Bianca Nenciu
b80765f1f4
DEV: Remove enable_whispers site setting ()
* DEV: Remove enable_whispers site setting

Whispers are enabled as long as there is at least one group allowed to
whisper, see whispers_allowed_groups site setting.

* DEV: Always enable whispers for admins if at least one group is allowed.
2022-12-16 18:42:51 +02:00
Rafael dos Santos Silva
4edeb7d9eb
DEV: Remove use of deprecated methods () 2022-12-15 22:08:05 -03:00
Osama Sayegh
1c03d6f9b9
FEATURE: Send notifications to admins when new features are released ()
This commit adds a new notification that gets sent to admins when the site gets new features after an upgrade/deploy. Clicking on the notification takes the admin to the admin dashboard at `/admin` where they can see the new features under the "New Features" section.

Internal topic: t/87166.
2022-12-15 20:12:53 +03:00
Krzysztof Kotlarek
aa3a9b6fea
FEATURE: Differentiate notification type for directly vs indirectly watched topic ()
When user is watching category or tag (watching or watching first post) notifications are moved to other tab.

To achieve that and distinguish between post create to directly watched topics and indirectly watched topics, new notification type called `watching_category_or_tag` was introduced.
2022-12-14 10:22:26 +11:00
Rafael dos Santos Silva
0434de6cee
DEV: Compatibility with Ruby 3.2 () 2022-12-13 16:03:53 -03:00
Alan Guo Xiang Tan
0abd772098
DEV: Pass in site guardian for Plugin::Instance.register_site_categories_callback ()
The guardian is useful for plugins to determine if the callback should
do anything. A common use case is to not do anything in the callback if
the user is anonymous.
2022-12-13 07:49:13 +08:00
jbrw
1a48ea767e
DEV: Allow additional TopicList preloaded associations ()
This provides a means to allow additional associations to be preloaded when generating a TopicList.
2022-12-12 09:08:13 -05:00
Alan Guo Xiang Tan
332ac0f299
UX: Exclude child catgegories of muted category in similar to search ()
Follow-up to 207b764ea3
2022-12-12 06:31:46 +08:00
Kris
07e4b7550d
DEV: relocate :root CSS custom properties () 2022-12-09 14:33:15 -05:00
Martin Brennan
b2acc416e7
FIX: Server-side hashtag lookups of secure categories for a user ()
* FIX: Use Category.secured(guardian) for hashtag datasource

Follow up to comments in , changing the category
hashtag datasource to use Category.secured(guardian) instead
of Site.new(guardian).categories here since the latter does
more work for not much benefit, and the query time is the
same. Also eliminates some Hash -> Model back and forth
busywork. Add some more specs too.

* FIX: Server-side hashtag lookup cooking user loading

When we were using the PrettyText.options.currentUser
and parsing back and forth with JSON for the hashtag
lookups server-side, we had a bug where the user's
secure categories were not loaded since we never actually
loaded a User model from the database, only parsed it
from JSON.

This commit fixes the issue by instead using the
PretyText.options.userId and looking up the user directly
from the database when calling hashtag_lookup via the
PrettyText::Helpers code when cooking server-side. Added
the missing spec to check for this as well.
2022-12-09 10:34:25 +10:00
Rafael dos Santos Silva
9f3760cfd8
FEATURE: Allow category moderators to set auto deletion topic timers ()
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2022-12-08 17:14:43 -03:00
Alan Guo Xiang Tan
fde9e6bc25
DEV: Migrate sidebar site settings ()
This new site setting replaces the
`enable_experimental_sidebar_hamburger` and `enable_sidebar` site
settings as the sidebar feature exits the experimental phase.

Note that we're replacing this without depreciation since the previous
site setting was considered experimental.

Internal Ref: /t/86563
2022-12-08 09:44:29 +08:00
Krzysztof Kotlarek
e313190fdb
FEATURE: better UI to manage 2fa ()
In this PR, we introduced an option, that when all authenticators are disabled, but backup codes still exists, user can authenticate with those backup codes. This was reverted as this is not expected behavior.

https://github.com/discourse/discourse/pull/18982

Instead, when the last authenticator is deleted, backup codes should be deleted as well. Because this disables 2fa, user is asked to confirm that action by typing text.

In addition, UI for 2fa preferences was refreshed.
2022-12-08 09:41:22 +11:00
Alan Guo Xiang Tan
207b764ea3
UX: Exclude search ignored and user muted categories in similarity search ()
When finding the candidates for `Topic.similar_to`, we will now ignore
topics in categories where `Category#search_priority` has been set to
ignore and also topics in categories which the user has specifically
muted.

Internal Ref: /t/87132
2022-12-07 11:33:01 +08:00
Andrei Prigorshnev
a76d864c51
FEATURE: Show live user status on inline mentions on posts ()
Note that we don't have a database table and a model for post mentions yet, and I decided to implement it without adding one to avoid heavy data migrations. Still, we may want to add such a model later, that would be convenient, we have such a model for mentions in chat.

Note that status appears on all mentions on all posts in a topic except of the case when you just posted a new post, and it appeared on the bottom of the topic. On such posts, status won't be shown immediately for now (you'll need to reload the page to see the status). I'll take care of it in one of the following PRs.
2022-12-06 19:10:36 +04:00
Bianca Nenciu
e5a18dddac
FIX: Update match_count of screened IP address ()
When a screened IP address is matched because it is either blocked or
allowed it should update match_count. This did not work because it
tried to validate the IP address and it failed as it matched with
itself.
2022-12-06 13:09:38 +02:00
Martin Brennan
e6f9504dd6
FEATURE: Allow user_status scope for UserApiKey ()
Follow-up to 6357a3ce33
where we allowed a general API key scope for user status
GET/PUT/DELETE, this commit allows the same for the
UserApiKey system.
2022-12-06 08:56:03 +10:00
Alan Guo Xiang Tan
ff40c890ac
DEV: Correct order of triggering topic trashed and recovered event ()
Previously we would trigger the event before the `Topic#deleted_at`
column has been updated making it hard for plugins to correctly work
with the model when its new state has not been persisted in the
database.
2022-12-06 05:56:16 +08:00
Blake Erickson
738f1958d8
FIX: Only modify secured sidebar links on user promotion/demotion ()
* FIX: Only modify secured sidebar links on user promotion/demotion

If a user is created populate their sidebar with the default
categories/tags that they have access to.

If a user is promoted to admin populate any new categories/tags that
they now have access to.

If an admin is demoted remove any categories/tags that they no longer
have access to.

This will only apply for "secured" categories. For example if these are
the default sitebar categories:

- general
- site feedback
- staff

and a user only has these sidebar categories:

- general

when they are promoted to admin they will only receive the "staff"
category. As this is a default category they didn't previously have
access to.

* Add spec, remove tag logic on update

Change it so that if a user becomes unstaged it used the "add" method
instead of the "update" method because it is essentially following the
on_create path.

On admin promotion/demotion remove the logic for updating sidebar tags because
we don't currently have the tag equivalent like we do for User.secure_categories.

Added the test case for when a user is promoted to admin it should
receive *only* the new sidebar categories they didn't previously have
access to. Same for admin demotion.

* Add spec for suppress_secured_categories_from_admin site setting

* Update tags as well on admin promotion/demotion

* only update tags when they are enabled

* Use new SidebarSectionLinkUpdater

We now have a SidebarSectionLinkUpdater
that was introduced in: fb2507c6ce

* remove empty line
2022-12-05 11:39:10 -07:00
Alan Guo Xiang Tan
fb2507c6ce
DEV: Centralise logic for updating sidebar section links ()
The centralization helps in reducing code duplication in our code base
and more importantly, centralizing logic for guardian checks into a
single spot.
2022-12-01 09:32:35 +08:00
Osama Sayegh
3ff6f6a5e1
FIX: Exclude claimed reviewables from user menu ()
Users who can access the review queue can claim a pending reviewable(s) which means that the claimed reviewable(s) can only be handled by the user who claimed it. Currently, we show claimed reviewables in the user menu, but this can be annoying for other reviewers because they can't do anything about a reviewable claimed by someone. So this PR makes sure that we only show in the user menu reviewables that are claimed by nobody or claimed by the current user.

Internal topic: t/77235.
2022-12-01 07:09:57 +08:00
Osama Sayegh
23bd993164
FEATURE: Separate notification indicators for new PMs and reviewables ()
This PR adds separate notification indicators for PMs and reviewables that have arrived since the last time the user opened the notifications menu.

The PM indicator is the strongest one of all three indicators followed by the reviewable indicator and then finally the blue indicator. This means that if there's a new PM and a new reviewable, then the PM indicator will be shown.

Meta topic: https://meta.discourse.org/t/no-green-or-red-notification-bubbles/242783?u=osama.

Internal topic: t/82995.
2022-12-01 07:05:32 +08:00
Isaac Janzen
aea492df5e
DEV: Add link rel preload to theme-javascripts ()
* DEV: Add link rel preload to theme-javascripts
2022-11-30 12:43:01 -06:00
Alan Guo Xiang Tan
7c321d3aad
PERF: Update Group#user_count counter cache outside DB transaction ()
While load testing our user creation code path in production, we
identified that executing the DB statement to update the `Group#user_count` column within a
transaction is creating a bottleneck for us. This is because the
creation of a user and addition of the user to the relevant groups are
done in a transaction. When we execute the DB statement to update
`Group#user_count` for the relevant group, a row level lock is held
until the transaction completes. This row level lock acts like a global
lock when the server is creating users that will be added to the same
group in quick succession.

Instead of updating the counter cache within a transaction which the
default ActiveRecord `counter_cache` option does, we simply update the
counter cache outside of the committing transaction.

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2022-11-30 11:52:08 -03:00
Bianca Nenciu
3048d3d07d
FEATURE: Track API and user API requests ()
Adds stats for API and user API requests similar to regular page views.
This comes with a new report to visualize API requests per day like the
consolidated page views one.
2022-11-29 13:07:42 +02:00
Roman Rizzi
07a9163ea8
FEATURE: Deleting a user with their posts also deletes chat messages. ()
This commit introduce a new API for registering callbacks, which we'll execute when a user gets destroyed, and the `delete_posts` opt is true. The chat plugin registers one callback and queues a job to destroy every message from that user in batches.
2022-11-28 13:32:57 -03:00
Martin Brennan
bbcb69461f
FIX: Existing users were mistakenly unable to redeem invite ()
Follow up to 40e8912395

In this previous commit I introduced a bug that prevented
a legitimate case for an existing user to redeem an invite,
where the email/domain were both blank and the invite was
still redeemable by the user. Fixes the issue and adds more
specs for that case.
2022-11-25 11:57:04 +10:00
Andrei Prigorshnev
6357a3ce33
FEATURE: User Status API ()
This adds API scope for the user status. This also adds a get method to the user status controller. We didn't need a dedicated method that returns status before because the server returns status with user objects, but I think we need to provide this method for API clients.
2022-11-24 19:16:28 +04:00
Gerhard Schlager
9e42473af4
FEATURE: Allow setting default_enabled for badges ()
Otherwise setting `enabled` in `Badge.seed` will always overwrite changes by admins.
2022-11-24 13:44:26 +01:00
Daniel Waterworth
f895f27b02
FIX: When filtering tags for visibility, respect tag group permissions () 2022-11-22 12:55:57 -06:00
TheJammiestDodger
be99c3eec7
FIX: Amend release_notes_link in app/models/admin_dashboard_general_data.rb ()
Update `release_notes_link` to current version

<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in
JavaScript). If your code does not include test coverage, please include
an explanation of why it was omitted. -->
2022-11-21 16:21:34 +00:00
Martin Brennan
d3f02a1270
FEATURE: Generic hashtag autocomplete lookup and markdown cooking ()
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.

**Serverside**

We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.

Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.

The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.

When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.

**Markdown**

The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.

This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.

**Chat Channels**

This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.

------

**Known Rough Edges**

- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 08:37:06 +10:00
Jordan Vidrine
44e27ddab8
more () 2022-11-18 13:28:13 -06:00
Sam
4f63bc8ed2
FEATURE: hidden site setting to suppress unsecured categories from admins ()
The hidden site setting `suppress_secured_categories_from_admin` will
suppress visibility of categories without explicit access from admins
in a few key areas (category drop downs and topic lists)

It is not intended to be a security wall since admins can amend any site
setting. Instead it is feature that allows hiding the categories from the
UI.

Admins will still be able to see topics in categories without explicit
access using direct URLs or flags.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-11-18 14:37:36 +11:00
Martin Brennan
40e8912395
FIX: Invite redemption error if user had already redeemed ()
When opening the invite acceptance page when the user
was already logged in, we were still showing the Accept
Invitation prompt even if the user had already redeemed
the invitation and was present in the `InvitedUser` table.

This would lead to errors when the user clicked on the button.

This commit fixes the issue by hiding the Accept Invitation
button and showing an error message instead indicating that
the user had already redeemed the invitation. This only applies
to multi-use invite links.
2022-11-17 15:51:58 +10:00
Penar Musaraj
5ee3e2932f
FIX: Bug with admin trust level growth report ()
When this report in the admin dashboard has lots of data ( > 75 days of activity), the dates were ordered incorrectly. This is apparently expected behaviour; when using GROUP BY without specifying the ordering, PG decides to order, and it so happens that it works under some conditions but not others. Explicit ordering fixes the problem. 

However, because this works in some conditions but not others, we can't really add a useful test.
2022-11-16 22:11:09 -05:00
Jordan Vidrine
38d6e8c071
FIX: Add Custom Primary-Numbers () 2022-11-16 11:33:23 -06:00
Krzysztof Kotlarek
87b297e76c
FIX: deprioritize reaction notifications ()
Fix for https://github.com/discourse/discourse/pull/19029

Reactions should be moved down similarly to likes
2022-11-16 15:37:51 +11:00
Krzysztof Kotlarek
2e655f8311
FEATURE: deprioritize like notifications on all list ()
On the all notifications list, likes should be deprioritized and marked as read.
2022-11-16 13:32:05 +11:00
Bianca Nenciu
ac272c041e
FEATURE: Add user tips for post and topic features ()
* DEV: Add utility to hide all user tips

* DEV: Add UserTip Glimmer component

* DEV: Add tests for existing user tips

* FEATURE: Add user tip for post menu

* FEATURE: Add user tip for topic notification level

* FEATURE: Add user tip for suggested topics

* FEATURE: Hide new popups for existing users
2022-11-15 17:36:08 +02:00
Joffrey JAFFEUX
6493ddce17
PERF: limits use of redis cache while building emojis list ()
We were doing get on Redis two times for each emoji while building the custom/standard/all lists which where resulting in ~3710 Redis calls. Given the emoji DB file is loaded in memory while we build/cache the emojis list this is unnecessary and slow.

As a simplification in pseudo code here is an explanation of what we were doing:

```ruby
emojis.each |emoji_name|
  aliases = get_aliases_from_redis_cache(emoji_name)
  is_tonable = get_is_tonable_from_redis_cache(emoji_name)
  build_emoji(emoji_name, aliases, is_tonable)
end
```

The two redis calls are now simplified to a simple hash access: `@db[emoji_name]`
2022-11-14 13:38:50 +01:00
Martin Brennan
a414520742
SECURITY: Prevent email from being nil in InviteRedeemer ()
This commit adds some protections in InviteRedeemer to ensure that email
can never be nil, which could cause issues with inviting the invited
person to private topics since there was an incorrect inner join.

If the email is nil and the invite is scoped to an email, we just use
that invite.email unconditionally.  If a redeeming_user (an existing
user) is passed in when redeeming an email, we use their email to
override the passed in email.  Otherwise we just use the passed in
email.  We now raise an error after all this if the email is still nil.
This commit also adds some tests to catch the private topic fix, and
some general improvements and comments around the invite code.

This commit also includes a migration to delete TopicAllowedUser records
for users who were mistakenly added to topics as part of the invite
redemption process.
2022-11-14 12:02:06 +10:00
Jarek Radosz
dc8a7e74f4
FIX: Allow attr updates of over-size-limit uploads () 2022-11-11 17:56:11 +01:00
Krzysztof Kotlarek
4db5525d25
FIX: do not lock account if backup codes are available ()
Currently, we have available three 2fa methods:
- Token-Based Authenticators
- Physical Security Keys
- Two-Factor Backup Codes

If the first two are deleted, user lose visibility of their backup codes, which suggests that 2fa is disabled.

However, when they try to authenticate, the account is locked, and they have to ask admin to fix that problem.

This PR is fixing the issue. User still sees backup codes in their panel and can use them to authenticate.

In next PR, I will improve UI to clearly notify the user when 2fa is fully disabled and when it is still active.
2022-11-11 13:00:06 +11:00
Jarek Radosz
3e0196cbbf
FIX: Shrinking images where smaller image upload exists () 2022-11-10 12:43:56 +01:00
Bianca Nenciu
4dad7816b2
FEATURE: Rename onboarding popups to user tips ()
This commit also hides the new user tips for existing users.
2022-11-09 20:20:34 +02:00
Gerhard Schlager
b0ea919b09 FIX: Use cached RegEx instead of recreating RegEx on every usage 2022-11-09 14:27:55 +01:00
Gerhard Schlager
ba232eab53 DEV: Remove unused code 2022-11-09 14:27:55 +01:00
Martin Brennan
c6764d8c74
FIX: Automatically generate category channel slugs ()
This commit automatically ensures that category channels
have slugs when they are created or updated based on the
channel name, category name, or existing slug. The behaviour
has been copied from the Category model.

We also include a backfill here with a simplified version
of Slug.for with deduplication to fill the slugs for already
created Category chat channels.

The channel slug is also now used for chat notifications,
and for the UI and navigation for chat. `slugifyChannel`
is still used, but now does the following fallback:

* Uses channel.slug if it is present
* Uses channel.escapedTitle if it is present
* Uses channel.title if it is present

In future we may want to remove this altogether
and always rely on the slug being present, but this
is currently not possible because we are not generating
slugs for DM channels at this point.
2022-11-09 10:28:31 +10:00
Martin Brennan
f8f55cef67
DEV: TODO followups ()
* Remove old bookmark column ignores to follow up b22450c7a8
* Change some group site setting checks to use the _map helper
* Remove old secure_media helper stub for chat
* Change attr_accessor to attr_reader for preloaded_custom_fields to follow up 70af45055a
2022-11-09 07:48:05 +10:00
Blake Erickson
cb8746c7e7
FIX: Update sidebar links when promoted to admin ()
It is likely that a new admin user was created as just a regular user
before being promoted to admin so this change will update the sidebar
link records for any users that are promoted to admin. This way if any
of the default side bar categories or tags are restricted to admins
these new admins will have those added to their sidebar as well.

You can easily replicate this issue locally (prior to this fix) by using
`rails admin:create` where it creates a user first, then it is promoted
to admin. This means it would receive the default categories of regular
user, but never receive the ones they should have access to as an admin.

As part of this change I did drop the `!` from
`SidebarSectionLink.insert_all` so that it would add any new records
that were missing, but not throw a unique constraint error trying to add
any existing records.

Follow up to: 1b56a55f50

And: e320bbe513
2022-11-07 16:39:24 -07:00
Vinoth Kannan
dea44ec923
FEATURE: new site setting to hide user profiles by default. ()
Previously, we didn't have a site-wide setting to set the default behavior for user profile visibility and user presence features. But we already have a user preference for that.
2022-11-06 16:44:17 +05:30
Osama Sayegh
391a456443
FIX: Write to group logs when a user is added to group by invite () 2022-11-03 07:58:12 +09:00
Roman Rizzi
0a5f548635
DEV: Move discourse-chat to the core repo. ()
As part of this move, we are also renaming `discourse-chat` to `chat`.
2022-11-02 10:41:30 -03:00
Daniel Waterworth
167181f4b7
DEV: Quote values when constructing SQL ()
All of these cases should already be safe, but still good to quote for
"defense in depth".
2022-11-01 14:05:13 -05:00
David Taylor
07ef1a80a1
SECURITY: Fix invite link email validation ()
See https://github.com/discourse/discourse/security/advisories/GHSA-x8w7-rwmr-w278

Co-authored-by: Martin Brennan <martin@discourse.org>
2022-11-01 16:33:32 +00:00
David Taylor
68b4fe4cf8
SECURITY: Expand and improve SSRF Protections ()
See https://github.com/discourse/discourse/security/advisories/GHSA-rcc5-28r3-23rr

Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
2022-11-01 16:33:17 +00:00
David Taylor
695b44269b
DEV: Do not remove debugger statements from themes () 2022-11-01 13:56:33 +00:00
Osama Sayegh
e120c94236
FIX: Don't attempt to add user again to a group when syncing groups via SSO ()
This commit fixes a regression introduced in 8979adc where under certain conditions the groups syncing logic in Discourse Connect would try to add users to groups they're already members of and cause errors when users try to sign in using Discourse Connect.
2022-10-28 13:27:12 +03:00
Natalie Tay
4b6a5f08fa
FIX: Allow users already in automatic groups to log in () 2022-10-27 14:59:27 +08:00
Alan Guo Xiang Tan
1b56a55f50
DEV: Sidebar default tags and categories are determined at user creation ()
The previous sidebar default tags and categories implementation did not
allow for a user to configure their sidebar to have no categories or
tags. This commit changes how the defaults are applied. When a user is being created,
we create the SidebarSectionLink records based on the `default_sidebar_categories` and
`default_sidebar_tags` site settings. SidebarSectionLink records are
only created for categories and tags which the user has visibility on at
the point of user creation.

With this change, we're also adding the ability for admins to apply
changes to the `default_sidebar_categories` and `default_sidebar_tags`
site settings historically when changing their site setting. When a new
category/tag has been added to the default, the new category/tag will be
added to the sidebar for all users if the admin elects to apply the changes historically.
Like wise when a tag/category is removed, the tag/category will be
removed from the sidebar for all users if the admin elects to apply the
changes historically.

Internal Ref: /t/73500
2022-10-27 06:38:50 +08:00
Alan Guo Xiang Tan
a473e352de
DEV: Introduce TopicGuardian#can_see_topic_ids method ()
Before this commit, there was no way for us to efficiently check an
array of topics for which a user can see. Therefore, this commit
introduces the `TopicGuardian#can_see_topic_ids` method which accepts an
array of `Topic#id`s and filters out the ids which the user is not
allowed to see. The `TopicGuardian#can_see_topic_ids` method is meant to
maintain feature parity with `TopicGuardian#can_see_topic?` at all
times so a consistency check has been added in our tests to ensure that
`TopicGuardian#can_see_topic_ids` returns the same result as
`TopicGuardian#can_see_topic?`. In the near future, the plan is for us
to switch to `TopicGuardian#can_see_topic_ids` completely but I'm not
doing that in this commit as we have to be careful with the performance
impact of such a change.

This method is currently not being used in the current commit but will
be relied on in a subsequent commit.
2022-10-27 06:13:21 +08:00
Osama Sayegh
8979adc3af
FIX: Log user addition/deletion from groups when they're changed via DiscourseConnect ()
Discourse Connect can be used to manage group memberships of users by including a `add_groups`, `remove_groups` or `groups` attribute in the Discourse Connect payload. However, additions/deletions of users from groups aren't logged to the groups logs (available at `/g/<group>/manage/logs`) which can cause confusions to admins they try to figure out when/how users were added or removed from a group. This commit makes Discourse Connect add entries to the groups logs when it makes changes to users' group memberships.
2022-10-25 11:25:26 +03:00
Osama Sayegh
ce53152e53
DEV: Include theme_uploads and theme_uploads_local objects in theme tests ()
Our theme system injects a magical `settings` object at the top of themes JS modules to allow theme authors to access the settings as configured by admins in the UI. Within this `settings` object, there are a couple of special objects `theme_uploads` and `theme_uploads_local` that contain URLs for all the assets/uploads that the theme has.

For test modules/files, the theme system also injects a `settings` object at the top of tests modules, but it's not the same object as the object that's injected in non-test files. The difference is that in tests we want the settings to have their default values as opposed to any custom values that may exist in the site's database. This ensures that test results are consistent no matter the site that runs them.

However, the `settings` object in tests files currently doesn't have the special objects `theme_uploads` and `theme_uploads_local` which means that if a theme includes an asset that's lazy-loaded, it's not possible to write tests for anything that depends on the lazy-loaded asset because the theme will not be able to load the asset during the tests since `theme_uploads_local` and `theme_uploads` don't exist. This PR adds these special objects inside the `settings` object for test files.

Internal topic: t/71825/52.
2022-10-20 08:00:29 +03:00
Blake Erickson
505aec123f
FEATURE: Hide welcome topic if it hasn't been edited () 2022-10-19 20:01:36 -06:00
David Taylor
8d3fe3ddc5
DEV: Bump theme compiler version for connector fix ()
This should have been included in c185043590
2022-10-19 10:52:39 +01:00
Joffrey JAFFEUX
a705e4815f
FIX: do not include group less emojis in standard list () 2022-10-19 09:53:56 +02:00
David Taylor
be3d6a56ce
DEV: Introduce minification and source maps for Theme JS ()
Theme javascript is now minified using Terser, just like our core/plugin JS bundles. This reduces the amount of data sent over the network.

This commit also introduces sourcemaps for theme JS. Browser developer tools will now be able show each source file separately when browsing, and also in backtraces.

For theme test JS, the sourcemap is inlined for simplicity. Network load is not a concern for tests.
2022-10-18 18:20:10 +01:00
David Taylor
cb87067c77 DEV: Introduce support for template colocation in themes 2022-10-18 10:10:49 +01:00
David Taylor
65a5c84a92 DEV: perform theme extra_js compilation all together
Previously, compiling theme 'extra_js' was done with a number of steps. Each theme_field would be compiled into its own value_baked column, and then the JavascriptCache content would be built by concatenating all of those compiled values.

This commit streamlines things by removing the value_baked step. The raw value of all extra_js theme_fields are passed directly to the ThemeJavascriptCompiler, and then the result is stored in the JavascriptCache.

In itself, this commit should not cause any behavior change. It is designed to open the door to more advanced compilation features which have interdependencies between different source files (e.g. template colocation, sourcemaps).
2022-10-18 10:10:49 +01:00
Krzysztof Kotlarek
243efa8931
FEATURE: allow user to set preferred sidebar list destination ()
User can choose between latest or new/unread and that preference will affect behavior of sidebar links.
2022-10-18 13:21:52 +11:00
Joffrey JAFFEUX
3bd94b2794
DEV: includes search_aliases in emoji object ()
This is an information we need on client side for example when filtering emojis.
2022-10-15 07:09:00 -05:00
Martin Brennan
beaf009b4a
FIX: action_code_path not being loaded for user-stream-item ()
This commit fixes an issue where we had a typo in the
UserAction.stream query which meant that action_code_path
was not loaded correctly. Once that was fixed, we were also
not actually using the action_code_path in the user-stream-item,
so that has been fixed here too.

The bug this caused was that, when the link for the action was
clicked within the user-stream-item, the user would be redirected
to a URL ending with `[missing%20%%7Bpath%7D%20value]` because
the I18n call did not have the path present.
2022-10-13 19:10:18 +10:00
Blake Erickson
efb116d2bd
FIX: Reset related site settings on general category delete ()
* FIX: Reset related site settings on general category delete

If the new seeded General category is deleted we also need to delete the
corresponding site setting for it so that we don't try and reference it.

This fixes a bug in the category dropdown composer.

This change creates the `clear_related_site_settings` after destroy
hook that could also be used by other features in the future, like maybe
when we have a `default_category_id` site_setting.

Looks like if `nil` out a site setting it is set to `0`?

```
[9] pry(main)> SiteSetting.general_category_id = nil
  SiteSetting Load (0.4ms)  SELECT "site_settings".* FROM "site_settings" WHERE "site_settings"."name" = 'general_category_id' LIMIT 1
=> nil
[10] pry(main)> SiteSetting.general_category_id
=> 0
```

That is why the tests check if the value is `< 1` and not `nil`.

* Use -1 instead of nil because it is the default
2022-10-12 11:09:45 -06:00
Bianca Nenciu
7611fec0da
FEATURE: Implement new onboarding popups ()
This commit introduces a new framework for building user tutorials as
popups using the Tippy JS library. Currently, the new framework is used
to replace the old notification spotlight and tips and show a new one
related to the topic timeline.

All popups follow the same structure and have a title, a description and
two buttons for either dismissing just the current tip or all of them
at once.

The state of all seen popups is stored in a user option. Updating
skip_new_user_tips will automatically update the list of seen popups
accordingly.
2022-10-12 18:38:45 +03:00
David Taylor
76c86a4269
FIX: Correctly handle HTTP errors during dominant color calculation ()
The previous fix in e83d35d6 was incorrect, and the stub in the test was never actually hit. This commit moves the error handling to the right place and updates the specs to ensure the stub is always used.
2022-10-12 15:50:44 +01:00
Alan Guo Xiang Tan
a7bdd0a58e
PERF: Exclude anon sidebar tags in site serializer for logged in user ()
This commits excludes the `anonymous_default_sidebar_tags` property in `SiteSerializer` when user
is not anonymous and when tagging has been disabled.
2022-10-11 08:45:22 +08:00
Bianca Nenciu
08ab09c928
FIX: Remove public topic invite functionality ()
This can no longer be used from the user interface and could be used to
generate useless topic invites notifications. This commit adds site 
setting max_topic_invitations_per_minute to prevent invite spam.
2022-10-10 19:21:51 +03:00
Arpit Jalan
140200ae83
FIX: do not show welcome CTA banner if the welcome topic is deleted () 2022-10-10 16:53:19 +05:30
Jan Cernik
08476f17ff
FEATURE: Add dark mode option for category logos ()
Adds a new upload field for a second dark mode category logo. 
This alternative will be used when the browser is in dark mode (similar to the global site setting for a dark logo).
2022-10-07 11:00:44 -04:00
David Taylor
e83d35d6f3
FIX: Improve error handling for calculate_dominant_color! ()
These errors tend to indicate that the upload is missing on the remote store. This is bad, but we don't want it to block the dominant-color calculation process. This commit catches errors when there is an HTTP error, and fixes the `base_store.rb` implementation when `FileHelper.download` returns nil.
2022-10-06 13:44:53 +01:00
Krzysztof Kotlarek
d5f6262c4f
FIX: watched topic overcome muted category ()
Previously, when categories were not muted by default, we were sending message about unmuted topics (topics which user explicitly set notification level to watching)

The same mechanism can be used to fix a bug. When the user was explicitly watching topic, but category was muted, then the user was not informed about new reply.
2022-10-06 11:10:43 +11:00
Roman Rizzi
94aba90c56
DEV: Delete reviewable associations when deleting () 2022-10-05 13:38:41 -03:00
Joffrey JAFFEUX
fbb9f983b1
DEV: removes legacy code ()
No methods from https://api.rubyonrails.org/classes/ActionView/Helpers/SanitizeHelper.html seem to be used in models/topic.rb
2022-10-05 15:16:10 +02:00
Osama Sayegh
4d05e3edab
DEV: Include pending reviewables in the main tab in the user menu ()
This commit makes pending reviewables show up in the main tab (a.k.a. "all notifications" tab). Pending reviewables along with unread notifications are always shown first and they're sorted based on their creation date (most recent comes first).

The dismiss button currently only shows up if there are unread notifications and it doesn't dismiss pending reviewables. We may follow up with another change soon that allows makes the dismiss button work with reviewables and remove them from the list without taking any action on them. 

Follow-up to 079450c9e4.
2022-10-05 12:30:02 +03:00
Bianca Nenciu
35a90b6a3f
FIX: Add better and more strict invite validators ()
* FIX: Add validator for email xor domain

* FIX: Add validator for max_redemptions_allowed

* FIX: Add validator for redemption_count
2022-09-30 13:35:00 +03:00
Osama Sayegh
5a5625460b
DEV: Add group messages and group_message_summary notifications in the messages tab in the user menu ()
This commit adds non-archived group messages and `group_message_summary` notifications in the messages tab in the user menu. With this change, the messages tab in the user menu now includes 3 types of items:

1. Unread `private_message` notifications (notifications when you receive a reply in a PM)
2. Unread and read `group_message_summary` notifications (notifications when there's a new message in a group inbox that you track)
3. Non-archived personal and group messages

Unread `private_message` notifications are always shown first, followed by unread `group_message_summary` notifications, and then everything else (messages and read `group_message_summary` notifications) sorted by recency (most recent first).

Internal topic: t/72976.
2022-09-30 08:44:04 +03:00
Osama Sayegh
079450c9e4
DEV: Do not show handled reviewables in the user menu ()
Currently, the reviewables tab in the user menu shows pending reviewables at the top of the menu and fills the remaining space in the menu with old/handled reviewables. This PR makes the revieables tab show only pending reviewables and hides the tab altogether from the menu if there are no pending reviewables. We're going to follow-up with another change soon that will show pending reviewables in the main tab of the user menu.

Internal topic: t/73220.
2022-09-30 06:10:07 +03:00
Martin Brennan
8ebd5edd1e
DEV: Rename secure_media to secure_uploads ()
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.

This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.

Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.

This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.

Deprecated settings:

* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 09:24:33 +10:00
Bianca Nenciu
20a17f248c
FIX: Make sure featured category topics are unique ()
In the past, CategoryFeaturedTopic.feature_topics raised an exception
sometimes because it tried to create multiple CategoryFeaturedTopic
records for the same topic.

This code should not raise any exceptions as long as the list of new
topic IDs is unique because all previous records are deleted first,
then recreated and everything happens inside a transaction. The previous
rescue block was dead code anyway because it tried to catch
PG::UniqueViolation instead of ActiveRecord::RecordNotUnique. This
commit includes a fix to ensure that the topic IDs are unique.
2022-09-27 14:01:22 +03:00
Martin Brennan
e62e93f83a
FEATURE: Introduce personal_message_enabled_groups setting ()
This will replace `enable_personal_messages` and
`min_trust_to_send_messages`, this commit introduces
the setting `personal_message_enabled_groups`
and uses it in all places that `enable_personal_messages`
and `min_trust_to_send_messages` currently apply.

A migration is included to set `personal_message_enabled_groups`
based on the following rules:

* If `enable_personal_messages` was false, then set
  `personal_message_enabled_groups` to `3`, which is
  the staff auto group
* If `min_trust_to_send_messages` is not default (1)
  and the above condition is false, then set the
  `personal_message_enabled_groups` setting to
  the appropriate auto group based on the trust level
* Otherwise just set `personal_message_enabled_groups` to
  11 which is the TL1 auto group

After follow-up PRs to plugins using these old settings, we will be
able to drop the old settings from core, in the meantime I've added
 DEPRECATED notices to their descriptions and added them
to the deprecated site settings list.

This commit also introduces a `_map` shortcut method definition
for all `group_list` site settings, e.g. `SiteSetting.personal_message_enabled_groups`
also has `SiteSetting.personal_message_enabled_groups_map` available,
which automatically splits the setting by `|` and converts it into
an array of integers.
2022-09-26 13:58:40 +10:00
Martin Brennan
e3d495850d
FEATURE: Overhaul email threading ()
See https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499
for thorough reasoning.

This commit changes how we generate Message-IDs and do email
threading for emails sent from Discourse. The main changes are
as follows:

* Introduce an outbound_message_id column on Post that
  is either a) filled with a Discourse-generated Message-ID
  the first time that post is used for an outbound email
  or b) filled with an original Message-ID from an external
  mail client or service if the post was created from an
  incoming email.
* Change Discourse-generated Message-IDs to be more consistent
  and static, in the format `discourse/post/:post_id@:host`
* Do not send References or In-Reply-To headers for emails sent
  for the OP of topics.
* Make sure that In-Reply-To is filled with either a) the OP's
  Message-ID if the post is not a direct reply or b) the parent
  post's Message-ID
* Make sure that In-Reply-To has all referenced post's Message-IDs
* Make sure that References is filled with a chain of Message-IDs
  from the OP down to the parent post of the new post.

We also are keeping X-Discourse-Post-Id and X-Discourse-Topic-Id,
headers that we previously removed, for easier visual debugging
of outbound emails.

Finally, we backfill the `outbound_message_id` for posts that have
a linked `IncomingEmail` record, using the `message_id` of that record.
We do not need to do that for posts that don't have an incoming email
since they are backfilled at runtime if `outbound_message_id` is missing.
2022-09-26 09:14:24 +10:00
David Taylor
42947ec6f1
FIX: Handle failed download when calculating image dominant color ()
This can happen when the upload size exceeds the maximum upload size, or there is a network issue during download
2022-09-23 12:42:07 +01:00
Loïc Guitaut
26fe047724 DEV: Use AR enums in reviewables related code
This is the first patch of many to replace our custom enums in Ruby by
the ones provided by `ActiveRecord`.
2022-09-22 14:44:27 +02:00
David Taylor
0f5a8cc526
DEV: Enforce dominant_color length in validation ()
The `add_column` `limit` parameter has no effect on a postgres `text` column. Instead we can perform the check in ActiveRecord.

We never expect this condition to be hit - users cannot control this value. It's just a safety net.
2022-09-21 11:01:21 +01:00
Martin Brennan
e69f7d2fd9
SECURITY: Limit user profile field length ()
Adds limits to location and website fields at model and DB level
to match the bio_raw field limits. A limit cannot be added at the
DB level for bio_raw because it is a postgres text field.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-09-21 12:07:06 +10:00
David Taylor
d0243f741e
UX: Use dominant color as image loading placeholder ()
We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images.

This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly.

Dominant color will be calculated:
1. When a new upload is created
2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored
3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job)

Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 10:28:17 +01:00
Joffrey JAFFEUX
09a434e2d8
DEV: emoji model should set the group of the emoji ()
We do it for custom emojis but not for standard emojis which is quite unexpected and limit the use cases we can have for it.
2022-09-14 13:10:48 +02:00
Osama Sayegh
1fa21ed415
DEV: Prioritize unread notifications in the experimental user menu ()
Right now the experimental user menu sorts notifications the same way that the old menu does: unread high-priority notifications are shown first in reverse-chronological order followed by everything else also in reverse-chronological order. However, since the experimental user menu has dedicated tabs for some notification types and each tab displays a badge with the count of unread notifications in the tab, we feel like it makes sense to change how notifications are sorted in the experimental user menu to this:

1. unread high-priority notifications
2. unread regular notifications
3. all read notifications (both high-priority and regular)
4. within each group, notifications are sorted in reverse-chronological order (i.e. newest is shown first).

This new sorting logic applies to all tabs in the experimental user menu, however it doesn't change anything in the old menu. With this change, if a tab in the experimental user menu shows an unread notification badge for a really old notification, it will be surfaced to the top and prevents confusing scenarios where a user sees an unread notification badge on a tab, but the tab doesn't show the unread notification because it's too old to make it to the list.

Internal topic: t72199.
2022-09-12 21:19:25 +03:00
David Taylor
f4e1d0c546
FIX: Ensure <script> handlebars templates are namespaced correctly ()
This regressed in 7e74dd0afe
2022-09-04 13:01:10 +02:00
Gerhard Schlager
fd6109a6e1
FIX: Theme components should work with empty locale files ()
Theme components didn't work with empty locale files (just the locale key without any translations) that are created by translator-bot.
2022-09-02 18:28:18 +02:00
David Taylor
7e74dd0afe
DEV: Use DiscourseJsProcessor for theme template compilation ()
Previously we were relying on a highly-customized version of the unmaintained Barber gem for theme template compilation. This commit switches us to use our own DiscourseJsProcessor, which makes use of more modern patterns and will be easier to maintain going forward.

In summary:
- Refactors DiscourseJsProcessor to move multiline JS heredocs into a companion `discourse-js-processor.js` file
- Use MiniRacer's `.call` method to avoid manually escaping JS strings
- Move Theme template AST transformers into DiscourseJsProcessor, and formalise interface for extending RawHandlebars AST transformations
- Update Ember template compilation to use a babel-based approach, just like Ember CLI. This gives each template its own ES6 module rather than directly assigning `Ember.TEMPLATES` values
- Improve testing of template compilation (and move some tests from `theme_javascript_compiler_spec.rb` to `discourse_js_processor_spec.rb`
2022-09-01 11:50:46 +01:00
Bianca Nenciu
5092c9804c
FIX: Return next bookmarks page only if it exists ()
It used to return the next URL anyway which lead to an additional
request. On the frontend, if the result set was empty, it kept retrying
until at least one result was returned. This bug is fixed in this commit
too.
2022-09-01 13:04:00 +03:00
Arpit Jalan
3aaf4dcfd0
FIX: hide welcome topic banner as soon as the welcome topic is edited ()
* FIX: hide welcome topic banner as soon as the welcome topic is edited

This commit adds a message bus listener on client to hide the welcome
topic banner as soon as the welcome topic is edited.

* update test

* only subscribe when show_welcome_topic_banner is true

* Do not lookup for messageBus service if it's not required

* Remove unneeded code

* Cache result for Site.show_welcome_topic_banner

* Update tests per latest changes

* Changes per PR review
2022-09-01 09:44:21 +05:30
Alan Guo Xiang Tan
0f0048e8e3
DEV: Enable new user menu when experimental sidebar hamburger is enabled ()
When `enable_experimental_sidebar_hamburger` site setting is enabled, we
will switch to rendering the new user menu.
2022-08-31 21:15:01 +03:00
Krzysztof Kotlarek
de8cd19438
FEATURE: unified user menu notifications count ()
Each new user menu notifications should have their own count. Therefore, we need to include all types to serializer and not only `grouped_unread_high_priority_notifications`

Additional PR will be created for chat and assign plugin, as they will have to switch to  `grouped_unread_notifications` as well.
2022-08-31 11:16:28 +10:00
David Taylor
6570fed175
DEV: Fix Ember version lookup ()
The source-of-truth for our ember version is now the installed node_module. The `ember_source` gem carries an old version of Ember and so the constant is no longer useful. We'll be dropping the gem soon.
2022-08-30 16:27:14 +01:00
David Taylor
e16c8ea2e7
DEV: Support inline-hbs compilation in themes ()
This commit makes a number of improvements to the DiscourseJsProcessor:

1. Remove dependence on the out-of-date Ember template compiler from the ember-rails gem; switch to modern template compiler
2. Refactor to make use of a proper module system with `define`/`require`
3. Introduce `babel-plugin-ember-template-compilation` to enable inline hbs compilation

The `mini-loader` is upgraded to support relative lookup and `require.has`, so that these new JS packages work correctly.
2022-08-29 19:53:42 +01:00
Bianca Nenciu
446eb40bb6
FIX: Do not invite whisper or small action posters ()
If a topic is converted to a private message, all posters were invited
to the new private message. This included users who only whispered or
posted small actions.
2022-08-29 15:52:54 +03:00
Bianca Nenciu
33ea8b4756
FIX: Skip topic allowed user for small actions ()
Topic allowed user records were created for small actions, which lead to
the system user being invited in many private topics when the user
removed themselves or if a group was invited but some members already
had access.

This commits skips creating topic allowed user. They are already skipped
for the whisper posts.
2022-08-29 13:01:16 +03:00
Bianca Nenciu
0d8ecab362
FIX: Restore trust level when leaving group ()
If a user was granted a trust level, joined a group that granted a trust
level and left the group, the trust level was reset. This commit tries
to restore the last known trust level before joining the group by
looking into staff logs.

This commit also migrates old :change_trust_level user history records
to use previous_value and new_value fields.
2022-08-29 13:00:48 +03:00
David Taylor
e7a84948b9
FIX: Avoid duplicate topic-list requests ()
When preloading topic_list data we were giving it a 'preload key' which was loosely based on the parameters of the list. However, it did not include all parameters, and mismatches between client/server-side logic would cause the preloaded data to be ignored.

This commit simplifies things by using a single key for all topic_list preloading. This works on the assumption that "The first topic_list the JS app will load is the one which was preloaded". That assumption also existed to some extent in the old design, so we don't expect any regressions here.
2022-08-24 11:54:01 +01:00
Krzysztof Kotlarek
2d58996a3b
FEATURE: anonymous sidebar categories and tags ()
Default sidebar tags for not authenticated users can be defined in admin panel. Otherwise, top 5 categories and tags are taken.

Optionally, if categories are set up in permanent order, then the first 5 categories are taken.
2022-08-23 16:20:46 +08:00
Alan Guo Xiang Tan
1d1a7db182
DEV: Fix flaky spec due to ordering of Array intersection ()
```
  1) CurrentUserSerializer#sidebar_category_ids includes visible default sidebar categories
     Failure/Error: expect(json[:sidebar_category_ids]).to eq([category.id, category_2.id])

       expected: [378, 379]
            got: [379, 378]
```

Note that in the Ruby doc it says "The order is preserved from the original array". In this case, we want to preserve the order of the site setting.
2022-08-23 16:20:10 +08:00
Krzysztof Kotlarek
caab916569
FEATURE: default sidebar categories and tags ()
Ability to set up default sidebar categories and tags for authenticated users
2022-08-22 08:46:20 +10:00
Isaac Janzen
48e2b984fb
DEV: Filter topics by tag for categories ()
* DEV: Filter topics by tag for categories and children
2022-08-17 11:51:02 -05:00
Roman Rizzi
1434fe3021
FIX: Recover from guardian check when deleting reviewable users. ()
Handles edge-case when a user is an admin and has an associated reviewable. Hitting this exception should be rare since we clear the reviewable when
granting staff to the user.
2022-08-16 11:50:06 -03:00
Bianca Nenciu
4b70594173
FIX: Reset flair group if user is removed from group ()
The flair used to stay set even if the user was removed from the group.
2022-08-12 15:45:09 +03:00
Roman Rizzi
58b135d6d3
DEV:Using symbols is deprecated, use strings instead () 2022-08-11 11:33:41 -03:00
David Taylor
d6bba1ea9d
DEV: Invalidate theme cache when S3 configuration changes ()
Compiled themes often include upload URLs which will need to be re-calculated
2022-08-11 11:03:57 +01:00
Bianca Nenciu
e029a9b36c
FEATURE: Allow private themes to be partially installed ()
A public key must be added to GitHub when installing private themes.
When the process happens asynchronously (for example if the admin does
not have admin permissions to the GitHub repository), installing
private themes becomes very difficult.

In this case, the Discourse admin can partially install the theme by
letting Discourse save the private key, create a placeholder theme and
give the admin a public key to be used as a deploy key. After the key
is installed, the admin can finish theme installation by pressing a
button on the theme page.
2022-08-10 13:30:18 +03:00
Krzysztof Kotlarek
cc84ea2444 SECURITY: Limit email invitations to topic 2022-08-10 11:33:16 +02:00
Bianca Nenciu
a0537816fb
FIX: Destroy all posts when hard deleting topic ()
Hard deleting topics that contained soft deleted posts or small actions
used to create orphan posts because only the first post was hard
deleted. This commit adds an error message if there are still posts left
in the topic that must be hard deleted first or hard deletes all small
actions too immediately (there is no other way of hard deleting a small
action because there is no wrench menu).
2022-08-10 12:11:50 +03:00
Osama Sayegh
473695ee4d
DEV: Add messages tab to the new user menu ()
Some of the changes in this PR are extracted from https://github.com/discourse/discourse/pull/17379.

Similar to the bookmarks tab in the new user menu, the messages tab also displays a mix of notifications and messages. When there are unread message notifications, the tab displays all of these notifications at the top and fills the remaining space in the menu with a list of the user's messages. The bubble/badge count on the messages tab indicates how many unread message notifications there are.
2022-08-10 08:25:39 +03:00
Arpit Jalan
d57bea4de3
FEATURE: add welcome topic cta banner () 2022-08-09 21:52:39 +05:30
David Taylor
47824bc69f
DEV: Bump asset compiler versions ()
This will trigger recompilation of theme/plugin assets following 7f9c2c0b
2022-08-09 12:06:15 +01:00
David Taylor
3c81683955 DEV: Rename UriHelper.escape_uri to .normalized_encode
This is a much better description of its function. It performs idempotent normalization of a URL. If consumers truly need to `encode` a URL (including double-encoding of existing encoded entities), they can use the existing `.encode` method.
2022-08-09 11:55:25 +01:00
Andrei Prigorshnev
3755bad03c
DEV: return user status on the user search route () 2022-08-09 14:54:33 +04:00
Osama Sayegh
4fdb275683
DEV: Add bookmarks tab to the new user menu ()
Some of the changes in this commit are extracted from https://github.com/discourse/discourse/pull/17379.

The bookmarks tab in the new user menu is different from the other tabs in that it can display a mixture of notifications and bookmarks. When there are unread bookmark reminder notifications, the tab displays all of these notifications at the top and fills the remaining space in the menu with the rest of the bookmarks. The bubble/badge count on the bookmarks tab indicates how many unread bookmark reminder notifications there are.

On the technical aspect, since this commit introduces a new `bookmark-item` component, we've done some refactoring so that all 3 "item" components (`notification-item`, `reviewable-item` and the new `bookmark-item`) inherit from a base component and get identical HTML structure so they all look consistent.

Internal tickets: t70584 and t65045.
2022-08-08 17:24:04 +03:00
Loïc Guitaut
5c37a5d0f2
FIX: Allow to add the same watched word with a different case ()
Currently we can’t add a case-sensitive watched word if another one
exists with a different case. For example, the existing watched word
`Meta` has been created and is case-sensitive. Now an admin tries to add
`metA` while marking it as case-sensitive too, this won’t work and the
word won’t be added.

This patch changes this behavior by allowing to add same words that have
different cases, so the example above will now work as expected.

We still check for uniqueness but case-sensitivy is now taken
into account. It means that if the watched word `meta` already exists
and is not case-sensitive then it will not be possible to add `Meta`
(case-sensitive or not) as `meta` already matches every possible
variations of this word.
2022-08-05 12:18:17 +02:00
Arpit Jalan
d600c36036
FIX: do not redeem expired invites on new user signup () 2022-08-05 07:50:48 +05:30
Osama Sayegh
5c2e909543
DEV: Don't publish to the /reviewable_counts channel ()
Follow-up to ce9eec8606.

I did a last-minute refactoring before merging the commit above where I extracted the Message Bus publish call into a new method, but forgot to delete the publish call after adding a call to the new method.
2022-08-03 18:23:43 +03:00
Osama Sayegh
062a5f316d
DEV: Wire up the dimiss button in the new notifications menu ()
This commit is a subset of the changes proposed in https://github.com/discourse/discourse/pull/17379.
2022-08-03 15:32:35 +03:00
Osama Sayegh
ce9eec8606
DEV: Combine all header notification bubbles into one in the new user menu ()
Extracted from https://github.com/discourse/discourse/pull/17379.
2022-08-03 08:57:59 +03:00
Gerhard Schlager
f3b2ee8e1b
FIX: Use default locale for footer of embedded topics ()
The content from the remote site and the footer get cached for 10 minutes, so Discourse should use the default locale instead of the user locale for the footer. Otherwise Discourse might cache the message in a different language.
2022-08-02 20:49:28 +02:00
Selase Krakani
862007fb18
FEATURE: Add support for case-sensitive Watched Words ()
* FEATURE: Add case-sensitivity flag to watched_words

Currently, all watched words are matched case-insensitively. This flag
allows a watched word to be flagged for case-sensitive matching.
To allow allow for backwards compatibility the flag is set to false by
default.

* FEATURE: Support case-sensitive creation of Watched Words via API

Extend admin creation and upload of Watched Words to support case
sensitive flag. This lays the ground work for supporting
case-insensitive matching of Watched Words.

Support for an extra column has also been introduced for the Watched
Words upload CSV file. The new column structure is as follows:

 word,replacement,case_sentive

* FEATURE: Enable case-sensitive matching of Watched Words

WordWatcher's word_matcher_regexp now returns a list of regular
expressions instead of one case-insensitive regular expression.

With the ability to flag a Watched Word as case-sensitive, an action
can have words of both sensitivities.This makes the use of the global
Regexp::IGNORECASE flag added to all words problematic.

To get around platform limitations around the use of subexpression level
switches/flags, a list of regular expressions is returned instead, one for each
case sensitivity.

Word matching has also been updated to use this list of regular expressions
instead of one.

* FEATURE: Use case-sensitive regular expressions for Watched Words

Update Watched Words regular expressions matching and processing to handle
the extra metadata which comes along with the introduction of
case-sensitive Watched Words.

This allows case-sensitive Watched Words to matched as such.

* DEV: Simplify type casting of case-sensitive flag from uploads

Use builtin semantics instead of a custom method for converting
string case flags in uploaded Watched Words to boolean.

* UX: Add case-sensitivity details to Admin Watched Words UI

Update Watched Word form to include a toggle for case-sensitivity.
This also adds support for, case-sensitive testing and matching of  Watched Word
in the admin UI.

* DEV: Code improvements from review feedback

 - Extract watched word regex creation out to a utility function
 - Make JS array presence check more explicit and readable

* DEV: Extract Watched Word regex creation to utility function

Clean-up work from review feedback. Reduce code duplication.

* DEV: Rename word_matcher_regexp to word_matcher_regexp_list

Since a list is returned now instead of a single regular expression,
change `word_matcher_regexp` to `word_matcher_regexp_list` to better communicate
this change.

* DEV:  Incorporate WordWatcher updates from upstream

Resolve conflicts and ensure apply_to_text does not remove non-word characters in matches
that aren't at the beginning of the line.
2022-08-02 10:06:03 +02:00
David Taylor
9d753cb89e
FIX: Ensure theme_uploads_local only has one / at beginning ()
Followup to c7dfb1c549
2022-07-28 22:20:52 +01:00
David Taylor
c7dfb1c549
DEV: Use relative URLs for theme_uploads_local ()
Relative URLs will work just fine for Web Workers, which were the original reason for introducing the `theme_uploads_local` feature

Making them relative will mean that `loadScript()` automatically uses the CDN (when enabled), which is doubly important because our CSP doesn't allow loading theme-javascripts from the host domain when the CDN is enabled.
2022-07-28 20:38:22 +01:00
Osama Sayegh
988a175e94
DEV: Add reviewables tab to the new user menu ()
This commit is a subset of the changes proposed in https://github.com/discourse/discourse/pull/17379.
2022-07-28 11:16:33 +03:00
Roman Rizzi
f1c3670d74
FIX: Publish membership update events when refreshing automatic groups. ()
Adding or removing users from automatic groups is now consistent with `Group#add` and `Group#remove`.
2022-07-27 11:34:08 -03:00
Alan Guo Xiang Tan
dcf84fce7b
PERF: Add index for TopicTimer#topic_id ()
When viewing a topic, we execute two queries to fetch the topic's
public topic timer and slow mode timer. The former query happens to be
able to use a unique index but the latter has to do a seq scan which is
slow. The query itself is not expensive but since viewing a topic is a
hot path, the little cuts add up overtime and the query itself
contributes significantly to the load of the database.
2022-07-27 16:21:11 +08:00
Alan Guo Xiang Tan
3bd5f2d411
DEV: Introduce SiteSetting to enable/disable Sidebar. ()
This commit removes the ability to enable/disable the Sidebar on a per
user basis and introduces a site wide setting. For testing purposes, sidebar can be enabled/disabled via the `enable_sidebar=1` or `enable_sidebar=0` query param.
2022-07-27 13:42:26 +08:00
Vinoth Kannan
90c969c90f
DEV: remove automatic_membership_retroactive column from group model. ()
The column is already dropped from the database table.
2022-07-26 21:26:22 +05:30
Bianca Nenciu
5f13ca5e54
FIX: Don't cook user fields to apply watched words ()
The previous method for reused the PrettyText logic which applied the
watched word logic, but had the unwanted effect of cooking the text too.
This meant that regular text values were converted to HTML.

Follow up to commit 5a4c35f627.
2022-07-26 18:15:42 +03:00
Roman Rizzi
5a0480efd7
DEV: Trigger an event when a user is granted admin or moderation. () 2022-07-26 10:20:34 -03:00
Jarek Radosz
0d3cf3333a
DEV: Add secondary sort to Group.visible_groups ()
Fix a flaky group controller spec and makes the API responses more stable.
2022-07-26 02:13:38 +02:00
Jean
424a274c12
FEATURE: add categories page style to order topics by created date () 2022-07-25 09:41:43 -04:00
Osama Sayegh
9103081eb7
DEV: Add likes, mentions and replies tabs to the new user menu ()
This commit is a subset of the changes proposed in https://github.com/discourse/discourse/pull/17379.
2022-07-25 15:19:53 +03:00
Osama Sayegh
306dca3a48
DEV: Add a helper method to clear every possible theme cache ()
Our theme system is very complex and it can take a while to figure out how to invalidate the various types of caches that are used throughout the theme system. So, having a single helper method that invalidates everything can be useful in emergency situations where there is no time to read through the code and figure out how to clear the various caches.

Internal ticket: t64732.
2022-07-22 09:46:52 +03:00
Alan Guo Xiang Tan
78427e0797
DEV: Refactor user_badge_granted DiscourseEvent logic ()
Follow-up to 02ce9b8a62
2022-07-22 09:06:02 +08:00
Joffrey JAFFEUX
02ce9b8a62
FIX: badge backfilling triggers :user_badge_granted DiscourseEvent () 2022-07-20 09:33:07 +08:00
Bianca Nenciu
a842b71a05
FIX: Do not redeem invites if user resets password ()
The invites should be redeemed during the signup process. This was a
problem because when user tried to redeem an admin invite it tried to
authenticate the user using information from the session that was not
available.
2022-07-19 22:25:01 +03:00
David Taylor
a4c4d3cb50 DEV: Patch Ruby Ember template compiler to work with recent ember 2022-07-19 10:00:59 +01:00
Leonardo Mosquera
aad1e76150
PERF: avoid extra object created on each User#validatable_user_fields call () 2022-07-18 12:35:47 -03:00
David Taylor
fab1c00c8f
DEV: Drop the deprecated themeSettings.blah syntax ()
This syntax has been printing deprecation messages since 880311dd4d
2022-07-18 10:10:23 +01:00
Martin Brennan
098ab29d41
FEATURE: Add plugin API to register About stat group ()
This commit introduces a new plugin API to register
a group of stats that will be included in about.json
and also conditionally in the site about UI at /about.

The usage is like this:

```ruby
register_about_stat_group("chat_messages", show_in_ui: true) do
  {
    last_day: 1,
    "7_days" => 10,
    "30_days" => 100,
    count: 1000,
    previous_30_days: 120
  }
end
```

In reality the stats will be generated any way the implementer
chooses within the plugin. The `last_day`, `7_days`, `30_days,` and `count`
keys must be present but apart from that additional stats may be added.
Only those core 4 stat keys will be shown in the UI, but everything will be shown
in about.json.

The stat group name is used to prefix the stats in about.json like so:

```json
"chat_messages_last_day": 2322,
"chat_messages_7_days": 2322,
"chat_messages_30_days": 2322,
"chat_messages_count": 2322,
```

The `show_in_ui` option (default false) is used to determine whether the
group of stats is shown on the site About page in the Site Statistics
table. Some stats may be needed purely for reporting purposes and thus
do not need to be shown in the UI to admins/users. An extension to the Site
serializer, `displayed_about_plugin_stat_groups`, has been added so this
can be inspected on the client-side.
2022-07-15 13:16:00 +10:00
Leonardo Mosquera
40222eb524
FIX: bug with multiselect user field validation ()
* FIX: properly validate multiselect user fields on user creation

* Add test cases

* FIX: don't check multiselect user fields for watched words

* Clarifiy/simplify tests

* Roll back apply_watched_words changes

Since this method no longer needs to deal with arrays for now. If/when
we add new user fields which uses them, we can deal with it then.
2022-07-14 19:36:54 -03:00
David Taylor
92e8a6d62b
DEV: Key the Theme DistributedCache on BASE_COMPILER_VERSION ()
All `DistributedCache` instances in Discourse are automatically keyed on the `Discourse.git_version`. Normally the theme compiler version is updated via a commit, and everything is fine. However, in some situations, it's possible for the BASE_COMPILER_VERSION to change without a change to the git_version (e.g. when applying patches directly to the codebase).

This commit adds the `BASE_COMPILER_VERSION` to the DistributedCache key to ensure that content from different compiler versions does not leak into other processes.
2022-07-14 12:43:55 +01:00
Alan Guo Xiang Tan
845642ac46
DEV: Add per-user feature flag for the new notifications menu ()
Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
2022-07-14 14:30:46 +08:00
Alan Guo Xiang Tan
92e1e43104
FIX: Improve reliability of topic tracking state ()
The `unread_not_too_old` attribute is a little odd because there should never be a case where
the user's first_unread_at column is less than the `Topic#updated_at`
column of an unread topic. The `unread_not_too_old` attribute is causing
a bug where topic states synced into `TopicTrackingState` do not appear
as unread because the attribute does not exsist on a normal `Topic`
object and hence never set.
2022-07-14 13:44:58 +08:00
Martin Brennan
6b2ea1b47b
FIX: Change UserCommScreener to use user_ids ()
It makes more sense to use user_ids for the UserCommScreener
introduced in fa5f3e228c since
in most cases the ID will be available, not the username. This
was discovered while starting work on a plugin that will
use this. In the cases where only usernames are available
the extra query is negligble.
2022-07-14 15:23:09 +10:00
Alan Guo Xiang Tan
ee89c68687
DEV: Unify way to reset Site.preloaded_category_custom_fields () 2022-07-14 08:54:31 +08:00
Martin Brennan
fa5f3e228c
DEV: Refactor user mute/ignore/disallow PM checks into central class ()
The idea behind this refactor is to centralise all of the user ignoring / muting / disallow PM checks in a single place, so they can be used consistently in core as well as for plugins like chat, while improving the main bulk of the checks to run in a single fast non-AR query.

Also fixed up the invite error when someone is muting/ignoring the user that is trying to invite them to the topic.
2022-07-13 13:58:01 +10:00
Loïc Guitaut
5a4c35f627 FIX: Apply all watched words rules to user fields
Currently we only apply watched words of the `Block` type to custom user
fields and user profile fields.

This patch enables all rules to be applied such as `Censor` or
`Replace`.
2022-07-11 11:51:57 +02:00
Jordan Vidrine
fec3df60cb
DEV: Add more color schemes () 2022-07-08 16:46:32 -05:00
Andrei Prigorshnev
ef923f1bb1
FEATURE: Publish everyone's status to everyone () 2022-07-07 17:37:05 +04:00
Andrei Prigorshnev
c59f1729a6
FEATURE: auto remove user status after predefined period () 2022-07-05 19:12:22 +04:00
Alan Guo Xiang Tan
3266350e80
FEATURE: Decouple category/tag presence in sidebar from notifi level () 2022-06-30 14:54:20 +08:00
Krzysztof Kotlarek
09932738e5
FEATURE: whispers available for groups ()
Before, whispers were only available for staff members.

Config has been changed to allow to configure privileged groups with access to whispers. Post migration was added to move from the old setting into the new one.

I considered having a boolean column `whisperer` on user model similar to `admin/moderator` for performance reason. Finally, I decided to keep looking for groups as queries are only done for current user and didn't notice any N+1 queries.
2022-06-30 10:18:12 +10:00
Sam
6ecfdc8f55
FIX: keep unique post checks separate for PMs vs topics ()
This allows for people to use PMs for drafting and then post them on topics.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-06-29 15:35:07 +10:00
Sérgio Saquetim
5840fb5c62
UX: Update topics stats automatically ()
Updates automatically data on the stats section of the topic.

It will update automatically the following information: likes, replies and last reply (timestamp and user)
2022-06-27 18:21:05 -03:00
Rafael dos Santos Silva
f56c44d1c7
FEATURE: Validate tags in WatchedWords ()
* FEATURE: Validate tags in WatchedWords

We didn't validate watched words automatic tagging, so it was possible
for an admin to created watched words with an empty tag list which would
result in an exception when users tried to create a new topic that
matched the misconfigured watched word.

Bug report: https://meta.discourse.org/t/lib-topic-creator-fails-when-the-word-math-appears-in-the-topic-title-or-text/231018?u=falco
2022-06-27 16:16:33 -03:00
Martin Brennan
a176b57be0
FIX: Use bookmarkable pattern for bookmark cleanup ()
We have a `cleanup!` class method on bookmarks that deletes
bookmarks X days after their related record (post/topic) are
deleted. This commit changes this method to use the
registered_bookmarkables for this instead, and each bookmarkable
type can delete related bookmarks in their own way.
2022-06-23 14:09:39 +10:00
Andrei Prigorshnev
033f72c65f
FEATURE: user status emoji () 2022-06-22 18:15:33 +04:00
Roman Rizzi
e0ba35350e
FEATURE: Custom unsubscribe options ()
With this change, plugins can create custom unsubscribe keys, extend the unsubscribe view with custom preferences, and decide how they are updated.
2022-06-21 15:49:47 -03:00
Arpit Jalan
59f0815c9d
FIX: tag groups were not visible to allowed group members ()
https://meta.discourse.org/t/tags-in-tag-groups-not-visible-to-assigned-user-group/230025/
2022-06-21 23:40:54 +05:30
Alan Guo Xiang Tan
115859964d SECURITY: Validate email constraints when trying to redeem an invite
In certain situations, a logged in user can redeem an invite with an email that
either doesn't match the invite's email or does not adhere to the email domain
restriction of an invite link. The impact of this flaw is aggrevated
when the invite has been configured to add the user that accepts the
invite into restricted groups.
2022-06-21 11:32:27 +01:00
Osama Sayegh
5176c689e9
UX: Change wording for 'regular' categories to 'normal' ()
At some point in the past we decided to rename the 'regular' notification state of topics/categories to 'normal'. However, some UI copy was missed when the initial renaming was done so this commit changes the spots that were missed to the new name.
2022-06-20 06:49:33 +03:00
Joe
804b8fd9f9
DEV: Defer loading core/plugin/theme JS files ()
This is pre-request work to introduce a splash screen while site assets load.

The only change this commit introduces is that it ensures we add the defer attribute to core/plugin/theme .JS files. This will allow us to insert markup before the browser starts evaluating those scripts later on. It has no visual or functional impact on core.

This will not have any impact on how themes and plugins work. The only exception is themes loading external scripts in the </head> theme field directly via script tags. Everything will work the same but those would need to add the defer attribute if they want to keep the benefits introduced in this PR.
2022-06-20 09:47:37 +08:00
David Taylor
6695d148d6
DEV: Cleanup topic thumbnail enqueuing ()
- Ensure only one job is enqueued when there are no extra sizes requested by themes
- Update and un-skip relevant tests
2022-06-17 11:07:32 +01:00
Alan Guo Xiang Tan
f618fdf17f
Revert "DEV: Centralize user updates to a single MessageBus channel. ()" ()
This reverts commit 94c3bbc2d1.

At this current point in time, we do not have enough data on whether
this centralisation is the trade-offs of coupling features into a single
channel.
2022-06-17 12:24:15 +08:00
Roman Rizzi
e5691362df
DEV: add smtp_should_reject to the receive_emails api key scope () 2022-06-16 13:46:56 -03:00
Martin Brennan
4d3c1ceb44
FEATURE: Log the SMTP response in EmailLog ()
When sending emails with delivery_method_options -> return_response
set to true, the SMTP sending code inside Mail will return the SMTP
response when calling deliver! for mail within the app. This commit
ensures that Email::Sender captures this response if it is returned
and stores it against the EmailLog created for the sent email.

A follow up PR will make this visible within the admin email UI.
2022-06-15 10:28:30 +10:00
Alan Guo Xiang Tan
0ae7b43018
PERF: Remove total for time to first response report. ()
The query is very inefficient without any constraints on large sites and
the average of all time to first response since the beginning of time is
not useful as well.
2022-06-14 13:27:48 +10:00
David Taylor
6650218e3d
FIX: Ensure that extract_upload_ids works with all short URLs ()
We do not zero-pad our base62 short URLs, so there is no guarantee that the length is 27. Instead, let's greedily match all consecutive base62 characters and look for a matching upload.

This reverts bd32656157 and 36f5d5eada.
2022-06-13 17:01:27 +01:00
Alan Guo Xiang Tan
e9a77e7f19
FEATURE: Add new/unread counts to tags section links exp sidebar () 2022-06-13 14:54:01 +08:00
Alan Guo Xiang Tan
94c3bbc2d1
DEV: Centralize user updates to a single MessageBus channel. ()
Introduces an interface to publish user updates on the server side and
helps to reduce the growing number of subscriptions on the client side.
2022-06-13 14:27:43 +08:00
Bianca Nenciu
9db8f00b3d
FEATURE: Create upload_references table ()
This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use.

* DEV: Create upload_references
* DEV: Use UploadReference instead of PostUpload
* DEV: Use UploadReference for SiteSetting
* DEV: Use UploadReference for Badge
* DEV: Use UploadReference for Category
* DEV: Use UploadReference for CustomEmoji
* DEV: Use UploadReference for Group
* DEV: Use UploadReference for ThemeField
* DEV: Use UploadReference for ThemeSetting
* DEV: Use UploadReference for User
* DEV: Use UploadReference for UserAvatar
* DEV: Use UploadReference for UserExport
* DEV: Use UploadReference for UserProfile
* DEV: Add method to extract uploads from raw text
* DEV: Use UploadReference for Draft
* DEV: Use UploadReference for ReviewableQueuedPost
* DEV: Use UploadReference for UserProfile's bio_raw
* DEV: Do not copy user uploads to upload references
* DEV: Copy post uploads again after deploy
* DEV: Use created_at and updated_at from uploads table
* FIX: Check if upload site setting is empty
* DEV: Copy user uploads to upload references
* DEV: Make upload extraction less strict
2022-06-09 09:24:30 +10:00
Blake Erickson
3941bad075
DEV: Remove this chained user scope ()
This reverts one of the changes introduced just now in:

27d7b0c6de

I don't think we need this `activated_not_suspended_not_staged` scope
because we can just compose it ourselves via method chaining like
`User.activated.not_suspended.not_staged`.
2022-06-07 15:21:42 -06:00
Blake Erickson
27d7b0c6de
DEV: Add new user scopes ()
Adds two new user scopes:

- `not_staged`
- `activated_not_suspended_not_staged`

This will allow us to easily grab activated users that are not suspended
or staged.

See this PR feedback:

https://github.com/discourse/discourse-chat/pull/913#discussion_r890692266
2022-06-07 12:58:58 -06:00
David Taylor
5238f6788c
FEATURE: Allow hotlinked media to be blocked ()
This commit introduces a new site setting: `block_hotlinked_media`. When enabled, all attempts to hotlink media (images, videos, and audio) will fail, and be replaced with a linked placeholder. Exceptions to the rule can be added via `block_hotlinked_media_exceptions`.

`download_remote_image_to_local` can be used alongside this feature. In that case, hotlinked images will be blocked immediately when the post is created, but will then be replaced with the downloaded version a few seconds later.

This implementation is purely server-side, and does not impact the composer preview.

Technically, there are two stages to this feature:

1. `PrettyText.sanitize_hotlinked_media` is called during `PrettyText.cook`, and whenever new images are introduced by Onebox. It will iterate over all src/srcset attributes in the post HTML and check if they're allowed. If not, the attributes will be removed and replaced with a `data-blocked-hotlinked-src(set)` attribute

2. In the `CookedPostProcessor`, we iterate over all `data-blocked-hotlinked-src(set)` attributes and check whether we have a downloaded version of the media. If yes, we update the src to use the downloaded version. If not, the entire media element is replaced with a placeholder. The placeholder is labelled 'external media', and is a link to the offsite media.
2022-06-07 15:23:04 +01:00
Alan Guo Xiang Tan
7da074d591
DEV: Implement "My Posts" section link for experimental sidebar () 2022-06-07 10:52:54 +08:00
Blake Erickson
35e17ce115
FIX: Email Send post has already been taken error ()
* FIX: Email Send post has already been taken error

Adding a failing test first before coming up with a good solution.

Related: 357011eb3b

The above commit changed

```
PostReplyKey.find_or_create_by_safe!
```

to

```
PostReplyKey.create_or_find_by!
```

But I don't think it is working as a 1-1 replacement because of the
`Validation failed: Post has already been taken` error we are receiving
with this change. Also we need to make sure we don't re-introduce any
concurrency issues.

Reported: https://meta.discourse.org/t/224706/13

* Remove rails unique constraint and rely on db index

I believe this is what is causing `create_or_find_by!` to fail. Because
we have a unique constraint in the db I think we can remove this rails
unique constraint?

* clean up spec wording
2022-06-06 13:13:26 -06:00
Alan Guo Xiang Tan
9f08a3b160
FIX: DiscourseConnect login did not auto approve based on email domain ()
This commit resolves a bug where users are not auto approved based on
`SiteSetting.auto_approve_email_domains` when
`SiteSetting.must_approve_users` has been enabled.
2022-06-06 15:16:01 +08:00
Alan Guo Xiang Tan
0fa0094531
FIX: Approves user when redeeming an invite for invites only sites ()
When a site has `SiteSetting.invite_only` enabled, we create a
`ReviewableUser`record when activating a user if the user is not
approved. Therefore, we need to approve the user when redeeming an
invite.

There are some uncertainties surrounding why a `ReviewableRecord` is
created for a user in an invites only site but this commit does not seek
to address that.

Follow-up to 7c4e2d33fa
2022-06-03 11:43:52 +08:00
Gerhard Schlager
7c4e2d33fa
SECURITY: Remove auto approval when redeeming an invite ()
This security fix affects sites which have `SiteSetting.must_approve_users`
enabled. There are intentional and unintentional cases where invited
users can be auto approved and are deemed to have skipped the staff approval process.
Instead of trying to reason about when auto-approval should happen, we have decided that
enabling the `must_approve_users` setting going forward will just mean that all new users
must be explicitly approved by a staff user in the review queue. The only case where users are auto
approved is when the `auto_approve_email_domains` site setting is used.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-06-02 16:10:48 +02:00
Andrei Prigorshnev
943cae82da
FEATURE: propagate user status via message bus () 2022-05-30 13:41:53 +04:00
Andrei Prigorshnev
5c596273a0
FEATURE: user status () 2022-05-27 13:15:14 +04:00
Sérgio Saquetim
102e3a8cf2
DEV: Add hook for plugins modify how slugs are computed ()
Allows plugins to hook into and change how topic slugs are computed.
2022-05-25 19:05:06 -03:00
Martin Brennan
a5779a7d0b
DEV: Bookmark cleanup ()
Gets rid of old bookmark app event and deletes anything
leftover from polymorphic bookmark changeover.
2022-05-24 13:52:42 +10:00
Martin Brennan
3f79b1c0f3
DEV: Ignore post_id and for_topic for bookmarks ()
Since fcc2e7ebbf we
no longer use these columns, will delete in a few
months.
2022-05-24 13:52:34 +10:00
David Taylor
0baabafa9d DEV: Map already-downloaded hotlinked images in post_process_cooked
Previously this mapping of **cooked** images was only being run for oneboxes. Now it runs for all images, so we can transform hotlinked images without needing to immediately update `raw`
2022-05-23 14:28:02 +01:00
Gerhard Schlager
eef17318c3
FIX: Applying default user options didn't work for boolean flags ()
It also ensures that only human users are updated and replaces usage of `send` with `public_send`. Also, it adds more specs for existing code.
2022-05-23 15:20:51 +02:00
Martin Brennan
fcc2e7ebbf
FEATURE: Promote polymorphic bookmarks to default and migrate ()
This commit migrates all bookmarks to be polymorphic (using the
bookmarkable_id and bookmarkable_type) columns. It also deletes
all the old code guarded behind the use_polymorphic_bookmarks setting
and changes that setting to true for all sites and by default for
the sake of plugins.

No data is deleted in the migrations, the old post_id and for_topic
columns for bookmarks will be dropped later on.
2022-05-23 10:07:15 +10:00
Osama Sayegh
eddb8891c4
FIX: Clear inline onebox cache when a post is rebaked ()
Meta topic: https://meta.discourse.org/t/prevent-to-linkify-when-there-is-a-redirect/226964/4?u=osama.
2022-05-20 14:09:50 +03:00
Gerhard Schlager
0af05c2682
DEV: Compatibility with TruffleRuby ()
Removes thread unsafe lazy initialization.
See https://meta.discourse.org/t/225447

It also removes an unused variable in `FlagSettings`.
2022-05-19 21:56:55 +02:00
Loïc Guitaut
73de203843 FIX: Apply 'hide email account' for invites 2022-05-17 09:56:06 +02:00
David Taylor
38216f6f0b
DEV: Make user field validation more specific ()
- Only validate if custom_fields are loaded, so that we don't trigger a db query
- Only validate public user fields, not all custom_fields

This commit also reverts the unrelated spec changes in ba148e08, which were required to work around these issues
2022-05-16 14:21:33 +01:00
David Taylor
991b62b6f1
DEV: Drop old hotlinked image data from post_custom_fields ()
`20220428094026_create_post_hotlinked_media` moved this data into a dedicated table
2022-05-12 15:34:35 +01:00
Daniel Waterworth
cc4171be11
DEV: Expose user action types where the user was the instigator () 2022-05-11 15:15:53 -05:00
Loïc Guitaut
ba148e082d FIX: Apply watched words to user fields
Currently we don’t apply watched words to custom user fields nor user
profile fields.
This led to users being able to use blocked words in their bio, location
or some custom user fields.

This patch addresses this issue by adding some validations so it’s not
possible anymore to save the User model or the UserProfile model if they
contain blocked words.
2022-05-10 11:37:52 +02:00
Martin Brennan
3b3c505f3c
FIX: Polymorphic bookmarks for bookmark report ()
This allows the category_id filter for the bookmark
report to work with polymorphic bookmarks. Honestly this
is a little hardcode-y at the moment but until we go and
make this report a lot more flexible with more filters
I don't think it's worth the work to add extra interfaces
to RegisteredBookmarkable and BaseBookmarkable to make
this more flexible. This is enough for now.
2022-05-10 11:14:59 +10:00
Martin Brennan
fbcc35b417
DEV: Remove PostAction/UserAction bookmark refs ()
We have not used anything related to bookmarks for PostAction
or UserAction records since 2020, bookmarks are their own thing
now. Deleting all this is just cleaning up old cruft.
2022-05-10 10:42:18 +10:00
Sam
2df3c65ba9
FIX: add support for pipelined and multi redis commands ()
Latest redis interoduces a block form of multi / pipelined, this was incorrectly
passed through and not namespaced.

Fix also updates logster, we held off on upgrading it due to missing functions
2022-05-10 08:19:02 +10:00
Roman Rizzi
6bdcd7afb2
FEATURE: Promote the "delete group" staff action log. ()
We used to log group deletion as custom, which means we can't later search for them. Include group ID in the details.
2022-05-09 12:12:52 -03:00
Martin Brennan
222c8d9b6a
FEATURE: Polymorphic bookmarks pt. 3 (reminders, imports, exports, refactors) ()
A bit of a mixed bag, this addresses several edge areas of bookmarks and makes them compatible with polymorphic bookmarks (hidden behind the `use_polymorphic_bookmarks` site setting). The main ones are:

* ExportUserArchive compatibility
* SyncTopicUserBookmarked job compatibility
* Sending different notifications for the bookmark reminders based on the bookmarkable type
* Import scripts compatibility
* BookmarkReminderNotificationHandler compatibility

This PR also refactors the `register_bookmarkable` API so it accepts a class descended from a `BaseBookmarkable` class instead. This was done because we kept having to add more and more lambdas/properties inline and it was very messy, so a factory pattern is cleaner. The classes can be tested independently as well.

Some later PRs will address some other areas like the discourse narrative bot, advanced search, reports, and the .ics endpoint for bookmarks.
2022-05-09 09:37:23 +10:00
Osama Sayegh
7e026e8ddf
DEV: Add last_seen_reviewable_id to the users table ()
We're adding this column now in preparation for a future commit(s) that will
redesign the avatar/notifications menu. The reason the column is added in a
separate commit is because the redesign changes are going to be complex with a
high risk of getting (temporarily) reverted and if they included a database
migration, they wouldn't revert cleanly/easily.

Internal ticket: t65045.
2022-05-06 10:11:16 +03:00
Alan Guo Xiang Tan
0328757ffb
FIX: Topic view breaks with topic timer to publish to restricted category. ()
When a user views a topic that contains a topic timer to publish to a
restricted category, an error occurs on the client side because the user
does not have access to information about the category.

This commit fixes it such that the topic timer is not shown to the user
if the user does not have access to the category.
2022-05-05 11:21:01 +08:00
Martin Brennan
48481dd6ed
DEV: Remove ignored columns ()
Bookmark columns deleted in b22450c7a8
TopicTimer columns deleted in d098f51ad3
Upload columns deleted in ef90575b91
2022-05-05 12:22:17 +10:00
David Taylor
19d2d55011
FIX: Ensure post_hotlinked_media index does not exceed size limit ()
On some installations, this would fail with 'index row size exceeds btree version 4 maximum'. This commit replaces the (post_id, url)` index with a `(post_id, md5(url))` index, which is much more space efficient.
2022-05-03 15:47:58 +01:00
David Taylor
c1db968740
DEV: Move hotlinked image information into a dedicated table ()
This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change.

For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 13:53:32 +01:00
Sam
616de83232
FIX: avoid concurrent usage of AR models ()
Flagged by the truffle team at: https://meta.discourse.org/t/thread-unsafe-current-user-usage-in-auth-defaultcurrentuserprovider/225671

This usage of AR is unsafe currently, as AR models are not safe for concurrent usage

Introduces a new query potentially every minute which should be acceptable.
2022-05-03 08:50:56 +10:00
Angus McLeod
9fc3d46003
Update wordpress scopes and add `session/scopes` endpoint ()
* Update wordpress scopes && add ``session/scopes`` endpointt

* Fix failing spec

* Add users#show scope to discourse_connect

* Update app/controllers/session_controller.rb

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>

Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
2022-05-02 12:15:32 -03:00
David Taylor
0f772bdf5b
FEATURE: Optionally skip using full_name when suggesting usernames ()
This commit introduces a new site setting: `use_name_for_username_suggestions` (default true)

Admins can disable it if they want to stop using Name values when generating usernames for users. This can be useful if you want to keep real names private-by-default or, when used in conjunction with the `use_email_for_username_and_name_suggestions` setting, you would prefer to use email-based username suggestions.
2022-04-29 14:00:13 +01:00
Loïc Guitaut
008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Alan Guo Xiang Tan
98c49acad5
DEV: Setup experimental sidebar skeleton ()
* hidden siteSetting to enable experimental sidebar
* user preference to enable experimental sidebar
* `experimental_sidebar_enabled` attribute for current user
* Empty glimmer component for Sidebar
2022-04-28 15:27:06 +08:00
David Taylor
5e34ce1282
DEV: Ensure theme tests are always loaded in a consistent order ()
If they aren't, then the digest will be different, causing unexpected issues
2022-04-26 17:05:57 +01:00
Joffrey JAFFEUX
596469a712
REFACTOR: live-development.js ()
- drop jquery
- replaces setTimeout by later
- removes dead code which is not even working today
2022-04-25 19:34:16 +02:00
David Taylor
922fbe82da
DEV: Ensure custom_fields_clean? returns false when values change ()
We were calling `dup` on the hash and using that to check for changes. However, we were not duplicating the values, so changes to arrays or nested hashes would not be detected.
2022-04-25 17:19:39 +01:00
Martin Brennan
3e4621c2cb
FEATURE: Polymorphic bookmarks pt. 2 (lists, search) ()
This pull request follows on from https://github.com/discourse/discourse/pull/16308. This one does the following:

* Changes `BookmarkQuery` to allow for querying more than just Post and Topic bookmarkables
* Introduces a `Bookmark.register_bookmarkable` method which requires a model, serializer, fields and preload includes for searching. These registered `Bookmarkable` types are then used when validating new bookmarks, and also when determining which serializer to use for the bookmark list. The `Post` and `Topic` bookmarkables are registered by default.
* Adds new specific types for Post and Topic bookmark serializers along with preloading of associations in `UserBookmarkList`
* Changes to the user bookmark list template to allow for more generic bookmarkable types alongside the Post and Topic ones which need to display in a particular way

All of these changes are gated behind the `use_polymorphic_bookmarks` site setting, apart from the .hbs changes where I have updated the original `UserBookmarkSerializer` with some stub methods.

Following this PR will be several plugin PRs (for assign, chat, encrypt) that will register their own bookmarkable types or otherwise alter the bookmark serializers in their own way, also gated behind `use_polymorphic_bookmarks`.

This commit also removes `BookmarkQuery.preloaded_custom_fields` and the functionality surrounding it. It was added in 0cd502a558 but only used by one plugin (discourse-assign) where it has since been removed, and is now used by no plugins. We don't need it anymore.
2022-04-22 08:23:42 +10:00
David Taylor
137e06a316
FIX: Ensure 'crop' always returns requested dimensions ()
Previously, 'crop' would resize the image to have the requested width, then crop the height to the requested value. This works when cropping images vertically, but not when cropping them horizontally.

For example, trying to crop a 500x500 image to 200x500 was actually resulting in a 200x200 image. Having an OptimizedImage with width/height columns mismatching the actual OptimizedImage width/height causes some unusual issues.

This commit ensures that a call to `OptimizedImage.crop(from, to, width, height)` will always return an image of the requested width/height. The `w x h^` syntax defines minimum width/height, while maintaining aspect ratio.
2022-04-19 11:21:24 +01:00
Martin Brennan
c6c633e041
FIX: Issues with incorrect unread and private message topic tracking state ()
This commit fixes two issues at play. The first was introduced
in f6c852b (or maybe not introduced
but rather revealed). When a user posted a new message in a topic,
they received the unread topic tracking state MessageBus message,
and the Unread (X) indicator was incremented by one, because with the
aforementioned perf commit we "guess" the correct last read post
for the user, because we no longer calculate individual users' read
status there. This meant that every time a user posted in a topic
they tracked, the unread indicator was incremented. To get around
this, we can just exclude the user who created the post from the
target users of the unread state message.

The second issue was related to the private message topic tracking
state, and was somewhat similar. Whenever a user created a new private
message, the New (X) indicator was incremented, and could not be
cleared until the page was refreshed. To solve this, we just don't
update the topic state for the user when the new_topic tracking state
message comes through if the user who created the topic is the
same as the current user.

cf. https://meta.discourse.org/t/bottom-of-topic-shows-there-is-1-unread-remaining-when-there-are-actually-0-unread-topics-remaining/220817
2022-04-19 11:37:01 +10:00
Gerhard Schlager
b7230d14a3 REFACTOR: Add full_url to Group
Lets stop writing the same code over and over again.
2022-04-14 11:53:57 +02:00
Gerhard Schlager
b3cda195b8 REFACTOR: Add full_url and display_name to User
Lets stop writing the same code over and over again.
2022-04-14 11:53:57 +02:00
Osama Sayegh
eb5a3cfded
FEATURE: Add 2FA support to the Discourse Connect Provider protocol ()
Discourse has the Discourse Connect Provider protocol that makes it possible to
use a Discourse instance as an identity provider for external sites. As a
natural extension to this protocol, this PR adds a new feature that makes it
possible to use Discourse as a 2FA provider as well as an identity provider.

The rationale for this change is that it's very difficult to implement 2FA
support in a website and if you have multiple websites that need to have 2FA,
it's unrealistic to build and maintain a separate 2FA implementation for each
one. But with this change, you can piggyback on Discourse to take care of all
the 2FA details for you for as many sites as you wish.

To use Discourse as a 2FA provider, you'll need to follow this guide:
https://meta.discourse.org/t/-/32974. It walks you through what you need to
implement on your end/site and how to configure your Discourse instance. Once
you're done, there is only one additional thing you need to do which is to
include `require_2fa=true` in the payload that you send to Discourse.

When Discourse sees `require_2fa=true`, it'll prompt the user to confirm their
2FA using whatever methods they've enabled (TOTP or security keys), and once
they confirm they'll be redirected back to the return URL you've configured and
the payload will contain `confirmed_2fa=true`. If the user has no 2FA methods
enabled however, the payload will not contain `confirmed_2fa`, but it will
contain `no_2fa_methods=true`.

You'll need to be careful to re-run all the security checks and ensure the user
can still access the resource on your site after they return from Discourse.
This is very important because there's nothing that guarantees the user that
will come back from Discourse after they confirm 2FA is the same user that
you've redirected to Discourse.

Internal ticket: t62183.
2022-04-13 15:04:09 +03:00
Bianca Nenciu
86c7e07428
FEATURE: Improve screened IPs roll up and extend for IPv6 ()
This commit improves the logic for rolling up IPv4 screened IP
addresses and extending it for IPv6. IPv4 addresses will roll up only
up to /24. IPv6 can rollup to /48 at most. The log message that is
generated contains the list of original IPs and new subnet.
2022-04-12 21:07:37 +03:00
Roman Rizzi
6f76a12e0a
FEATURE: Let sites add a sitemap.xml file. ()
* FEATURE: Let sites add a sitemap.xml file.

This PR adds the same features discourse-sitemap provides to core. Sitemaps are only added to the robots.txt file if the `enable_sitemap` setting is enabled and `login_required` disabled.

After merging , this change will take priority over the sitemap plugin because it will disable itself. We're also using the same sitemaps table, so our migration won't try to create it
again using `if_not_exists: true`.
2022-04-12 10:33:59 -03:00
David Taylor
7b1686ad7d
FIX: Ensure category_required_tag_groups are destroyed with tag_groups () 2022-04-11 13:28:06 +01:00
OsamaSayegh
2f7a307237 FIX: Prevent duplicates in API scope allowed URLs
It's possible in Rails to map a single route to multiple controller
actions with different constraints. We do this in at least 1 place in
our application for the root route (/) to make it possible to change the
page that root route displays.

This means that if you get the list of routes of your application,
you'll get the same route for each time the route is defined. And if
there's an API scope for 2 (or more) controller actions that map to the
same route, the route will be listed twice in the Allowed URLs list of
the scope.

To prevent this, this PR adds the allowed URLs in a set so that
duplicate routes are automatically removed.
2022-04-07 02:09:11 +03:00
OsamaSayegh
5eaf4b8553 FIX: Include routes in an API scope's allowed URLs even if they have no format constraints
The Allowed URLs list of an API scope only includes routes that
constraint the format for the route to JSON. However, some routes define
no format constraints, but that doesn't mean they can't be used by an
API key.

This commit amends the logic for the Allowed URLs list so that it
includes routes that have no format constraints or the format
constraints include JSON.
2022-04-07 02:09:11 +03:00
Sam
cedcdb0057
FEATURE: allow for local theme js assets ()
Due to default CSP web workers instantiated from CDN based assets are still
treated as "same-origin" meaning that we had no way of safely instansiating
a web worker from a theme.

This limits the theme system and adds the arbitrary restriction that WASM
based components can not be safely used.

To resolve this limitation all js assets in about.json are also cached on
local domain.

{
  "name": "Header Icons",
  "assets" : {
    "worker" : "assets/worker.js"
  }
}

This can then be referenced in JS via:

settings.theme_uploads_local.worker

local_js_assets are unconditionally served from the site directly and
bypass the entire CDN, using the pre-existing JavascriptCache

Previous to this change this code was completely dormant on sites which
used s3 based uploads, this reuses the very well tested and cached asset
system on s3 based sites.

Note, when creating local_js_assets it is highly recommended to keep the
assets lean and keep all the heavy working in CDN based assets. For example
wasm files can still live on the CDN but the lean worker that loads it can
live on local.

This change unlocks wasm in theme components, so wasm is now also allowed
in `theme_authorized_extensions`

* more usages of upload.content

* add a specific test for upload.content

* Adjust logic to ensure that after upgrades we still get a cached local js
on save
2022-04-07 07:58:10 +10:00
Rafael dos Santos Silva
1203121ac1
PERF: Throttle updates to API key last_used_at ()
Co-authored-by: David Taylor <david@taylorhq.com>
2022-04-06 11:01:52 -03:00
David Taylor
68c74e9b93
FEATURE: Allow multiple required tag groups for a category ()
Previously we only supported a single 'required tag group' for a category. This commit allows admins to specify multiple required tag groups, each with their own minimum tag count.

A new category_required_tag_groups database table replaces the existing columns on the categories table. Data is automatically migrated.
2022-04-06 14:08:06 +01:00
Loïc Guitaut
357011eb3b DEV: Clean up freedom patches
This patch removes some of our freedom patches that have been deprecated
for some time now.
Some of them have been updated so we’re not shipping code based on an
old version of Rails.
2022-04-06 10:07:14 +02:00
Blake Erickson
ec2930712d
FIX: 500 error when creating a user with an integer username ()
Via the API it is possible to create a user with an integer username. So
123 instead of "123". This causes the following 500 error:

```
NoMethodError (undefined method `unicode_normalize' for 1:Integer)
app/models/user.rb:276:in `normalize_username'
```

See: https://meta.discourse.org/t/222281
2022-04-04 15:15:32 -06:00
Sam
b023d88b09
FIX: Abort theme creation if unable to create uploads ()
Previous to this change if any of the assets were not allowed extensions
they would simply be silently ignored, this could lead to broken themes
that are very hard to debug
2022-04-01 12:03:14 +11:00
Alan Guo Xiang Tan
e7c3d01aaa
DEV: Restore order assertion in category serializer tests. ()
Our group fabrication creates groups with name "my_group_#{n}" where n
is the sequence number of the group being created. However, this can
cause the test to be flaky if and when a group with name `my_group_10`
is created as it will be ordered before
`my_group_9`. This commits makes the group names determinstic to
eliminate any flakiness.

This reverts commit 558bc6b746.
2022-04-01 08:58:06 +08:00
Mark VanLandingham
b974375239
DEV: Skip notifications without topic_id in ensure_consistency () 2022-03-30 09:56:35 -05:00
Martin Brennan
b8828d4a2d
FEATURE: Polymorphic bookmarks pt. 1 (CRUD) ()
This commit introduces a new use_polymorphic_bookmarks site setting
that is default false and hidden, that will be used to help continuous
development of polymorphic bookmarks. This setting **should not** be
enabled anywhere in production yet, it is purely for local development.

This commit uses the setting to enable create/update/delete actions
for polymorphic bookmarks on the server and client side. The bookmark
interactions on topics/posts are all usable. Listing, searching,
sending bookmark reminders, and other edge cases will be handled
in subsequent PRs.

Comprehensive UI tests will be added in the final PR -- we already
have them for regular bookmarks, so it will just be a matter of
changing them to be for polymorphic bookmarks.
2022-03-30 12:43:11 +10:00
Osama Sayegh
e40c4bb7f9
FIX: Validate category tag restrictions before sending new topics to review ()
Tags (and tag groups) can be configured so that they can only be used in specific categories and (optionally) restrict topics in these categories to be able to add/use only these tags. These restrictions work as expected when a topic is created without going through the review queue; however, if the topic has to be reviewed by a moderator then these restrictions currently aren't checked before the topic is sent to the review queue, but they're checked later when a moderator tries to approve the topic. This is because if a user manages to submit a topic that doesn't meet the restrictions, moderators won't be able to approve and it'll be stuck in the review queue.

This PR prevents topics that don't meet the tags requirements from being sent to the review queue and shows the poster an error message that indicates which tags that cannot be used.

Internal ticket: t60562.
2022-03-28 21:25:26 +03:00
Alan Guo Xiang Tan
fc40a572bb
DEV: Register question_answer_user_commented notification type. ()
The notification type is used by https://github.com/discourse/discourse-question-answer
2022-03-28 16:03:19 +08:00
Martin Brennan
230e82e948
DEV: Make Bookmark#post_id column nullable ()
As we are gradually moving to having a polymorphic
bookmarkable relationship on the Bookmark table,
we need to make the post_id column nullable to be
able to develop and test the new columns, and
for cutover/migration purposes later as well.
2022-03-28 13:09:13 +10:00
Dan Ungureanu
76ece494f9
DEV: Fix "serialize to JSON safely" deprecation ()
Job arguments must match after a serialize-deserialize cycle and
symbols were converted to strings during this process.
2022-03-25 09:07:21 +08:00
Bianca Nenciu
cbaf7c949b
FIX: Make sure max_oneboxes_per_post is enforced ()
PostAnalyzer and CookedPostProcessor both replace URLs with oneboxes.
PostAnalyzer did not use the max_oneboxes_per_post site and setting and
CookedPostProcessor replaced at most max_oneboxes_per_post URLs ignoring
the oneboxes that were replaced already by PostAnalyzer.
2022-03-23 17:36:08 +02:00
Martin Brennan
8040b95e8c
DEV: Re-add polymorphic bookmark columns ()
This commit is a redo of2f1ddadff7dd47f824070c8a3f633f00a27aacde
which we reverted because it blew up an internal CI check. I looked
into it, and it happened because the old migration to add the bookmark
columns still existed, and those columns were dropped in a post migrate,
so the two migrations to add the columns were conflicting before
the post migrate was run.

------

This commit only includes the creation of the new columns and index,
and does not add any triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

Add an experimental site setting for using polymorphic bookmarks,
and make sure in the places where bookmarks are created or updated
we fill in the columns. This setting will be used in subsequent
PRs as well.
Listing and searching bookmarks based on polymorphic associations
Creating post and topic bookmarks using polymorphic associations,
and changing special for_topic logic to just rely on the Topic
bookmarkable_type
Querying bookmark reminders based on polymorphic associations
Make sure various other areas like importers, bookmark guardian,
and others all rely on the associations
Prepare plugins that rely on the Bookmark model to use polymorphic
associations
The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-23 12:43:08 +10:00
Alan Guo Xiang Tan
4e6f7a1a6e
Revert "DEV: Re-add polymorphic bookmark columns ()" ()
This reverts commit 2f1ddadff7.
2022-03-22 13:13:42 +08:00
Martin Brennan
2f1ddadff7
DEV: Re-add polymorphic bookmark columns ()
This commit is a redo of e21c640a3c
which we reverted to not include half-done work in a release.

This commit is slightly different though, in that it only includes
the creation of the new columns and index, and does not add any
triggers, backfilling, or new data.

A backfill will be done in the final PR when we switch this over.
Intermediate PRs will look something like this:

1. Add an experimental site setting for using polymorphic bookmarks,
   and make sure in the places where bookmarks are created or updated
   we fill in the columns. This setting will be used in subsequent
   PRs as well.
2. Listing and searching bookmarks based on polymorphic associations
3. Creating post and topic bookmarks using polymorphic associations,
   and changing special for_topic logic to just rely on the Topic
   bookmarkable_type
4. Querying bookmark reminders based on polymorphic associations
5. Make sure various other areas like importers, bookmark guardian,
   and others all rely on the associations
6. Prepare plugins that rely on the Bookmark model to use polymorphic
   associations

The final core PR will remove all the setting gates and switch over
to using the polymorphic associations, backfill the bookmarks
table columns, and ignore the old post_id and for_topic colummns.
Then it will just be a matter of dropping the old columns down the
line.
2022-03-22 14:26:13 +10:00
Roman Rizzi
581d435d09
Revert "FEATURE: Let reviewables override the score type title. ()" ()
This reverts commit 746f8f3797.
2022-03-21 16:32:47 -03:00
Roman Rizzi
746f8f3797
FEATURE: Let reviewables override the score type title. ()
Plugins like chat add custom score type to override the title in the UI, but that should be reserved for situations when you need to manage the flag priority separately, which is configurable in the queue settings page.

Currently, if a plugin creates a custom score type, it won't be able to associate a priority, so there's no real gain from doing so. Priorities are tightly related to post-action types, which is something we might want to revise. For now, this change lets plugins move away from custom score types without compromises.
2022-03-21 14:29:57 -03:00
Bianca Nenciu
30f3e78834
FIX: Reset last sent for existent bookmarks ()
The meaning of reminder_at and reminder_last_sent_at changed after
commit 6d422a8033. A bookmark reminder
will fire only if reminder_last_sent_at is null, but before that it
fired everytime reminder_at was set. This is no longer true because
sometimes reminder_at continues to exist even after a reminder fired.
2022-03-18 16:31:35 +02:00
Rafael dos Santos Silva
100a84d345
DEV: Reserve assigned notification type ()
* DEV: Reserve assigned notification type

* update constants
2022-03-16 17:08:10 -03:00
Mark VanLandingham
cea0b9cefb
DEV: Reserve usernames for chat and sunset chat_group_mention () 2022-03-16 08:55:21 -05:00
Jarek Radosz
383f164672
FIX: Always include the first poster when converting to PM ()
Regressed in 
2022-03-15 10:51:10 +01:00
Sam
de9a031073
FEATURE: use canonical links in posts.rss feed ()
* FEATURE: use canonical links in posts.rss feed

Previously we used non canonical links in posts.rss

These links get crawled frequently by crawlers when discovering new
content forcing crawlers to hop to non canonical pages just to end up
visiting canonical pages

This uses up expensive crawl time and adds load on Discourse sites

Old links were of the form:

`https://DOMAIN/t/SLUG/43/21`

New links are of the form

`https://DOMAIN/t/SLUG/43?page=2#post_21`

This also adds a post_id identified element to crawler view that was
missing.

Note, to avoid very expensive N+1 queries required to figure out the
page a post is on during rss generation, we cache that information.

There is a smart "cache breaker" which ensures worst case scenario is
a "page drift" - meaning we would publicize a post is on page 11 when
it is actually on page 10 due to post deletions. Cache holds for up to
12 hours.

Change only impacts public post RSS feeds (`/posts.rss`)
2022-03-15 20:17:06 +11:00
David Taylor
8664712c1a
PERF: Fix n+1 for categories + featured topics ()
`topic.featured_topic` and `topic.category` are used by `TopicGuardian#can_see_topic?`
2022-03-14 22:23:39 +00:00
Bianca Nenciu
7a416257df
FIX: Invalidate site settings cache in all instances ()
Previous cache implementation did not support multisite instances and
the cache was invalidated only in the instance where the change took
place.
2022-03-11 17:16:56 +02:00
Penar Musaraj
94750c81fa
FIX: Update group inbox notifications on archive/unarchive () 2022-03-11 11:57:47 +01:00
Osama Sayegh
8d96761a4f
FIX: Include engine mount path for API scopes added by plugins ()
In the API keys page where admins can create API keys with restricted scopes, each scope shows a list of URLs that it allows. But currently, this list of allowed URLs shows incomplete URLs for scopes that are added by plugins. For example, the allowed URL for the "run queries" scope of the data-explorer plugin is shown as `/queries/:id/run` when the correct URL for this scope is `/admin/plugins/explorer/queries/:id/run`. The first 3 segments of the path are the mount path of the plugin's engine and it's missing because the routes set of the engine doesn't include the mount path. To fix this, this commit gets the mount path and prepends it to the URL so the complete URL is shown to the user.

It's not possible to write tests for this change because plugins are not loaded in the test environment by default when core's tests suite is running.
2022-03-10 22:01:22 +03:00
Bianca Nenciu
6d422a8033
FEATURE: Highlight expired bookmark reminders ()
The user can select what happens with a bookamrk after it expires. New
option allow bookmark's reminder to be kept even after it has expired.
After a bookmark's reminder notification is created, the reminder date
will be highlighted in red until the user resets the reminder date.
User can do that using the new Clear Reminder button from the dropdown.
2022-03-08 19:44:18 +02:00
Blake Erickson
d760fd4074
DEV: Add API scope for categories endpoint ()
This change adds support for the categories endpoint to have an api
scope. Only adds GET scope for listing categories and for fetching a
single category.

See: https://meta.discourse.org/t/218080/4
2022-03-04 14:29:47 -07:00
David Taylor
eb2e3b510d
FEATURE: Introduce 'Subcategories with featured topics' view ()
This categories view is designed for sites which make heavy use of subcategories, and use top-level categories mainly for grouping
2022-03-04 21:11:59 +00:00
Blake Erickson
07e80b52ef
DEV: Add api scoping for updating topics. ()
This change adds a new api scope for updating topics.

See: https://meta.discourse.org/t/219805
2022-03-04 10:44:56 -07:00
Sam
3bf5692c72
FEATURE: prioritize group search order based on prefix match ()
Our @mention user search prioritized users based on prefix matches.

So if searching for `sa` we will display `sam`, `asam` in that order

Previously, we did not prioritize group matches based on prefix. This change ensures better parity.

Implementation notes:

1. User search only prioritizes based on username prefix, not name prefix. TBD if we want to change that.
2. @mention on client side will show 0 group matches if we fill up all the spots with user matches. TBD if we want to unconditionally show the first / second group match.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-03-03 16:57:52 +11:00
Krzysztof Kotlarek
ea3a58d051
FIX: indirectly muted categories for topic-tracking-state ()
Topics belonging to indirectly muted categories should be excluded from topic-tracking-state report.
2022-03-02 15:02:09 +11:00
Jarek Radosz
6f6406ea03
DEV: Fix random typos () 2022-02-28 10:20:58 +08:00
Jeff Wong
d1bdb6c65d
FEATURE: upload an avatar option for uploading avatars with selectable avatars ()
* FEATURE: upload an avatar option for uploading avatars with selectable avatars

Allow staff or users at or above a trust level to upload avatars even when the site
has selectable avatars enabled.

Everyone can still pick from the list of avatars. The option to upload is shown
below the selectable avatar list.

refactored boolean site setting into an enum with the following values:

disabled: No selectable avatars enabled (default)
everyone: Show selectable avatars, and allow everyone to upload custom avatars
tl1: Show selectable avatars, but require tl1+ and staff to upload custom avatars
tl2: Show selectable avatars, but require tl2+ and staff to upload custom avatars
tl3: Show selectable avatars, but require tl3+ and staff to upload custom avatars
tl4: Show selectable avatars, but require tl4 and staff to upload custom avatars
staff: Show selectable avatars, but only allow staff to upload custom avatars
no_one: Show selectable avatars. No users can upload custom avatars

Co-authored-by: Régis Hanol <regis@hanol.fr>
2022-02-24 12:57:39 -08:00
Sam
d4d3580761
PERF: perform all cached counting in background ()
Previously cached counting made redis calls in main thread and performed
the flush in main thread.

This could lead to pathological states in extreme heavy load.

This refactor reduces load and cleans up the interface
2022-02-22 16:45:25 +00:00
David Taylor
f6c852bf8e
PERF: Use a shared message for replies to tracked topics ()
Previously we were publishing one messagebus message per user which was 'tracking' a topic. On large sites, this can easily be 1000+ messages. The important information in the message is common between all users, so we can manage with a single message on a shared channel, which will be much more efficient.

For user-specific values (notification_level and last_read_post_number), the JS app can infer values which are 'good enough'. Correct values will be loaded as soon as a topic-list containing the topic is visited.
2022-02-22 15:27:46 +00:00
Joffrey JAFFEUX
f5ec32bc8c
FEATURE: adds the user_promoted event to webhooks () 2022-02-22 10:57:18 +01:00
Martin Brennan
fa0c796baf
DEV: Fix SMTP bounce regexp ()
Never trust me with regexp. Follow up to
01ef1d08fc,
which did not take into account codes in
the format X.X.XX (with the 2 digits on the end)
2022-02-22 08:54:01 +10:00
Bianca Nenciu
90c3695ab0
FEATURE: Rename Reset Read bulk action to Defer ()
It is enabled only if defer is enabled in user options too and if the
button shows up in the topic's footer.
2022-02-21 22:45:01 +02:00
Bianca Nenciu
94883dd326
FIX: Show links with 0 clicks in top links section ()
This is useful to moderators to see if users post spam links.
2022-02-21 22:15:38 +02:00
Bianca Nenciu
ae1d2d957f
FEATURE: Replace share post popup with share modal ()
This uniformizes the topic share modal and the post link popup. It also
introduces a new feature which can notify the user of a post.
2022-02-21 22:14:28 +02:00
David Taylor
5d6d3fb244
DEV: Remove child theme settings/variables from parent compilation ()
aa1442fdc3 split theme stylesheets so that every component gets its own stylesheet. Therefore, there is now no need for parent themes to collate the settings/variables of its children during scss compilation.

Technically this is a breaking change for any themes which depend on the settings/variables of their child components. That was never a supported/recommended arrangement, so we don't expect this to cause issues.
2022-02-21 11:15:35 +00:00
David Taylor
c8d956374d FIX: Use fresh theme setting values when compiling stylesheets
If a theme is updated to introduce a new setting AND immediately make use of it in a stylesheet, then an error was being shown. This is because the stylesheet compilation was using the theme's cached settings, and the cache is only cleared **after** the theme has finished compiling.

This commit updates the SCSS compilation to use uncached values for settings. A similar fix was applied to other parts of theme compilation back in 2020: (a51b8d9c66)
2022-02-21 09:45:14 +00:00
David Taylor
51afa579f7 DEV: Use the theme cache helper for settings
The previous Discourse.cache usage was different to how other theme-related caching is handled, and also requires reaching out to redis every time. The common theme cache is held in memory (as a DistributedCache)
2022-02-21 09:45:14 +00:00
Martin Brennan
01ef1d08fc
FIX: Conform EmailLog#bounce_error_code to RFC ()
This commit makes sure that the email log's bounce_error_code
conforms to the SMTP error code RFC on save, so that
it is always in the format X.X.X or XXX without any
additional string details. Also included is a migration
to fix this issue for past records.
2022-02-21 11:26:39 +10:00
Bianca Nenciu
38cbca3f67
FIX: Count clicks on links with query params ()
This did not work sometimes if a topic had the same URL with and without
query params because it did not try to select the best matching URL.
2022-02-18 14:47:56 +02:00
Bianca Nenciu
53f9a1a469
FEATURE: Add settings to scale daily flags limit ()
Similar site settings exist for likes and edits and the new ones work
in a similar way.

By default, users below TL2 have a limit of 20, the limit is increased
by 1.5 for TL2 users up to 30, by 2 for TL3 users up to 40 and by 3 for
TL4 users up to 60.
2022-02-18 14:44:32 +02:00
Michael Brown
3bf3b9a4a5 DEV: pull email address validation out to a new EmailAddressValidator
We validate the *format* of email addresses in many places with a match against
a regex, often with very slightly different syntax.

Adding a separate EmailAddressValidator simplifies the code in a few spots and
feels cleaner.

Deprecated the old location in case someone is using it in a plugin.

No functionality change is in this commit.

Note: the regex used at the moment does not support using address literals, e.g.:
* localpart@[192.168.0.1]
* localpart@[2001:db8::1]
2022-02-17 21:49:22 -05:00
megothss
c71c107649
FIX: Don't accept accents in slug if generation_method == 'ascii' ()
* FIX: Don't accept accents in slug if generation_method == 'ascii'

Fixes bug reported in:
- https://meta.discourse.org/t/404-when-trying-to-edit-category-with-accent-in-slug/214762
- https://meta.discourse.org/t/formatting-and-accents-in-urls/215734/5

Assuming `SiteSetting.slug_generation_method == 'ascii'.

If the user provides a slug containing non-ascii characters while
creating the category, the user will receive a 404 error just
after saving the category since the slug will be escaped anyway but
Category.find_by_slug_path won't escape the category slug
causing the Edit Page of the category to be inaccessible.

This commit checks the provided slug and raises an error if the
provided slugcontains non-ascii characters ensuring that the
provided value is consistent with the site settings.

It also changes Category.find_by_slug_path to always escape the slug,
since if present, it is escaped anyway in Category.ensure_slug to
prevent the 404 in the Edit Category Page in case the user already
have some category with a non-ascii slug.

* Removed trailing whitespace
2022-02-17 13:46:06 +11:00
Krzysztof Kotlarek
a7d43cf1ec
FEATURE: mute subcategory when parent category is muted ()
When parent category or grandparent category is muted, then category should be muted as well.

Still, it can be overridden by setting individual subcategory notification level.

CategoryUser record is not created, mute for subcategories is purely virtual.
2022-02-17 00:42:02 +01:00
Dan Ungureanu
effbd6d3e4
FEATURE: Show error if invite to topic is invalid ()
This can happen if the topic to which a user is invited is in a private
category and the user was not invited to one of the groups that can see
that specific category.

This used to be a warning and this commit makes it an error.
2022-02-16 18:35:02 +02:00
Bianca Nenciu
5eaf214594
FEATURE: New plugin API to check if upload is used ()
This commit introduces two new APIs for handling unused uploads, one
can be used to exclude uploads in bulk when the data model allow and
the other one excludes uploads one by one.
2022-02-16 09:00:30 +02:00
Martin Brennan
2d30dd439f
DEV: Add chat_quoted notification type ()
This is needed for the notification sent when quoting
chat messages inside a post.
2022-02-16 15:22:08 +10:00
Sam
33a0ad1b69
PERF: introduce site/global emoji cache ()
Previously calls such as `Emoji["smile"]` would force a full dehydration of
objects from Redis.

This introduces a version safe site and global emoji cache so lookups are
cheap. It eliminates iterating through the list of emojis and pulling from
redis.

Distributed cache uses a normalized name as the key and stores an Array tuple
with version and Emoji. Successful hits always confirm version matches.

Interface to Emoji object remains unchanged.

We opted for 2 caches to improve reuse on multisites. misses though will be
stored in both caches. If there is a hit on the global cache we can avoid
looking up in site local cache and storing a miss there.
2022-02-16 12:46:17 +11:00
David Taylor
94a47d037f
PERF: Reduce number of EXPIRE calls from CachedCounting ()
Previously we were calling `EXPIRE` every time we incremented a given key. Instead, we can call EXPIRE once when the key is first populated. A LUA script is used to make this as efficient as possible.

Consumers of this Concern use daily keys. Since we're now calling EXPIRE only at the beginning of the day, rather than throughout the day, the expire time has been increased from 3 to 4 days.
2022-02-15 16:55:21 +00:00
Rafael dos Santos Silva
4d3da70bc6
PERF: Use Redis SET EX GET instead of LUA script for counting ()
This will prevent Discourse from booting on Redis < 6.2.0
2022-02-15 10:36:07 -03:00
Martin Brennan
4086ee551e
DEV: Add bounce_error_code to EmailLog ()
Whenever we got a bounced email in the Email::Receiver we
previously would just set bounced: true on the EmailLog and
discard the status/diagnostic code. This commit changes this
flow to store the bounce error code (defined in the RFC at
https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml)
not just in the Email::Receiver, but also via webhook events
from other mail services and from SNS.

This commit does not surface the bounce error in the UI,
we can do that later if necessary.
2022-02-15 14:17:26 +10:00
Osama Sayegh
7ed899fed9
FIX: Ensure theme JavaScript cache get consistent SHA1 digest ()
There is a couple of layers of caching for theme JavaScript in Discourse:

The first layer is the `javascript_caches` table in the database. When a theme
with JavaScript files is installed, Discourse stores each one of the JavaScript
files in the `theme_fields` table, and then concatenates the files, compiles
them, computes a SHA1 digest of the compiled JavaScript and store the results
along with the SHA1 digest in the `javascript_caches` table.

Now when a request comes in, we need to render `<script>` tags for the
activated theme(s) of the site. To do this, we retrieve the `javascript_caches`
records of the activated themes and generate a `<script>` tag for each record.
The `src` attribute of these tags is a path to the `/theme-javascripts/:digest`
route which simply responds with the compiled JavaScript that has the requested
digest.

The second layer is a distributed cache whose purpose is to make rendering
`<script>` a lot more efficient. Without this cache, we'd have to query the
`javascript_caches` table to retrieve the SHA1 digests for every single
request. So we use this cache to store the `<script>` tags themselves so that
we only have to retrieve the `javascript_caches` records of the activated
themes for the first request and future requests simply get the cached
`<script>` tags.

What this commit does it ensures that the SHA1 digest in the
`javascript_caches` table stay the same across compilations by adding an order
by id clause to the query that loads the `theme_fields` records. Currently, we
specify no order when retrieving the `theme_fields` records so the order in
which they're retrieved can change across compilations and therefore cause the
SHA1 to change even though the individual records have not changed at all.

An inconsistent SHA1 digest across compilations can cause the database cache
and the distributed cache to have different digests and that causes the
JavaScript to fail to load (and if the theme heavily customizes the site, it
gives the impression that the site is broken) until the cache is cleared.

This can happen in busy sites when 2 concurrent requests recompile the
JavaScript files of a theme at the same time (this can happen when deploying a
new Discourse version) and request A updates the database cache after request B
did, and request B updates the distributed cache after request A did.

Internal ticket: t60783.

Co-authored-by: David Taylor <david@taylorhq.com>
2022-02-14 16:23:06 +03:00
David Taylor
af24c10314 DEV: Improve theme error handling UX
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
2022-02-14 10:11:19 +00:00
Alan Guo Xiang Tan
1d6a45d1f9
DEV: Bump Theme::BASE_COMPILER_VERSION. ()
In 8e5b945b0f, we reverted the commit but
at the same time resulted in Theme::BASE_COMPILER_VERSION going
backwards which caused problems with themes caching.

This commit bumps the version to clear all the caches.

Follow-up to 8e5b945b0f
2022-02-11 17:33:04 +11:00
Alan Guo Xiang Tan
8e5b945b0f
Revert "DEV: Improve theme error handling UX" ()
`PrettyText.cook` is breaking on some sites. Revert for now while we
investigate.

This reverts commit c81d369ab6.
2022-02-11 11:30:36 +08:00
Alan Guo Xiang Tan
b876ff6281
FIX: Update user stat counts when post/topic visibility changes. ()
Breakdown of fixes in this commit:

* `UserStat#topic_count` was not updated when visibility of
the topic changed.

* `UserStat#post_count` was not updated when post was hidden or
unhidden.

* `TopicConverter` was only incrementing or decrementing the counts by 1
even if a user has multiple posts in the topic.

* The commit turns off the verbose logging by default as it is just
noise to normal users who are not debugging this problem.
2022-02-11 09:00:58 +08:00
David Taylor
c81d369ab6 DEV: Improve theme error handling UX
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
2022-02-10 22:56:11 +00:00
Ayke Halder
5ff3a9c4bb
DEV: add native lazy loading for emojis () 2022-02-09 12:18:59 +01:00
Bianca Nenciu
f704deca17
FIX: Clear drafts only when post is created by real user ()
This commits adds a new advance_draft to PostCreator that controls if
the draft sequence will be advanced or not. If the draft sequence is
advanced then the old drafts will be cleared. This used to happen for
posts created by plugins or through the API and cleared user drafts
by mistake.
2022-02-09 10:37:38 +02:00
Blake Erickson
71f7f7ed49
FEATURE: Add external_id to topics ()
* FEATURE: Add external_id to topics

This commit allows for topics to be created and fetched by an
external_id. These changes are API only for now as there aren't any
front changes.

* add annotations

* add external_id to this spec

* Several PR feedback changes

- Add guardian to find topic
- 403 is returned for not found as well now
- add `include_external_id?`
- external_id is now case insensitive
- added test for posts_controller
- added test for topic creator
- created constant for max length
- check that it redirects to the correct path
- restrain external id in routes file

* remove puts

* fix tests

* only check for external_id in webhook if exists

* Update index to exclude external_id if null

* annotate

* Update app/controllers/topics_controller.rb

We need to check whether the topic is present first before passing it to the guardian.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-02-08 20:55:32 -07:00
David Taylor
ecd88e9abb
DEV: Make :send_system_message arguments strings () 2022-02-07 20:18:17 +00:00
Martin Brennan
0a738bd5bc
FEATURE: Allow sending group SMTP emails with from alias ()
This commit allows group SMTP emails to be sent with a
different from email address that has been set up as an
alias in the email provider. Emails from the alias will
be grouped correctly using Message-IDs in the mail client,
and replies to the alias go into the correct group inbox.
2022-02-07 13:52:01 +10:00
Alan Guo Xiang Tan
b7eacaed21
FIX: Handle addressable error when parsing an invalid URL. ()
Passing in an invalid URL would result in an `Addressable::URI::InvalidURIError` which we were not catching.
2022-02-07 11:25:42 +08:00
Alan Guo Xiang Tan
5bd55acf83
FIX: Add DB constraints for post & topic counter cache for UserStat ()
Ensures that `UserStat#post_count` and `UserStat#topic_count` does not
go below 0. When it does like it did now, we tend to have bugs in our
code since we're usually coding with the assumption that the count isn't
negative.

In order to support the constraints, our post and topic fabricators in
tests will now automatically increment the count for the respective
user's `UserStat` as well. We have to do this because our fabricators
bypasss `PostCreator` which holds the responsibility of updating `UserStat#post_count` and
`UserStat#topic_count`.
2022-02-07 11:23:34 +08:00
David Taylor
9ddd1f739e
DEV: Update :critical_user_email calls to use strings ()
Symbols are converted to strings anyway, so there is no change in behaviour. The latest version of sidekiq introduced a warning for this.
2022-02-04 23:43:53 +00:00
David Taylor
5d82a695d0
DEV: Use string timestamp when enqueuing update_top_redirection ()
Job arguments go via JSON, and so DateTime objects will appear as strings in the Job's `#execute` method. The latest version of Sidekiq has started warning about this to reduce developer confusion.
2022-02-04 23:14:35 +00:00
Alan Guo Xiang Tan
4aa9a813ec
FIX: Liking whispers should not contribute to Topic#like_count. ()
Non-staff users are not allowed to see whisper so this change prevents
non-staff user from seeing a like count that does not make sense to
them. In the future, we might consider adding another like count column
for staff user.

Follow-up to 4492718864
2022-02-03 16:24:33 +11:00
Krzysztof Kotlarek
bb01563682
FIX: post mover validation color and message ()
When the record is not saved, we should display a proper message.
One potential reason can be plugins for example discourse-calendar is specifying that only first post can contain event
2022-02-02 16:22:52 +11:00
Martin Brennan
0b8d0a14d7
DEV: Add markdown_additional_options to Site ()
Sometimes plugins need to have additional data or options available
when rendering custom markdown features/rules that are not available
on the default opts.discourse object. These additional options should
be namespaced to the plugin adding them.

```
Site.markdown_additional_options["chat"] = { limited_pretty_text_markdown_rules: [] }
```

These are passed down to markdown rules on opts.discourse.additionalOptions.

The main motivation for adding this is the chat plugin, which currently stores
chat_pretty_text_features and chat_pretty_text_markdown_rules on
the Site object via additions to the serializer, and the Site object is
not accessible to import via markdown rules (either through
Site.current() or through container.lookup). So, to have this working
for both front + backend code, we need to attach these additional options
from the Site object onto the markdown options object.
2022-01-28 13:02:02 +10:00
David Taylor
c6f8729b5c
DEV: Move OAuth2UserInfo deprecation to after_save ()
We initialize models as part of the warmup process in production, so this was being logged on every boot. We only want to log if a plugin is actually using the model, so after_save is a safer bet.
2022-01-25 10:29:31 +00:00
Bianca Nenciu
48e5d1af03
FIX: Improve top links section from user summary ()
* Do not extract links for hotlinked images
* Include only links that have been clicked at least once in user
summary
2022-01-24 11:33:23 +11:00
Blake Erickson
4bf6789bd7
DEV: Do not use hard-coded everyone group id ()
Follow up to: 12f041de5d

Probably best to lookup the "everyone" group_id instead of hard-coding
it to `0`. Also now its more clear what this `0` means.
2022-01-21 15:56:45 -07:00
David Taylor
6c3df84a93
DEV: In themes:update, only update themes which are out-of-date ()
Running `update_from_remote` and `save!` cause a number of side-effects, including instructing all clients to reload CSS files. If there are no changes, then this is wasteful, and can even cause a 'flicker' effect on clients as they reload CSS.

This commit checks if any updates are available before triggering `update_from_remote` / `save!`. This should be much faster, and stop the 'flickering' UX from happening on every themes:update run.

It also improves the output of the command to include the from/to commit hashes, which may be useful for debugging issues. For example:

```
Checking 'Alien Night | A Dark Discourse Theme' for 'default'... already up to date
Checking 'Star Wars' for 'default'... updating from d8a170dd to 66b9756f
Checking 'Media Overlay' for 'default'... already up to date
```
2022-01-21 18:23:26 +00:00
Martin Brennan
70af45055a
DEV: Clear custom field preload proxy on preload_custom_fields ()
If a model class calls preload_custom_fields twice then
we have to clear this otherwise the fields are cached inside the
already existing proxy and no new ones are added, so when we check
for custom_fields[KEY] an error is likely to occur
2022-01-21 14:29:51 +10:00
Martin Brennan
c1ae214c7b
DEV: Add more info to N1 custom field error ()
This commit makes a more specific N1NotPreLoadedError from
StandardError to raise when a custom field is loaded before
being preloaded, so it is easier to test that this does
not happen from plugins. Also adds the name of the class
trying to load the custom field to the error message.
2022-01-21 13:21:13 +10:00
Alan Guo Xiang Tan
e4e37257cc FIX: Handle malformed URLs in TopicEmbed.absolutize_urls. 2022-01-21 11:18:54 +08:00
Osama Sayegh
688be607c9
FIX: Allowed URLs for API scopes added by plugins () 2022-01-20 22:29:03 +03:00
David Taylor
820564826e
FIX: Ensure that login does not fail for users with invite records ()
In the unlikely, but possible, scenario where a user has no email_tokens, and has an invite record for their email address, login would fail. This commit fixes the `Invite` `user_doesnt_already_exist` validation so that it only applies to new invites, or when changing the email address.

This regressed in d8fe0f4199 (based on `git bisect`)
2022-01-20 10:54:38 +00:00
Blake Erickson
12f041de5d
FIX: Tag watching for everyone tag groups ()
* FIX: Tag watching for everyone tag groups

Tags in tag groups that have permissions set to everyone were not able
to be saved correctly. A user on their preferences page would mark the
tags that they wanted to save, but the watched_tags in the response
would be empty. This did not apply to admins, just regular users. Even
though the watched tags were being saved in the db, the user serializer
response was filtering them out. When a user refreshed their preferences
pages it would show zero watched tags.

This appears to be a regression introduced by:

0f598ca51e

The issue that needed to be fixed is that we don't track the "everyone"
group (which has an id of 0) in the group_users table. This is because
everyone has access to it, so why fill a row for every single user, that
would be a lot. The fix was to update the query to include tag groups
that had permissions set to the "everyone" group (group_id 0).

I also added another check to the existing spec for updating
watched tags for tags that aren't in a tag group so that it checks the
response body. I then added a new spec which updates watched tags for
tags in a tag group which has permissions set to everyone.

* Resolve failing tests

Improve SQL query syntax for including the "everyone" group with the id
of 0.

This commit also fixes a few failing tests that were introduced. It
turns out that the Fabrication of the Tag Group Permissions was faulty.
What happens when creating the tag groups without any permissions is
that it sets the permission to "everyone". If we then follow up with
fabricating a tag group permission on the tag group instead of having a
single permission it will have 2 (everyone + the group specified)! We
don't want this. To fix it I removed the fabrication of tag group
permissions and just set the permissions directly when creating the tag
group.

* Use response.parsed_body instead of JSON.parse
2022-01-18 15:02:29 -07:00
Mark VanLandingham
fcc80137ce
DEV: Add chat_group_mention notification type () 2022-01-18 08:26:27 -06:00
Roman Rizzi
5ee31cbf7d
FIX: Mark invites flash messages as HTML safe. ()
* FIX: Mark invites flash messages as HTML safe.
This change should be safe as all user inputs included in the errors are sanitized before sending it back to the client.

Context: https://meta.discourse.org/t/html-tags-are-explicit-after-latest-update/214220

* If somebody adds a new error message that includes user input and doesn't sanitize it, using html-safe suddenly becomes unsafe again. As an extra layer of protection, we make the client sanitize the error message received from the backend.

* Escape user input instead of sanitizing
2022-01-18 09:38:31 -03:00
Rafael dos Santos Silva
8cfd791e0f
FEATURE: Add missing emojis ()
* FEATURE: Add missing emojis

* More missing emojis
2022-01-14 17:51:13 -03:00
David Taylor
b3e52f99e6
FIX: LOAD_PLUGINS=0 in dev/prod, warn in plugin:pull_compatible_all ()
The `plugin:pull_compatible_all` task is intended to take incompatible plugins and downgrade them to an earlier version. Problem is, when running the rake task in development/production environments, the plugins have already been activated. If an incompatible plugin raises an error in `plugin.rb` then the rake task will be unable to start.

This commit centralises our LOAD_PLUGINS detection, adds support for LOAD_PLUGINS=0 in dev/prod, and adds a warning to `plugin:pull_compatible_all` if it's run with plugins enabled.
2022-01-11 12:30:22 +00:00
Bianca Nenciu
5d35c38db2
FEATURE: Search screened IP address in blocks ()
An admin could search for all screened ip addresses in a block by
using wildcards. 192.168.* returned all IPs in range 192.168.0.0/16.
This feature allows admins to search for a single IP address in all
screened IP blocks. 192.168.0.1 returns all IP blocks that match it,
for example 192.168.0.0/16.

* FEATURE: Remove roll up button for screened IPs

* FIX: Match more specific screened IP address first
2022-01-11 09:16:51 +02:00
Alan Guo Xiang Tan
6fb89c153a Revert "DEV: Remove stale ignored_columns from models."
This reverts commit 9f5c8644d0.

Have to revert because the ignored columns have not been dropped.
2022-01-11 11:00:58 +08:00
Alan Guo Xiang Tan
9f5c8644d0 DEV: Remove stale ignored_columns from models. 2022-01-11 10:38:10 +08:00
Alan Guo Xiang Tan
ab2c17236a DEV: Follow Discourse's convention when monkey patching.
Having to load `ip_addr` is confusing especially when that file exists
to monkey patch Ruby's `IpAddr` class. Moving it to our freedom patches
folder which is automatically loaded on initialization.
2022-01-11 09:48:27 +08:00
Rafael dos Santos Silva
06bba76719
DEV: Update emojis constants ()
Follow up to d1cfabd so the new emojis are usable by the app
2022-01-10 14:53:52 -03:00
Alan Guo Xiang Tan
f99ade3ce5 DEV: Ignore dropped Bookmark columns.
This is a partial revert of 099b679fc5.
`Bookmark#topic_id` and `Bookmark#reminder_type` was dropped in
b22450c7a8 so we need to continue ignoring
the dropped columns so as to ensure a seamless deploy. Otherwise,
ActiveRecord's schema cache will still contain references to
`Bookmark#topic_id` when the column is dropped in a post migration.
2022-01-10 16:04:22 +08:00
Arpit Jalan
554ff07786
FIX: when composing a message do not suggest deleted posts users () 2022-01-07 17:06:11 +05:30
Martin Brennan
04c7776650
DEV: Rolling back bookmarkable column changes ()
It is too close to release of 2.8 for incomplete
feature shenanigans. Ignores and drops the columns and drops
the trigger/function introduced in
e21c640a3c.
Will pick this feature back up post-release.
2022-01-07 12:16:43 +10:00
David Taylor
78d0ec35a5
DEV: Deprecate OAuth2Authenticator and OAuth2UserInfo ()
These have been superseded by ManagedAuthenticator and UserAssociatedAccount. For more information, see https://meta.discourse.org/t/106695
2022-01-06 16:50:18 +00:00
Andrei Prigorshnev
16d9b2755c
DEV: rename single_sign_on classes to discourse_connect () 2022-01-06 16:28:46 +04:00
Martin Brennan
e21c640a3c
DEV: Add polymorphic bookmarkable columns ()
We are planning on attaching bookmarks to more and
more other models, so it makes sense to make a polymorphic
relationship to handle this. This commit adds the new
columns and backfills them in the bookmark table, and
makes sure that any new bookmark changes fill in the columns
via DB triggers.

This way we can gradually change the frontend and backend
to use these new columns, and eventually delete the
old post_id and for_topic columns in `bookmarks`.
2022-01-06 08:56:05 +10:00
Peter Zhu
c5fd8c42db
DEV: Fix methods removed in Ruby 3.2 ()
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
2022-01-05 18:45:08 +01:00
Martin Brennan
099b679fc5
DEV: Clean up old bookmark code ()
The rake task deleted here was added back in Feb 2020
when bookmarks were first converted from PostAction
records, it is no longer needed. The ignored columns
were removed in ed83d7573e.
2022-01-05 10:02:02 +10:00
Martin Brennan
b22450c7a8
DEV: Drop old bookmark columns ()
As per 22208836c5
these are no longer needed, a follow up commit will remove the ignored
columns.
2022-01-04 11:19:27 +10:00
Martin Brennan
20fe5eceb8
FEATURE: Scheduled group email credential problem check ()
This commit adds a check that runs regularly as per
2d68e5d942 which tests the
credentials of groups with SMTP or IMAP enabled. If any issues
are found with those credentials a high priority problem is added to the
admin dashboard.

This commit also formats the admin dashboard differently if
there are high priority problems, bringing them to the top of
the list and highlighting them.

The problem will be cleared if the issue is fixed before the next
problem check, or if the group's settings are updated with a valid
credential.
2022-01-04 10:14:33 +10:00
Arpit Jalan
702685b6a0
SECURITY: only show user suggestions with regular post () 2022-01-03 13:37:40 +05:30
Osama Sayegh
8e6988163f
FIX: Don't allow NULL values for notification_level in category_users () 2021-12-29 09:19:39 +11:00
Osama Sayegh
0192efb37b
FIX: Remove length constraint of fancy_title in topics ()
The `fancy_title` column in the `topics` table currently has a constraint that limits the column to 400 characters. We need to remove that constraint because it causes some automatic topics/PMs from the system to fail when using Discourse in locales that need more than 400 characters to the translate the content of those automatic messages.

Internal ticket: t58030.
2021-12-21 21:24:23 +03:00
Andrei Prigorshnev
c202252190
FEATURE: when suggesting usernames skip input that consist entirely of disallowed characters () 2021-12-21 21:13:05 +04:00
Martin Brennan
2d68e5d942
FEATURE: Scheduled problem checks for admin dashboard ()
This commit introduces scheduled problem checks for the admin dashboard, which are long running or otherwise cumbersome problem checks that will be run every 10 minutes rather than every time the dashboard is loaded. If these scheduled checks add a problem, the problem will remain until it is cleared or until the scheduled job runs again.

An example of a check that should be scheduled is validating credentials against an external provider.

This commit also introduces the concept of a `priority` to the problems generated by `AdminDashboardData` and the scheduled checks. This is `low` by default, and can be set to `high`, but this commit does not change any part of the UI with this information, only adds a CSS class.

I will be making a follow up PR to check group SMTP credentials.
2021-12-20 09:59:11 +10:00
Andrei Prigorshnev
9365c4b364
DEV: make sure we handle staged users correctly in DiscourseConnect ()
Some time ago, we made this fix to external authentication –  https://github.com/discourse/discourse/pull/13706. We didn't address Discourse Connect (https://meta.discourse.org/t/discourseconnect-official-single-sign-on-for-discourse-sso/13045) at that moment, so I wanted to fix it for Discourse Connect as well.

Turned out though that Discourse Connect doesn't contain this problem and already handles staged users correctly. This PR adds tests that confirm it. Also, I've extracted two functions in Discourse Connect implementation along the way and decided to merge this refactoring too (the refactoring is supported with tests).
2021-12-16 19:44:07 +04:00
Martin Brennan
4519f3f137
FIX: Add more actions to the uploads API key scope ()
The uploads API key create scope did not cover the
external upload API endpoints, or the direct S3
multipart endpoints, and this commit adds them.

cf. https://meta.discourse.org/t/upload-create-api-key-insufficient/211896
2021-12-15 14:08:11 +10:00
Mark VanLandingham
175ced5096
DEV: Add chat invitation notification type () 2021-12-14 09:01:53 -06:00
Dan Ungureanu
adb6202c94
FIX: Check if invite domain is valid ()
* FIX: Check if invite domain is valid

Previous regex checked for generic hostname, which is too generic for
this case.
2021-12-13 16:39:14 +11:00
Dan Ungureanu
3d4aee1487
DEV: Drop unused column email_tokens.token () 2021-12-13 16:29:47 +11:00
Angus McLeod
df3886d6e5
FEATURE: Experimental support for group membership via google auth ()
This commit introduces a new site setting "google_oauth2_hd_groups". If enabled, group information will be fetched from Google during authentication, and stored in the Discourse database. These 'associated groups' can be connected to a Discourse group via the "Membership" tab of the group preferences UI. 

The majority of the implementation is generic, so we will be able to add support to more authentication methods in the near future.

https://meta.discourse.org/t/managing-group-membership-via-authentication/175950
2021-12-09 12:30:27 +00:00
Dan Ungureanu
d8fe0f4199
FEATURE: Restrict link invites to email domain ()
Allow multiple emails to redeem a link invite only if the email domain
name matches the one specified in the link invite.
2021-12-08 17:06:57 +02:00
Bianca Nenciu
b1c11d5787
FIX: Select correct topic draft for user ()
The old query could return multiple rows.
2021-12-08 15:23:44 +02:00
Bianca Nenciu
049bc33838
FIX: Update has_topic_draft when draft is updated ()
Current user state regarding the new topic draft was not updated when
the draft was created or destroyed.
2021-12-08 14:40:35 +02:00
Loïc Guitaut
74387e83b6 DEV: Stop polluting all Ruby classes
The `ReviewableScore` model was defining class methods on `self.class`
from a singleton context so instead of defining methods on
`ReviewableScore` it was defining them on `Class`, so basically on every
existing class.

This patch resolves this issue. Using `enum` from `ActiveRecord` in the
future will avoid this kind of problems.
2021-12-08 11:32:25 +01:00
Dan Ungureanu
9a6ec1d0c6 PERF: Add index on email_tokens.token_hash 2021-12-07 10:17:45 +08:00
Alan Guo Xiang Tan
4e67297a7c FIX: Missing allowed urls when displaying granualar API key scopes.
Follow-up to 3791fbd919
2021-12-07 10:17:17 +08:00
Alan Guo Xiang Tan
44588255fc FEATURE: Introduce API scopes for badges. 2021-12-07 10:17:17 +08:00
Daniel Waterworth
bd10f113e9
DEV: Raise errors for (black|white)list accesses ()
These have been deprecated for a while
2021-12-02 12:16:55 -06:00
Michelle Bueno Saquetim Vendrame
9b5836aa1d
Add three reports ()
* Add report top_users_by_received_likes

* Add report top_users_by_received_likes_from_inferior_trust_level

* Add report top_users_by_likes_received_from_a_variety_of_people

* Add test to report_top_users_by_received_likes

* add top_users_by_likes_received_from_a_variety_of_people report test

* add top_users_by_likes_received_from_inferior_trust_level report tests
2021-12-02 22:41:55 +05:30
Daniel Waterworth
2f04a9b9fb
DEV: Remove site_setting_saved event ()
We said we would drop it from 2.4, so this is long overdue

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2021-12-02 09:33:03 -06:00
Andrei Prigorshnev
1c0022c195
FIX: extract and fix overriding of usernames by external auth () 2021-12-02 17:42:23 +04:00
Osama Sayegh
1d69261bc0
FIX: Set auto_update to false for non-git themes/components ()
Related to: 20f736aa11.

`auto_update` is true by default at the database level, but it doesn't make sense for `auto_update` to be true on themes that are not imported from a Git repository.
2021-12-01 19:58:13 +03:00
Natalie Tay
0f598ca51e
SECURITY: Only show tags to users with permission () 2021-12-01 10:26:56 +08:00
Krzysztof Kotlarek
9cabd3721b
FEATURE: ability to add description to tags ()
Ability to add description to tags, which will be displayed on hover.
2021-12-01 09:18:56 +11:00
Roman Rizzi
1fc06520bd
REFACTOR: Improve support for consolidating notifications. ()
* REFACTOR: Improve support for consolidating notifications.

Before this commit, we didn't have a single way of consolidating notifications. For notifications like group summaries, we manually removed old ones before creating a new one. On the other hand, we used an after_create callback for likes and group membership requests, which caused unnecessary work, as we need to delete the record we created to replace it with a consolidated one.

We now have all the consolidation rules centralized in a single place: the consolidation planner class. Other parts of the app looking to create a consolidable notification can do so by calling Notification#consolidate_or_save!, instead of the default Notification#create! method.

Finally, we added two more rules: one for re-using existing group summaries and another for deleting duplicated dashboard problems PMs notifications when the user is tracking the moderator's inbox. Setting the threshold to one forces the planner to apply this rule every time.

I plan to add plugin support for adding custom rules in another PR to keep this one relatively small.

* DEV: Introduces a plugin API for consolidating notifications.

This commit removes the `Notification#filter_by_consolidation_data` scope since plugins could have to define their criteria. The Plan class now receives two blocks, one to query for an already consolidated notification, which we'll try to update, and another to query for existing ones to consolidate.

It also receives a consolidation window, which accepts an ActiveSupport::Duration object, and filter notifications created since that value.
2021-11-30 13:36:14 -03:00
Loïc Guitaut
a5fbb90df4 FEATURE: Display pending posts on user’s page
Currently when a user creates posts that are moderated (for whatever
reason), a popup is displayed saying the post needs approval and the
total number of the user’s pending posts. But then this piece of
information is kind of lost and there is nowhere for the user to know
what are their pending posts or how many there are.

This patch solves this issue by adding a new “Pending” section to the
user’s activity page when there are some pending posts to display. When
there are none, then the “Pending” section isn’t displayed at all.
2021-11-29 10:26:33 +01:00
Jarek Radosz
2971d03307
DEV: Don't create unnecessary scope methods ()
Skipping methods we don't use gives us mem/perf gains (minuscule but still), but more importantly fixes warnings about `Poll#open` (created by `enum :status`) conflicting with some internal AR method. 😃
2021-11-26 16:34:07 +01:00
Jarek Radosz
1441226b1a
DEV: Don't polute all ActiveRecord classes ()
`pending`, `approved`, `rejected`, `ignored`, and `deleted` scope method were accessible on all model classes… 😂

Fixes `Creating scope :pending. Overwriting existing method DiscoursePostEvent::EventDate.pending.` warnings.
2021-11-26 10:17:10 +08:00
Penar Musaraj
d99deaf1ab
FEATURE: show recent searches in quick search panel () 2021-11-25 15:44:15 -05:00
Dan Ungureanu
fa8cd629f1
DEV: Hash tokens stored from email_tokens ()
This commit adds token_hash and scopes columns to email_tokens table.
token_hash is a replacement for the token column to avoid storing email
tokens in plaintext as it can pose a security risk. The new scope column
ensures that email tokens cannot be used to perform a different action
than the one intended.

To sum up, this commit:

* Adds token_hash and scope to email_tokens

* Reuses code that schedules critical_user_email

* Refactors EmailToken.confirm and EmailToken.atomic_confirm methods

* Periodically cleans old, unconfirmed or expired email tokens
2021-11-25 09:34:39 +02:00
Bianca Nenciu
3ea8937157
FEATURE: Add email normalization rules setting ()
When this setting is turned on, it will check that normalized emails
are unique. Normalized emails are emails without any dots or plus
aliases.

This setting can be used to block use of aliases of the same email
address.
2021-11-24 11:30:06 +02:00
Bianca Nenciu
73760c77d9
FEATURE: Mention @here to notify users in topic ()
Use @here to mention all users that were allowed to topic directly or
through group, who liked topics or read the topic. Only first 10 users
will be notified.
2021-11-23 22:25:54 +02:00
Alan Guo Xiang Tan
8226ab1099
PERF: Updating first unread PM for user not respecting limits. ()
In b8c8909a9d, we introduced a regression
where users may have had their `UserStat.first_unread_pm_at` set
incorrectly. This commit introduces a migration to reset `UserStat.first_unread_pm_at` back to
`User#created_at`.

Follow-up to b8c8909a9d.
2021-11-23 12:51:54 +08:00
Dan Ungureanu
d420a7b2c8
DEV: Reuse code for TrustLevelAndStaffSetting ()
The code that checked this permission was duplicated everytime a new
settings of this type was added. This commit changes the behavior of
some functionality because some feature checks were bypassed for staff
members.
2021-11-22 20:18:53 +02:00
RogerBW
fd66df5997
FEATURE: adds uploads scope for API keys ()
* FEATURE: adds uploads scope for API keys

* Add basic test, change "image" to "file"
2021-11-22 10:49:08 -07:00
Jarek Radosz
d91d67a442
FEATURE: Allow theme settings to request refresh ()
Similar to site settings, adds support for `refresh` option to theme settings.

```yaml
super_feature_enabled:
  type: bool
  default: false
  refresh: true
```
2021-11-22 13:16:56 +01:00
Martin Brennan
9f8ee8f137
FIX: Don't publish notifications to MessageBus for inactive users ()
We are pushing /notification-alert/#{user_id} and /notification/#{user_id}
messages to MessageBus from both PostAlerter and User#publish_notification_state.
This can cause memory issues on large sites with many users. This commit
stems the bleeding by only sending these alert messages if the user
in question has been seen in the last 30 days, which eliminates a large
chunk of users on some sites.
2021-11-22 14:38:49 +11:00
Alan Guo Xiang Tan
4b4973ee0d
PERF: Reduce records queried in UserStat.update_first_unread_pm. ()
The inefficiency here is that we were previously fetching all the
records from `TopicAllowedUser` before filtering against a limited subset of
users based on `User#last_seen_at`.
2021-11-19 15:30:39 +11:00
Vinoth Kannan
fc1c76cfcc
FIX: exclude moderator_action post for reply count in user summary. ()
Previously, incorrect reply counts are displayed in the "top categories" section of the user summary page since we included the `moderator_action` and `small_action` post types.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-11-18 13:42:03 +05:30
Osama Sayegh
b86127ad12
FEATURE: Apply rate limits per user instead of IP for trusted users ()
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).

This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).

For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.

The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.

Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.

Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.

Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.

Internal ticket number: t54739.
2021-11-17 23:27:30 +03:00
Bianca Nenciu
cc1b45f58b
FIX: Convert URLs embedded topics to absolute form ()
Sometimes the expanded post contained broken relative URLs because they
were not converted to their absolute form.
2021-11-17 16:39:49 +11:00
Vinoth Kannan
c47a526371
FIX: exclude topics from muted tag in category featured list. ()
Topics from muted tags were visible in the categories page's featured topics section since we didn't filter it before.
2021-11-16 12:10:50 +05:30
Martin Brennan
31035010af
FIX: Use correct group out of multiple for SMTP sender ()
When there are multiple groups on a topic, we were selecting
the first from the topic allowed groups to act as the sender
email address when sending group SMTP replies via PostAlerter.
However, this was not ordered, and since there is no created_at
column on TopicAllowedGroup we cannot order this nicely, which
caused just a random group to be used (based on whatever postgres
decided it felt like that morning).

This commit changes the group used for SMTP sending to be the
group using the email_username of the to address of the first
incoming email for the topic, if there are more than one allowed
groups on the topic. Otherwise it just uses the only SMTP enabled
group.
2021-11-16 10:21:49 +10:00
Daniel Waterworth
4724f3cbf7
DEV: Remove deprecated method () 2021-11-12 09:07:44 -06:00
Daniel Waterworth
e7c0bbb9c0
DEV: Let's always give a drop_from param to deprecate ()
So that we know when deprecations can be removed in the future.
2021-11-12 08:52:59 -06:00
Jarek Radosz
61d14a7694
DEV: Fix 3N+1 query in /admin/customize/themes () 2021-11-11 18:11:23 +01:00
Bianca Nenciu
3791fbd919
FEATURE: Add read-only scope to API keys ()
This commit adds a global read-only scope that can be used to create
new API keys.
2021-11-10 17:48:00 +02:00
David Taylor
5ac10e2e79
DEV: Update DiscourseConnect nonce errors to be more descriptive () 2021-11-09 17:39:05 +00:00
Krzysztof Kotlarek
8b93da9fe0
FIX: rename action_code_href to action_code_path ()
Small actions should use path instead of absolute url. getURL function is necessary to insert a potential subfolder prefix.
2021-11-08 14:32:17 +11:00
Krzysztof Kotlarek
fe8087e523
FEATURE: small action post accepts href ()
Optionally add href to small action.
It can be used by discourse-assign to link to correct post from translation
2021-11-08 08:24:44 +11:00
Vinoth Kannan
53b7220638
FIX: exclude suppressed category topics in digest even if unmuted. ()
Previously, suppressed category topics are included in the digest emails if the user visited that topic before and the `TopicUser` record is created with any notification level except 'muted'.
2021-11-03 12:47:09 +05:30
jbrw
cfc62dbace
FIX: allowed_theme_ids should not be persisted in GlobalSettings ()
* FIX: allowed_theme_ids should not be persisted in GlobalSettings

It was observed that the memoized value of `GlobalSetting.allowed_theme_ids` would be persisted across requests, which could lead to unpredictable/undesired behaviours in a multisite environment.

This change moves that logic out of GlobalSettings so that the returned theme IDs are correct for the current site.

Uses get_set_cache, which ultimately uses DistributedCache, which will take care of multisite issues for us.
2021-10-29 11:46:52 -04:00
Bianca Nenciu
1c3c0f04d9
FEATURE: Pull hotlinked images in user bios () 2021-10-29 17:58:05 +03:00
Bianca Nenciu
cb0958fcea
FIX: Hide links to muted topics and in categories list ()
* FIX: Hide links to muted topics

* FIX: Hide muted topics in categories list on mobile
2021-10-29 17:52:23 +03:00
Jarek Radosz
19c9b892dc
DEV: Instantiate relation early to save a query ()
Previously it would do `SELECT 1 AS one` and then `SELECT "color_scheme_colors".*`. Now it only does the latter.
2021-10-29 03:03:22 +02:00
Jarek Radosz
85d0ddb9eb
FIX: Avoid another N+1 query in Site.json_for ()
A follow-up to , this time for logged-in users and/or non-login-required sites.
2021-10-28 20:28:31 +02:00
Vinoth Kannan
deee715a2c
FIX: update flair group of all members if primary group setting changed. ()
Previously, if we enable the `primary_group` setting on a group then the `flair_group_id` of its' members are not affected.
2021-10-28 22:56:44 +05:30
Vinoth Kannan
c62242c6b3
FIX: should not receive topic invites from ignored users. ()
Previously, ignored users can send notifications by inviting the ignorer to topics or PMs.
2021-10-28 21:03:00 +05:30
Dan Ungureanu
ba3078e098
PERF: Use different column for better query plan ()
Using topics.id provides a better query plan than posts.topic_id which
speeds up search by almost 50%.
2021-10-28 11:30:30 +03:00
Roman Rizzi
df3eb93973
DEV: Sanitize HTML admin inputs ()
* DEV: Sanitize HTML admin inputs

This PR adds on-save HTML sanitization for:

Client site settings
translation overrides
badges descriptions
user fields descriptions

I used Rails's SafeListSanitizer, which [accepts the following HTML tags and attributes](018cf54073/lib/rails/html/sanitizer.rb (L108))

* Make sure that the sanitization logic doesn't corrupt settings with special characters
2021-10-27 11:33:07 -03:00
Andrei Prigorshnev
19d95c64af
DEV: simplify username suggester ()
This PR doesn't change any behavior, but just removes code that wasn't in use. This is a pretty dangerous place to change, since it gets called during user's registration. At the same time the refactoring is very straightforward, it's clear that this code wasn't doing any work (it still needs to be double-checked during review though). Also, the test coverage of UserNameSuggester is good.
2021-10-27 14:41:24 +04:00
Dan Ungureanu
69f0f48dc0
DEV: Fix rubocop issues () 2021-10-27 11:39:28 +03:00
Jarek Radosz
c7768b6d16
FIX: Avoid N+1 query in Site.json_for () 2021-10-26 22:46:53 +02:00
Bianca Nenciu
0c6f9d7c67
FIX: Show right message when permanently deleting topic () 2021-10-26 18:31:15 +03:00
Dan Ungureanu
f003e31e2f
PERF: Optimize search in private messages query ()
* PERF: Remove JOIN on categories for PM search

JOIN on categories is not needed when searchin in private messages as
PMs are not categorized.

* DEV: Use == for string comparison

* PERF: Optimize query for allowed topic groups

There was a query that checked for all topics a user or their groups
were allowed to see. This used UNION between topic_allowed_users and
topic_allowed_groups which was very inefficient. That was replaced with
a OR condition that checks in either tables more efficiently.
2021-10-26 10:16:38 +03:00
Martin Brennan
2b40049abb
FIX: Remove superfluous topic allowed users on group invite ()
When inviting a group to a topic, there may be members of
the group already in the topic as topic allowed users. These
can be safely removed from the topic, because they are implicitly
allowed in the topic based on their group membership.

Also, this prevents issues with group SMTP emails, which rely
on the topic_allowed_users of the topic to send to and cc's
for emails, and if there are members of the group as topic_allowed_users
then that complicates things and causes odd behaviour.

We also ensure that the OP of the topic is not removed from
the topic_allowed_users when a group they belong to is added,
as it will make it harder to add them back later.
2021-10-22 08:57:51 +10:00
Osama Sayegh
8fb823c30f
FIX: Make the verbose_auth_token_logging setting off by default ()
The `generate`, `rotate` and `suspicious`  auth token logs are now always logged regardless of the `verbose_auth_token_logging` setting because we rely no these to detect suspicious logins.
2021-10-20 17:20:39 +03:00
Andrei Prigorshnev
b9164d22ee
FEATURE: add a hidden setting that enables using email as a source for username suggestions ()
This is a follow-up to https://github.com/discourse/discourse/pull/14541. This adds a hidden setting for restoring the old behavior for those users who rely on it. We'll likely deprecate this setting at some point in the future.
2021-10-19 14:37:39 +04:00
Jean
6275658e3d
FEATURE: Add setting to disable notifications for topic category edits () 2021-10-18 09:04:01 -04:00
Bianca Nenciu
c4843fc1c1
FEATURE: Allow admins to permanently delete posts and topics ()
Sometimes administrators want to permanently delete posts and topics
from the database. To make sure that this is done for a good reasons,
administrators can do this only after one minute has passed since the
post was deleted or immediately if another administrator does it.
2021-10-13 12:53:23 +03:00
Andrei Prigorshnev
88ecb83382
FEATURE: stop using email as source for username and name suggestions for Single Sign On ()
We don't want to be using emails as source for username and name suggestions in cases when it's possible that a user have no chance to intervene and correct a suggested username. It risks exposing email addresses.
2021-10-12 17:25:54 +04:00
Krzysztof Kotlarek
300ed6ea75
FIX: extract filter pm and categories from UserAction ()
Those two methods are useful for different places like discourse-reactions:

213d90b82f (diff-00e1224615af30993a63193531b5073c11d5c20e91c46e2a59c1cd25e8952da6R203)
2021-10-12 17:37:09 +11:00
Vinoth Kannan
eeecb2460d
FIX: nil the baked version after moving the posts. ()
Previosuly, quotes from original topics are rendered incorrectly since the moved posts are not rebaked.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-10-12 17:31:18 +11:00
Krzysztof Kotlarek
0301b775a8
FIX: topic_tracking_state not erroring when missing user_stat ()
publish_unread should not error when UserStat is missing
2021-10-11 13:20:55 +11:00
Yasuo Honda
2944d2cdd6
FIX: Ruby 3 does not freeze interpolated string ()
Ruby 2.7 or earlier `+contents` returns self.dup
when `frozen_string_literal: true`. However, Ruby 3.0 returns self
because this string is interpolated one, which is not frozen anymore.

This commit uses self.dup to return duplicated string regardless Ruby
versions.
https://bugs.ruby-lang.org/issues/17104
2021-10-11 13:20:18 +11:00
Krzysztof Kotlarek
cb5b0cb9d8
FEATURE: save local date to calendar ()
It allows saving local date to calendar.
Modal is giving option to pick between ics and google. User choice can be remembered as a default for the next actions.
2021-10-06 14:11:52 +11:00
Blake Erickson
fe676f334a
FEATURE: Return subcategories on categories endpoint ()
* FEATURE: Return subcategories on categories endpoint

When using the API subcategories will now be returned nested inside of
each category response under the `subcategory_list` param. We already
return all the subcategory ids under the `subcategory_ids` param, but
you then would have to make multiple separate API calls to fetch each of
those subcategories. This way you can get **ALL** of the categories
along with their subcategories in a single API response.

The UI will not be affected by this change because you need to pass in
the `include_subcategories=true` param in order for subcategories to be
returned.

In a follow up PR I'll add the API scoping for fetching categories so
that a readonly API key can be used for the `/categories.json` endpoint. This
endpoint should be used instead of the `/site.json` endpoint for
fetching a sites categories and subcategories.

* Update PR based on feedback

- Have spec check for specific subcategory
- Move comparison check out of loop
- Only populate subcategory list if option present
- Remove empty array initialization
- Update api spec to allow null response

* More PR updates based on feedback

- Use a category serializer for the subcategory_list
- Don't include the subcategory_list param if empty
- For the spec check for the subcategory by id
- Fix spec to account for param not present when empty
2021-10-05 12:12:31 -06:00
Yasuo Honda
dbbfad7ed0 FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
Alan Guo Xiang Tan
4cade9d7f6
FIX: Topic.similar_to results in invalid query for certain locales. ()
For `zh_CN`, we use the `cppjieba_rb` gem to remove stop words so
calling `Search.prepare_data` may result in an empty string.
2021-10-04 11:40:22 +08:00
Roman Rizzi
4c2d5158c5
FIX: Follow the canonical URL when importing a remote topic. ()
FinalDestination now supports the `follow_canonical` option, which will perform an initial GET request, parse the canonical link if present, and perform a HEAD request to it.

We use this mode during embeds to avoid treating URLs with different query parameters as different topics.
2021-10-01 12:48:21 -03:00
Andrei Prigorshnev
b609f6c11c
FIX: restrict other user's notification routes ()
It was possible to see notifications of other users using routes:
- notifications/responses
- notifications/likes-received
- notifications/mentions
- notifications/edits

We weren't showing anything private (like notifications about private messages), only things that're publicly available in other places. But anyway, it feels strange that it's possible to look at notifications of someone else. Additionally, there is a risk that we can unintentionally leak something on these pages in the future.

This commit restricts these routes.
2021-09-29 16:24:28 +04:00
Alan Guo Xiang Tan
a1745e05ae
FIX: Do not publish post for PM topic tracking if not new for user. () 2021-09-29 13:54:24 +08:00
Alan Guo Xiang Tan
cd64e88711
PERF: Improve database query perf when loading topics for a category. ()
* PERF: Improve database query perf when loading topics for a category.

Instead of left joining the `topics` table against `categories` by filtering with `categories.id`,
we can improve the query plan by filtering against `topics.category_id`
first before joining which helps to reduce the number of rows in the
topics table that has to be joined against the other tables and also
make better use of our existing index.

The following is a before and after of the query plan for a category
with many subcategories.

Before:

```
                                                                                                       QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------
 Limit  (cost=1.28..747.09 rows=30 width=12) (actual time=85.502..2453.727 rows=30 loops=1)
   ->  Nested Loop Left Join  (cost=1.28..566518.36 rows=22788 width=12) (actual time=85.501..2453.722 rows=30 loops=1)
         Join Filter: (category_users.category_id = topics.category_id)
         Filter: ((topics.category_id = 11) OR (COALESCE(category_users.notification_level, 1) <> 0) OR (tu.notification_level > 1))
         ->  Nested Loop Left Join  (cost=1.00..566001.58 rows=22866 width=20) (actual time=85.494..2453.702 rows=30 loops=1)
               Filter: ((COALESCE(tu.notification_level, 1) > 0) AND ((topics.category_id <> 11) OR (topics.pinned_at IS NULL) OR ((t
opics.pinned_at <= tu.cleared_pinned_at) AND (tu.cleared_pinned_at IS NOT NULL))))
               Rows Removed by Filter: 1
               ->  Nested Loop  (cost=0.57..528561.75 rows=68606 width=24) (actual time=85.472..2453.562 rows=31 loops=1)
                     Join Filter: ((topics.category_id = categories.id) AND ((categories.topic_id <> topics.id) OR (categories.id = 1
1)))
                     Rows Removed by Join Filter: 13938306
                     ->  Index Scan using index_topics_on_bumped_at on topics  (cost=0.42..100480.05 rows=715549 width=24) (actual ti
me=0.010..633.015 rows=464623 loops=1)
                           Filter: ((deleted_at IS NULL) AND ((archetype)::text <> 'private_message'::text))
                           Rows Removed by Filter: 105321
                     ->  Materialize  (cost=0.14..36.04 rows=30 width=8) (actual time=0.000..0.002 rows=30 loops=464623)
                           ->  Index Scan using categories_pkey on categories  (cost=0.14..35.89 rows=30 width=8) (actual time=0.006.
.0.040 rows=30 loops=1)
                                 Index Cond: (id = ANY ('{11,53,57,55,54,56,112,94,107,115,116,117,97,95,102,103,101,105,99,114,106,1
13,104,98,100,96,108,109,110,111}'::integer[]))
               ->  Index Scan using index_topic_users_on_topic_id_and_user_id on topic_users tu  (cost=0.43..0.53 rows=1 width=16) (a
ctual time=0.004..0.004 rows=0 loops=31)
                     Index Cond: ((topic_id = topics.id) AND (user_id = 1103877))
         ->  Materialize  (cost=0.28..2.30 rows=1 width=8) (actual time=0.000..0.000 rows=0 loops=30)
               ->  Index Scan using index_category_users_on_user_id_and_last_seen_at on category_users  (cost=0.28..2.29 rows=1 width
=8) (actual time=0.004..0.004 rows=0 loops=1)
                     Index Cond: (user_id = 1103877)
 Planning Time: 1.359 ms
 Execution Time: 2453.765 ms
(23 rows)
```

After:

```
                                                                                                                            QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1.28..438.55 rows=30 width=12) (actual time=38.297..657.215 rows=30 loops=1)
   ->  Nested Loop Left Join  (cost=1.28..195944.68 rows=13443 width=12) (actual time=38.296..657.211 rows=30 loops=1)
         Filter: ((categories.topic_id <> topics.id) OR (topics.category_id = 11))
         Rows Removed by Filter: 29
         ->  Nested Loop Left Join  (cost=1.13..193462.59 rows=13443 width=16) (actual time=38.289..657.092 rows=59 loops=1)
               Join Filter: (category_users.category_id = topics.category_id)
               Filter: ((topics.category_id = 11) OR (COALESCE(category_users.notification_level, 1) <> 0) OR (tu.notification_level > 1))
               ->  Nested Loop Left Join  (cost=0.85..193156.79 rows=13489 width=20) (actual time=38.282..657.059 rows=59 loops=1)
                     Filter: ((COALESCE(tu.notification_level, 1) > 0) AND ((topics.category_id <> 11) OR (topics.pinned_at IS NULL) OR ((topics.pinned_at <= tu.cleared_pinned_at) AND (tu.cleared_pinned_at IS NOT NULL))))
                     Rows Removed by Filter: 1
                     ->  Index Scan using index_topics_on_bumped_at on topics  (cost=0.42..134521.06 rows=40470 width=24) (actual time=38.267..656.850 rows=60 loops=1)
                           Filter: ((deleted_at IS NULL) AND ((archetype)::text <> 'private_message'::text) AND (category_id = ANY ('{11,53,57,55,54,56,112,94,107,115,116,117,97,95,102,103,101,105,99,114,106,113,104,98,100,96,108,109,110,111}'::integer[])))
                           Rows Removed by Filter: 569895
                     ->  Index Scan using index_topic_users_on_topic_id_and_user_id on topic_users tu  (cost=0.43..1.43 rows=1 width=16) (actual time=0.003..0.003 rows=0 loops=60)
                           Index Cond: ((topic_id = topics.id) AND (user_id = 1103877))
               ->  Materialize  (cost=0.28..2.30 rows=1 width=8) (actual time=0.000..0.000 rows=0 loops=59)
                     ->  Index Scan using index_category_users_on_user_id_and_last_seen_at on category_users  (cost=0.28..2.29 rows=1 width=8) (actual time=0.004..0.004 rows=0 loops=1)
                           Index Cond: (user_id = 1103877)
         ->  Index Scan using categories_pkey on categories  (cost=0.14..0.17 rows=1 width=8) (actual time=0.001..0.001 rows=1 loops=59)
               Index Cond: (id = topics.category_id)
 Planning Time: 1.633 ms
 Execution Time: 657.255 ms
(22 rows)
```

* PERF: Optimize index on topics bumped_at.

Replace `index_topics_on_bumped_at` index with a partial index on `Topic#bumped_at` filtered by archetype since there is already another index that covers private topics.
2021-09-28 10:05:00 +08:00
Vinoth Kannan
28be284b27
FIX: use active record update_attribute instead of mini sql. ()
* DEV: use active record `save!` instead of mini sql.

The "save" method will trigger the before_save callback "match_primary_group_changes" for User model. Else `flair_group_id` won't be removed from the user.

* check whether the method `match_primary_group_changes` called or not.
2021-09-21 09:29:12 +08:00
Martin Brennan
0c42a1e5f3
FEATURE: Topic-level bookmarks ()
Allows creating a bookmark with the `for_topic` flag introduced in d1d2298a4c set to true. This happens when clicking on the Bookmark button in the topic footer when no other posts are bookmarked. In a later PR, when clicking on these topic-level bookmarks the user will be taken to the last unread post in the topic, not the OP. Only the OP can have a topic level bookmark, and users can also make a post-level bookmark on the OP of the topic.

I had to do some pretty heavy refactors because most of the bookmark code in the JS topics controller was centred around instances of Post JS models, but the topic level bookmark is not centred around a post. Some refactors were just for readability as well.

Also removes some missed reminderType code from the purge in 41e19adb0d
2021-09-21 08:45:47 +10:00
Vinoth Kannan
a8b2e7e343
DEV: trash category definition topic instead of destroying. ()
After deleting a category, we should soft-delete the category definition topic instead of hard deleting it. Else it causes issues while doing the user merge action if the source user has an orphan post that belongs to the deleted topic.
2021-09-20 11:20:49 +05:30
Alan Guo Xiang Tan
903439a14e
PERF: Fix N+1 queries in SiteSerialier. ()
`SiteSerializer#groups` loads `Group#flair_upload` for each group so
we need to eagerload this.

Follow-up to e8a9917db1
2021-09-17 10:34:04 +08:00
andrea
119bdc12ea Added S3 region eu-south-1 Milanù
Added S3 region eu-south-1 Milan
2021-09-16 12:33:43 -04:00
Vinoth Kannan
a6de4a5ce9
DEV: use upload id to save in theme setting instead of URL. ()
When we use URL instead it creates the problem while changing the CDN hostname.
2021-09-16 07:58:53 +05:30
Martin Brennan
41e19adb0d
DEV: Ignore reminder_type for bookmarks ()
We don't actually use the reminder_type for bookmarks anywhere;
we are just storing it. It has no bearing on the UI. It used
to be relevant with the at_desktop bookmark reminders (see
fa572d3a7a)

This commit marks the column as readonly, ignores it, and removes
the index, and it will be dropped in a later PR. Some plugins
are relying on reminder_type partially so some stubs have been
left in place to avoid errors.
2021-09-16 09:56:54 +10:00
Martin Brennan
d1d2298a4c
DEV: Add for_topic column to bookmarks ()
This new column will be used to indicate that a bookmark
is at the topic level. The first post of a topic can be
bookmarked twice after this change -- with for_topic set
to true and with for_topic set to false.

A later PR will use this column for logic to bookmark the
topic, and then topic-level bookmark links will take you
to the last unread post in the topic.

See also 22208836c5
2021-09-15 11:29:22 +10:00
Martin Brennan
22208836c5
DEV: Ignore bookmarks.topic_id column and remove references to it in code ()
We don't need no stinkin' denormalization! This commit ignores
the topic_id column on bookmarks, to be deleted at a later date.
We don't really need this column and it's better to rely on the
post.topic_id as the canonical topic_id for bookmarks, then we
don't need to remember to update both columns if the bookmarked
post moves to another topic.
2021-09-15 10:16:54 +10:00
Mark VanLandingham
68bb7c5a66
DEV: Support translated title in desktop/notifications () 2021-09-14 09:57:38 -05:00
Dan Ungureanu
f517b6997c
FEATURE: Cook drafts excerpt in user activity ()
The previous excerpt was a simple truncated raw message. Starting with
this commit, the raw content of the draft is cooked and an excerpt is
extracted from it. The logic for extracting the excerpt mimics the the
`ExcerptParser` class, but does not implement all functionality, being
a much simpler implementation.

The two draft controllers have been merged into one and the /draft.json
route has been changed to /drafts.json to be consistent with the other
route names.
2021-09-14 15:18:01 +03:00
Martin Brennan
581482003a
DEV: Change uploads.filesize column to bigint ()
This is necessary to allow for large file uploads via
the direct S3 upload mechanism, as we convert the external
file to an Upload record via ExternalUploadManager once
it is complete.

This will allow for files larger than 2,147,483,647 bytes (2.14GB)
to be referenced in the uploads table.

This is a table locking migration, but since it is not as highly
trafficked as posts, topics, or users, the disruption should be minimal.
2021-09-14 12:20:56 +10:00
Rafael dos Santos Silva
2e0992c757
DEV: Allow TopicEmbed.import to optionally receive a list of tags ()
This will be used by the rss-polling plugin
2021-09-13 17:01:59 -03:00
Alan Guo Xiang Tan
bc23dcd30b
FIX: Don't publish PM archive events to acting user. ()
When a user archives a personal message, they are redirected back to the
inbox and will refresh the list of the topics for the given filter.
Publishing an event to the user results in an incorrect incoming message
because the list of topics has already been refreshed.

This does mean that if a user has two tabs opened, the non-active tab
will not receive the incoming message but at this point we do not think
the technical trade-offs are worth it to support this feature. We
basically have to somehow exclude a client from an incoming message
which is not easy to do.

Follow-up to fc1fd1b416
2021-09-10 09:20:50 +08:00
Alan Guo Xiang Tan
412587f70a
FEATURE: Publish read topic tracking events for private messages. ()
Follow-up to fc1fd1b416
2021-09-09 09:16:53 +08:00
Alan Guo Xiang Tan
1e05175364
PERF: Avoid running query unnecessarily when updating bookmark. ()
* Avoid loading an entire ActiveRecord object when saving and updating.
* Avoid running a DB query when `post_id` or `user_id` is not changed.
2021-09-09 10:50:26 +10:00
Alan Guo Xiang Tan
5de64b3630
FIX: Correct highest post number for read topic tracking state. () 2021-09-08 11:55:12 +08:00
Krzysztof Kotlarek
e3793e6d7c
FIX: better filter for groups search ()
Follow up of https://github.com/discourse/discourse/pull/14216

Allow plugins to register custom filter with block
2021-09-08 09:38:45 +10:00
Martin Brennan
d429dcfaca
DEV: Stop ignoring bookmarks.delete_when_reminder_sent ()
This was ignored a long time ago in
8dc3543134
2021-09-07 12:38:18 +10:00
Andrei Prigorshnev
8f9e692e41
FEATURE: enable users to choose unseen as a default view () 2021-09-06 21:51:52 +04:00
Krzysztof Kotlarek
f859fd6bde
FEATURE: allow plugins to extend Groups ()
* add_permitted_group_param API for plugins
* add groups-interaction-custom-options outlet
* custom search can use custom group scope
2021-09-06 10:18:51 +10:00
Alan Guo Xiang Tan
b47a4d0207
PERF: Improve query perf when fetching unread for PM topic tracking state. ()
This change helps the planner to use the
index_topics_on_timestamps_private index on topics.

Follow-up to f66007ec83
2021-09-03 15:12:13 +08:00
Rafael dos Santos Silva
ef0471d7ae
DEV: Allow passing cook_method to TopicEmbed.import to override default ()
DEV: Allow passing cook_method to TopicEmbed.import to override default

This will be used in the rss-polling plugin when we want to have
oneboxes on feed content, like youtube for example.
2021-09-01 15:46:39 -03:00
Daniel Waterworth
d11f19f099
PERF: Remove redundant post_timings_summary index ()
It's redundant since post_timings_unique exists which has a superset of
the columns with the same prefix.
2021-08-26 10:50:34 -05:00
Penar Musaraj
85b8fea262
UX: Add Styling step to wizard ()
Refactors three wizard steps (colors, fonts, homepage style) into one new step called Styling.
2021-08-25 17:10:12 -04:00
Bianca Nenciu
197532dc31
FIX: Add plugin event to topic list user lookup ()
This can be used to change the list of topic posters. For example,
discourse-solved can use this to move the user who posted the solution
after the original poster.
2021-08-25 13:16:08 +03:00
Alan Guo Xiang Tan
f66007ec83
FEATURE: Display unread and new counts for messages. ()
There are certain design decisions that were made in this commit.

Private messages implements its own version of topic tracking state because there are significant differences between regular and private_message topics. Regular topics have to track categories and tags while private messages do not. It is much easier to design the new topic tracking state if we maintain two different classes, instead of trying to mash this two worlds together.

One MessageBus channel per user and one MessageBus channel per group. This allows each user and each group to have their own channel backlog instead of having one global channel which requires the client to filter away unrelated messages.
2021-08-25 11:17:56 +08:00
Martin Brennan
d66b258b0e
DEV: Do not destroy external upload stub on error in debug mode ()
We do not want to destroy the external upload stub records
in debug mode because they allow for investigation of problems
occuring.
2021-08-25 11:11:19 +10:00
Martin Brennan
d295a16dab
FEATURE: Uppy direct S3 multipart uploads in composer ()
This pull request introduces the endpoints required, and the JavaScript functionality in the `ComposerUppyUpload` mixin, for direct S3 multipart uploads. There are four new endpoints in the uploads controller:

* `create-multipart.json` - Creates the multipart upload in S3 along with an `ExternalUploadStub` record, storing information about the file in the same way as `generate-presigned-put.json` does for regular direct S3 uploads
* `batch-presign-multipart-parts.json` - Takes a list of part numbers and the unique identifier for an `ExternalUploadStub` record, and generates the presigned URLs for those parts if the multipart upload still exists and if the user has permission to access that upload
* `complete-multipart.json` - Completes the multipart upload in S3. Needs the full list of part numbers and their associated ETags which are returned when the part is uploaded to the presigned URL above. Only works if the user has permission to access the associated `ExternalUploadStub` record and the multipart upload still exists.

  After we confirm the upload is complete in S3, we go through the regular `UploadCreator` flow, the same as `complete-external-upload.json`, and promote the temporary upload S3 into a full `Upload` record, moving it to its final destination.
* `abort-multipart.json` - Aborts the multipart upload on S3 and destroys the `ExternalUploadStub` record if the user has permission to access that upload.

Also added are a few new columns to `ExternalUploadStub`:

* multipart - Whether or not this is a multipart upload
* external_upload_identifier - The "upload ID" for an S3 multipart upload
* filesize - The size of the file when the `create-multipart.json` or `generate-presigned-put.json` is called. This is used for validation.

When the user completes a direct S3 upload, either regular or multipart, we take the `filesize` that was captured when the `ExternalUploadStub` was first created and compare it with the final `Content-Length` size of the file where it is stored in S3. Then, if the two do not match, we throw an error, delete the file on S3, and ban the user from uploading files for N (default 5) minutes. This would only happen if the user uploads a different file than what they first specified, or in the case of multipart uploads uploaded larger chunks than needed. This is done to prevent abuse of S3 storage by bad actors.

Also included in this PR is an update to vendor/uppy.js. This has been built locally from the latest uppy source at d613b849a6. This must be done so that I can get my multipart upload changes into Discourse. When the Uppy team cuts a proper release, we can bump the package.json versions instead.
2021-08-25 08:46:54 +10:00
Bianca Nenciu
ff367e22fb
FEATURE: Make allow_uploaded_avatars accept TL ()
This gives admins more control over who can upload custom profile
pictures.
2021-08-24 10:46:28 +03:00
Joffrey JAFFEUX
2bbc97fda5
DEV: UserOption.user_tzinfo ()
Provides a safe way to retrieve the timezone of a user.

This is not used in core yet, but used in multiple plugins.
2021-08-19 21:56:14 +02:00
Arpit Jalan
7db3888f17
FIX: log proper error message when SSO nonce verification fails () 2021-08-18 18:44:12 +05:30
Rafael dos Santos Silva
560c13211a
DEV: Allow passing a category parameter when importing a topic ()
This will be used in the rss pooling plugin to address the feature
request at https://meta.discourse.org/t/-/200644?u=falco
2021-08-17 18:17:07 -03:00
Alan Guo Xiang Tan
11b421525c DEV: Remove an unnecessary join in TopicTrackingState.report take 2.
This reverts commit f438cb8e65.
2021-08-16 13:26:45 +08:00
Martin Brennan
f438cb8e65
Revert "DEV: Remove an unnecessary join in TopicTrackingState.report." ()
This reverts commit 06ee0e5a51.
2021-08-16 13:54:34 +10:00
Alan Guo Xiang Tan
06ee0e5a51 DEV: Remove an unnecessary join in TopicTrackingState.report. 2021-08-16 10:55:44 +08:00
Alan Guo Xiang Tan
0198ffd2b1 DEV: Remove unused code from TopicTrackingState.
`TopicTrackingState` contains only class methods so `attr_accessor` has
no meaning for the class. Probably stale code from a long time ago.
2021-08-13 08:35:37 +08:00
Roman Rizzi
29bb79de37
FIX: Query the items in the queue to calculate a user's flagged post count. ()
When a staff member clicks on a user's number of flagged posts, we redirect them to the review queue, so it makes sense to count the number of items there to calculate the count.

We used to look at post action items to calculate this number, which doesn't match the number of items in the queue if old flags exist.
2021-08-12 14:20:46 -03:00
Alan Guo Xiang Tan
aed65ec16d SECURITY: User's read state for topic is leaked to unauthorized clients.
A user's read state for a topic such as the last read post number and the notification level is exposed.
2021-08-12 12:16:45 +08:00
Alan Guo Xiang Tan
9a60c83535 FIX: TopicTrackingState.report not including unread for staff posts.
Follow-up to e15c86e8c5
2021-08-12 11:10:49 +08:00
Alan Guo Xiang Tan
e157925308 DEV: Remove unused attributes when publishing read/new. 2021-08-11 11:12:23 +08:00
Arpit Jalan
3006de39d1
REVERT "FIX: do not show private group flair on user avatars" ()
This reverts commit fe3e18f981 and 0d8fd9ace6
2021-08-10 17:25:11 +05:30
Andrei Prigorshnev
09ad3ed41d
FEATURE: revert disallowing putting URLs in titles for TL0 users ()
This reverts a part of changes introduced by https://github.com/discourse/discourse/pull/13947

In that PR I:
1. Disallowed topic feature links for TL-0 users
2. Additionally, disallowed just putting any URL in topic titles for TL-0 users

Actually, we don't need the second part. It introduced unnecessary complexity for no good reason. In fact, it tries to do the job that anti-spam plugins (like Akismet plugin) should be doing.

This PR reverts this second change.
2021-08-06 20:07:42 +04:00
Arpit Jalan
0d8fd9ace6
FIX: do not show flair bg color if flair is not visible ()
follow up to fe3e18f981
2021-08-06 20:53:23 +05:30
Andrei Prigorshnev
0c0a11b66a
FEATURE: Disallow putting urls in the title for TL-0 users ()
This disallows putting URLs in topic titles for TL0 users, which means that:

If a TL-0 user puts a link into the title, a topic featured link won't be generated (as if it was disabled in the site settings)
Server methods for creating and updating topics will be refusing featured links when they are called by TL-0 users
TL-0 users won't be able to put any link into the topic title. For example, the title "Hey, take a look at https://my-site.com" will be rejected.

Also, it improves a bit server behavior when creating or updating feature links on topics in the categories with disabled featured links. Before the server just silently ignored a featured link field that was passed to him, now it will be returning 422 response.
2021-08-05 13:38:39 +04:00
jbrw
fb14e50741
SECURITY: Destroy EmailToken when EmailChangeRequest is destroyed () 2021-08-04 19:14:56 -04:00
Jarek Radosz
07c6b720bc
DEV: Remove PostProcessed trigger option ()
It was deprecated 5 years ago in e55e2aff94

I've seen it still being used in the wild, even though it doesn't do anything anymore as I understand it.
2021-08-04 22:24:47 +02:00
Bianca Nenciu
d9843d757a
FIX: Update draft count when sequence is increased ()
* FIX: Update draft count when sequence is increased

Sometimes users ended up having a draft count higher than the actual
number of drafts.

* FIX: Do not update draft count twice

The call to DraftSequence.next! above already does it.
2021-08-04 13:30:37 +03:00
Vinoth Kannan
1da0aa838f
FIX: use update_attribute method to trigger callbacks. ()
Group flair is not removed while removing a user from the group since the `before_save` callback methods are not triggered while using the `update_columns` method.
2021-08-04 11:54:46 +05:30
Jean
e7b8e75583
FEATURE: Add post edits count to user activity () 2021-08-02 10:15:53 -04:00
Alan Guo Xiang Tan
016efeadf6
FEATURE: New and Unread messages for user personal messages. ()
* FEATURE: New and Unread messages for user personal messages.

Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2021-08-02 12:41:41 +08:00
Arpit Jalan
fe3e18f981
FIX: do not show private group flair on user avatars ()
Meta ref: https://meta.discourse.org/t/visible-flair-for-invisible-groups-is-that-on-purpose/167674
2021-08-02 06:21:00 +05:30
Jean
ac777440fd
FIX: Validate value of custom dropdown user fields - dropdowns and multiple selects () 2021-07-30 13:50:47 -04:00
Bianca Nenciu
300db3d3fa
FIX: Update draft count after creating a post ()
When a post is created, the draft sequence is increased and then older
drafts are automatically executing a raw SQL query. This skipped the
Draft model callbacks and did not update user's draft count.

I fixed another problem related to a raw SQL query from Draft.cleanup!
method.
2021-07-29 17:06:11 +03:00
Martin Brennan
b500949ef6
FEATURE: Initial implementation of direct S3 uploads with uppy and stubs ()
This adds a few different things to allow for direct S3 uploads using uppy. **These changes are still not the default.** There are hidden `enable_experimental_image_uploader` and `enable_direct_s3_uploads`  settings that must be turned on for any of this code to be used, and even if they are turned on only the User Card Background for the user profile actually uses uppy-image-uploader.

A new `ExternalUploadStub` model and database table is introduced in this pull request. This is used to keep track of uploads that are uploaded to a temporary location in S3 with the direct to S3 code, and they are eventually deleted a) when the direct upload is completed and b) after a certain time period of not being used. 

### Starting a direct S3 upload

When an S3 direct upload is initiated with uppy, we first request a presigned PUT URL from the new `generate-presigned-put` endpoint in `UploadsController`. This generates an S3 key in the `temp` folder inside the correct bucket path, along with any metadata from the clientside (e.g. the SHA1 checksum described below). This will also create an `ExternalUploadStub` and store the details of the temp object key and the file being uploaded.

Once the clientside has this URL, uppy will upload the file direct to S3 using the presigned URL. Once the upload is complete we go to the next stage.

### Completing a direct S3 upload

Once the upload to S3 is done we call the new `complete-external-upload` route with the unique identifier of the `ExternalUploadStub` created earlier. Only the user who made the stub can complete the external upload. One of two paths is followed via the `ExternalUploadManager`.

1. If the object in S3 is too large (currently 100mb defined by `ExternalUploadManager::DOWNLOAD_LIMIT`) we do not download and generate the SHA1 for that file. Instead we create the `Upload` record via `UploadCreator` and simply copy it to its final destination on S3 then delete the initial temp file. Several modifications to `UploadCreator` have been made to accommodate this.

2. If the object in S3 is small enough, we download it. When the temporary S3 file is downloaded, we compare the SHA1 checksum generated by the browser with the actual SHA1 checksum of the file generated by ruby. The browser SHA1 checksum is stored on the object in S3 with metadata, and is generated via the `UppyChecksum` plugin. Keep in mind that some browsers will not generate this due to compatibility or other issues.

    We then follow the normal `UploadCreator` path with one exception. To cut down on having to re-upload the file again, if there are no changes (such as resizing etc) to the file in `UploadCreator` we follow the same copy + delete temp path that we do for files that are too large.

3. Finally we return the serialized upload record back to the client

There are several errors that could happen that are handled by `UploadsController` as well.

Also in this PR is some refactoring of `displayErrorForUpload` to handle both uppy and jquery file uploader errors.
2021-07-28 08:42:25 +10:00
Bianca Nenciu
760c9a5698
FEATURE: Show draft count in user menu and activity ()
This commit adds the number of drafts a user has next to the "Draft"
label in the user preferences menu and activity tab. The count is
updated via MessageBus when a draft is created or destroyed.
2021-07-27 14:05:33 +03:00
Vinoth Kannan
5a93893b08
FIX: use correct URL in schema markup for post images. ()
Currently, it wrongly adds Discourse base URL in prefix even for CDN URLs.
2021-07-26 21:39:51 +05:30
Krzysztof Kotlarek
88aa0136e3
FIX: do not raise exception when svg path is nil ()
Bug was introduced here: f7ab852e12

If path is nil, it should not raise an exception and continue logging the error.
2021-07-26 12:35:27 +10:00
Alan Guo Xiang Tan
ffb3e7b356 FIX: Clear Site#categories cache when git sha changes.
Category custom fields are cached so we need to break the cache when a
new plugin is added/removed.
2021-07-26 09:15:07 +08:00
Robin Ward
7b45a5ce55 FIX: Better and more secure validation of periods for TopicQuery
Co-authored-by: Martin Brennan <mjrbrennan@gmail.com>
2021-07-23 14:24:44 -04:00
Joffrey JAFFEUX
5eb6e9281a
FIX: manually adds frowning_face_with_open_mouth for apple () 2021-07-21 23:27:20 +02:00
Penar Musaraj
2ce2c83bc9
FIX: Show user filter hints when typing @ in search ()
Will show the last 6 seen users as filtering suggestions when typing @ in quick search. (Previously the user suggestion required a character after the @.)

This also adds a default limit of 6 to the user search query, previously the backend was returning 20 results but a maximum of 6 results was being shown anyway.
2021-07-21 09:14:53 -04:00
mintsaxon
7162ecfb04 FEATURE: Per-category default slow mode duration for topics.
When configured, all topics in the category inherits the slow mode
duration from the category's default.

Note that currently there is no way to remove the slow mode from the
topics once it has been set.
2021-07-21 12:32:07 +08:00
Andrei Prigorshnev
1a8c949900
UX: suspend forever time period messages ()
When the Forever option is selected for suspending a user, the user is suspended for 1000 years. Without customizing the site’s text, this time period is displayed to the user in the suspension email that is sent to the user, and if the user attempts to log back into the site. Telling someone that they have been suspended for 1000 years seems likely to come across as a bad attempt at humour.

This PR special case messages when a user suspended or silenced forever.
2021-07-20 14:42:08 +04:00
Mark VanLandingham
80950d7b28
DEV: Add chat_mention notification type () 2021-07-19 14:52:12 -05:00
Alan Guo Xiang Tan
a1047f5ef4
FEATURE: Add new plugin API to allow plugins to extend Site#categories () 2021-07-19 13:54:19 +08:00
Rafael dos Santos Silva
e12b00eab7
FEATURE: Stop checking referer for embeds ()
Flips content_security_policy_frame_ancestors default to enabled, and
removes HTTP_REFERER checks on embed requests, as the new referer
privacy options made the check fragile.
2021-07-16 15:25:49 -03:00
Dan Ungureanu
079d2af55f
FIX: Clear stale status of reloaded reviewables ()
* FIX: Clear stale status of reloaded reviewables

Navigating away from and back to the reviewables reloaded Reviewable
records, but did not clear the "stale" attribute.

* FEATURE: Show user who last acted on reviewable

When a user acts on a reviewable, all other clients are notified and a
generic "reviewable was resolved by someone" notice was shown instead of
the buttons. There is no need to keep secret the username of the acting
user.
2021-07-16 19:57:12 +03:00
David Taylor
422fa1b1d8 FIX: Use correct setting for DiscourseConnect card background overrides 2021-07-16 16:19:09 +01:00
Penar Musaraj
361c8be547
PERF: Add scheduled job to delete old stylesheet cache rows () 2021-07-16 10:58:01 -04:00
David Taylor
444e21b12d
FEATURE: Add 'users.list' API scope () 2021-07-16 11:10:04 +08:00
Penar Musaraj
f7ab852e12
FIX: Issues with custom icons in themes ()
Fixes two issues:
- ignores invalid XML in custom icon sprite SVG file (and outputs an error if sprite was uploaded via admin UI)
- clears SVG sprite cache when deleting an `icons-sprite` upload in a theme
2021-07-14 15:18:29 -04:00
Gerhard Schlager
6805cc287e FIX: Allow uppercase letters in automatic group names
`User.username_exists?` always converts to lowercase during lookup, so this change should be safe.
2021-07-13 20:29:07 +02:00
David Taylor
800c6e1a68 PERF: Improve topic_user.liked update performance when moving posts
Previously we would re-calculate topic_user.liked for all users who have ever viewed the source or destination topic. This can be very expensive on large sites. Instead, we can use the array of moved post ids to find which users are actually affected by the move, and restrict the update query to only check/update their records.

On an example site this reduced the `update_post_action_cache` time from ~27s to 300ms
2021-07-13 12:30:38 +01:00
David Taylor
3d049245af PERF: Improve post_timing performance when moving posts
Scanning for all possible invalid post_timing records in the destination topics can be a very expensive operation. The main aim is to avoid the data clashing with soon-to-be-moved posts, so we can reduce the scope of the query by targeting only rows which would actually cause a clash. post_timings has an index on (topic_id, post_number), so this is very fast.

On an example site, this reduced the query from ~6s to <10ms
2021-07-13 12:30:38 +01:00
Dan Ungureanu
2d904ade6d
FIX: Let staged users choose their username ()
When a staged user tried to redeem an invite, a different username was
suggested and manually typing the staged username failed because the
username was not available.
2021-07-12 07:57:38 +10:00
David Taylor
04db58fbd8
UX: Translate the action type in watched words regex error ()
Previously we were using the untranslated key from the code
2021-07-09 14:34:08 +01:00
Martin Brennan
9f275c12ab
FIX: Handle storage providers not implementing ACLs ()
When secure media is enabled or when upload secure status
is updated, we also try and update the upload ACL. However
if the object storage provider does not implement this we
get an Aws::S3::Errors::NotImplemented error. This PR handles
this error so the update_secure_status method does not error
out and still returns whether the secure status changed.
2021-07-09 11:31:44 +10:00
Bianca Nenciu
87c1e98571
FEATURE: Let users select flair ()
User flair was given by user's primary group. This PR separates the
two, adds a new field to the user model for flair group ID and users
can select their flair from user preferences now.
2021-07-08 10:46:21 +03:00
Mark VanLandingham
14a13dc192
FIX: Check type of existing reviewables when new reviewable is created () 2021-07-07 11:45:00 -05:00
Vinoth Kannan
33eae4cbd8
FEATURE: add period filter in top topics route for tags. ()
And also move all the "top topics by period" routes to query string param.

/top/monthly => /top?period=monthly
/c/:slug/:id/l/top/monthly => /c/:slug/:id/l/top?period=monthly
/tag/:slug/l/top/daily => /tag/:slug/l/top?period=daily (new)
2021-07-06 15:25:11 +05:30
Dan Ungureanu
34387c5a38
FEATURE: Warn if invited user cannot see topic ()
Users can invite people to topics from secured category, but they will
not be redirected to the topic after signing up unless they have the
permissions to view the topic. This commit shows a warning when invite
is saved if the topic is in a secured category and none of the invite
groups are allowed to see it.
2021-07-06 12:49:26 +03:00
David Taylor
f999ef2d52
DEV: Drop user_options.disable_jump_reply column ()
24ef4f7b removed the use of this column in 2019
2021-07-06 10:47:17 +01:00
David Taylor
4b55b59931 DEV: Update core model annotations
Generated using `bin/rake annotate:clean`
2021-07-06 10:11:06 +01:00
Alan Guo Xiang Tan
37b8ce79c9
FEATURE: Add last visit indication to topic view page. ()
This PR also removes grey old unread bubble from the topic badges by
dropping `TopicUser#highest_seen_post_number`.
2021-07-05 14:17:31 +08:00
Martin Brennan
0f688f45bd
FIX: Handle edge cases for group SMTP email job ()
Skip group SMTP email (and add log) if:

* topic is deleted
* post is deleted
* smtp has been disabled for the group

Skip without log if:

* enable_smtp site setting is false
* disable_emails site setting is yes

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-07-05 14:56:32 +10:00
Alan Guo Xiang Tan
f26acb4b63
DEV: Remove User class methods that have been deprecated. () 2021-07-05 10:44:53 +08:00
Alan Guo Xiang Tan
b42e9e5c86
DEV: Remove broken associations on Invite model. ()
No relevant foreign keys exists on the invites table for the
associations to even work.
2021-07-01 10:50:16 +08:00
jbrw
4728962f7d
FIX: Don’t translate TrustLevel name when generating links ()
We want to put the name of the trust level in to generated URLs, not the human-readable form.

i.e.:

`/admin/users/list/newuser`

rather than:

`/admin/users/list/new user`
2021-06-30 14:19:15 -04:00
Andrei Prigorshnev
11baf872ed
FIX: do not close the merged topic if the first post wasn't merged ()
When a topic is fully merged into another topic we close it and schedule it for deleting. But last time I changed this place I added a bug – when merging all posts in topic except the first one the topic was closing too.

If the OP is not merged into another topic, the original topic shouldn't be closed and marked for deletion. This PR fixes this.
2021-06-30 18:28:18 +04:00
Arpit Jalan
1ea2880276
FEATURE: add staff action logs for watched words () 2021-06-30 11:22:46 +05:30
Martin Brennan
d098f51ad3
DEV: Drop duration column from topic timers ()
The duration column has been ignored since the commit
4af77f1e38
for topic_timers, we use duration_minutes instead.

Also removing the duration key from Topic.set_or_create_timer. The only
plugin to use this was discourse-solved, which doesn't use it any
longer
since
c722b94a97
2021-06-29 09:27:12 +10:00