Commit Graph

4548 Commits

Author SHA1 Message Date
Martin Brennan
af92444948
FIX: Make sure bookmark serializer works with deleted topics + posts (#9195) 2020-03-13 10:44:39 +10:00
Martin Brennan
2237ba8c9d
FIX: Add topic deleted check to email/sender (#9166)
It already had a deleted post check and log reason, add a topic one too to avoid errors
2020-03-13 10:04:15 +10:00
David Taylor
ec2d49d48a
DEV: Allow plugins to add theme modifiers via db migrations (#9192) 2020-03-12 16:35:28 +00:00
Stasiek Michalski
1b8793e7a4
FEATURE: Add support for custom gravatar-like services (#9137)
Adds 3 config values that allow to set a custom provider of Gravatar-like API accessible from gravatar_base_url. The gravatar_name is purely cosmetic, but helps with associating name with the service that actually provides the avatars. gravatar_login_url is a link relative to gravatar_base_url, which provides the user with the login to the Gravatar service
2020-03-12 11:23:55 -04:00
Martin Brennan
03c012bd97 Unscope post + topic relation for bookmark
* we want to be able to get these records for the bookmark
  even if they are trashed, for serialization for the bookmark
  list
2020-03-12 16:00:45 +10:00
Martin Brennan
793f39139a
FEATURE: Send notifications for time-based and At Desktop bookmark reminders (#9071)
* This PR implements the scheduling and notification system for bookmark reminders. Every 5 minutes a schedule runs to check any reminders that need to be sent before now, limited to **300** reminders at a time. Any leftover reminders will be sent in the next run. This is to avoid having to deal with fickle sidekiq and reminders in the far-flung future, which would necessitate having a background job anyway to clean up any missing `enqueue_at` reminders.

* If a reminder is sent its `reminder_at` time is cleared and the `reminder_last_sent_at` time is filled in. Notifications are only user-level notifications for now.

* All JavaScript and frontend code related to displaying the bookmark reminder notification is contained here. The reminder functionality is now re-enabled in the bookmark modal as well.

* This PR also implements the "Remind me next time I am at my desktop" bookmark reminder functionality. When the user is on a mobile device they are able to select this option. When they choose this option we set a key in Redis saying they have a pending at desktop reminder. The next time they change devices we check if the new device is desktop, and if it is we send reminders using a DistributedMutex. There is also a job to ensure consistency of these reminders in Redis (in case Redis drops the ball) and the at desktop reminders expire after 20 days.

* Also in this PR is a fix to delete all Bookmarks for a user via `UserDestroyer`
2020-03-12 10:16:00 +10:00
Rafael dos Santos Silva
3f9b310beb
FIX: Typo on draft save 2020-03-11 12:12:28 -03:00
David Taylor
d1474e94a1
FEATURE: Allow themes to specify modifiers in their about.json file (#9097)
There are three modifiers:
- serialize_topic_excerpts (boolean)
- csp_extensions (array of strings)
- svg_icons (array of strings)

When multiple themes are active, the values will be combined. The combination method varies based on the setting. CSP/SVG arrays will be combined. serialize_topic_excerpts will use `Enumerable#any`.
2020-03-11 13:30:45 +00:00
Dan Ungureanu
0754c7c404
FIX: Various fixes to support posts with no user (#8877)
* Do not grant badges for posts with no user
* Ensure instructions are correct in Change Owner modal
* Hide user-dependent actions from posts with no user
* Make PostRevisor work with posts with no user
* Ensure posts with no user can be deleted
* discourse-narrative-bot should ignore posts with no user
* Skip TopicLink creation for posts with no user
2020-03-11 14:03:20 +02:00
Blake Erickson
6fb4c333b0
FIX: Throw error when removing a user from group fails (#9162)
This commit ensures that an error is thrown when a user fails to be
removed from a group instead of silently failing.

This means when using the api you will receive a 400 instead of a 200 if
there is a failure. The remove group endpoint allows the removal of
multiple users, this change means that if you try to delete 10 users,
but 1 of them fails you will receive a 400 instead of 200 even though
the other 9 were removed successfully. Rather than adding a bunch more
complexity I think this is more than adequate for most use cases.
2020-03-10 15:25:00 -06:00
Penar Musaraj
e69b6379ad
FEATURE: Broader support for post uploads in video markup (#9152)
Ensures URLs in the following HTML attributes are included in post uploads:
- video poster
- source src
- track src
2020-03-10 09:01:40 -04:00
Jarek Radosz
85e03a7f68
DEV: Replace Time.new with Time.now (#9142)
(or `Time.zone.now`)
2020-03-09 17:37:49 +01:00
Blake Erickson
43b54c631d
DEV: Reserve webhook event types to be used in plugins (#9110)
* DEV: Reserve webhook event types to be used in plugins

Based on feedback on the following PR's:

https://github.com/discourse/discourse-solved/pull/85

https://github.com/discourse/discourse-assign/pull/61

This commit reserves ID's to be used for webhook event types to ensure
that some other webhook or plugin doesn't end up using the same ID.

* Fix broken test

I don't think this test has to test ALL event types to verify that this
feature is working. Now that we added some event types that plugins are
using this test was failing for missing fabricators that exist in the
respective plugins.

* remove loop and just test first record
2020-03-06 10:16:19 -07:00
Rafael dos Santos Silva
54f67661ac FEATURE: Option to connect to Redis using SSL 2020-03-05 20:49:05 -03:00
Vinoth Kannan
d953c908d2 FEATURE: add child theme components in theme metadata.
Now theme creators can add an array of child theme components in about.json file for a top level theme.
2020-03-05 18:28:18 +05:30
David Taylor
65cc61be7a
PERF: Allow preloading 'recent time read' for a user (#9076)
This will be used when serializing multiple user cards
2020-03-03 13:57:46 +00:00
David Taylor
d23f7af3cb
PERF: Allow user serializer to make use of preloaded custom fields (#9074) 2020-03-03 13:56:54 +00:00
Martin Brennan
0388653a4d
DEV: Upload and secure media retroactive rake task improvements (#9027)
* Add uploads:sync_s3_acls rake task to ensure the ACLs in S3 are the correct (public-read or private) setting based on upload security

* Improved uploads:disable_secure_media to be more efficient and provide better messages to the user.

* Rename uploads:ensure_correct_acl task to uploads:secure_upload_analyse_and_update as it does more than check the ACL

* Many improvements to uploads:secure_upload_analyse_and_update

* Make sure that upload.access_control_post is unscoped so deleted posts are still fetched, because they still affect the security of the upload.

* Add escape hatch for capture_stdout in the form of RAILS_ENABLE_TEST_STDOUT. If provided the capture_stdout code will be ignored, so you can see the output if you need.
2020-03-03 10:03:58 +11:00
Joffrey JAFFEUX
11425f8adc
FEATURE: alows to add a description link to a report (#9065)
This commit adds a description link to users_per_trust_level report linking to our blog  article on the subject https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/
2020-03-02 14:30:51 -05:00
David Taylor
f9cc3dc4b7
PERF: Allow passing an existing list of user field ids when loading (#8970)
* PERF: Allow passing an existing list of user field ids when loading

This avoids the need for running `UserField.pluck(:id)` for each user that is serialized

* Memoize user_fields to avoid rebuilding hash ever time
2020-03-02 14:22:49 -05:00
Martin Brennan
8123538c94
DEV: Minor review fixes and fix bookmark spec logging (#9045)
As per:

https://review.discourse.org/t/fix-never-allow-custom-emoji-to-be-marked-secure-8965/9072
https://review.discourse.org/t/feature-improving-bookmarks-part-2-topic-bookmarking-8954/9038
2020-03-02 15:40:29 +10:00
Dan Ungureanu
60184a290c
FIX: Sync preload key format for category topic lists
The server and client used two different formats for preload keys. The
server was using 'topic_list_c/SLUG/l/latest', but the client was using
'topic_list_c/SLUG/ID/l/latest'.

This commit is an addition to 374534f00e.
2020-02-28 11:10:03 +02:00
Dan Ungureanu
60908a94ec
FIX: Skip 'invited' small action if user is in an invited group (#9056)
Inviting a user that is already invited through a group used to generate
a small action and a notification. This commit skips that small action.
2020-02-27 14:45:20 +02:00
Vinoth Kannan
acf337d583
FEATURE: auto archive group message if topic is closed. (#9046)
Co-Authored-By: Régis Hanol <regis@hanol.fr>
2020-02-27 11:09:37 +05:30
Sam Saffron
372f6f4f22
FEATURE: limit number of notifications per user to 10,000
Introduces a new site setting `max_notifications_per_user`.

Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.

To disable this safeguard set the setting to 0.

Enforcement happens weekly.

This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
2020-02-24 11:42:50 +11:00
Robin Ward
a47e0a3fda FIX: TOTP could not be used on sites with colons in their names
This is because the TOTP gem identifies as a colon as an addressable
protocol. The solution for now is to remove the colon in the issuer
name.

Changing the issuer changes the token values, but now it was completely
broken for colons so this should not be breaking anyone new.
2020-02-20 16:35:30 -05:00
Sam Saffron
28292d2759
PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
Martin Brennan
56b16bc68e
FIX: Never allow custom emoji to be marked secure (#8965)
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
2020-02-14 11:17:09 +10:00
Martin Brennan
e1e74abd4f
FEATURE: Improving bookmarks part 2 -- Topic Bookmarking (#8954)
### UI Changes

If `SiteSetting.enable_bookmarks_with_reminders` is enabled:

* Clicking "Bookmark" on a topic will create a new Bookmark record instead of a post + user action
* Clicking "Clear Bookmarks" on a topic will delete all the new Bookmark records on a topic
* The topic bookmark buttons control the post bookmark flags correctly and vice-versa
Disabled selecting the "reminder type" for bookmarks in the UI because the backend functionality is not done yet (of sending users notifications etc.)

### Other Changes

* Added delete bookmark route (but no UI yet)
* Added a rake task to sync the old PostAction bookmarks to the new Bookmark table, which can be run as many times as we want for a site (it will not create duplicates).
2020-02-13 16:26:02 +10:00
Robin Ward
726d97b29d FIX: Linking to a category via hashtag had a broken URL. 2020-02-12 14:23:09 -05:00
Mark VanLandingham
3e89774908
DEV: Use .hbr for raw template file extension (#8883) 2020-02-11 13:38:12 -06:00
Roman Rizzi
3413ec0a5c
FEATURE: Pending queued posts are included even if they don't pass the minimum priority threshold (#8925) 2020-02-11 15:29:22 -03:00
Dan Ungureanu
ecaf2c2f4e
FIX: Make category slug validation less strict (#8915)
This was changed recently and caused issues saving old categories which
already had digits at the beginning of the slug (for example, '30-days').
2020-02-11 17:01:12 +02:00
Jeff Wong
1a1bb7a2c9
FEATURE: Add logging when claiming and unclaiming reviewable flagged posts (#8920) 2020-02-10 15:40:01 -08:00
Jarek Radosz
6cfd16656f
FIX: Ignore group mentions inside quotes (#8905)
Also includes:
* DEV: Reuse found elements
2020-02-10 18:31:42 +01:00
Joffrey JAFFEUX
4de7d5ff90 FIX: removes limit for trust level growth report (#8908) 2020-02-10 11:56:29 +01:00
David Taylor
5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Penar Musaraj
99fd65328c FIX: Skip absolutizing URLs when source URI is invalid 2020-02-07 10:54:24 -05:00
Joffrey JAFFEUX
20944e69e4
FEATURE: adds trust_level_growth report (#8878) 2020-02-06 19:44:30 +01:00
Blake Erickson
926d5f1c0a REFACTOR: Edit title respects min trust to edit post
Follow up to: 241d8f6452
2020-02-05 10:36:24 -07:00
Jarek Radosz
53529a3427
DEV: Upgrade Ember to version 3.12.2 (#8753)
* DEV: Use Ember 3.12.2
* Add Ember version to ThemeField's DEPENDENT_CONSTANTS
* DEV: Use `id` instead of `elementId` (See: https://github.com/emberjs/ember.js/issues/18147)
* FIX: Don't leak event listeners (bug introduced in 999e2ff)
2020-02-05 14:51:00 +01:00
Blake Erickson
241d8f6452 FIX: Edit title respects min trust to edit post
This fix ensures that the site setting `post_edit_time_limit` does not
bypass the limit of the site setting `min_trust_to_edit_post`. This
prevents a bug where users that did not meet the minimum trust level to
edit could edit the title of topics.
2020-02-04 16:31:16 -07:00
Gerhard Schlager
71849242fa PERF: Speed up moving posts on large databases
Old exection plan:
```
Delete on post_replies pr  (cost=6.59..20462.62 rows=2254 width=24) (actual time=2.580..2.580 rows=0 loops=1)
  ->  Nested Loop  (cost=6.59..20462.62 rows=2254 width=24) (actual time=0.086..2.557 rows=4 loops=1)
        Join Filter: (p.topic_id <> r.topic_id)
        Rows Removed by Join Filter: 328
        ->  Nested Loop  (cost=6.16..16845.77 rows=2254 width=26) (actual time=0.020..1.886 rows=332 loops=1)
              ->  Nested Loop  (cost=5.74..13257.09 rows=2254 width=20) (actual time=0.016..1.361 rows=332 loops=1)
                    ->  Seq Scan on moved_posts mp  (cost=0.00..19.70 rows=970 width=10) (actual time=0.002..0.028 rows=263 loops=1)
                    ->  Bitmap Heap Scan on post_replies pr  (cost=5.74..13.63 rows=2 width=14) (actual time=0.004..0.005 rows=1 loops=263)
                          Recheck Cond: ((reply_post_id = mp.old_post_id) OR (post_id = mp.old_post_id))
                          Heap Blocks: exact=278
                          ->  BitmapOr  (cost=5.74..5.74 rows=2 width=0) (actual time=0.004..0.004 rows=0 loops=263)
                                ->  Bitmap Index Scan on index_post_replies_on_reply_post_id  (cost=0.00..2.87 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=263)
                                      Index Cond: (reply_post_id = mp.old_post_id)
                                ->  Bitmap Index Scan on index_post_replies_on_post_id_and_reply_post_id  (cost=0.00..2.87 rows=1 width=0) (actual time=0.002..0.002 rows=1 loops=263)
                                      Index Cond: (post_id = mp.old_post_id)
              ->  Index Scan using posts_pkey on posts p  (cost=0.42..1.59 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=332)
                    Index Cond: (id = pr.post_id)
        ->  Index Scan using posts_pkey on posts r  (cost=0.42..1.59 rows=1 width=14) (actual time=0.001..0.002 rows=1 loops=332)
              Index Cond: (id = pr.reply_post_id)
Planning Time: 0.305 ms
Execution Time: 2.600 ms
```

New execution plan:
```
Delete on post_replies pr  (cost=15.34..6538275.37 rows=364157 width=12) (actual time=1.961..1.961 rows=0 loops=1)
  ->  Nested Loop  (cost=15.34..6538275.37 rows=364157 width=12) (actual time=0.048..1.827 rows=187 loops=1)
        ->  Seq Scan on moved_posts mp  (cost=0.00..19.70 rows=970 width=10) (actual time=0.004..0.029 rows=188 loops=1)
        ->  Bitmap Heap Scan on post_replies pr  (cost=15.34..6736.72 rows=375 width=14) (actual time=0.009..0.009 rows=1 loops=188)
              Recheck Cond: ((reply_post_id = mp.old_post_id) OR (post_id = mp.old_post_id))
              Filter: ((SubPlan 1) <> (SubPlan 2))
              Heap Blocks: exact=187
              ->  BitmapOr  (cost=15.34..15.34 rows=377 width=0) (actual time=0.003..0.003 rows=0 loops=188)
                    ->  Bitmap Index Scan on index_post_replies_on_reply_post_id  (cost=0.00..4.33 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=188)
                          Index Cond: (reply_post_id = mp.old_post_id)
                    ->  Bitmap Index Scan on index_post_replies_on_post_id_and_reply_post_id  (cost=0.00..10.82 rows=376 width=0) (actual time=0.001..0.001 rows=0 loops=188)
                          Index Cond: (post_id = mp.old_post_id)
              SubPlan 1
                ->  Index Scan using posts_pkey on posts p  (cost=0.43..8.45 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=187)
                      Index Cond: (id = pr.post_id)
              SubPlan 2
                ->  Index Scan using posts_pkey on posts r  (cost=0.43..8.45 rows=1 width=4) (actual time=0.002..0.003 rows=1 loops=187)
                      Index Cond: (id = pr.reply_post_id)
Planning Time: 0.136 ms
Execution Time: 1.990 ms
```
2020-02-04 12:30:43 +01:00
Robin Ward
ee17138c0f FIX: Better error messages when name is too long
Previously you'd get a server side generic error due to a password check
failing. Now the input element has a maxlength attribute and the server
side will respond with a nicer error message if the value is too long.
2020-02-03 14:14:32 -05:00
Robin Ward
37888d9818 FIX: Never return the same reply more than once via reply_ids
If our reply tree somehow ends up with cycles or other odd
structures, we only want to consider a reply once, at the first
level in the tree that it appears.
2020-02-03 13:41:18 -05:00
David Taylor
7640914552
UX: Include muted categories on the category page by default (#8842)
* DEV: Add data-notification-level attribute to category UI

* Show muted categories on the category page by default

This reverts commit ed9c21e42c.

* Remove redundant spec - muted categories are now visible by default
2020-02-03 10:40:02 -08:00
Robin Ward
f83362b05b FIX: Don't return post replies from other topics
It seems in some situations replies have been moved to other topics but
the `PostReply` table has not been updated. I will try and fix this in a
follow up PR, but for now this fix ensures that every time we ask a post
for its replies that we restrict it to the same topic.
2020-02-03 13:12:27 -05:00
Stephen Chung
98e9302c26
Log error when optimized image file fails to store. (#8840) 2020-02-03 12:28:45 -05:00
Roman Rizzi
df43ac901d
FIX: We don't want to update the post read count and user stats if the post timing wasn't created due to a conflict. (#8824) 2020-01-31 10:23:24 -03:00
Martin Brennan
1150cd4621
FIX: Stop secure media URLs being censored too liberally in emails (#8817)
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.

Refactored code so this secure URL detection happens in one place.
2020-01-30 16:19:14 +10:00