Commit Graph

2327 Commits

Author SHA1 Message Date
Michael Brown
d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Blake Erickson
03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
Roman Rizzi
671f882fa3
FIX: We don't create a Post object if the queued post gets rejected. We need to count review items directly. (#9856) 2020-05-22 11:50:28 -03:00
Martin Brennan
f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Martin Brennan
df68d11c38
FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
Guo Xiang Tan
96c02caba7
DEV: Change use of Redis flushall to flushdb.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
Gerhard Schlager
6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Gerhard Schlager
27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
Guo Xiang Tan
d01c336899
DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
Krzysztof Kotlarek
ea63fa7de7
FIX: extract reset_last_seen_cache! in user_spec (#9808)
Helper method to avoid code duplication
2020-05-18 09:13:37 +10:00
Guo Xiang Tan
165fa13c9d
FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
Krzysztof Kotlarek
65a3fdbc57
FIX: randomly failing user_spec (#9754)
* FIX: randomly falling user_spec

When we evaluate `update_last_seen!` we relay on Redis to not run that code too often

https://github.com/discourse/discourse/blob/master/app/models/user.rb#L753

The problem is that not all specs which are running `update_last_seen!` are not cleaning after themselves

For examples specs in that block https://github.com/discourse/discourse/blob/master/spec/models/user_spec.rb#L901

So it can be replicated when you run a few times
`bundle exec rspec  ./spec/models/user_spec.rb -e "should not update the first seen value if it doesn't exist" -e "should have 0 for days_visited"`

We should delete Redis key after each spec which is evaluating `update_last_seen!`
2020-05-15 09:24:07 +10:00
Sam Saffron
7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Martin Brennan
9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Sam Saffron
a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan
d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Martin Brennan
37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Gerhard Schlager
ec2f3169ff FIX: Using the default_locale in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Robin Ward
612284cef3
DEV: Remove Discourse.RAW_TEMPLATES (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor
03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek
9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
Krzysztof Kotlarek
37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Neil Lalonde
9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
mentalstring
67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Sam Saffron
cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Martin Brennan
cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Krzysztof Kotlarek
52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Vinoth Kannan
df0c386f8a
UX: drop the automatic_membership_retroactive column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Dan Ungureanu
e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Jarek Radosz
c08753dc34
DEV: Fix topic_timer_spec (#9483)
File it under "how did it even work".
2020-04-21 14:38:35 +10:00
Mark VanLandingham
06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward
25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward
56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
David Taylor
0e4497b6be
DEV: Update tl3 spec to remove hard-coded primary keys
In some test runs, the hard-coded ids could correspond to real topics created earlier in the tests, and cause random test failures
2020-04-17 17:24:14 +01:00
Neil Lalonde
074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Robin Ward
b6b92a562c
FEATURE: New site setting embed_unlisted (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Kane York
138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Robin Ward
e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Roman Rizzi
8e1bdc9458
FIX: Prevent low score flags from auto-closing a topic if the reviewable default visibility is higher than low (#9375) 2020-04-08 10:44:31 -03:00
Neil Lalonde
c45574c9e7
FIX: exclude private messages from TL3 requirements
It wasn't intended that people should be able to earn trust level
3 without participating in public topics. When counting topic
views and likes given/received, don't count private topics.
2020-04-07 17:36:07 -04:00
Krzysztof Kotlarek
a811976023
FIX: reset gravatar cache by adding random param to URL (#9370)
Users noticed that sometimes, avatar from Gravatar is not correctly updated - https://meta.discourse.org/t/updated-image-on-gravatar-not-seeing-it-update-on-site/54357

A potential reason for that is that even if you update your avatar in Gravatar, URL stays the same and if the cache is involved, service is still receiving the old photo.

For example. In my case, when I click the button to refresh avatar the
new Upload record is created with `origin` URL to new avatar, and `url` to
old one

I made some tests in the rails console and adding random param to Gravatar URL is deceiving cache and correct, the newest avatar is downloaded
2020-04-08 07:35:42 +10:00
Roman Rizzi
e52e7f6e75
FIX: Include pending queued users regardless of their score (#9372) 2020-04-07 11:42:12 -03:00
David Taylor
3814ca06a8
DEV: Allow using .js extension for javascript modules in themes (#9358) 2020-04-06 17:24:59 +01:00
Dan Ungureanu
b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
David Taylor
3cfe086a94
FIX: Limit personal message participants when converting from topic (#9343)
Previously all topic posters would be added which could lead to major performance issues. Now if there are too many posters, only the acting user will be added as a participant.
2020-04-03 16:42:01 +01:00
Martin Brennan
b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Arpit Jalan
b2a0d34bb7
FEATURE: add setting auto_approve_email_domains to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Krzysztof Kotlarek
a0f59a55cc
FIX: when a post is moved copy notifications level (#9311)
This is a revert of 2c011252f1

More information on meta: https://meta.discourse.org/t/when-a-reply-is-moved-to-a-new-topic-the-followers-of-the-previous-topic-are-automatically-follower-of-the-new-topic-as-well/130025
2020-03-31 16:19:47 +11:00
Jarek Radosz
104b647ca4 DEV: Fix another time-sensitive flaky spec 2020-03-27 20:39:25 +01:00