Commit Graph

7542 Commits

Author SHA1 Message Date
Krzysztof Kotlarek
d8360b4c82 SECURITY: vulnerability in WildcardUrlChecker 2019-12-13 09:29:09 -05:00
David Taylor
f47909a2b6 DEV: Check for pending migrations before running rspec suite
This should catch a few scenarios which can waste a lot of time in development
 - Forgetting to run migrations
 - Missing plugin migrations
 - Missing post_deploy migrations
2019-12-13 12:50:18 +00:00
Gerhard Schlager
c068804a0a Fixing the specs. Take 2. 2019-12-13 13:05:17 +01:00
Gerhard Schlager
2e8c23f0d0 Fix spec 2019-12-13 12:40:52 +01:00
Roman Rizzi
b6a2875749
FIX: Granting staff status should auto-approve users waiting approval (#8533)
* FIX: Granting staff status should auto-approve users waiting approval

* Update app/models/concerns/roleable.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2019-12-12 16:26:38 -03:00
Robin Ward
76f676879c FIX: Constraint error when inserting the same topic group twice
If that happens, update the value instead of raising an error.
2019-12-12 13:10:46 -05:00
David Taylor
4c9ca24ccf
FEATURE: Hash API keys in the database (#8438)
API keys are now only visible when first created. After that, only the first four characters are stored in the database for identification, along with an sha256 hash of the full key. This makes key usage easier to audit, and ensures attackers would not have access to the live site in the event of a database leak.

This makes the merge lower risk, because we have some time to revert if needed. Once the change is confirmed to be working, we will add a second commit to drop the `key` column.
2019-12-12 11:45:00 +00:00
Bianca Nenciu
3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Martin Brennan
9f4c9bafa1
FIX: Hide old bookmark button on post-menu if SiteSetting.enable_bookmarks_with_reminders
Hide old bookmark post-menu item if the site setting for the new bookmark reminders is enabled and change icon for the new bookmark functionality to the same as the old bookmark button
Fix null @topic_view error in post serializer for post_bookmark, as new posts do not have a @topic_view
2019-12-12 16:28:01 +10:00
Sam Saffron
9d6a9bba96 DEV: correct flaky test
This is a slight workaround which helps somewhat now but is pending a larger
fix.

When this spec ran in parallel mode uploads could start cross talking and
an upload you expect to be there may vanish.

This works around the issue by making the upload unique every time it is
created

It also folds up an expensive test into the main one.
2019-12-12 16:54:22 +11:00
Martin Brennan
edbc356593
FIX: Replace deprecated URI.encode, URI.escape, URI.unescape and URI.unencode (#8528)
The following methods have long been deprecated in ruby due to flaws in their implementation per http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/29293?29179-31097:

URI.escape
URI.unescape
URI.encode
URI.unencode
escape/encode are just aliases for one another. This PR uses the Addressable gem to replace these methods with its own encode, unencode, and encode_component methods where appropriate.

I have put all references to Addressable::URI here into the UrlHelper to keep them corralled in one place to make changes to this implementation easier.

Addressable is now also an explicit gem dependency.
2019-12-12 12:49:21 +10:00
Mark VanLandingham
09d9baa6d7
FIX: Update S3 stubs for more aws-sdk API changes (#8534) 2019-12-11 11:26:52 -08:00
dependabot-preview[bot]
863d3f89de Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0 (#8502)
* Build(deps): Bump aws-sdk-s3 from 1.36.1 to 1.59.0

Bumps [aws-sdk-s3](https://github.com/aws/aws-sdk-ruby) from 1.36.1 to 1.59.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-s3/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/v1.36.1...v1.59.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Dependabot/bundler/aws sdk s3 1.59.0 (#8532)
2019-12-11 10:18:11 -08:00
Dan Ungureanu
554b0f366d
SECURITY: Ensure only image uploads can be inlined
This prevents malicious files (for example special crafted XMLs) to be
used in XSS attacks.
2019-12-11 17:08:58 +02:00
Dan Ungureanu
43ddb6b36d
SECURITY: Remove event handlers from SVG files 2019-12-11 17:08:57 +02:00
Dan Ungureanu
4e130f1e03
SECURITY: Remove event handlers from SVG files 2019-12-11 17:04:50 +02:00
Dan Ungureanu
adfa793731
SECURITY: Ensure only image uploads can be inlined
This prevents malicious files (for example special crafted XMLs) to be
used in XSS attacks.
2019-12-11 17:04:49 +02:00
dependabot-preview[bot]
b90a592146 DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490)
Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 06:13:17 -08:00
Martin Brennan
6261339da9
Improving bookmarks part 1 (#8466)
Note: All of this functionality is hidden behind a hidden, default false, site setting called `enable_bookmarks_with_reminders`. Also, any feedback on Ember code would be greatly appreciated!

This is part 1 of the bookmark improvements. The next PR will address the backend logic to send reminder notifications for bookmarked posts to users. This PR adds the following functionality:

* We are adding a new `bookmarks` table and `Bookmark` model to make the bookmarks a first-class citizen and to allow attaching reminders to them.
* Posts now have a new button in their actions menu that has the icon of an actual book
* Clicking the button opens the new bookmark modal.
* Both name and the reminder type are optional.
* If you close the modal without doing anything, the bookmark is saved with no reminder.
* If you click the Cancel button, no bookmark is saved at all.
* All of the reminder type tiles are dynamic and the times they show will be based on your user timezone set in your profile (this should already be set for you).
* If for some reason a user does not have their timezone set they will not be able to set a reminder, but they will still be able to create a bookmark.
* A bookmark can be deleted by clicking on the book icon again which will be red if the post is bookmarked.

This PR does NOT do anything to migrate or change existing bookmarks in the form of `PostActions`, the two features live side-by-side here. Also this does nothing to the topic bookmarking.
2019-12-11 14:04:02 +10:00
Neil Lalonde
20464116ae FIX: tag input doesn't show all top 5 permitted tags 2019-12-10 10:30:01 -05:00
Sam Saffron
0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
Krzysztof Kotlarek
81c7d6a462
FIX: Muted tags are respected by TopicTrackingState (#8467)
When the tag is muted and topic contains that tag, we should not mark that message as NEW.

There are 3 possible settings which site admin can set.
remove_muted_tags_from_latest - always
It means that if the topic got at least one muted tag, we should not mark that topic as NEW

remove_muted_tags_from_latest - only muted
Similar to above, however, if at least one tag is not muted, the topic is marked as NEW

remove_muted_tags_from_latest - never
Basically, mute tag setting is ignored and all topics are set as NEW
2019-12-10 09:50:05 +11:00
Mark VanLandingham
14cb386f1e
FEATURE: Featured topic for user profile & card (#8461) 2019-12-09 11:15:47 -08:00
Dan Ungureanu
aa24be1a9a
DEV: Extend plugin API for uploads (#8440)
* DEV: Add API to alter uploads Markdown

* DEV: Extract data attributes from image / download Markdown

For example '[test|attachment|hello=world]' will generate an 'a' element
with a data attribute: 'data-hello=world'.

This commit also makes MarkdownIt to transform '|attachment' into
'class="attachment"'. This transformation used to be a part of the
process which resolves short URLs (i.e. upload://).

* DEV: Export imageNameFromFileName
2019-12-09 16:20:03 +02:00
Dan Ungureanu
ebe6fa95be
FIX: Optimize images in Onebox (#8471)
This commit ensures that images in Onebox are being optimized, but not
converted to lightbox too.
2019-12-09 15:39:25 +02:00
Sam Saffron
494fe335d3 DEV: allow handling crawler reqs with no user agent
Followup to e440ec25 we treat no user agent as crawler reqs.
2019-12-09 18:40:10 +11:00
Sam Saffron
e440ec2519 FIX: crawler requests not tracked for non UTF-8 user agents
Non UTF-8 user_agent requests were bypassing logging due to PG always
wanting UTF-8 strings.

This adds some conversion to ensure we are always dealing with UTF-8
2019-12-09 17:43:51 +11:00
Jarek Radosz
067c4a7c10 DEV: Fix CookedPostProcessor specs
Broken in #8480
2019-12-09 03:16:25 +01:00
Gerhard Schlager
c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
Sam Saffron
a06fccae1b DEV: update dependencies and add notes about exceptions
Previously it was unclear why certain gems are being held back cause Gemfile
had no comment explaining it.

I tried to add some explanation from memory and remove some exceptions that
seemed to be superfluous.

This upgrades shoulda to latest, it appears to work once a couple of assertions
are removed

Also update http accept language used to auto detect language from http header
this is tested

Zeitwerk small update seems fine
2019-12-06 13:00:28 +11:00
Vinoth Kannan
f7084a4339 FEATURE: add site setting to remove X-Frame-Options header. 2019-12-06 03:15:09 +05:30
Vinoth Kannan
11c4d5eda7 Fix the build - take 3.
754e1238b0
2019-12-05 20:35:39 +05:30
Vinoth Kannan
e6dfcda0bc DEV: handle all notification consolidations in new 'NotificationConsolidator' class.
481c8314f0
2019-12-05 14:36:06 +05:30
Jarek Radosz
d07f039468 FIX: Secure Upload URLs in lightbox (#8451)
This fixes the following issues:

* The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image
* Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 09:13:09 +10:00
Neil Lalonde
875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Vinoth Kannan
15c2755b7b Fix the WebHookPostSerializer spec.
400f79cffc
2019-12-04 23:30:04 +05:30
Robin Ward
888d56774a
DEV: HTML Builders should respect if a plugin is enabled or not (#8454)
Previously they would return the HTML regardless of whether the plugin
was enabled or not.
2019-12-04 12:26:23 -05:00
David Taylor
be1510b09d FIX: Use filtered posts when determining the next page
This bug was causing some unusual behavior when the last post is filtered (e.g. from an ignored user). In some situations this would cause suggested topics to be omitted from the payload.

The next_page specs have been updated to remove most of the stubs
2019-12-04 12:52:24 +00:00
Krzysztof Kotlarek
46fc45de99
FIX: inbound link when the only slug available (#8457)
Problem mentioned in [meta](https://meta.discourse.org/t/inbound-links-dont-show-up-when-topic-id-is-not-included/100551)

When there is an internal link without ID, only slug, we should still try to create reflection link.
2019-12-04 17:13:20 +11:00
Gerhard Schlager
9ebb69e8eb FIX: Respect enable_inline_emoji_translation setting in titles 2019-12-03 20:39:11 +01:00
romanrizzi
c86ca3609e FIX: Filter readers avatars correctly when the post is a whisper 2019-12-03 10:50:02 -03:00
Joffrey JAFFEUX
5cb00d5528 DEV: s/\$redis/Discourse\.redis
With manual merge conflicts
2019-12-03 14:26:57 +01:00
Sam Saffron
14db879a31 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-12-03 14:03:30 +01:00
Sam Saffron
ef791a5b1f DEV: use Discourse.cache over Rails.cache
With manual merge
2019-12-03 14:03:21 +01:00
Joffrey JAFFEUX
0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Arpit Jalan
016732cced FEATURE: improve suspect user discovery 2019-12-02 16:39:28 +05:30
Bianca Nenciu
57f108899a
FEATURE: Make site texts controller handle pages and locales (#8408)
Some endpoints are returning i18n keys instead of translated messages
and with these changes, the site_texts endpoint can help translating
those.

Pagination part is needed for better wildcard support. For example,
looking for 'js.notifications' would set 'has_more' to true, but return
only the first 50 messages with no way of fetching the remaining.
2019-11-30 15:16:36 +02:00
David Taylor
06c2e28bbb DEV: Remove two more references to instagram_user_infos
Follow-up to ad6f33e5b1
2019-11-29 17:12:06 +00:00
David Taylor
a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Robin Ward
7fee3c61de
Support for custom messages and redirects when creating posts (#8434)
* Support for custom messages and redirects when creating posts

When a post/topic is created Discourse serializes a `NewPostResult`
object. Normally this contains a status like `created_post` or
errors describing why the post could not be created.

There are times when a plugin might want to take the inputted post
and do something in the background. In this case, the plugin
can return a custom `message` and `route_to` attribute in the
`NewPostResult`.

If present, the message will be displayed in an alert, and when "Ok" is
clicked the user will be routed to the new URL.

* Destroy the draft in parallel
2019-11-29 09:30:54 -05:00
Dan Ungureanu
1e0c2235a3
FIX: Optimize quoted images (#8427)
Only images that were part of a lightbox used to be optimized. This
patch ensures that quoted images are also optimized.
2019-11-29 15:18:42 +02:00
Krzysztof Kotlarek
b120728999
FEATURE: Ability to add components to all themes (#8404)
* FEATURE: Ability to add components to all themes

This is the first and functional step from that topic https://dev.discourse.org/t/adding-a-theme-component-is-too-much-work/15398/16

The idea here is that when a new component is added, the user can easily assign it to all themes (parents).

To achieve that, I needed to change a site-setting component to accept `setDefaultValues` action and `setDefaultValuesLabel` translated label.
Also, I needed to add `allowAny` option to disable that for theme selector.

I also refactored backend to accept both parent and child ids with one method to avoid duplication (Renamed `add_child_theme!` to more general `add_relative_theme!`)

* FIX: Improvement after code review

* FIX: Improvement after code review2

* FIX: use mapBy and filterBy directly
2019-11-28 16:19:01 +11:00
Vinoth Kannan
481c8314f0 FEATURE: consolidate group membership request notifications. 2019-11-28 04:02:35 +05:30
Martin Brennan
901054fd75
FIX: Cache failed onebox URL request server-side (#8421)
We already cache failed onebox URL requests client-side, we now want to cache this on the server-side for extra protection. failed onebox previews will be cached for 1 hour, and any more requests for that URL will fail with a 404 status. Forcing a rebake via the Rebake HTML action will delete the failed URL cache (like how the oneboxer preview cache is deleted).
2019-11-28 07:48:29 +10:00
Martin Brennan
e7c7a05097
FIX: Mark secure media upload insecure automatically if used for theme component (#8413)
When uploading a file to a theme component, and that file is existing and has already been marked as secure, we now automatically mark the file as secure: false, change the ACL, and log the action as the user (also rebake the posts for the upload)
2019-11-28 07:32:17 +10:00
Gerhard Schlager
d12f2580de FIX: Serve crawler view to Google PageSpeed 2019-11-27 22:15:34 +01:00
Roman Rizzi
1c9d18f094
FIX: Correctly ignore/approve replies when acting on a flagged post (#8425) 2019-11-27 17:19:44 -03:00
David Taylor
a227083c1c
FEATURE: Limit the number of active sessions for a user (#8411)
If a user has more than 60 active sessions, the oldest sessions will be terminated automatically. This protects performance when logging in and when loading the list of recently used devices.
2019-11-27 12:39:31 +00:00
Sam Saffron
88ecb650a9 DEV: Implement a faster Discourse.cache
This is a bottom up rewrite of Discourse cache to support faster performance
and a limited surface area.

ActiveSupport::Cache::Store accepts many options we do not use, this partial
implementation only picks the bits out that we do use and want to support.

Additionally params are named which avoids typos such as "expires_at" vs "expires_in"

This also moves a few spots in Discourse to use Discourse.cache over setex
Performance of setex and Discourse.cache.write is similar.
2019-11-27 16:11:49 +11:00
Sam Saffron
0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
Leo McArdle
2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
Sam Saffron
941162e90e FIX: draft not clearing when replying to new topic
This amends our API so we provide it with the draft key when saving a post
this means post creator can clean up the draft consistently even if we are
doing fancy stuff like replying to a new topic or new pm or whatever.

There will be some followup work to clean it up so client never calls destroy
on draft during normal operation and the #create/#update endpoints takes care of it
every time
2019-11-26 18:23:20 +11:00
Sam Saffron
ebc82eb63e DEV: correct spec failures in PG 12
PG 12 changes internals in a subtle way, time jitter is noticed in a few new
spots (which is normal) and default ordering is a bit different which is meant
to be random anyway.
2019-11-26 16:39:14 +11:00
Krzysztof Kotlarek
6e403f20ee
FIX: Don't error when the empty current value in dif (#8406)
If current value is nil we should use `&.` combined with `dig` to protect diff from erroring

It is happening when for example theme is delete (new value is empty)
2019-11-26 09:17:14 +11:00
Dan Ungureanu
a992caf741
DEV: Replace magic values (#8398)
Follow-up to 35942f7c7c.
2019-11-25 14:32:19 +02:00
Dan Ungureanu
ae9e881333
DEV: Add test to ensure :after_auth event is triggered (#8400)
Follow-up to ee8669d778.
2019-11-25 14:31:57 +02:00
Martin Brennan
afb5533581
FEATURE: Add timezone to core user_options (#8380)
* Add timezone to user_options table

* Also migrate existing timezone values from UserCustomField,
  which is where the discourse-calendar plugin is storing them

* Allow user to change their core timezone from Profile

* Auto guess & set timezone on login & invite accept & signup

* Serialize user_options.timezone for group members. this is so discourse-group-timezones can access the core user timezone, as it is being removed in discourse-calendar.

* Annotate user_option with timezone

* Validate timezone values
2019-11-25 10:49:27 +10:00
Krzysztof Kotlarek
43ddf60cdf
FIX: When dismissing category inform via MessageBus (#8371)
When category is dismissed, `dismiss_new` message is sent to fronted to clean state.

In addition, I noticed that when old dismiss new button is clicked, no message is sent so I decided to kill two birds with one stone.
2019-11-25 06:17:31 +11:00
Roman Rizzi
c92f0b8775
FEATURE: New API to apply custom filters to the review queue (#8392) 2019-11-22 16:33:10 -03:00
Martin Brennan
23714e77c4
FIX: Always return secure_proxy_without_cdn url for secure media (#8394)
There was an issue on dev where when uploading secure media, the href of the media was correctly being replaced in the CookedPostProcessor, but the srcset urls were not being replaced correctly. This is because UrlHelper.cook_url was returning the asset host URL for the media for secure media instead of returning early with the proxied secure proxy url.
2019-11-22 15:29:31 +10:00
Penar Musaraj
11d22293fb FIX: Allow private media uploads to be reused in login_required sites
In non-login-required sites, we prevent secure uploads already used in PMs from being used in public topics.

In login_required sites, secure uploads should be reusable in any topic, PM or not.
2019-11-21 09:14:06 -05:00
Martin Brennan
5ad6dc01c1
FEATURE: Support RS256 algorithm for webauthn (#8385)
* Add support for RS256 webauthn algorithm (-257)
2019-11-21 15:45:36 +10:00
Sam Saffron
b57e108e84 FEATURE: improve email change workflow
- Show old and new email address during the process
- Ensure correct user is logged on when attempting to make email changes
- Support reloading a page during the email reset process without resubmit
of form
- Improve tests
- Fixed issue where redirect back to site was not linking correctly in
subfolder setups

Internal refactor of single action into 4 distinct actions that are simpler
to reason about.

This also removes the step that logs on an account after you confirm an
email change, since it is no longer needed which leaves us with safer
internals.

This left me no choice but to amend translations cause the old route was
removed.
2019-11-21 16:28:35 +11:00
Sam Saffron
423ad5f0a4 FIX: do not log if an invalid mime type is passed to app
Previously our custom exception handler was unable to handle situations
where an invalid mime type was sent, resulting in a warning log

This ensures we pretend a request is HTML for the purpose of rendering
the error page if an invalid mime type from a scanner is shipped to the app
2019-11-21 15:51:34 +11:00
Martin Brennan
d45bd7f131
FIX: Abort CensoredWordsValidator early if censored_words_regexp nil (#8375)
* Abort CensoredWordsValidator early if censored_words_regexp nil. Sometimes censored_words_regex can end up nil, erroring the validator. This handles the nil condition and also adds a spec for the validator
2019-11-20 20:05:06 +10:00
Sam Saffron
dd89a13e22 DEV: disable flaky spec
This is erratically failing in our production CI
2019-11-20 08:56:07 +11:00
Martin Brennan
02cb01406e
FIX: Allow secure uploads if global s3 setting active and enable_s3_uploads validations (#8373)
The secure media functionality relied on `SiteSetting.enable_s3_uploads?` which, as we found in dev, did not take into account global S3 settings via `GlobalSetting.use_s3?`. We now use `SiteSetting.Upload.enable_s3_uploads` instead to be more consistent.

Also, we now validate `enable_s3_uploads` changes, because if `GlobalSetting.use_s3?` is true users should NOT be enabling S3 uploads manually.
2019-11-20 07:46:44 +10:00
David Taylor
46841888b7 FIX: For a single authenticator, do not interrupt registration flow
Followup to 0a14b9b42a
2019-11-19 19:15:11 +00:00
Neil Lalonde
565a967192 FIX: email excerpts for posts starting with a quote were displaying a username
If a post starts with a post quote and has no other text content,
then the email excerpt was the name of the person quoted and
nothing else. The intention was to show the contents of the
first paragraph or div after the quote.

With this change, a quote followed by an image will use the
image as the excerpt. A quote followed by a onebox will use the
onebox.
2019-11-19 12:35:23 -05:00
Blake Erickson
266e486037 FIX: Turn off auto bumping for topics with scheduled bumps
If a topic has a timer scheduled to bump a topic it should be excluded
from being auto bumped.
2019-11-19 07:27:49 -07:00
Neil Lalonde
a4dbec5bf9 FIX: unable to remove required tag group from a category 2019-11-18 15:54:00 -05:00
Neil Lalonde
228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
Penar Musaraj
0e759add0f DEV: Better topic participants summary spec 2019-11-18 09:43:14 -05:00
Vinoth Kannan
3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Sam Saffron
7d389df5e7 DEV: correct spec to allow for new default
b4bfc27b changes the default so the spec should be changed as well.
2019-11-18 16:05:58 +11:00
Martin Brennan
af091c49e9
FIX: Ensure revisions are made to store edit reasons and no reasons get wiped (#8363)
* Fix an issue where if an edit was made to a post with a reason provided, and then another edit was made with no reason, the original edit reason got wiped out
* We now always make a post revision (even with ninja edits) if an edit reason has been provided and it is different from the current edit reason

Co-Authored-By: Sam <sam.saffron@gmail.com>
2019-11-18 13:08:54 +10:00
Penar Musaraj
102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Vinoth Kannan
56b19ba740 UX: instead of total user count display only the count of users going to be affected.
edec922803
2019-11-18 00:09:38 +05:30
Penar Musaraj
99b4f79a90 FIX: Include 5 participants in topic summary 2019-11-15 15:11:09 -05:00
Roman Rizzi
4cf3c9cccb
FEATURE: Filter reviewables by date range (#8354) 2019-11-15 15:29:59 -03:00
Martin Brennan
5c59247c3a Block enabling force 2FA if local logins disabled & vice-versa (#8355) 2019-11-15 17:05:10 +11:00
Sam Saffron
e7cf4579a8 DEV: improve usability of subfolder specs
Previously people were not consistent about mocking which left internals in
a fragile state when running subfolder specs.

This introduces a simple helper `set_subfolder` which you can use to set
the subfolder for the spec. It takes care of proper configuration of subfolder
and teardown.

```
# usage
set_subfolder "/my_amazing_subfolder"
```

You should no longer stub base_uri or global_settings
2019-11-15 16:48:24 +11:00
Jeff Wong
40d14ba166 Fix the actual failing test 2019-11-14 16:20:45 -08:00
Jeff Wong
25dd045ba5 Fix failing test 2019-11-14 16:11:42 -08:00
Jeff Wong
786fcceb62 Return full URLs in review serializer for both post and topic
Return full URLs, this fixes reviewable links to support subfolder
2019-11-14 15:58:21 -08:00
Blake Erickson
3b16eb7abb FIX: Confirm new email with backup codes enabled
This is a fix for this bug:

https://meta.discourse.org/t/-/133185?u=blake

where rails would throw a missing template error when trying to confirm
a new email address when you had two factor backup codes enabled.

Apparently this feature broke during this commit:

68d35b14f4

when a partial that contained a lot of javascript was removed most
likely because it didn't comply with our Content Security Policy, so as
a fix I rewrote the previous js functionality without using any
javascript and then added a spec to verify that the correct backup code
form is displayed when that page is loaded.
2019-11-14 16:27:42 -07:00
romanrizzi
ff355ad204 FIX: Don't show bots as post readers 2019-11-14 17:53:52 -03:00
Penar Musaraj
067696df8f DEV: Apply Rubocop redundant return style 2019-11-14 15:10:51 -05:00
David Taylor
f2da630c19 FIX: SVG Sprite version hash should be based on bundle result
This version hash is used for the filename, and so browsers/CDNs cache based on it. Previously the version hash was based only on the list of requested icons. This can cause issues in a couple of situations, most commonly when developing themes with custom icons:

- A requested icon does not exist, and then later is added to the theme. The bundle output changes, but the hash did not
- The SVG content of an icon changes, but the name of the icon does not. The bundle output changes, but the hash did not
2019-11-14 13:20:16 +00:00
Krzysztof Kotlarek
f434de2536
FIX: Tracking Topic State know about category_seen_at (#8351)
If category got last_seen_at is set TrackingTopicState should know about it and exclude those topics from marking them as new
2019-11-14 16:11:34 +11:00
Martin Brennan
e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
Krzysztof Kotlarek
6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
David Taylor
d095c2cee7 FIX: Allow importing themes with subdirectories in extra_js
The folder/file detection was broken by 9fea43e46a. This commit fixes and adds relevant specs
2019-11-13 23:45:09 +00:00
David Taylor
0a14b9b42a FEATURE: Automatically redirect to authenticator when there is only one
This brings the behavior in line with native Discourse SSO. If login is required, and a user tries to visit the forum, they will be directed straight to the external login page without requiring any clicks.
2019-11-13 17:28:12 +00:00
Neil Lalonde
7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
David Taylor
836b3f4d82
FIX: Do not deactivate admin accounts with recent posts or api keys (#8342)
This prevents 'bot' users being deactivated
2019-11-12 16:56:01 +00:00
Mark VanLandingham
3dd2f2f701
DEV: Remove RSS feed polling in favor of plugin (#8233) 2019-11-12 09:49:02 -06:00
David Taylor
5f927ceeb3
DEV: Display a warning when themes hard-code optimized image links (#8304) 2019-11-12 14:30:19 +00:00
Dan Ungureanu
bbcce08712
FIX: Update quotes after moving posts (#8326) 2019-11-12 15:16:39 +02:00
Krzysztof Kotlarek
69266f60ed FIX: tag and category watchers regression (#8336)
I made a regression here 17366d3bcc (diff-ddeebb36d131f89ca91be9d04c2baefaR10)

When the tag is added, people watching specific tag are notified but also people watching specific category.

Therefore, `notify_post_users` should accept options who should be notified.

So when `category` is added to the topic, users watching topic and users watching category are notified.

When `tag` is added to the topic, users watching topic and users watching tag are notified

Finally, when a new post is created, everybody is notified, topic watchers, category watchers, tag watchers.
2019-11-12 16:44:46 +11:00
Gerhard Schlager
6ebffaaf6e FIX: Better error handling for invalid locale bundle versions 2019-11-11 22:30:32 +01:00
Dan Ungureanu
6672dcc985
FIX: Keep emoji images in group bio excerpt (#8329) 2019-11-11 18:42:08 +02:00
Sam Saffron
0c0192e7da DEV: disable all upload recovery specs
These specs are inherently fragile when they are run in a concurrent mode
in CI.

One process will create an upload, another will destroy the upload on disk
at the same time. We need a new safe mechanism here.
2019-11-11 16:04:18 +11:00
Sam Saffron
91daafc674 DEV: correct implementation of expiry api
Previously we were always hard-coding expiry, this allows the secure session
to correctly handle custom expiry times

Also adds a ttl method for looking up time to live
2019-11-11 11:18:12 +11:00
Dan Ungureanu
dfc002d331
FIX: Count current penalty if it started more than 6 months ago (#8313) 2019-11-08 12:23:57 +02:00
Martin Brennan
56d3e29a69
FIX: Badge and user title interaction fixes (#8282)
* Fix user title logic when badge name customized
* Fix an issue where a user's title was not considered a badge granted title when the user used a badge for their title and the badge name was customized. this affected the effectiveness of revoke_ungranted_titles! which only operates on badge_granted_titles.
* When a user's title is set now it is considered a badge_granted_title if the badge name OR the badge custom name from TranslationOverride is the same as the title
* When a user's badge is revoked we now also revoke their title if the user's title matches the badge name OR the badge custom name from TranslationOverride
* Add a user history log when the title is revoked to remove confusion about why titles are revoked
* Add granted_title_badge_id to user_profile, now when we set badge_granted_title on a user profile when updating a user's title based on a badge, we also remember which badge matched the title
* When badge name (or custom text) changes update titles of users in a background job
* When the name of a badge changes, or in the case of system badges when their custom translation text changes, then we need to update the title of all corresponding users who have a badge_granted_title and matching granted_title_badge_id. In the case of system badges we need to first get the proper badge ID based on the translation key e.g. badges.regular.name
* Add migration to backfill all granted_title_badge_ids for both normal badge name titles and titles using custom badge text.
2019-11-08 15:34:24 +10:00
Martin Brennan
64b4a7ba45
FIX: Ensure enforce 2FA for staff satisfied by security keys (#8316)
* If a staff user created only a security key as their single 2FA option. they continued to be prompted to create a 2FA option because we only considered this condition satisfied if a TOTP was added.
* The condition is now satisfied if TOTP OR security keys are enabled.
2019-11-08 15:11:53 +10:00
Vinoth Kannan
ba5b78a348
FEATURE: support to mute all categories by default. (#8295)
Instead of enabling `suppress_from_latest` setting on many categories now we can enable `mute_all_categories_by_default` site setting. Then users should opt-in to categories for them to appear in the latest and categories pages.
2019-11-08 08:28:11 +05:30
David Taylor
60a235d128
DEV: Allow execute_command to receive a block (#8303)
This makes it easy to run multiple commands with the same keyword arguments. The main use is for using `chdir` across multiple commands. The `Dir.chdir` method is not concurrency safe because it switches the working directory of the entire process.
2019-11-07 15:47:16 +00:00
Krzysztof Kotlarek
17366d3bcc
FEATURE: notify tag watchers when tag was added to post (#8299)
Issue was mentioned in this [meta topic](https://meta.discourse.org/t/send-a-notification-to-watching-users-when-adding-tag/125314)

It is working well when category is changed because NotifyCategoryChange job already got that code:
```
if post&.topic&.visible?
  post_alerter = PostAlerter.new
  post_alerter.notify_post_users(post, User.where(id: args[:notified_user_ids]))
  post_alerter.notify_first_post_watchers(post, post_alerter.category_watchers(post.topic))
end
```

For NotifyTagChange job notify post users were missing so it worked only when your notification was set to `watching first post`
2019-11-07 08:20:15 +11:00
Mark VanLandingham
bf778d66b5
FIX: unicode group names encoded for url (#8302) 2019-11-06 14:25:45 -06:00
David Taylor
914e50db49 DEV: Update users controller spec following user_search update 2019-11-06 17:32:10 +00:00
Penar Musaraj
0dfc594784 FIX: skip invalid URLs when checking for audio/video in search blurbs
Fixes 500 errors on search queries introduced in 580a4a8
2019-11-06 10:32:15 -05:00
Krzysztof Kotlarek
5ad0dd0c23 FIX: tweak restorer spec to make it stableish (#8300) 2019-11-06 17:05:33 +11:00
Gerhard Schlager
61b1f9c36b FEATURE: Load translation overrides without JS eval 2019-11-05 19:16:38 +01:00
Penar Musaraj
ca6adfbdd6 DEV: Ignore renames when diffing theme changes
Renamed assets are quite noisy when checking whether a remote theme has local changes.
2019-11-05 11:52:04 -05:00
David Taylor
52c5cf33f8
FEATURE: Overhaul of admin API key system (#8284)
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
2019-11-05 14:10:23 +00:00
Vinoth Kannan
671f303b53
FEATURE: Add welcome message for admins. (#8293) 2019-11-05 18:15:55 +05:30
Neil Lalonde
b6d140e4bd UX: tag input suggests required tags if none have been selected
This is a follow-up to the new feature that allows a category to
require a certain number of tags from a tag group. The tag input will
shows results from the required group if none have been chosen yet.
Once a require tag is selected, the tag input will include other
results as usual. Staff users can ignore this restriction, so the input
behaviour is unchanged for them.
2019-11-04 16:51:54 -05:00
Penar Musaraj
74869b8a7f FIX: Do not consider mobile app traffic as crawler visits
Followup to a4eb523a
2019-11-04 09:16:50 -05:00
Joe
ce0bac7a3d FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286)
* use image alt as a fallback when there's no title

* update spec

we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored.

The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 10:15:14 +11:00
Joffrey JAFFEUX
24d4241e35 FIX: prevents whitelisted_generic_onebox_spec to fail with zeitwerk (#8288) 2019-11-04 09:15:09 +11:00
Vinoth Kannan
2cb805a683 DEV: Add option to keep onebox body content in post excerpt. 2019-11-02 17:14:04 +05:30
Daniel Waterworth
200cef90ea FIX: TopicQuery doesn't react well to subcategories without definitions
Also:

Move includes call higher which makes it possible to run all of the
intermediate queries for easier debugging.

Add tests for TagsController with categories in the path.
2019-11-02 08:34:43 +00:00
Daniel Waterworth
2b8a013e32 DEV: Reduce duplication 2019-11-01 13:07:51 +00:00
Daniel Waterworth
f9a05e4ae7 DEV: Switch flaky blame to restorer_spec.rb 2019-11-01 12:27:38 +00:00
Daniel Waterworth
eae7c0e395 DEV: Mark flaky test 2019-11-01 11:35:57 +00:00
Daniel Waterworth
b6d2b78a6b DEV: More prefabrication 2019-11-01 10:40:58 +00:00
David Taylor
4312bbe1e7
FIX: Do not load plugin CSS/JS assets when disabled (#8275)
Follow-up to 839916aa49 and 5bd6b70d98
2019-11-01 09:50:31 +00:00
Vinoth Kannan
72aa26c8c5
FEATURE: New site settings for default tags in user preferences. (#8283) 2019-11-01 12:40:13 +05:30
Sam Saffron
25a3bd333e DEV: correct fragile test
This test would fail sometimes if for some reason topic id 1 was in the db

No reason to pretend here, this only makes for a fragile test suite.
2019-11-01 15:40:00 +11:00
Martin Brennan
f753643cb1
FIX: Unread topics not clearing when whisper is last post (#8271)
Meta thread: https://meta.discourse.org/t/cant-dismiss-unread-if-last-post-is-an-assign-or-whisper/131823/7

* when sending a whisper, the highest_staff_post_number is set
in the next_post_number method for a Topic, but the
highest_post_number is left alone. this leaves a situation
where highest_staff_post_number is > highest_post_number
* when TopicsBulkAction#dismiss_posts was run, it was only setting the topic_user
highest_seen_post_number using the highest_post_number from the topic, so if
the user was staff and the last post in a topic was a whisper
their highest seen number was not set, and the topic stayed unread

Found through testing that the bug wasn't to do with Assign/Unassign as they do not affect the post numbers, only whispering does.
2019-11-01 09:19:43 +10:00
Neil Lalonde
d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
Daniel Waterworth
d84c34ad75 DEV: Server-side category routing changes
The routes for categories are changing. The scheme that I intend to move
us to is:

/c/*slug_path/(:id)/ENDPOINT
/c/*slug_path/(:id)

This commit adds support for the new scheme to the server side without
dropping support for existing URLs. It is necessary to support existing
URLs for two reasons:

 * This commit does not change any client side routing code,
 * Posts that contain category hashtags that refer to a root category
   are baked into URLs that do not fit this new scheme, (/c/[id]-[slug])
2019-10-31 18:06:22 +00:00
David Taylor
1f88ecf6d8
FIX: Cache Discourse.system_user separately for each multisite tenant (#8276)
This was not causing any known issue, because the system user ID is always the same across all sites. However, we should cache this on a per-site basis to be safe.
2019-10-31 15:16:26 +00:00
Penar Musaraj
f8b72d9835 DEV: Refactor excluding audio/video URLs from search result blurbs
Followup to 580a4a82
2019-10-31 09:13:24 -04:00
Sam Saffron
c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Vinoth Kannan
d355506123 FEATURE: Option to update user preferences of all users when default site setting changed. 2019-10-31 06:48:03 +05:30
Joffrey JAFFEUX
0ba28e344b FIX: tags can be filtered on categoryId without a q param (#8264) 2019-10-31 10:16:23 +10:00
Sam Saffron
d8f7f363cd FEATURE: stop updating last_posted_at on users for messages and whispers
This ensures we only update last_posted_at which is user facing for non messages
and non whispers.

We still update this date for secure categories, we do not revert it for
deleted posts.
2019-10-31 09:01:26 +11:00
Daniel Waterworth
32107a9a72 FIX: Correct slug validation
We were allowing new categories to use slugs like "2342-category".
2019-10-30 17:13:28 +00:00
Penar Musaraj
580a4a827b Exclude audio/video URLs from search result blurbs
Displays translatable "[audio]" or "[video]" placeholders instead of ugly (and often long) URLs.
2019-10-30 13:07:16 -04:00
Régis Hanol
92213813d4 Revert "FIX: Require q param in /tags/filter/search route (#8263)"
This reverts commit 968a1419df.
2019-10-30 10:37:18 +01:00
Martin Brennan
968a1419df
FIX: Require q param in /tags/filter/search route (#8263)
* Require q param in /tags/filter/search route.
* If not provided this route was causing a 500 error when
  DiscourseTagging.clean_tag was called, because .downcase
  was being called on the param (which was nil).
* Now return a 400 error instead.
2019-10-30 16:32:48 +10:00
Arpit Jalan
2ae71b7a87 FEATURE: allow sending bulk invites to staged users 2019-10-30 11:40:03 +05:30
Krzysztof Kotlarek
c32bd8ae48 FEATURE: Remove attachments and truncate raw field for incoming emails (#8253)
Adds the settings: 

raw_email_max_length, raw_rejected_email_max_length, delete_rejected_email_after_days. 

These settings control retention of the "raw" emails logs.

raw_email_max_length ensures that if we get incoming email that is huge we will truncate it removing uploads from the raw log.

raw_rejected_email_max_length introduces an even more aggressive truncation for rejected incoming mail. 

delete_rejected_email_after_days controls how many days we will keep rejected emails for (default 90)
2019-10-30 16:54:35 +11:00
Jeff Wong
74dc37c07c FIX: upserting custom fields using keywords converts the array key to a string 2019-10-29 11:35:38 -07:00
Daniel Waterworth
29e41dc511 Revert "DEV: Mark flaky tests as pending"
This reverts commit 0c47529713.
2019-10-29 13:32:42 +00:00
Krzysztof Kotlarek
c5bcad10a6
FIX: more flaky jobs_base_spec.rb (#8259)
This commit is follow up of 417be323a2

We need more steps to properly switch from multi-site to single-site
2019-10-29 21:32:43 +11:00
Daniel Waterworth
0c47529713 DEV: Mark flaky tests as pending 2019-10-29 10:00:00 +00:00
Krzysztof Kotlarek
417be323a2 FIX: flaky jobs_base_spec.rb (#8258)
I was searching for a reason for randomly failing jobs_base_spec.rb. The reason was that after restorer_spec, the database is not restored to default.
After restorer spec RailsMultisite::ConnectionManagement.all_dbs is returning array of ['default', 'second']

Then base job execution is evaluated twice
```
dbs = RailsMultisite::ConnectionManagement.all_dbs
dbs.each do |db|
     execute(opts)
end
```
2019-10-29 18:25:00 +11:00
Mark VanLandingham
4eb54f08b2
FEATURE: Site setting/UI to allow users to set their primary group (#8244)
* FEATURE: Site setting/ui to allow users to set their primary group

* prettier and remove logic from account template

* added 1 to 43 to make web_hook_user_serializer_spec pass
2019-10-28 12:46:27 -05:00
Leo McArdle
e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
David Taylor
6de49b88df FIX: Respond to user search correctly when category_id is blank
Previously it would search for category_id=0, which does not exist. With the new permission checks, this returns a 404
2019-10-28 13:18:47 +00:00
David Taylor
f0617f96ed DEV: Update users controller spec following user_search update 2019-10-28 12:32:56 +00:00
David Taylor
c38c37bcc3 SECURITY: Check permissions when autocompleting mentions 2019-10-28 12:20:26 +00:00
David Taylor
eec464d8d1 SECURITY: Check permissions when autocompleting mentions 2019-10-28 11:01:47 +00:00
Sam Saffron
3d85cc1e69 PERF: run expensive clean up uploads less frequently
Previously every hour we would run a full scan of the entire DB searching
for expired uploads that need to be moved to the tombstone folder.

This commit amends it so we only run the job 2 times per clean_orpha_uploads_grace_period_hours

There is a upper bound of 7 days so even if the grace period is set really
high it will still run at least once a week.

By default we have a 48 grace period so this amends it to run this cleanup
daily instead of hourly. This eliminates 23 times we run this ultra expensive
query.
2019-10-28 11:14:52 +11:00
Neil Lalonde
4c2d6e19ba PERF: cache new users counts in summary emails
The query to count how many new users there are since a given date
is expensive. It's the least personalized stat and the one we fallback
to last when no better number can be found for the target user.
Give up accuracy so we can aggressively cache the user counts
that appear in this email.
2019-10-25 16:33:36 -04:00
Roman Rizzi
070a3dcf9b
FIX: When running the wizard and using a custom theme, fallback to the color_scheme name if the base_scheme_id is nil (#8236) 2019-10-25 09:29:51 -03:00
David Taylor
f5d6236ad2 DEV: Use before_all to run code before prefabrication
Followup to e9897d295f
2019-10-24 12:27:40 +01:00
David Taylor
e9897d295f DEV: Refactor user search spec to use prefabrication
Break up single large example into multiple examples, using fab! to maintain performance. On my machine, this speeds up the test slightly, and also makes it more readable.
2019-10-24 11:58:33 +01:00
Krzysztof Kotlarek
b8688c4af7
FIX: Rubocop rule on restorer spec (#8238) 2019-10-24 12:03:27 +11:00
Krzysztof Kotlarek
f530378df3 FIX: Restore for non-multisite is not raising an error on reconnect step (#8237)
That commit introduced a bug to the system: f69dacf979

Restore works fine for multisite, however, stopped working for non-multisite.

Reason for that was that `establish_connection` method got a check if the multisite instance is available:
```
    def self.instance
      @instance
    end

    def self.establish_connection(opts)
      @instance.establish_connection(opts) if @instance
    end
```
However, the reload method don't have that check
```
    def self.reload
      @instance = new(instance.config_filename)
    end
```

To solve it, let's ensure we are in a multisite environment before call reload
2019-10-24 11:46:22 +11:00
Neil Lalonde
cfe26eb301 FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
Neil Lalonde
f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Vinoth Kannan
726bf37a12 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:50:10 +05:30
Vinoth Kannan
31577b2131 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:35:38 +05:30
Daniel Waterworth
1a72a61822 FIX: Fixed testsuite
The backup restorer tests weren't cleaning up after themselves and
playing poorly with prefabrication.
2019-10-23 09:43:05 +01:00
Krzysztof Kotlarek
f69dacf979 FIX: Reconnect in restore process connects to correct DB (#8218)
Simplified flow of restore is like that
```
migrate_database
reconnect
extract_uploads
```

Problem with incorrect current database started with this fix https://github.com/discourse/discourse/commit/025d4ee91f4

Dump task is reconnecting to default database https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L429

And then, we are trying to reconnect to the original database with that code:
```
def reconnect_database
  log "Reconnecting to the database..."
  RailsMultisite::ConnectionManagement::establish_connection(db: @current_db)
end
```

This reconnect is not switching us back to correct database because of that check
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L181
Basically, it finds existing handler and it thinks that we are connected to correct DB and this step can be skipped.

To solve it, we can reload RailsMultisite::ConnectionManagement which creates a new instance of that class
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L38
2019-10-23 17:23:50 +11:00
Sam Saffron
950da34826 DEV: waiting for 10ms is hardly enough
tests that test thread behavior and rely on scheduling need to allow for
a wider amount of error margin
2019-10-23 16:18:41 +11:00
Arpit Jalan
1e9d9d9346
FIX: respect tl3 links no follow setting (#8232) 2019-10-22 22:41:04 +05:30
Blake Erickson
7d09af7eda
FIX: Prevent null-byte searches causing 500 error (#8226)
This fix ensures that searches that contain a null byte return a 400
error instead of causing a 500 error.

For some reason from rspec we will reach the raise statement inside
of the `rescue_from ArgumentError` block, but outside of rspec it will
not execute the raise statement and so a 500 is thrown instead of
reaching the `rescue_from Discourse::InvalidParameters` block inside of
the application controller.

This fix raises Discourse::InvalidParameters directly from the search
controller instead of relying on `PG::Connection.escape_string` to
raise the `ArgumentError`.
2019-10-22 08:44:52 -06:00
Blake Erickson
3201613f13 FEATURE: Include image url in topic serializer
The `image_url` is already included when fetching a list of topics, and
this commit adds it to the individual topic serializer so that it is
available via the api.

See

https://meta.discourse.org/t/single-topic-api-endpoint-should-contain-image-url/131020

for more details.
2019-10-21 22:02:49 -06:00
Blake Erickson
ef0fe51e05
FIX: Include user id in notification webhook (#8195)
The payload when receiving a notification webhook is pointless without
knowing which user the notification is for. This fix adds the user_id to
the notification serializer so that when you receive a notification
webhook you can properly identify which user the notification is for.

See

https://meta.discourse.org/t/getting-the-target-user-for-notification-webhook-events/129052?u=blake

for more details.
2019-10-21 16:24:41 -06:00
Dan Ungureanu
ceb74bef8f
DEV: Add test.
Follow-up to 2f54dd5e3d.
2019-10-21 21:33:58 +03:00
Michael Brown
40b0b549ff DEV: update the comments re the encoded slug changes
* followup to 2e73985d
2019-10-21 13:39:45 -04:00
Rafael dos Santos Silva
2e73985d1b FIX: Make category updates slug validation idempotent
Co-authored-by: Michael Brown <supermathie@gmail.com>
2019-10-21 14:33:19 -03:00
Dan Ungureanu
3ad07aacfa
FIX: Reload only notifications when refreshing notification count (#8221)
Previously, we used to reload the whole User instance which discarded
any changes made (for example setting 'unstage' to false).
2019-10-21 20:25:46 +03:00
Vinoth Kannan
5e55e75aed FIX: 'only_hidden_tags_changed?' method returned 'true' even when tags are not changed.
While editing the first post it does't bumped the topic when the new post revision created. Because we wrongly assumed that the hidden tags are changed even when no tags are updated.
2019-10-21 17:57:31 +05:30
Daniel Waterworth
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Rafael dos Santos Silva
2304dcf993 FIX: Do not truncate encoded slugs
Trying to truncate encoded slugs will mean that we have to keep the URL
valid, which can be tricky as you have to be aware of multibyte
characters.

Since we already have upper bounds for the title, the slug won't grow
for more than title*6 in the worst case. The slug column in the topic
table can store that just fine.

Added a test to ensure that a generated slug is a valid URL too, so we
don't introduce regressions in the future.
2019-10-17 13:38:31 -03:00
Mark VanLandingham
bd969332e0 FIX: Display site text overrides for non '_MF' keys (#8189)
FIX: Transform pluralized keys to `.other`, to check valid interpolation
2019-10-17 18:34:07 +02:00
David Taylor
11a9e0fc70 DEV: Add test for creating user with custom fields and user fields 2019-10-17 15:46:23 +01:00
Sam Saffron
cb0443792b DEV: correct revision debouncer for backup draft
We were debouncing on update_at which we kept on pushing forward, this
was leading to not enough revisions.
2019-10-17 17:41:28 +11:00
Sam Saffron
f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Daniel Waterworth
7a0c06691c FIX: Account for nil when looking up subcategories 2019-10-16 20:04:54 +01:00
Neil Lalonde
5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Neil Lalonde
61f6a6e836 Remove focus from specs 2019-10-16 14:28:04 -04:00
Sam Saffron
d5d8db7fa8 FEATURE: improve honeypot and challenge logic
This feature amends it so instead of using one challenge and honeypot
statically per site we have a rotating honeypot and challenge value which
changes every hour.

This means you must grab a fresh copy of honeypot and challenge value once
an hour or account registration will be rejected.

We also now cycle the value of the challenge when after successful account
registration forcing an extra call to hp.json between account registrations

Client has been made aware of these changes.

Additionally this contains a JavaScript workaround for:
https://bugs.chromium.org/p/chromium/issues/detail?id=987293

This is client side code that is specific to Chrome user agent and swaps
a PASSWORD type honeypot with a TEXT type honeypot.
2019-10-16 16:53:44 +11:00
Krzysztof Kotlarek
0bf55fe2ff
Revert "FIX: public_file_server.enabled is false in test (#8192)" (#8196)
This reverts commit 5a8fdd02fe.
2019-10-16 10:39:31 +11:00
Krzysztof Kotlarek
5a8fdd02fe FIX: public_file_server.enabled is false in test (#8192)
After a small conversation, we decided that we can set `public_file_server.enabled` to false in the `test` environment to have the same value as `production`.
2019-10-16 09:18:48 +11:00
Daniel Waterworth
5f5b232cde FIX: Category.find_by_slug
find_by_slug should ensure that the parent actually exists when its
looking for a parent.
2019-10-15 16:44:24 +01:00
Vinoth Kannan
b2f682f35e
FEATURE: Option to update category preferences of all users when site setting changed (#8180) 2019-10-15 18:41:27 +05:30
Daniel Waterworth
0de7e4339c FIX: Subcategory permissions validation
When a category has a subcategory, we ensure that no one who can see the
subcategory cannot see the parent. However, we don't take into account
the fact that, when no CategoryGroups exist, the default is that
everyone has full permissions.
2019-10-14 21:13:22 +01:00
Daniel Waterworth
c49b20a1a2 FIX: Validation of category tree depth
This prevents the creation of sub-sub-categories in multiple tabs
2019-10-14 21:13:22 +01:00
Gerhard Schlager
2c011252f1 FIX: Move notification level only when user posted
Moving posts also moves the read state (`topic_users` table) to the destination topic. This changes that behavior so that only users who posted in the destination topic will have the original notification level (probably "watching") of the original topic. The notification level for all other users will be set to "regular".
2019-10-14 15:06:09 +02:00
Dan Ungureanu
96b8710b39
DEV: Fix heisentest (ensure that user ID really does not exist). 2019-10-14 12:25:43 +03:00
Vinoth Kannan
519fe290e3 SPEC: 'lookup_upload_urls' method should use cdn url if available.
e4fe864c0b
2019-10-14 12:57:33 +05:30
Vinoth Kannan
129e308dac fix the incorrect specs
Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 12:49:11 +05:30
Vinoth Kannan
f92a6f7ac5 FIX: 'local_cdn_url' method should work for local relative urls too. 2019-10-14 11:39:16 +05:30
Krzysztof Kotlarek
99086edf85
FIX: Allow themes to upload and serve js files (#8188)
If you set `config.public_file_server.enabled = false` when you try to get uploaded js file you will get an error:
`Security warning: an embedded <script> tag on another site requested protected JavaScript. If you know what you're doing, go ahead and disable forgery protection on this action to permit cross-origin JavaScript embedding.`

The reason is that content type is `application/javascript` and in Rails 5 guard looked like that:
https://github.com/rails/rails/blob/5-2-stable/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L278-L280
However, in Rails 6 `application` was added to regex:
https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L282-L284

This pull request is related to https://meta.discourse.org/t/uploaded-js-file-for-theme-causes-a-rejection/129753/8
2019-10-14 15:40:33 +11:00
Roman Rizzi
01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Mark VanLandingham
f63db1c4c8
FIX: Accurate sub_total calculation for reviewable_scores (#8184) 2019-10-11 11:07:19 -05:00
Rafael dos Santos Silva
76ab0350f1
FIX: Properly encoded slugs when configured to (#8158)
When an admin changes the site setting slug_generation_method to
encoded, we weren't really encoding the slug, but just allowing non-ascii
characters in the slug (unicode).

That brings problems when a user posts a link to topic without the slug, as
our topic controller tries to redirect the user to the correct URL that contains
the slug with unicode characters. Having unicode in the Location header in a
response is a RFC violation and some browsers end up in a redirection loop.

Bug report: https://meta.discourse.org/t/-/125371?u=falco

This commit also checks if a site uses encoded slugs and clear all saved slugs
in the db so they can be regenerated using an onceoff job.
2019-10-11 12:38:16 -03:00
David Taylor
67787799bb DEV: Allow plugins to mark user custom fields as editable only by staff
This adds a staff_only parameter to the register_editable_user_custom_field API. The default is false, to maintain backwards compatibility.
2019-10-11 09:57:55 +01:00
Nacho Caballero
7d2f5240d9 FIX: Show a correct diff when editing consecutive paragraphs (#8177) 2019-10-11 03:50:37 -04:00
Krzysztof Kotlarek
694a5bf229 FIX: Prevent from creation of duplicated TopicAllowedUsers (#8169)
Ensure that we don't try to create duplicated TopicAllowedUsers

Related to https://meta.discourse.org/t/error-message-topic-allowed-users-is-invalid/130382/5

Spec amended to cover a case creating an overlap.
2019-10-11 17:44:29 +11:00
Roman Rizzi
5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166)
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
2019-10-09 11:41:16 -03:00
David Taylor
68fc799380 DEV: Allow specifying button class in reviewable action definitions (#8093)
This avoids the need for using `@extend` in SCSS, which can be problematic in plugins

For context, see https://review.discourse.org/t/fix-make-compatible-with-debundled-plugin-css-assets-feature/5297/7
2019-10-08 15:06:15 -03:00
Gerhard Schlager
bee000bcec FIX: Existing post timings could prevent moving posts
Post timings are created by `topic_id` and `post_number` and it's possible that the destination topic already contains post timings for non-existent posts. For example, this can happen if the destination topic was previously split and Discourse recorded post timings for moved posts in the destination topic.

This commit ensures that all timings which reference non-existent posts are deleted from the destination topic before the posts are moved.
2019-10-08 21:07:29 +02:00
Roman Rizzi
b805037825
FIX: Decrement posts read count when destroying post timings (#8172) 2019-10-08 15:39:23 -03:00
Régis Hanol
349c1cd085 FIX: remove site setting 'shadowed-by-global' option (#8061) 2019-10-08 12:43:26 -04:00
Mark VanLandingham
057f698e37 FIX: Store user_accuracy_bonus to clarify explanations (#8165)
* FIX: Store user_accuracy_bonus to clarify explanations

* Fixed specs from rounding change

* migration cleanup

* user_accuracy_bonus column not nullable
2019-10-08 09:49:07 -04:00
Dan Ungureanu
fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
David Taylor
d2bceff133
FEATURE: Use full page redirection for all external auth methods (#8092)
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.

For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
2019-10-08 12:10:43 +01:00
Jarek Radosz
20514f2e44
DEV: Update markdown-it from 8.4.1 to 10.0.0 (#8164) 2019-10-08 13:00:22 +02:00
Joffrey JAFFEUX
f6f0fcc8e3 DEV: attemps to male top-topic-spec more resilient (#8168) 2019-10-08 11:38:13 +11:00
Sam Saffron
78c5adc093 DEV: stop hardcoding ids in specs
hardcoded ids are unreliable for running specs and lead to erratic failures
2019-10-08 09:37:07 +11:00
Jarek Radosz
bfe0bc0cbd
FIX: Check for category conflicts in SiteSetting validations (#8137)
It was possible to add a category to more than one default group, e.g. "default categories muted" and "default categories watching first post".

The bug was caused by category validations inadvertently comparing strings and numbers.
2019-10-06 20:50:07 +02:00
Mark VanLandingham
868303e5d9 FEATURE: Weighted reviewable user accuracy (#8156)
* FEATURE: Inaccurate users have negative review accuracy

* FIX: disallow negative reviewable score even if the accuracy would make it negative
2019-10-04 15:10:56 -04:00
Sam Saffron
926bd712bf DEV: improve flaky spec
This handles a lot of date calculations, keep the baseline time consistent
so it is not prone to issues when running out UTC threshold
2019-10-04 11:11:03 +10:00
Sam Saffron
a49e484519 DEV: correct flaky test testing message content
This test was expecting user id ordering which is not really required.
Order of users does not matter
2019-10-04 09:56:28 +10:00
tshenry
df02930c67
Correct filename typo 2019-10-03 15:08:56 -07:00
Osama Sayegh
e27f332318
PERF: speed up about page render time and limit category mods (#8112)
* PERF: speed up about page render time and limit category mods

* Remove return

* Remove widgets

* Convert admins and mods lists

* Rename component

* Apply Joffrey's patch

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>

* Make limit 100
2019-10-03 21:48:56 +03:00
Roman Rizzi
10565e4623
SECURITY: Safely decompress files. (#8124)
* FEATURE: Adds an extra protection layer when decompressing files.

* Rename exporter/importer to zip importer. Update old locale

* Added a new composite class to decompress a file with multiple strategies

* Set max file size inside a site setting

* Ensure that file is deleted after compression

* Sanitize path and files before compressing/decompressing
2019-10-03 10:19:35 -03:00
Justin DiRose
63fabdb6f2 FIX: first post true if user creates topic first (#8139)
Reported here: https://meta.discourse.org/t/user-was-banned-for-posting-a-reply-within-3-seconds/128823/12
The problem here is the user could create a topic, then reply and get
silenced on the second time (though technically their first post) for being below the
min_first_post_typing_time threshold.
2019-10-02 15:51:40 -04:00
Sam Saffron
55ee9abecb DEV: clean up dependencies in spec
Follow up on zeitwork we needed to be a bit more explicit about a few
dependencies internally.

On certain orders the test suite could fail.
2019-10-02 14:50:54 +10:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Martin Brennan
68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Ryan Boder
b7ebe574df FEATURE: Add topic and category context to post webhook payload (#8110)
Adds additional fields topic_archetype and category_slug to the post
webhook so that handlers have some context about the post event without
having to call back to the API.

Discussed [here](https://meta.discourse.org/t/webhooks-how-best-to-differentiate-a-pm-from-a-public-post/76363/13).
2019-10-02 11:41:33 +10:00
Krzysztof Kotlarek
f331b5eab2 FEATURE: topic title is validated for blocked words (#8127)
Currently, the topic is only validated for censored words and should be validated for blocked words as well.

Blocked word validation is now used by both Post and Topic. To avoid code duplication, I extracted blocked words validation code into separate Validator, and use it in both places.

The only downside is that even if the topic contains blocked words validation message is saying "Your post contains a word that's not allowed: tomato" but I think this is descriptive enough.
2019-10-02 10:38:34 +10:00
romanrizzi
ba3dbcc117 Spec should not depend on aliases 2019-10-01 18:35:41 -03:00
romanrizzi
45513fb29a Spec should not depend on aliases 2019-10-01 18:33:53 -03:00
Gerhard Schlager
9e4fb262cf FIX: Respect unicode whitelist when suggesting username 2019-10-01 20:33:09 +02:00
Jarek Radosz
d407bcab36 FIX: Correctly escape category description text (#8107)
* FIX: Correctly escape category description text

This bug has been introduced in db14e10943.

* Remove unnecessary `html_safe`

`Theme.lookup_field` already returns html-safe strings: 7ad338e3e6/app/models/theme.rb (L237-L242)

* Rename `description` where it's acutally `descriptionText`
2019-10-01 12:04:39 -04:00
Robin Ward
cd20d0fdfd SECURITY: Don't allow base_uri as embeddable host if none exist 2019-10-01 17:58:39 +02:00
Robin Ward
d5c5ca46b6 SECURITY: Don't allow base_uri as embeddable host if none exist 2019-09-29 20:51:59 -04:00
Vinoth Kannan
02731ef33e FIX: include video tags and short urls in 'have_uploads' method.
While checking the existence of upload in posts we must include <video> tags and 'short-url' format of upload URLs.
2019-09-24 23:17:59 +05:30
Vinoth Kannan
301c5a303f FIX: include 'short_path' as src in each_upload_url method. 2019-09-22 15:32:28 +05:30