Commit Graph

212 Commits

Author SHA1 Message Date
Martin Brennan
6a2bde4d48 Fix broken secure media specs 2020-02-21 10:01:32 +10:00
Robin Ward
041168c9b6 FIX: Group mentions were not being cooked the same was as previewed
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
2020-02-18 15:45:02 -05:00
Martin Brennan
56b16bc68e
FIX: Never allow custom emoji to be marked secure (#8965)
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
2020-02-14 11:17:09 +10:00
Robin Ward
37888d9818 FIX: Never return the same reply more than once via reply_ids
If our reply tree somehow ends up with cycles or other odd
structures, we only want to consider a reply once, at the first
level in the tree that it appears.
2020-02-03 13:41:18 -05:00
Robin Ward
f83362b05b FIX: Don't return post replies from other topics
It seems in some situations replies have been moved to other topics but
the `PostReply` table has not been updated. I will try and fix this in a
follow up PR, but for now this fix ensures that every time we ask a post
for its replies that we restrict it to the same topic.
2020-02-03 13:12:27 -05:00
Martin Brennan
7c32411881
FEATURE: Secure media allowing duplicated uploads with category-level privacy and post-based access rules (#8664)
### General Changes and Duplication

* We now consider a post `with_secure_media?` if it is in a read-restricted category.
* When uploading we now set an upload's secure status straight away.
* When uploading if `SiteSetting.secure_media` is enabled, we do not check to see if the upload already exists using the `sha1` digest of the upload. The `sha1` column of the upload is filled with a `SecureRandom.hex(20)` value which is the same length as `Upload::SHA1_LENGTH`. The `original_sha1` column is filled with the _real_ sha1 digest of the file. 
* Whether an upload `should_be_secure?` is now determined by whether the `access_control_post` is `with_secure_media?` (if there is no access control post then we leave the secure status as is).
* When serializing the upload, we now cook the URL if the upload is secure. This is so it shows up correctly in the composer preview, because we set secure status on upload.

### Viewing Secure Media

* The secure-media-upload URL will take the post that the upload is attached to into account via `Guardian.can_see?` for access permissions
* If there is no `access_control_post` then we just deliver the media. This should be a rare occurrance and shouldn't cause issues as the `access_control_post` is set when `link_post_uploads` is called via `CookedPostProcessor`

### Removed

We no longer do any of these because we do not reuse uploads by sha1 if secure media is enabled.

* We no longer have a way to prevent cross-posting of a secure upload from a private context to a public context.
* We no longer have to set `secure: false` for uploads when uploading for a theme component.
2020-01-16 13:50:27 +10:00
Vinoth Kannan
3b7f5db5ba
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
Bianca Nenciu
3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Mark VanLandingham
09d9baa6d7
FIX: Update S3 stubs for more aws-sdk API changes (#8534) 2019-12-11 11:26:52 -08:00
dependabot-preview[bot]
b90a592146 DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490)
Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 06:13:17 -08:00
Dan Ungureanu
a992caf741
DEV: Replace magic values (#8398)
Follow-up to 35942f7c7c.
2019-11-25 14:32:19 +02:00
Penar Musaraj
102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Vinoth Kannan
129e308dac fix the incorrect specs
Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 12:49:11 +05:30
Vinoth Kannan
02731ef33e FIX: include video tags and short urls in 'have_uploads' method.
While checking the existence of upload in posts we must include <video> tags and 'short-url' format of upload URLs.
2019-09-24 23:17:59 +05:30
Vinoth Kannan
301c5a303f FIX: include 'short_path' as src in each_upload_url method. 2019-09-22 15:32:28 +05:30
Vinoth Kannan
a31281a888 fix the build
aa012d12dc
2019-09-02 16:19:20 +05:30
Vinoth Kannan
aa012d12dc FIX: include 'short_url' as src if upload url not exist
The URL '/images/transparent.png' will be used in the cooked content if upload record not found. In that case we have to use 'short_url' as image src in 'post.each_upload_url' method.
2019-09-02 15:11:22 +05:30
Guo Xiang Tan
8a6ee09008 FIX: Post#each_upload_url yields incorrect path to block when CDN is enabled. 2019-07-31 10:00:52 +08:00
Penar Musaraj
7b0517895e FEATURE: Add "Group owners" to posting options for groups
Context: https://meta.discourse.org/t/121589

This new setting option lets group owners message/mention large groups
without granting that privilege to all members.
2019-07-08 17:14:11 -04:00
Guo Xiang Tan
73a45048a0 FIX: Upload#short_url generates incorrect URL when extension is nil. 2019-06-19 09:10:50 +08:00
Vinoth Kannan
1881e895dc SPEC: correctly skips invalid upload urls
788f995f30
2019-06-11 20:15:40 +05:30
Vinoth Kannan
788f995f30 FIX: skip external urls which has upload url in query string.
Add spec tests for post.each_upload_url method. e8fafbc123
2019-06-11 19:55:02 +05:30
Guo Xiang Tan
f0620e7118 FEATURE: Support [description|attachment](upload://<short-sha>) in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Vinoth Kannan
0e677daaee FIX: include posts with data-orig-src attribute in have_uploads scope query. 2019-05-16 16:39:38 +05:30
Sam Saffron
88650a1259 PERF: avoid checking for consecutive replies in test
This check can issue up to 2 queries per post created, we have specific
tests for it so we can avoid.

This also rolls back #4da6ca4d
2019-05-09 13:28:53 +10:00
Daniel Waterworth
4da6ca4d9f PERF: Skip post validation by default when fabricating posts (#7508)
This speeds up tests by 10%
2019-05-09 10:15:38 +10:00
Daniel Waterworth
e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Sam Saffron
f8eddd40ad PERF: remove avg_time calculations and regular jobs from posts and topics
After careful analysis of large data-sets it became apparent that avg_time
had no impact whatsoever on "best of" topic scoring. Calculating avg_time
was a very costly operation especially on large databases.

We have some longer term plans of introducing other weighting that is read
time based into our scoring for "best of" and "top" topics, but in the
interim to stop a large amount of work that is not achieving any value we
are removing the jobs.

Column removal will follow once we decide on a new replacement metric.
2019-05-06 15:59:01 +10:00
Vinoth Kannan
e8fafbc123 List and restore missing post uploads from S3 inventory. 2019-05-04 01:16:20 +05:30
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Daniel Waterworth
ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Dan Ungureanu
57d1dea8a2
FEATURE: Let staff add custom post notices. (#7377) 2019-04-19 17:53:58 +03:00
Vinoth Kannan
8e40c35eb8 FIX: 'have_uploads' scope should include all uploads without multisite 'upload_path' prefix 2019-04-15 01:54:55 +05:30
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron
9ebabc1de8 FEATURE: unconditionally update Topic updated_at when posts change in topic
Previously we would bypass touching `Topic.updated_at` for whispers and post
recovery / deletions.

This meant that certain types of caching can not be done where we rely on
this information for cache accuracy.

For example if we know we have zero unread topics as of yesterday and whisper
is made I need to bump this date so the cache remains accurate

This is only half of a larger change but provides the groundwork.

Confirmed none of our serializers leak out Topic.updated_at so this is safe
spot for this info

At the moment edits still do not change this but it is not relevant for the
unread cache.

This commit also cleans up some specs to use the new `eq_time` matcher for
millisecond fidelity comparison of times

Previously `freeze_time` would fudge this which is not that clean.
2019-03-28 17:28:01 +11:00
Robin Ward
fa5a158683 REFACTOR: Move queue_jobs out of SiteSetting
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
Robin Ward
d1d9a4f128 Add new run_jobs_synchronously! helper for tests
Previously if you wanted to have jobs execute in test mode, you'd have
to do `SiteSetting.queue_jobs = false`, because the opposite of queue
is to execute.

I found this very confusing, so I created a test helper called
`run_jobs_synchronously!` which is much more clear about what it does.
2019-03-11 16:58:35 -04:00
Dan Ungureanu
b28b418363
FIX: Various improvements to post notices.
- Notices are visible only by poster and trust level 2+ users.
- Notices are not generated for non-human or staged users.
- Notices are deleted when post is deleted.
2019-03-11 11:19:58 +02:00
Guo Xiang Tan
8384075184 DEV: Fix incorrect test. 2019-03-11 11:00:47 +08:00
Dan Ungureanu
35942f7c7c
FEATURE: Special call-out for new / returning posters. (#7115) 2019-03-08 10:48:35 +02:00
Sam
70269c7c97 FEATURE: tighter limits on per cluster post rebakes
We have the periodical job that regularly will rebake old posts. This is
used to trickle in update to cooked markdown. The problem is that each rebake
can issue multiple background jobs (post process and pull hotlinked images)

Previously we had no per-cluster limit so cluster running 100s of sites could
flood the sidekiq queue with rebake related jobs.

New system introduces a hard limit of 300 rebakes per 15 minutes across a
cluster to ensure the sidekiq job is not dominated by this.

We also reduced `rebake_old_posts_count` to 80, which is a safer default.
2019-01-04 09:24:46 +11:00
Guo Xiang Tan
482013a1d4 FIX: Group mentions missing after post processing. 2018-11-26 12:57:07 +08:00
Guo Xiang Tan
3f636b2d19 FIX: Check whether group is mentionable by user when cooking post. 2018-11-22 16:16:33 +08:00
Sam
33541c4096 FEATURE: unconditionally omit no-follow for staff
Previously TL2 and below staff would have links
no-followed which was never intended
2018-09-17 12:02:20 +10:00
Guo Xiang Tan
434035f167 FIX: Link post to uploads in PostCreator.
* This ensures that uploads are linked to their post on creation
  instead of a background job which may be delayed if Sidekiq
  is facing difficulties.
2018-09-06 11:18:11 +08:00
Gerhard Schlager
14af90df5b UX: Stop putting usernames in edit reason when changing post owner 2018-08-20 12:28:04 +02:00
Neil Lalonde
135c803f49 FIX: don't send PM if flagged post is deleted but flags were deferred or cleared 2018-07-26 15:12:31 -04:00
OsamaSayegh
f2cc05c6c6 FIX: ignore self-quotes from the same post when saving (#6082) 2018-07-10 16:17:28 +08:00
Guo Xiang Tan
6ddd214476 FIX: Post#summary returning posts from other topics. 2018-06-21 12:00:54 +08:00
Jeff Wong
2ff226e509 FIX: consider staged users mature (no spam checks) after 1 day 2018-06-19 09:41:10 -07:00
Andrew Schleifer
2f7a8ed86c use different example domain 2018-06-19 11:15:27 -05:00
Arpit Jalan
784202170b fix the build 2018-06-19 21:40:08 +05:30
Michael Brown
ae5d255f83 FIX: Reference example.com instead of somesite.com in examples
* somesite.com actually exists...
* example.com should be used in examples and is harmless to visit
2018-06-19 10:37:24 -04:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Jeff Wong
68e4e6a575 FIX: staged users are still tl0 but do not trigger spam if 1 week old. 2018-06-18 17:20:04 -07:00
Jeff Wong
9e55767f6a FIX: don't punish a user for being previously staged for spam flags. 2018-06-15 12:25:25 -07:00
Robin Ward
fd54c92a52 FEATURE: New site setting, whitelisted_link_domains
If provided, users who normally couldn't post links (say, due to a
low trust level), can post links to those specific hosts.
2018-06-13 16:11:22 -04:00
Sam
df815d6c0e DEV: prefer using ordering in relation over default scope 2018-05-29 09:34:12 +10:00
Gerhard Schlager
ed4c0c4a63 FEATURE: Add option to delete all replies of flagged post 2018-04-24 11:08:05 -04:00
Robin Ward
31d0998506 FIX: Don't allow links with no href 2018-03-28 12:32:16 -04:00
Sam
e162cd16b6 FEATURE: editing_grace_period_max_diff to force revisions in grace period
If a user performs a substantive edit of 20 chars or more during grace period
we will store a revision to track the change

This allows for better auditing of changes that happen during the grace period
2018-03-07 18:34:34 +11:00
Robin Ward
c74d10cf34 FIX: Consider live links in <code> as links when counting 2018-03-06 11:46:12 -05:00
Robin Ward
5c40ae9e63 FIX: Links in quotes should be counted for rate limits 2018-02-20 20:42:01 -05:00
Robin Ward
3ea272f4f1 New setting: minimum trust level to embed images in a post 2018-02-20 20:00:06 -05:00
Robin Ward
5466389f4e FIX: Consider oneboxes links wrt to min_trust_level_to_post_links 2018-02-08 18:27:40 -05:00
Robin Ward
1bab15c757 FEATURE: A site setting for a minimum TL to post links 2018-02-06 18:07:58 -05:00
Régis Hanol
b91f83eb7d Ignore auto-quote/reply when counting replies 2017-12-15 00:38:14 +01:00
Régis Hanol
092c976d7c FIX: prevent 💥 when selecting replies to posts quoting themselves 2017-12-15 00:23:51 +01:00
Régis Hanol
5db3d39b05 FIX: Post.reply_ids should also handle quotes 2017-12-14 00:43:48 +01:00
Régis Hanol
1b4483c942 FEATURE: Added 'select +below' and 'select +all replies' options to selecting posts 2017-12-13 22:12:06 +01:00
Sam
229a10e142 Missed a whitelist, compensate for strict classes 2017-10-16 10:46:01 +11:00
Gerhard Schlager
f3d3129113 FIX: Use default locale for edit reason when owner of post gets changed 2017-09-14 17:17:37 +02:00
Neil Lalonde
15a74d6d3e FIX: don't enforce newuser_spam_host_threshold on private messages 2017-08-10 17:19:08 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam Saffron
d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam
3613ebee5b Revert "correct specs"
This reverts commit 2c6284ece3.
2017-07-12 18:09:57 -04:00
Sam
2c6284ece3 correct specs 2017-07-12 18:03:56 -04:00
Guo Xiang Tan
13f3de4bf6 Nuke all SiteSetting.stubs from our codebase. 2017-07-07 15:09:14 +09:00
Robin Ward
7b6242bfbb Minor cleanup of /users/ rename 2017-03-30 10:23:24 -04:00
Robin Ward
45a257815a Convert front end paths from /users/ to /u/ 2017-03-30 10:23:24 -04:00
Rimian Perkins
25516874b5 FIX: Escape regexp chars in SiteSetting.censored_words. 2017-01-31 10:14:51 +08:00
Arpit Jalan
2d0c99636a do not add rel noreferrer 2016-11-20 18:19:14 +05:30
Arpit Jalan
7cb76f7333 FIX: add rel noopener and noreferrer in addition to nofollow 2016-11-20 17:07:27 +05:30
Guo Xiang Tan
efea296c7a FIX: Do not cook post if Post#raw has not been changed. 2016-10-24 12:02:38 +08:00
Arpit Jalan
4a2f0e772c add specs for post ownership change without revision 2016-08-20 01:27:48 +05:30
Robin Ward
a546395397 REFACTOR: Migrate markdown functionality in ES6 2016-07-11 12:57:05 -04:00
Arpit Jalan
6f0137dec9 FEATURE: disable post editing when the post has active flag 2016-03-30 23:28:49 +05:30
Robert Riemann
9c39647cd2 fix: support for hyphens in group name
The group mention @ORG-team triggers notifications for the group @ORG. This fix changes the RegExp, so that the group name is correctly extracted.

see: https://meta.discourse.org/t/group-mentions-that-begin-with-the-same-characters-may-be-incorrect/39892/12?u=rriemann
2016-02-23 23:55:31 +01:00
Arpit Jalan
97e4f7f6d3 Enums that are used in tables need to be stable 2016-01-08 20:43:11 +05:30
Andy Waite
3e50313fdc Prepare for separation of RSpec helper files
Since rspec-rails 3, the default installation creates two helper files:
* `spec_helper.rb`
* `rails_helper.rb`

`spec_helper.rb` is intended as a way of running specs that do not
require Rails, whereas `rails_helper.rb` loads Rails (as Discourse's
current `spec_helper.rb` does).

For more information:

https://www.relishapp.com/rspec/rspec-rails/docs/upgrade#default-helper-files

In this commit, I've simply replaced all instances of `spec_helper` with
`rails_helper`, and renamed the original `spec_helper.rb`.

This brings the Discourse project closer to the standard usage of RSpec
in a Rails app.

At present, every spec relies on loading Rails, but there are likely
many that don't need to. In a future pull request, I hope to introduce a
separate, minimal `spec_helper.rb` which can be used in tests which
don't rely on Rails.
2015-12-01 20:39:42 +00:00
Robin Ward
d65ec1af2e Rename ninja_edit_window to editing_grace_period :'( 2015-11-24 14:28:42 -05:00
Luciano Sousa
b52f12948a models with rspec3 syntax 2015-01-05 13:04:23 -03:00
Régis Hanol
a8a0be0b34 FIX: change the unlisted/invisible topic state only when unhiding the first post 2014-11-12 16:34:30 +01:00
Régis Hanol
e7f251c105 LOTS of changes to properly handle post/topic revisions
FIX: history revision can now properly be hidden
FIX: PostRevision serializer is now entirely dynamic to properly handle
hidden revisions
FIX: default history modal to "side by side" view on mobile
FIX: properly hiden which revision has been hidden
UX: inline category/user/wiki/post_type changes with the revision
details
FEATURE: new '/posts/:post_id/revisions/latest' endpoint to retrieve
latest revision
UX: do not show the hide/show revision button on mobile (no room for
them)
UX: remove CSS transitions on the buttons in the history modal
FIX: PostRevisor now handles all the changes that might create new
revisions
FIX: PostRevision.ensure_consistency! was wrong due to off by 1
mistake...
refactored topic's callbacks for better readability
extracted 'PostRevisionGuardian'
2014-10-27 22:06:43 +01:00
Sam
1cc37e32b9 FEATURE: add max_reply_history to limit number of replies
that can be expanded, when clicking "in-reply-to"
2014-10-27 09:44:42 +11:00
Régis Hanol
de76b512c1 fix most deprecations in the specs (still some left) 2014-09-25 17:44:48 +02:00
Régis Hanol
0b13f6572f FEATURE: staff option to unhide a post 2014-09-22 18:55:13 +02:00
Sam
59d04c0695 Internal renaming of elder,leader,regular,basic to numbers
Changed internals so trust levels are referred to with

TrustLevel[1], TrustLevel[2] etc.

This gives us much better flexibility naming trust levels, these names
are meant to be controlled by various communities.
2014-09-05 15:20:52 +10:00
Jeff Atwood
fcca64c0cf rename site settings for trust levels as numbers 2014-09-04 13:16:51 -07:00
Régis Hanol
e64d3b8a42 FIX: disagree flag should unhide hidden post 2014-08-11 10:48:00 +02:00