Commit Graph

6469 Commits

Author SHA1 Message Date
Vinoth Kannan
8e40c35eb8 FIX: 'have_uploads' scope should include all uploads without multisite 'upload_path' prefix 2019-04-15 01:54:55 +05:30
Vinoth Kannan
87b53e170b FIX: skip <br> inside <p> if next character is \n 2019-04-14 14:44:54 +05:30
Jeff Wong
2863abefdd FIX: detect local assets for subfolder installs 2019-04-13 10:35:23 -07:00
Arpit Jalan
95ada3f190 FIX: save registration IP address for invited users 2019-04-13 13:04:25 +05:30
Robin Ward
ef1af53e05 FIX: If creating an active user via the API, create reviewables 2019-04-12 15:25:02 -04:00
Robin Ward
a5d9afe397 FEATURE: Include a user's pending posts in the topic view
Also includes a refactor to TopicView's serializer which was not
building our attributes using serializers properly.
2019-04-12 14:25:12 -04:00
Robin Ward
1760069419 FIX: Our rspec suite should not require a network connection to github 2019-04-12 14:25:12 -04:00
Penar Musaraj
55942224ed FIX: strip spoilers in notification excerpts 2019-04-12 12:00:49 -04:00
David Taylor
268d4d4c82
FEATURE: Multiple SCSS file support for themes (#7351)
Theme developers can include any number of scss files within the /scss/ directory of a theme. These can then be imported from the main common/desktop/mobile scss.
2019-04-12 11:36:08 +01:00
David Taylor
24ef4f7b2b Remove support for disable_jump_reply user setting (#7359) 2019-04-12 09:03:06 +10:00
Robin Ward
6f7b6a3333 FIX: Disagree and Restore should not revert edits 2019-04-11 16:21:18 -04:00
Robin Ward
d21dd521d2 FIX: Don't try to create a reviewable user twice 2019-04-11 12:30:54 -04:00
Robin Ward
331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Arpit Jalan
7143572e0c
FIX: correctly retrieve 'login required' setting value on wizard (#7355)
* FIX: correctly retrieve 'login required' setting value on wizard

FEATURE: extract 'invite only' setting in a separate checkbox control

* Update invite_only checkbox locale on wizard.

Co-Authored-By: techAPJ <arpit@techapj.com>
2019-04-11 20:25:08 +05:30
Arpit Jalan
318aba1f53 fix flaky spec 2019-04-11 12:25:41 +05:30
Sam Saffron
baa7a9836c FEATURE: remove "COMPRESS_BROTLI" optional behavior
The compress brotli functionality is no longer optional, this has worked
well for years. The name of the ENV var is also confusing cause it does
not have a `DISCOURSE_` prefix which caused issues with the web upgrader

Brotli support is now unconditionally on
2019-04-11 12:36:18 +10:00
Robin Ward
87de5d9e8f Record the reason that a post was put into the queue 2019-04-10 17:42:49 -04:00
Dan Ungureanu
47c50d9a95
DEV: Add test for post notices. (#7352) 2019-04-10 22:06:27 +03:00
Robin Ward
abf4ca9774 FIX: Should be able to recover a user deleted post if raw didn't change 2019-04-10 13:10:40 -04:00
Robin Ward
cab83596ec FIX: Sites with SSO that required approval were not creating reviewables 2019-04-10 12:53:50 -04:00
Régis Hanol
f665862ee8 SPEC: ensure we don't send activation reminder to staged users 2019-04-10 17:40:11 +02:00
Régis Hanol
4fe562ece5 fix build 2019-04-10 17:37:38 +02:00
Robin Ward
5d99346740 FIX: Always allow us to reject users, even if they are deleted 2019-04-10 11:00:39 -04:00
Bianca Nenciu
3d545d66df FEATURE: Send user activation reminders. (#7280) 2019-04-10 16:53:52 +02:00
Vinoth Kannan
af15ba86a0 Fix the build.
8d5c900142
2019-04-10 19:14:41 +05:30
Roman Rizzi
536b90e0ef
FEATURE: Support searching custom staff actions (#7346) 2019-04-10 09:53:17 -03:00
Tarek Khalil
442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Roman Rizzi
82e051077d Show link to the original topic when it was deleted (#7319)
* Moved i18n keys to core. We show a links that points to the original topic when it was deleted

* Use link-to since it's more idiomatic
2019-04-10 11:25:45 +02:00
Vinoth Kannan
d0fe42e2ef FIX: should look through posts for image markdown
Downloaded onebox images only included in the cooked HTML content. So we have to check 'post.cooked' instead of 'raw'. bfdd0fe64c
2019-04-10 13:52:35 +05:30
Sam Saffron
f6db30a5da FEATURE: mark last notification unread when removing timings
This calls a bit more attention to the deferred topic by amending the "read"
state of the last notification on the topic.
2019-04-09 12:42:21 +10:00
Vinoth Kannan
f5ac01253a FIX: use new changed method name and remove whitespaces 2019-04-09 02:25:26 +05:30
Vinoth Kannan
914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Roman Rizzi
aec457e09a
DEV: Expose a way to extend a method that returns a list 2019-04-08 14:42:36 -03:00
Robin Ward
df1ab9259b FIX: Take action should agree with all pending flags 2019-04-08 12:39:18 -04:00
David Taylor
3987691b5e DEV: Correct specs which had exceptions raised in jobs 2019-04-08 16:00:33 +01:00
David Taylor
20cda5d689 DEV: Raise exceptions when jobs fail in test mode
Previously jobs would fail silently in test mode. Now they will raise the exception and cause the relevant test to fail. This identified a few broken tests, which I will fix in a followup commit
2019-04-08 15:59:04 +01:00
Guo Xiang Tan
fd6b594625 Fix the build. 2019-04-08 21:30:17 +08:00
David Taylor
f524f8f811
Remove Yahoo login support from core and deprecate OpenID2.0 (#7310)
- Plugin developers using OpenID2.0 should migrate to OAuth2 or OIDC. OpenID2.0 APIs will be removed in v2.4.0

- For sites requiring Yahoo login, it can be implemented using the OpenID Connect plugin: https://meta.discourse.org/t/103632

For more information, see https://meta.discourse.org/t/113249
2019-04-08 10:38:25 +01:00
David Taylor
6a05f190c6
PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
Guo Xiang Tan
108c231d1c FIX: Clean up topic_search_data of trashed topics.
This keeps the index and table smaller.
2019-04-08 16:53:39 +08:00
Guo Xiang Tan
c4997ce85f FIX: Don't try and reindex posts that have been trashed. 2019-04-08 16:38:43 +08:00
Robin Ward
8181ad646b FIX: Queued Posts were not saving tags properly 2019-04-05 14:07:47 -04:00
Robin Ward
cbc311e4ed UX: Update the reviewable count before the message bus
In certain edge cases, the message bus won't send the message to the
user about the updated review count and it can go out of sync.

This patch synchronizes the review count every time:

1. The user visits the "Needs Review" page

2. Every time the user performs an action
2019-04-05 10:35:38 -04:00
Tarek Khalil
131eba0366 FIX: Hide post replies content for ignored users (#7320) 2019-04-05 16:13:36 +02:00
Erick Guan
9dd325f805 FIX: skip some checks for CJK locale in TextSentinel (#7322) 2019-04-05 15:07:49 +02:00
Maja Komel
ca33d091b3 FIX: don't trigger notifications when changing category/tags of unlisted topics (#7323) 2019-04-05 15:06:38 +02:00
Sam Saffron
5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Guo Xiang Tan
d299197392 DEV: Add missing test cases for Topic.for_digest and Topic.secured. 2019-04-05 09:00:48 +08:00
Robin Ward
c9ddc6ce30 FIX: Make sure the site setting works and fix build 2019-04-04 17:44:10 -04:00
Neil Lalonde
a0f7953f0c remove extra lines 2019-04-04 17:03:57 -04:00
Neil Lalonde
47eefb9f37 remove wip comments 2019-04-04 16:47:02 -04:00
Neil Lalonde
83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Robin Ward
df48c657fd FIX: Reviewable counts were not updating properly
Sometimes sidekiq is so fast that it starts jobs before transactions
have comitted. This patch moves the message bus stuff until after things
have comitted.
2019-04-04 16:07:34 -04:00
Robin Ward
2055804e95 FIX: The option to delete replies was missing from the new review queue 2019-04-04 15:51:36 -04:00
Vinoth Kannan
64ec4a6a1b fix the EnsurePostUploadsExistence job spec 2019-04-04 21:53:58 +05:30
Vinoth Kannan
df6ef856e6 DEV: save missing s3 uploads count in redis 2019-04-04 19:05:57 +05:30
Vinoth Kannan
b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Robin Ward
111a502231 FIX: Deleting Users should work nicely with Reviewable Users
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.

Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
2019-04-03 16:42:39 -04:00
Robin Ward
c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Robin Ward
1bce97a596 FIX: Allow queued posts from deleted users to be rejected 2019-04-03 11:40:46 -04:00
Robin Ward
62956003c3 FEATURE: Allow users to customize bonuses for reviewable types
A new settings section in the review queue allows admins to specify that
certain types of flags should be weighted higher than others.
2019-04-03 11:18:34 -04:00
Guo Xiang Tan
c10861da2f DEV: Fix randomly failing search spec. 2019-04-03 16:42:14 +08:00
Arpit Jalan
7b194743d7
FIX: clean up user export csv upload records in scheduled job (#7309) 2019-04-03 13:31:19 +05:30
Sam Saffron
1c57ae6657 FIX: grant first quote at the date post was created
Previously due to #b2dc65f9534ea date on the quoted_posts table could not
be trusted.

This changes it so we use the date on the actual post as the grant date.

Note: there is an edge case where you create a post and only add a quote
a week later. In this case the badge will not be awarded at the correct
time (it will display it was granted a week ago).
That said this is far more rare than the current situation.
2019-04-03 16:41:52 +11:00
Sam Saffron
b2dc65f953 FIX: on rebakes should not recreate quoted_posts records
Previously every rebake would remove and recreate records in this table
This caused created_at and updated_at to keep changing

Yes, I know the SQL is somewhat complex, but this makes quote extraction
more efficient cause we do everything in 2 round trips.

This also removes some concurrency protection we should no longer need
2019-04-03 16:17:15 +11:00
Guo Xiang Tan
d151425353
PERF: Delete search data of posts from trashed topics periodically. (#7302)
This keeps both the index and table smaller.
2019-04-03 10:10:41 +08:00
venarius
c185109380 FIX: Restricted site text better error 2019-04-02 11:16:27 -04:00
Robin Ward
123c05cb07 FIX: category id wasn't stored when enqueueing a topic 2019-04-02 11:12:17 -04:00
Robin Ward
6ebadaed2c FIX: Do not allow invite_only and enable_sso at the same time
This functionality was never supported but before the new review queue
it didn't have any errors. Now the combination of settings is prevented
and existing sites with sso enabled will be migrated to remove invite
only.
2019-04-02 10:26:27 -04:00
Robin Ward
af04318aff FIX: Allow us to serialize reviewables with deleted topics 2019-04-02 09:36:49 -04:00
Arpit Jalan
d68d29f37a FIX: only staff can banner topics 2019-04-02 12:46:55 +05:30
Guo Xiang Tan
6a95d3fded FIX: UploadRecovery should look through posts for img src and bbcode. 2019-04-02 11:41:00 +08:00
Penar Musaraj
1c0f885a5b FIX: double render error with delegated authentication
Makes sure delegated authentication is checked before other login redirects

Updates specs to cover login_required = true cases
2019-04-01 22:13:53 -04:00
Guo Xiang Tan
d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan
a2e6a8a51e DEV: Refactor test to assert for actual content.
The other assertion is already asserting for `Post#url` and I think it
makes the test clearer here.
2019-04-02 07:43:13 +08:00
Guo Xiang Tan
aa2311a7b0 FIX: Don't reindex posts belonging to a deleted topic for search.
Posts belonging to a deleted topic can't be index for search so we need
to avoid loading those post ids.
2019-04-02 07:36:53 +08:00
Guo Xiang Tan
3fc5dbb045 FIX: Don't attempt to reindex posts that have an empty raw.
If the post ids keep loading, we might end up in a situations where
we're always loading the same post ids over and over again without
indexing anything new.

Follow up to daeda80ada.
2019-04-02 07:13:33 +08:00
Robin Ward
d5a61ab167 FIX: Don't log a second pending action
If two people flagged the same post you'd get a "Pending" history event
logged which looked bad.
2019-04-01 17:07:47 -04:00
Robin Ward
76669bb5a6 FIX: Don't refer to pending review items as flags
They could be queued posts or users, and the notice should reflect that
properly.
2019-04-01 14:46:56 -04:00
Robin Ward
6470c85fda UX: Link directly to the post, not to the topic 2019-04-01 14:29:14 -04:00
Penar Musaraj
fdf4145d4b
FEATURE: Delegated authentication via user api keys (#7272) 2019-04-01 13:18:53 -04:00
Joffrey JAFFEUX
e986e96227
DEV: removes old dashboard (#7295) 2019-04-01 12:39:49 +02:00
Guo Xiang Tan
2a69ab4a4c FIX: Keep alt and title in lightbox when indexing for search.
Follow up to cfd507822f
2019-04-01 16:20:33 +08:00
Guo Xiang Tan
e87ca59401 FIX: Relevance search will now consider document length in ranking.
The default ranking options ranks by the number of matches which is
highly problematic when posts are stuffed with a keyword. The ranking
will now be divided by the document length which is a much fairer way to
rank.
2019-04-01 14:37:45 +08:00
Maja Komel
88128f1ced UX: show which groups are missing permissions for parent category (#7252) 2019-04-01 15:34:52 +11:00
Guo Xiang Tan
16215f9d3b DEV: Correct spec added in cfd507822f.
Remove stub.
2019-04-01 10:32:25 +08:00
Guo Xiang Tan
cfd507822f
PERF: Improve quality of PostSearchData#raw_data. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Guo Xiang Tan
7ac76fe935 DEV: Remove warning.
Library has already been loaded in application.rb.
2019-04-01 10:11:08 +08:00
Guo Xiang Tan
daeda80ada
FIX: Don't index posts with empty Post#raw for search. (#7263)
* DEV: Remove unnecessary join in `Jobs::ReindexSearch`.

* FIX: Don't index posts with empty `Post#raw` for search.
2019-04-01 10:06:27 +08:00
Guo Xiang Tan
730ebdfcba DEV: Refactor Jobs::EmitWebHookEvent specs. 2019-04-01 09:46:39 +08:00
Vinoth Kannan
904ba266cf SPEC: Add test case in emit_web_hook_event_spec for commit 4c6bfb9 2019-03-31 16:28:40 +05:30
Bianca Nenciu
034b8a7ecc FIX: Let users delete topics.
Follow-up to 31053f30de.
2019-03-29 22:00:36 +02:00
Robin Ward
370355d754 FIX: Allow users with posts to be rejected 2019-03-29 13:53:46 -04:00
Bianca Nenciu
31053f30de FEATURE: Let users delete their own topics. (#7267) 2019-03-29 17:10:05 +01:00
Maja Komel
4a3daacb1b FIX: reset embedding settings when no embeddable host, log host changes (#7264) 2019-03-29 17:05:51 +01:00
Roman Rizzi
7740b1570b FIX: Avoid the deleted_at scope when recovering a topic from a recently recovered post 2019-03-29 09:40:15 -04:00
Tarek Khalil
b1cb95fc23
FEATURE: Introduce ignore duration selection (#7266)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-29 10:14:53 +00:00
Guo Xiang Tan
f458cba4cb FIX: Admin search logs should filter by date instead of timestamp.
The client side filters by date so it is confusion when the data changes as each second passes.
2019-03-29 11:50:25 +08:00
Guo Xiang Tan
d8faf5f79e FIX: SearcLog.term_details generating incorrect data because of case.
Also match on equality rather than "LIKE ?" which is quite strange.
2019-03-29 11:31:01 +08:00
Guo Xiang Tan
8c2fa99f78 FIX: Remove :term from admin/search_logs/term/:term route.
Search log terms is a string that can contain characters like `/` which
messes with the route.
2019-03-29 09:48:20 +08:00