Commit Graph

6099 Commits

Author SHA1 Message Date
Guo Xiang Tan
ce4c8e957b PERF: Avoid looking up the same group multiple times during bulk invite.
Also cache the guardian check because it does a query to check if the
user is an owner of the group.
2019-04-16 09:42:25 +08:00
David Taylor
af86cf46dc FEATURE: Allow API requests to specify the DISCOURSE_VISIBLE header
This allows API consumers to automatically update the user's last_seen time. The default behaviour is unchanged.
2019-04-15 17:34:34 +01:00
Penar Musaraj
24fd710f04 Revert "FIX: strip spoilers in notification excerpts"
This reverts commit 55942224ed.
2019-04-15 10:15:17 -04:00
Arpit Jalan
989325ea7c FIX: show available details when rescuing error 2019-04-15 14:09:52 +05:30
Arpit Jalan
07b961ab13 DEV: rescue avatar rake tasks with error backtrace 2019-04-15 14:06:29 +05:30
Dan Ungureanu
e92cd5318b FEATURE: Add setting to strip whitespaces from incoming emails. (#7375)
Some email clients add leading whitespaces which get are transformed in
code blocks when processed.
2019-04-15 16:26:00 +10:00
Vinoth Kannan
87b53e170b FIX: skip <br> inside <p> if next character is \n 2019-04-14 14:44:54 +05:30
Arpit Jalan
a423e9b14a FIX: rescue avatars:refresh rake task 2019-04-14 14:26:15 +05:30
Jeff Wong
2863abefdd FIX: detect local assets for subfolder installs 2019-04-13 10:35:23 -07:00
Robin Ward
a5d9afe397 FEATURE: Include a user's pending posts in the topic view
Also includes a refactor to TopicView's serializer which was not
building our attributes using serializers properly.
2019-04-12 14:25:12 -04:00
Penar Musaraj
55942224ed FIX: strip spoilers in notification excerpts 2019-04-12 12:00:49 -04:00
David Taylor
268d4d4c82
FEATURE: Multiple SCSS file support for themes (#7351)
Theme developers can include any number of scss files within the /scss/ directory of a theme. These can then be imported from the main common/desktop/mobile scss.
2019-04-12 11:36:08 +01:00
David Taylor
24ef4f7b2b Remove support for disable_jump_reply user setting (#7359) 2019-04-12 09:03:06 +10:00
Robin Ward
6f7b6a3333 FIX: Disagree and Restore should not revert edits 2019-04-11 16:21:18 -04:00
Robin Ward
331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Arpit Jalan
7143572e0c
FIX: correctly retrieve 'login required' setting value on wizard (#7355)
* FIX: correctly retrieve 'login required' setting value on wizard

FEATURE: extract 'invite only' setting in a separate checkbox control

* Update invite_only checkbox locale on wizard.

Co-Authored-By: techAPJ <arpit@techapj.com>
2019-04-11 20:25:08 +05:30
Sam Saffron
84f12ac3f7 DEV: No need to carry two brotli implementations around
old version of brotli binary is long gone, require brotli 1.0.0 and up
2019-04-11 16:53:29 +10:00
Arpit Jalan
a20f58554b IMPORT: create category definitions in import:ensure_consistency task 2019-04-11 12:06:37 +05:30
Sam Saffron
baa7a9836c FEATURE: remove "COMPRESS_BROTLI" optional behavior
The compress brotli functionality is no longer optional, this has worked
well for years. The name of the ENV var is also confusing cause it does
not have a `DISCOURSE_` prefix which caused issues with the web upgrader

Brotli support is now unconditionally on
2019-04-11 12:36:18 +10:00
Robin Ward
87de5d9e8f Record the reason that a post was put into the queue 2019-04-10 17:42:49 -04:00
Robin Ward
abf4ca9774 FIX: Should be able to recover a user deleted post if raw didn't change 2019-04-10 13:10:40 -04:00
Penar Musaraj
eae22548de
Footer navigation for iOS PWAs and DiscourseHub app (#7347) 2019-04-10 12:23:18 -04:00
Vinoth Kannan
8d5c900142 DEV: add unique missing uploads index in post custom fields
https://review.discourse.org/t/feature-send-missing-post-uploads-stat-to-prometheus/2609/6?u=vinothkannans
2019-04-10 18:09:35 +05:30
Bianca Nenciu
c523a12187 FIX: Fix assets:precompile rake task. 2019-04-10 14:03:26 +03:00
Tarek Khalil
442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Bianca Nenciu
a723699519 FIX: Ensure TarReader is closed. 2019-04-10 13:37:04 +03:00
Bianca Nenciu
4555d0c598 FEATURE: Refresh MaxmindDb during assets:precompile. (#7340) 2019-04-10 11:37:29 +02:00
Bianca Nenciu
b5008586c5 DEV: Remove HTML parser from Tautologistics. (#7344) 2019-04-10 11:21:22 +02:00
Vinoth Kannan
d0fe42e2ef FIX: should look through posts for image markdown
Downloaded onebox images only included in the cooked HTML content. So we have to check 'post.cooked' instead of 'raw'. bfdd0fe64c
2019-04-10 13:52:35 +05:30
Vinoth Kannan
4878ee95e5 DEV: optimize sql query to narrow down the filtering of post with uploads 2019-04-10 12:58:10 +05:30
Guo Xiang Tan
bac1bcc79f PERF: Add index_reply_id_on_post_replies.
Speeds up the reference posts query in `Email::Sender#send`.
2019-04-10 13:58:29 +08:00
Vinoth Kannan
bfdd0fe64c FIX: empty the missing list on each post loop 2019-04-10 11:20:26 +05:30
Vinoth Kannan
f5ac01253a FIX: use new changed method name and remove whitespaces 2019-04-09 02:25:26 +05:30
Vinoth Kannan
914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Roman Rizzi
aec457e09a
DEV: Expose a way to extend a method that returns a list 2019-04-08 14:42:36 -03:00
Robin Ward
df1ab9259b FIX: Take action should agree with all pending flags 2019-04-08 12:39:18 -04:00
David Taylor
20cda5d689 DEV: Raise exceptions when jobs fail in test mode
Previously jobs would fail silently in test mode. Now they will raise the exception and cause the relevant test to fail. This identified a few broken tests, which I will fix in a followup commit
2019-04-08 15:59:04 +01:00
David Taylor
f524f8f811
Remove Yahoo login support from core and deprecate OpenID2.0 (#7310)
- Plugin developers using OpenID2.0 should migrate to OAuth2 or OIDC. OpenID2.0 APIs will be removed in v2.4.0

- For sites requiring Yahoo login, it can be implemented using the OpenID Connect plugin: https://meta.discourse.org/t/103632

For more information, see https://meta.discourse.org/t/113249
2019-04-08 10:38:25 +01:00
David Taylor
6a05f190c6
PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
Guo Xiang Tan
d53cc07de1 Version bump to v2.3.0.beta8. 2019-04-08 12:28:34 +08:00
Robin Ward
8181ad646b FIX: Queued Posts were not saving tags properly 2019-04-05 14:07:47 -04:00
Neil Lalonde
347e3e5a46 Version bump to v2.3.0.beta7 2019-04-05 12:24:14 -04:00
Erick Guan
9dd325f805 FIX: skip some checks for CJK locale in TextSentinel (#7322) 2019-04-05 15:07:49 +02:00
Sam Saffron
0744d2ef73 PERF: speed up suggested unread
Previously we tried using a sub-query which has some terrible performance
implications, by adding 1 extra query we can eliminate the PG issue

A join to user_stats also is prone to the same slowdown
2019-04-05 15:55:29 +11:00
Sam Saffron
5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Neil Lalonde
83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Vinoth Kannan
35431a8ddb FIX: set missing count in redis even if zero 2019-04-04 20:05:57 +05:30
Vinoth Kannan
df6ef856e6 DEV: save missing s3 uploads count in redis 2019-04-04 19:05:57 +05:30
Maja Komel
b0053f3a1c FEATURE: bump onebox version, add styling for new reddit image onebox 2019-04-04 11:24:30 +02:00
Dax74
c5b76cd138
Add Yearly review plugin
Add Yearly review plugin in the official list
2019-04-03 11:30:49 +02:00
Guo Xiang Tan
e8a4d72281 FIX: Avoid penalizing long documents too much in search.
This is a follow up to e87ca59401.
2019-04-03 14:09:57 +08:00
Sam Saffron
1c57ae6657 FIX: grant first quote at the date post was created
Previously due to #b2dc65f9534ea date on the quoted_posts table could not
be trusted.

This changes it so we use the date on the actual post as the grant date.

Note: there is an edge case where you create a post and only add a quote
a week later. In this case the badge will not be awarded at the correct
time (it will display it was granted a week ago).
That said this is far more rare than the current situation.
2019-04-03 16:41:52 +11:00
Robin Ward
123c05cb07 FIX: category id wasn't stored when enqueueing a topic 2019-04-02 11:12:17 -04:00
David Taylor
ef3dd83367 DEV: Remove trailing blank line 2019-04-02 15:49:23 +01:00
David Taylor
02ed5e78e1 DEV: Use parallel-compatible formatter for logging autospec failures
The old method would cause the parallel processes to overwrite each other. The parallel formatter allows multiple processes to write to the same file.
2019-04-02 15:34:30 +01:00
Robin Ward
6ebadaed2c FIX: Do not allow invite_only and enable_sso at the same time
This functionality was never supported but before the new review queue
it didn't have any errors. Now the combination of settings is prevented
and existing sites with sso enabled will be migrated to remove invite
only.
2019-04-02 10:26:27 -04:00
Arpit Jalan
d68d29f37a FIX: only staff can banner topics 2019-04-02 12:46:55 +05:30
Guo Xiang Tan
6a95d3fded FIX: UploadRecovery should look through posts for img src and bbcode. 2019-04-02 11:41:00 +08:00
Guo Xiang Tan
1f9799c979 FIX: Handle more cases in UploadRecovery. 2019-04-02 11:29:26 +08:00
Guo Xiang Tan
4b0ac91bfb DEV: Remove duplicated scope.
`.joins(:topic)` will automatically add the `deleted_at IS NULL` scope.
2019-04-02 10:48:17 +08:00
Guo Xiang Tan
d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
David Taylor
25feb287b8 DEV: Add parallel spec support to autospec
Set PARALLEL_SPEC=1 to use this functionality
2019-04-01 11:06:47 -04:00
Régis Hanol
2b721dff57 FIX: whitelist 'feGaussianBlur' and 'filter' svg elements 2019-04-01 12:56:47 +02:00
Guo Xiang Tan
e87ca59401 FIX: Relevance search will now consider document length in ranking.
The default ranking options ranks by the number of matches which is
highly problematic when posts are stuffed with a keyword. The ranking
will now be divided by the document length which is a much fairer way to
rank.
2019-04-01 14:37:45 +08:00
Guo Xiang Tan
cfd507822f
PERF: Improve quality of PostSearchData#raw_data. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Bianca Nenciu
034b8a7ecc FIX: Let users delete topics.
Follow-up to 31053f30de.
2019-03-29 22:00:36 +02:00
Bianca Nenciu
31053f30de FEATURE: Let users delete their own topics. (#7267) 2019-03-29 17:10:05 +01:00
Roman Rizzi
7740b1570b FIX: Avoid the deleted_at scope when recovering a topic from a recently recovered post 2019-03-29 09:40:15 -04:00
Joffrey JAFFEUX
7c97df6a89
FIX: correctly uses woman version of previously genderless emojis (#7285) 2019-03-29 10:53:29 +01:00
David Taylor
a6617a63af FIX: Expire theme cache after remap 2019-03-28 22:31:58 +00:00
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Neil Lalonde
6e2e095b53 Version bump to v2.3.0.beta6 2019-03-28 11:05:39 -04:00
Gerhard Schlager
aacbb93b21 FIX: moment.js uses a more specific language code for Armenian 2019-03-28 14:47:37 +01:00
Gerhard Schlager
d43f4206c7 FEATURE: Add Armenian language 2019-03-28 14:24:14 +01:00
Sam Saffron
9ebabc1de8 FEATURE: unconditionally update Topic updated_at when posts change in topic
Previously we would bypass touching `Topic.updated_at` for whispers and post
recovery / deletions.

This meant that certain types of caching can not be done where we rely on
this information for cache accuracy.

For example if we know we have zero unread topics as of yesterday and whisper
is made I need to bump this date so the cache remains accurate

This is only half of a larger change but provides the groundwork.

Confirmed none of our serializers leak out Topic.updated_at so this is safe
spot for this info

At the moment edits still do not change this but it is not relevant for the
unread cache.

This commit also cleans up some specs to use the new `eq_time` matcher for
millisecond fidelity comparison of times

Previously `freeze_time` would fudge this which is not that clean.
2019-03-28 17:28:01 +11:00
David Taylor
0d3531c2f1 FEATURE: Allow auth providers to define specific icons
Previously we relied on the provider name matching the name of the icon. Now icon names are explicitly set. Plugin providers which do not define an icon will get the default "sign-in-alt" icon
2019-03-27 13:25:04 +00:00
Tarek Khalil
ef2362a30f
FEATURE: Introducing new UI for changing User's notification levels (#7248)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-27 09:41:50 +00:00
Gerhard Schlager
4f04ae5692 FIX: Failed to show details about some bounced emails
Bounces sent to reply_by_email_address could not be found.
2019-03-26 18:00:27 +01:00
Guo Xiang Tan
4774633dac DEV: Remove StatsSocket.
Removed in favor of https://github.com/discourse/discourse-prometheus.
2019-03-26 18:16:58 +08:00
Gerhard Schlager
dc90133d29 FIX: Forcing permissions of seeded categories shouldn't fail
Less restrictive permissions of subcategories could make the seeding of categories fail.
2019-03-26 10:39:07 +01:00
Guo Xiang Tan
dae0bb4c67 FIX: Post blurb incorrect when search contains a phrase match.
If the blurb generated is not around the search term, we will not be
able to highlight it on the client side.
2019-03-26 17:01:52 +08:00
Guo Xiang Tan
71e431de9e DEV: Fix hardcoded value introduced in cc2bac86e9. 2019-03-26 07:45:21 +08:00
Gerhard Schlager
cd4fd447ca Make Rubocop happy 2019-03-25 17:04:45 +01:00
Guo Xiang Tan
cc2bac86e9 FIX: Dry run broken for uploads:migrate_to_s3. 2019-03-25 22:38:24 +08:00
Tim Lange
5a9dd923cc FIX: Onebox discourse user not respecting enable names (#7245) 2019-03-25 12:50:14 +05:30
Guo Xiang Tan
ac661e856a
FEATURE: Allow categories to be prioritized/deprioritized in search. (#7209) 2019-03-25 10:59:55 +08:00
Penar Musaraj
51e08feb7e DEV: Refactor icons used in lightbox HTML
Uses <svg> elements instead of hacky CSS pseudoelements

Adds a migration to mark posts with lightboxes as needing a rebake
2019-03-22 11:52:06 -04:00
David Taylor
a9d5ffbe3d FIX: Prevent critical emails bypassing disable, and improve email test logic
- The test_email job is removed, because it was always being run synchronously (not in sidekiq)
- 34b29f62 added a bypass for critical emails, to match the spec. This removes the bypass, and removes the spec.
- This adapts the specs for 72ffabf6, so that they check for emails being sent
- This reimplements c2797921, allowing test emails to be sent even when emails are disabled
2019-03-22 17:28:43 +08:00
tshenry
48d0465f72 FEATURE: Improve avatar education link (#7240)
* Make profile_path more direct to setting

* Add id to profile picture setting for easy linking
2019-03-22 02:20:48 -07:00
Guo Xiang Tan
19c3c25db1 FIX: Handle BBCode in migrate_to_s3 task as well. 2019-03-22 16:47:06 +08:00
Guo Xiang Tan
2d34be24be FIX: Rebake lightbox and use short upload urls in migrate_to_s3. 2019-03-22 13:09:59 +08:00
David Taylor
3f9e7eb326 FIX: Respect the disable_emails=non-staff site setting correctly
This reverts commit c279792130.

This commit inadvertently removed all of the non-staff email logic, rather than just for the 'test email' button. 

https://meta.discourse.org/t/112231/5
2019-03-21 21:44:14 +00:00
Joffrey JAFFEUX
c18a6069c2
DEV: upgrades bootstrap-modal (#7235) 2019-03-21 14:18:12 +01:00
Tarek Khalil
a31a35b334 FEATURE: Ignored user notification behaviour should be as a muted user (#7227) 2019-03-21 12:15:34 +01:00
Guo Xiang Tan
4e594f2b2b FIX: Destroy optimized images in upload:migrate_to_s3 rake task.
`OptimizedImage` are currently not renegerated when the image changes
store.
2019-03-21 16:50:15 +08:00
Tim Lange
f7b156ffbd UX: Better emoji escaping for topic title (#7218)
* FIX: Fixed failing discourse-prometheus-alert-receiver plugin specs
2019-03-21 09:11:33 +01:00
Guo Xiang Tan
8e5e5d7d35 DEV: Make setting up of multisite DB in test env clearer. 2019-03-21 09:58:07 +08:00
Sam Saffron
600313373b DEV: skip creating multisite db when running JS tests only 2019-03-21 12:19:12 +11:00
Tarek Khalil
1dd0fa0c4e
REFACTOR: Move redundant ignored user check into guardian (#7219)
* REFACTOR: Move redundant ignored user check into guardian
2019-03-20 19:55:46 +00:00
David Taylor
56b4ee43d3 DEV: Update sassc gem 2019-03-20 16:04:26 +00:00