Commit Graph

4742 Commits

Author SHA1 Message Date
Sam Saffron
70c6831125 PERF: add a filtered index for banners
This ensures we can very quickly figure out which topics are banners if
a banner is set.

Previously you would have to scan an entire table to find banners
2019-11-01 11:22:19 +11:00
Neil Lalonde
d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
Daniel Waterworth
d84c34ad75 DEV: Server-side category routing changes
The routes for categories are changing. The scheme that I intend to move
us to is:

/c/*slug_path/(:id)/ENDPOINT
/c/*slug_path/(:id)

This commit adds support for the new scheme to the server side without
dropping support for existing URLs. It is necessary to support existing
URLs for two reasons:

 * This commit does not change any client side routing code,
 * Posts that contain category hashtags that refer to a root category
   are baked into URLs that do not fit this new scheme, (/c/[id]-[slug])
2019-10-31 18:06:22 +00:00
Sam Saffron
c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Sam Saffron
fc06cfa90b DEV: remove unused method
confirmed not used in all_the_plugins not used in core
2019-10-31 08:59:32 +11:00
Daniel Waterworth
32107a9a72 FIX: Correct slug validation
We were allowing new categories to use slugs like "2342-category".
2019-10-30 17:13:28 +00:00
Daniel Waterworth
cacf99e144 FIX: Generate category url correctly when slugs are empty 2019-10-30 14:12:28 +00:00
Gerhard Schlager
8e6ad99e92 DEV: Update annotations 2019-10-30 12:59:59 +01:00
Arpit Jalan
2ae71b7a87 FEATURE: allow sending bulk invites to staged users 2019-10-30 11:40:03 +05:30
Jeff Wong
74dc37c07c FIX: upserting custom fields using keywords converts the array key to a string 2019-10-29 11:35:38 -07:00
Mark VanLandingham
437edfc415
FEATURE: Welcome moderator message - add copy! (#8246) 2019-10-28 08:58:45 -05:00
David Taylor
eec464d8d1 SECURITY: Check permissions when autocompleting mentions 2019-10-28 11:01:47 +00:00
Roman Rizzi
070a3dcf9b
FIX: When running the wizard and using a custom theme, fallback to the color_scheme name if the base_scheme_id is nil (#8236) 2019-10-25 09:29:51 -03:00
Sam Saffron
5ae35f9906 FIX: allow storage of non unique rows in oauth2_user_infos
Certain DBs have duplicates already, if we want to ensure uniqueness here
we need to decide first how to clean up existing data and confirm all the
plugins expect this.
2019-10-25 11:57:34 +11:00
Neil Lalonde
cfe26eb301 FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
Neil Lalonde
f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Dan Ungureanu
a7301c8671 PERF: Add index on group to category_groups (#8231) 2019-10-23 10:30:43 +01:00
Dan Ungureanu
09a569aaba PERF: Add unique index oauth2_user_infos(user_id, provider) (#8230) 2019-10-23 10:27:56 +01:00
Sam Saffron
676c432b0f FIX: notifications are missing under certain conditions
This simplifies the code for refreshing notification counts. It now
unconditionally looks up the user object which protects against stale objects
in memory.

This also removes a pile of conditional logic we no longer need.
2019-10-23 16:09:55 +11:00
Dan Ungureanu
2f54dd5e3d
FIX: Load user model when some attributes are missing.
Follow-up to 3ad07aacfa.
2019-10-21 21:05:25 +03:00
Robin Ward
ab53b485b4 Add setter for excerpt_size so plugins can set it more easily 2019-10-21 13:44:32 -04:00
Michael Brown
40b0b549ff DEV: update the comments re the encoded slug changes
* followup to 2e73985d
2019-10-21 13:39:45 -04:00
Rafael dos Santos Silva
2e73985d1b FIX: Make category updates slug validation idempotent
Co-authored-by: Michael Brown <supermathie@gmail.com>
2019-10-21 14:33:19 -03:00
Dan Ungureanu
3ad07aacfa
FIX: Reload only notifications when refreshing notification count (#8221)
Previously, we used to reload the whole User instance which discarded
any changes made (for example setting 'unstage' to false).
2019-10-21 20:25:46 +03:00
Daniel Waterworth
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
tshenry
95114f87ae
Improve spam_hosts copy (#8203)
- Increase size of the reviewable's conversation excerpt to prevent truncation of the new copy
- Remove the `domain` parameter from the `flag_linked_posts_as_spam` method in the user model since it is no longer needed
- Remove the `domain` interpolation variable from all translation files
- Add "All posts from this user that include links should be reviewed." to server.en.yml for added clarity on why the posts entered the queue
2019-10-18 09:31:15 -07:00
Rafael dos Santos Silva
f97b6e85da FIX: Handle encoded slugs for subcategories 2019-10-17 17:02:17 -03:00
Mark VanLandingham
bd969332e0 FIX: Display site text overrides for non '_MF' keys (#8189)
FIX: Transform pluralized keys to `.other`, to check valid interpolation
2019-10-17 18:34:07 +02:00
Sam Saffron
cb0443792b DEV: correct revision debouncer for backup draft
We were debouncing on update_at which we kept on pushing forward, this
was leading to not enough revisions.
2019-10-17 17:41:28 +11:00
Sam Saffron
3314aa9f88 DEV: annotate models
Some comments were out of data, updated
2019-10-17 16:58:22 +11:00
Sam Saffron
f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Robin Ward
f14020dd0a Allow the topic excerpt size to be customizable by plugins 2019-10-16 16:28:28 -04:00
Rafael dos Santos Silva
6e9c8fe854
FIX: More encoded slug fixes (#8191)
* FIX: Do not encode the URL twice

Now that we encode slugs in the server we don't need this anymore.

Reverts fe5na33

* FIX: More places do deal with encoded slugs

* the param is a string now, not a hash

* FIX: Handle the nil slug on /categories

* DEV: Add seeded? method to identity default categories

* DEV: Use SiteSetting to keep track of seeded categories
2019-10-16 17:08:43 -03:00
Daniel Waterworth
7a0c06691c FIX: Account for nil when looking up subcategories 2019-10-16 20:04:54 +01:00
Daniel Waterworth
7ba914f1e1 FIX: Add migrations to fix index on category slugs
Slugs can be the empty string, but the added index didn't account for
that. This commit changes the migration, stopping it from being unique
so that it can be applied everywhere and adds another migration that
recreates the index properly.
2019-10-16 14:02:11 +00:00
Daniel Waterworth
c71da3f335 FIX: Add unique index to prevent duplicate slugs for categories 2019-10-15 19:44:50 +00:00
Daniel Waterworth
5f5b232cde FIX: Category.find_by_slug
find_by_slug should ensure that the parent actually exists when its
looking for a parent.
2019-10-15 16:44:24 +01:00
Daniel Waterworth
0de7e4339c FIX: Subcategory permissions validation
When a category has a subcategory, we ensure that no one who can see the
subcategory cannot see the parent. However, we don't take into account
the fact that, when no CategoryGroups exist, the default is that
everyone has full permissions.
2019-10-14 21:13:22 +01:00
Daniel Waterworth
c49b20a1a2 FIX: Validation of category tree depth
This prevents the creation of sub-sub-categories in multiple tabs
2019-10-14 21:13:22 +01:00
Gerhard Schlager
2c011252f1 FIX: Move notification level only when user posted
Moving posts also moves the read state (`topic_users` table) to the destination topic. This changes that behavior so that only users who posted in the destination topic will have the original notification level (probably "watching") of the original topic. The notification level for all other users will be set to "regular".
2019-10-14 15:06:09 +02:00
Gerhard Schlager
f1742617fb PERF: Faster moving of read state
This should improve the performance of moving the read state of lots of posts to a new/existing topic.
2019-10-14 15:06:09 +02:00
Mark VanLandingham
f63db1c4c8
FIX: Accurate sub_total calculation for reviewable_scores (#8184) 2019-10-11 11:07:19 -05:00
David Taylor
67787799bb DEV: Allow plugins to mark user custom fields as editable only by staff
This adds a staff_only parameter to the register_editable_user_custom_field API. The default is false, to maintain backwards compatibility.
2019-10-11 09:57:55 +01:00
Krzysztof Kotlarek
694a5bf229 FIX: Prevent from creation of duplicated TopicAllowedUsers (#8169)
Ensure that we don't try to create duplicated TopicAllowedUsers

Related to https://meta.discourse.org/t/error-message-topic-allowed-users-is-invalid/130382/5

Spec amended to cover a case creating an overlap.
2019-10-11 17:44:29 +11:00
Mark VanLandingham
d710316ed1 FIX: Order UserFields by position, by default (#8176)
* FIX: site user_fields sorted by position

* FIX: Sort UserField by position for Site
2019-10-09 13:49:28 -04:00
Gerhard Schlager
10e509e47f FIX: Don't swallow the original error when moving posts
Dropping the temp table in an `ensure` block hides the actual exception. Creating the table with `ON COMMIT DROP` makes the temp table disappear automatically at the end of the transaction. We only need the explicit `DROP` in tests, because tests already run inside a transaction, so the temp table won't be dropped after each test which leads to spec failures.
2019-10-09 17:02:17 +02:00
Gerhard Schlager
bee000bcec FIX: Existing post timings could prevent moving posts
Post timings are created by `topic_id` and `post_number` and it's possible that the destination topic already contains post timings for non-existent posts. For example, this can happen if the destination topic was previously split and Discourse recorded post timings for moved posts in the destination topic.

This commit ensures that all timings which reference non-existent posts are deleted from the destination topic before the posts are moved.
2019-10-08 21:07:29 +02:00
Robin Ward
eae57652a4 FIX: Bump Compiler Version
In 98719be we fixed raw handlebars, but they won't be recompiled without
this bump.
2019-10-08 14:50:14 -04:00
Roman Rizzi
b805037825
FIX: Decrement posts read count when destroying post timings (#8172) 2019-10-08 15:39:23 -03:00
Mark VanLandingham
057f698e37 FIX: Store user_accuracy_bonus to clarify explanations (#8165)
* FIX: Store user_accuracy_bonus to clarify explanations

* Fixed specs from rounding change

* migration cleanup

* user_accuracy_bonus column not nullable
2019-10-08 09:49:07 -04:00
Dan Ungureanu
fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
Mark VanLandingham
868303e5d9 FEATURE: Weighted reviewable user accuracy (#8156)
* FEATURE: Inaccurate users have negative review accuracy

* FIX: disallow negative reviewable score even if the accuracy would make it negative
2019-10-04 15:10:56 -04:00
Jarek Radosz
5e88baebb6 DEV: Remove SiteSetting.default_categories_selected (#8138)
Added on Aug 21, 2015 in bef80633b1
The only usage removed on Aug 26, 2015 in 4ba89eec27

As far I can tell it isn't used by core or any official plugins.
2019-10-04 15:57:17 +10:00
Osama Sayegh
e27f332318
PERF: speed up about page render time and limit category mods (#8112)
* PERF: speed up about page render time and limit category mods

* Remove return

* Remove widgets

* Convert admins and mods lists

* Rename component

* Apply Joffrey's patch

Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>

* Make limit 100
2019-10-03 21:48:56 +03:00
Roman Rizzi
10565e4623
SECURITY: Safely decompress files. (#8124)
* FEATURE: Adds an extra protection layer when decompressing files.

* Rename exporter/importer to zip importer. Update old locale

* Added a new composite class to decompress a file with multiple strategies

* Set max file size inside a site setting

* Ensure that file is deleted after compression

* Sanitize path and files before compressing/decompressing
2019-10-03 10:19:35 -03:00
Mark VanLandingham
1e2aa263a9 FIX: update user vote count on topic trash/recover (#8144) 2019-10-03 07:53:48 -04:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Martin Brennan
68d35b14f4 FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099)
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).

Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
2019-10-01 19:08:41 -07:00
Joffrey JAFFEUX
3ee6e859ec FIX: prevents trigger post read count update on non existing post (#8128) 2019-10-02 10:57:34 +10:00
Krzysztof Kotlarek
f331b5eab2 FEATURE: topic title is validated for blocked words (#8127)
Currently, the topic is only validated for censored words and should be validated for blocked words as well.

Blocked word validation is now used by both Post and Topic. To avoid code duplication, I extracted blocked words validation code into separate Validator, and use it in both places.

The only downside is that even if the topic contains blocked words validation message is saying "Your post contains a word that's not allowed: tomato" but I think this is descriptive enough.
2019-10-02 10:38:34 +10:00
Gerhard Schlager
f82576deea FIX: Reset watched site settings when default locale changes
Some site settings (e.g. `unicode_username_character_whitelist`) depend on the default locale, so we need to reset the watched settings when the locale changes.
2019-10-01 20:41:16 +02:00
Jarek Radosz
d407bcab36 FIX: Correctly escape category description text (#8107)
* FIX: Correctly escape category description text

This bug has been introduced in db14e10943.

* Remove unnecessary `html_safe`

`Theme.lookup_field` already returns html-safe strings: 7ad338e3e6/app/models/theme.rb (L237-L242)

* Rename `description` where it's acutally `descriptionText`
2019-10-01 12:04:39 -04:00
Robin Ward
d5c5ca46b6 SECURITY: Don't allow base_uri as embeddable host if none exist 2019-09-29 20:51:59 -04:00
Vinoth Kannan
02731ef33e FIX: include video tags and short urls in 'have_uploads' method.
While checking the existence of upload in posts we must include <video> tags and 'short-url' format of upload URLs.
2019-09-24 23:17:59 +05:30
Vinoth Kannan
301c5a303f FIX: include 'short_path' as src in each_upload_url method. 2019-09-22 15:32:28 +05:30
Robin Ward
3c6a5836c2 FIX: Sensitivity did not work by default
Forums without previously calculated scores would return the same values
for low/medium/high sensitivity. Now those are scaled based on the
default value.

The default value has also been changed from 10.0 to 12.5 based on
observing data from live discourse forums.
2019-09-19 13:26:17 -04:00
David Taylor
479fdaaea1
DEV: Allow specifying button class in reviewable action definitions (#8093)
This avoids the need for using `@extend` in SCSS, which can be problematic in plugins

For context, see https://review.discourse.org/t/fix-make-compatible-with-debundled-plugin-css-assets-feature/5297/7
2019-09-18 11:28:59 +01:00
Robin Ward
5bf3a00328 FIX: Ignored flags should not count in your accuracy score 2019-09-17 14:54:20 -04:00
David Taylor
3da9b99dbf FIX: Live reload plugin stylesheets when the color scheme changes 2019-09-17 09:54:55 +01:00
David Taylor
081c36a459 FIX: Do not include theme variables in plugin SCSS, and fix register_css 2019-09-17 09:54:52 +01:00
David Taylor
98719bee10 FIX: Load raw hbs templates correctly from theme javascripts folder 2019-09-13 18:01:16 +01:00
Neil Lalonde
7331e0865a FIX: user directory should not include unapproved users
When the "must approve users" setting is enabled, new users who were
waiting to be approved could show up in the user directory.
2019-09-11 15:18:17 -04:00
Joffrey JAFFEUX
a25869969a
DEV: adds event hook when add/remove user to group (#8038) 2019-09-10 11:58:08 -05:00
Bianca Nenciu
0d22beb81d
FIX: Improve Onebox detection (#8019)
Follow-up to 7c83d2eeb2.
2019-09-10 13:59:48 +03:00
Roman Rizzi
7d5f3c1338 UX/PERF: Update readers count when a post from another user is read. Don't fetch the post data again just to update the count. (#8078) 2019-09-09 11:29:15 +10:00
Gerhard Schlager
52461abad9 FIX: Move read state when moving posts
* Moves / copies post timings
* Moves / copies topic users
* Fixes a small bug in the calculation of post numbers
2019-09-06 20:52:44 +02:00
Rimian Perkins
6bbd83067d FEATURE: New post editing period for >= tl2 users (#8070)
* FEATURE: Add tl2 threshold for editing new posts

* Adds a new setting and for tl2 editing posts (30 days same as old value)
* Sets the tl0/tl1 editing period as 1 day

* FIX: Spec uses wrong setting

* Fix site setting on guardian spec

* FIX: post editing period specs

* Avoid shared examples
* Use update_columns to avoid callbacks on user during tests
2019-09-06 07:44:12 -04:00
Robin Ward
bde0ef865f
FEATURE: Adds a pop up that shows a more detailed score for reviewables (#8035)
If you click a (?) icon beside the reviewable status a pop up will
appear with expanded informatio that explains how the reviewable got its
score, and how it compares to system thresholds.
2019-09-04 09:56:25 -06:00
Sam Saffron
dc9110cc43 FEATURE: track date api key was last used
Start tracking the date an api key was last used. This has already been
the case for user_api_keys.

This information can provide us with the ability to automatically expire
unused api keys after N days.
2019-09-03 18:11:30 +10:00
Vinoth Kannan
aa012d12dc FIX: include 'short_url' as src if upload url not exist
The URL '/images/transparent.png' will be used in the cooked content if upload record not found. In that case we have to use 'short_url' as image src in 'post.each_upload_url' method.
2019-09-02 15:11:22 +05:30
Roman Rizzi
e515324afa
Feature/Fix: Flagged posts user notifications (#8041)
* FIX: User should get notified when a post is deleted

* FEATURE: Notify posters when restoring flagged posts

* Fix typo

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Improve tests
2019-08-30 10:27:52 -03:00
Sam Saffron
e9cf0de588 FEATURE: do not replace &harr; with an emoji
&harr; technically can be an emoji, but usually will not be used in that way
2019-08-30 15:06:23 +10:00
Osama Sayegh
ff8cc244d8
FIX: remote theme record not saved when checking for updates (#8054) 2019-08-30 00:32:54 +03:00
romanrizzi
2b3dd1b945 FIX: Avoid publishing changes if the topic was deleted 2019-08-29 13:38:23 -03:00
Roman Rizzi
ebb389ef8a
UX: Read indicator improvements. (#8049)
* The read indicator now shows up when no member has read the last post of the topic (written by a non-member)
* The read indicator works on mobile and receives live updates from message bus
* The icon we display in the topic list was changed
* Added a title to the indicator to indicate its purpose when hovering over it
2019-08-29 12:03:43 -03:00
David Taylor
98fbc019a3
FIX: Ensure live-reloading of theme CSS works first time (#8052)
The client-side theme-selector would always apply the first in a series of file change notifications. This has been fixed, so it now applies the most recent notification.

Duplicate notifications were being sent because
- The remote_theme autosave was causing every change notification to be doubled
- Color scheme change notifications were being sent every time a theme was uploaded, even if the colors were unchanged

These duplicate notifications have been fixed, and a spec added to ensure it does not regress in future
2019-08-29 15:47:08 +01:00
Sam Saffron
4fce6484fe PERF: reduce window of consistency on user actions
Databases can have a lot of user actions, self joining and running an
aggregate on millions of rows can be very costly

This optimisation will reduce the regular window of consistency down to 13
hours, this ensures the job runs much faster
2019-08-29 13:27:04 +10:00
Régis Hanol
e3eb49729c FIX: don't blow up if the topic does not exists anymore
PERF: only select the only field we use

introduced in 7c741fa0d6 (cc @romanrizzi)
2019-08-28 18:07:56 +02:00
David Taylor
2bb08d93e4 FIX: When activating a user, ensure the change is reflected immediately
When activating a user via an external provider, this would cause the "this account is not activated" message to show on the first attempt, even though the account had been activated correctly.
2019-08-28 14:07:16 +01:00
David Taylor
ca9ef58318 FIX: Allow dashboard to load even when git version cannot be found 2019-08-28 12:37:42 +01:00
Roman Rizzi
7c741fa0d6
FEATURE: Publish read state on group messages. (Originally introduced in #7989) (#8025)
* Revert "Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)""

This reverts commit 36425eb9f0.

* Fix: Show who read only if the attribute is enabled

* PERF: Precalculate the last post  readed by a group member

* Use book-reader icon instear of far-eye

* FIX: update topic groups correctly

* DEV: Tidy up read indicator update on write
2019-08-27 09:09:00 -03:00
Roman Rizzi
3259ea60a6
DEV: Remove code deprecated by the new Reviewable API (#8023)
* Remove flag hooks and endpoints

* Remove #reject_bulk for users

* Remove code for quued_posts_controller
2019-08-26 10:33:26 -03:00
Gerhard Schlager
cc7b24b88b FIX: Regularly reset unknown extension of uploads
Discourse will try to detect the extension the next time the upload is needed. Maybe there was just a transient error the last time.
2019-08-21 10:23:20 +02:00
Sam Saffron
719a93c312 FEATURE: treat theme_uploads as settings in JavaScript
This change allows themes and components access to theme assets.

This means that inside theme js you can now get the URL for an asset with:

```
settings.theme_uploads.name
```
2019-08-21 16:51:10 +10:00
romanrizzi
36425eb9f0 Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)"
This reverts commit 5dda5c2f7c.
2019-08-20 13:29:22 -03:00
Roman Rizzi
5dda5c2f7c
FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)
* Reenable: "FEATURE: Publish read state on group messages. (#7989)"

This reverts commit 67f5cc1ce8.

* FIX: Read indicator only appears when the group setting is enabled
2019-08-20 11:57:25 -03:00
romanrizzi
67f5cc1ce8 Revert "FEATURE: Publish read state on group messages. (#7989)"
This reverts commit 1630dae2db.
2019-08-20 10:24:34 -03:00
Roman Rizzi
1630dae2db
FEATURE: Publish read state on group messages. (#7989)
* Enable or disable read state based on group attribute

* When read state needs to be published, the minimum unread count is calculated in the topic query. This way, we can know if someone reads the last post

* The option can be enabled/disabled from the UI

* The read indicator will live-updated using message bus

* Show read indicator on every post

* The read indicator now shows read count and can be expanded to see user avatars

* Read count gets updated everytime someone reads a message

* Simplify topic-list read indicator logic

* Unsubscribe from message bus on willDestroyElement, removed unnecesarry values from post-menu, and added a comment to explain where does minimum_unread_count comes from
2019-08-20 09:46:57 -03:00
Dan Ungureanu
0b1146add4
UX: Hide "Create Tag" option if user cannot create tag. (#7723) 2019-08-19 11:40:56 +03:00
Sam Saffron
9a9e31f927 FIX: properly set notification levels on group invite
Previously we relied on side effects to set tracking state correctly
when inviting groups to messages

Also has a minor optimisation in that we use pluck instead of pulling in
full record
2019-08-16 18:23:51 +10:00
Sam Saffron
a590061aae FIX: when inviting groups to message respect tracking state
Previously we would unconditionally issue an "invited_to_pm" notification
to all non muting users.

New behavior

- Watching and Watching first post get notified
- Tracking get a new "summary" message
- The rest get nothing

This is consistent with topic creation and way clearer
2019-08-16 17:22:18 +10:00
Robin Ward
b4878cde6f FEATURE: Add a webhook for user notifications
If enabled, this will fire a webhook whenever a user's notification has
been created. This could potentially be a lot of data depending on your
forum, and should be used carefully since it includes everything all users
will see in their feeds.
2019-08-15 14:47:25 -04:00
Joffrey JAFFEUX
0a6d1b687c FIX: ensures reports can't modify records (#8006) 2019-08-15 17:27:53 +10:00
Sam Saffron
e53a171916 FIX: hold s3 related distributed locks longer
These operations are pretty expensive and can take multiple minutes due to
networking.

Hold distributed mutex for much longer.
2019-08-15 11:48:44 +10:00
Vinoth Kannan
88359b0f16
FEATURE: add support for group members visibility level (#8004)
There are 5 visibility levels (similar to group visibility)

public (default)
logged-in users
members only
staff
owners

Admins & group owners always have visibility to group members.
2019-08-14 19:00:04 +05:30
Bianca Nenciu
ba396a5384
DEV: Use ResultSet with staff action logs. (#7661) 2019-08-13 20:55:05 +03:00
David Taylor
961213ba52 FIX: Correct ordering for post_edits report, and remove query limit 2019-08-13 16:53:16 +01:00
David Taylor
572e928cba FIX: Correct query for post_edits dashboard report
- Use query builder to improve readability
- Remove subquery, so that all `where` filters happen alongside the limit
- Add 'edited at' column to the report
2019-08-13 16:11:17 +01:00
Gerhard Schlager
2a95c5c5d6 FIX: Don't update watching_first_post notifications when moving first post
The first post isn't moved. It gets copied during a move. Notifications of this special type should still link to the original first post.
2019-08-12 22:59:43 +02:00
Gerhard Schlager
d686318133 FIX: Prevent failed remaps during restores
Additional changes:
* Verbose logging of remaps during restores
* Exclude the backup_metadata table from restores
2019-08-12 17:15:01 +02:00
David Taylor
d348368ab6
FEATURE: Allow themes to override color transformation variables (#7987)
Theme developers can now add any of the transformed color variables to their color scheme in about.json. For example

```
  "color_schemes": {
    "Light": {
      "primary": "333333",
      "secondary": "ffffff",
      "primary-low": "ff0000"
    }
  },
```

would override the primary-low variable when compiling SCSS for the color scheme. The primary-low variable will also be visible in administrator color palette UI.
2019-08-12 11:02:38 +01:00
Gerhard Schlager
e4f14ca3d7 FIX: Disallow user self-delete when user posted in PMs
All posts created by the user are counted unless they are deleted,
belong to a PM sent between a non-human user and the user or belong
to a PM created by the user which doesn't have any other recipients.

It also makes the guardian prevent self-deletes when SSO is enabled.
2019-08-10 12:30:16 +02:00
romanrizzi
d01c938e1a Revert "FIX: Use #dup instead of #+@ since content could be an instance of Nokogiri::XML::Element."
This reverts commit 50afe59306.
2019-08-09 11:35:22 -03:00
romanrizzi
50afe59306 FIX: Use #dup instead of #+@ since content could be an instance of Nokogiri::XML::Element. 2019-08-09 11:13:09 -03:00
Dan Ungureanu
3008ecccbd FIX: Notify on Reviewable update. (#7980)
If a post is flagged after an action was already performed on it, it
will update the previous Reviable instance and not create a new one.
The notification logic was implemented in the :create callback which was
completely skipped in this case.
2019-08-08 16:04:34 +02:00
Joffrey JAFFEUX
5d72561f4f
FIX: allows replacement of digits and symbols emojis (#7978) 2019-08-07 11:38:58 +02:00
Sam Saffron
2408d55551 FIX: embedding topics would fail with some HTML
When truncating content we try to search for first paragraph, if HTML had
no P it would fallback to first div which may have nested elements.
2019-08-07 12:45:55 +10:00
Sam Saffron
3c44e54c3f DEV: followup on secure category mention
Follows up on #f7809207

- Priority mention support only available for groups with less than 500 members
- Proper use attribute on model
2019-08-07 10:31:25 +10:00
Robin Ward
f1b3e72581 FIX: Don't hide/close topics if they don't meet minimum visibility
There are situations where depending on site settings, actions could be
taken due to flags (for example, hiding a post) but those actions were
not visibile in the review queue due to visibility settings.

This patch makes sure that the minimum score required for an action such
as hiding a post needs to meet the visibility for a moderator to see it.
2019-08-06 15:36:39 -04:00
Joffrey JAFFEUX
a475c384d8
FIX: ensures flag-ratio report shows users with disagreed > agreed (#7977)
Without causing a division by zero error
2019-08-06 15:02:45 +02:00
Bianca Nenciu
37e7998a82
FEATURE: Send notification when member was accepted to group. (#7614) 2019-08-06 13:29:46 +03:00
Sam Saffron
f780920759 FEATURE: mention in secure category to prioritize groups
This feature allows @ mentions to prioritize showing members of a group who
have explicit permission to a category.

This makes it far easier to @ mention group member when composing topics in
categories where only the group has access.

For example:

If Sam, Jane an Joan have access to bugs category.

Then `@` will auto complete to (jane,joan,sam) ordered on last seen at

This feature works on new topics and existing topics. There is an explicit
exclusion of trust level 0,1,2 groups cause they get too big.
2019-08-06 17:57:56 +10:00
David Taylor
39e0442de9 FIX: Various watched words improvements
- Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback)
- Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins
- Censor tests are moved to ruby, to ensure everything works end-to-end
- If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-08-02 15:29:12 +01:00
Gerhard Schlager
50db6a1d62 FIX: Correctly update replies when first post gets moved 2019-08-01 22:07:21 +02:00
Gerhard Schlager
4113b57cfe REFACTOR: Use less queries when moving posts 2019-08-01 22:04:45 +02:00
Robin Ward
6f367dde26
UX: Rename "Keep Post" to "Keep Post Hidden" when hidden (#7767)
* UX: Rename "Keep Post" to "Keep Post Hidden" when hidden

This is based on this feedback:
https://meta.discourse.org/t/category-group-review-moderation/116478/19

When a post is hidden this makes the operation much more clear.

* REFACTOR: Better support for aliases for actions

Allow calls on alias actions and delegate to the original one.
This is less code but also simplifies tests where the action might
be "agree_and_keep" or "agree_and_keep_hidden" which are the same.
2019-08-01 11:23:23 -04:00
Guo Xiang Tan
7bd93eba3e FIX: Gravatar uploads being dependent on authorized_extensions. 2019-08-01 16:24:09 +08:00
Osama Sayegh
13e74151a9
FEATURE: list category moderators on the about page (#7916)
https://meta.discourse.org/t/category-group-review-moderation/116478?u=osama
2019-07-31 16:46:58 +03:00
Guo Xiang Tan
8a6ee09008 FIX: Post#each_upload_url yields incorrect path to block when CDN is enabled. 2019-07-31 10:00:52 +08:00
Neil Lalonde
9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Guo Xiang Tan
ef46231214 Fix the build. 2019-07-29 20:02:18 +08:00
Guo Xiang Tan
faea594436 DEV: Extract common regexps for multisite. 2019-07-29 19:01:36 +08:00
Guo Xiang Tan
8a64b0c8e8 Revert "DEV: Remove unused kwarg and properly check for local missing uploads."
This reverts commit 97769f3d02.

The code is confusing but this change is quite risky. Defer for now
until we can look at it properly.
2019-07-29 14:35:34 +08:00
Guo Xiang Tan
97769f3d02 DEV: Remove unused kwarg and properly check for local missing uploads. 2019-07-29 14:21:06 +08:00
Julien Ma
dcb0e5f1e5 Fix "Host is invalid" error when TLD >10 chars (#7948)
Related to https://meta.discourse.org/t/host-is-invalid-error-when-tld-is-longer-than-7-characters/46081.

Using Discourse `v2.4.0.beta2 +119`, I can't add an host (when embedding, cf. `/admin/customize/embedding`) ending with `.engineering`.
Turns out current regex limits to 10 characters.

Fix is dumb: it only allows for up to 24 chars, which is the **current** max TLD length, see https://stackoverflow.com/a/22038535/1907212.

---

Maybe a better (and longer-term) fix would be to allow for up to 64 chars, which I understand comes from the RFC.
I'm not at ease with regexes, so can't be sure about it, but [this suggestion](https://meta.discourse.org/t/host-is-invalid-error-when-tld-is-longer-than-7-characters/46081/8?u=julienma) seems pretty good:

> rules of DNS labels are:
>
> - All labels are 1 to 63 characters, case insensitive A to Z, 0 to 9 and - (hyphen), all from ASCII.
> - No labels may start with a hyphen.
> - No top level domain label may start with a number.
>
>That means a regexp for a valid domain name would look like:
>
>`/^([a-z0-9][a-z0-9-]{0,62}\.)+[a-z][a-z0-9-]{0,62}\.?$/`
>
>Domains that are just a TLD are sufficiently bizarre as to be worth ignoring.
2019-07-26 16:29:48 -04:00
Arpit Jalan
0603636cea FIX: include default label when exporting reports 2019-07-26 12:57:13 +05:30
Vinoth Kannan
2ba4de2d45 REVERT: DEV: should ignore missing post uploads when a user export destroyed
Reverts 793915fe6a. We no longer need this since we're destroying each posts in commit 028121b95b.
2019-07-25 19:41:25 +05:30
Kyle Zhao
0e1d6151b9 FIX: Frozen string error in TopicEmbed.import (#7938)
When `SiteSetting.embed_truncate` is enabled (by default), the truncated
string is mutatable and does not raise an error.

However, when the setting is disabled, the `contents` string is frozen
and immutable, and will raise a `FrozenError`.
2019-07-25 09:21:01 -04:00
Gerhard Schlager
845fd42153 FIX: Update reply count when moving posts 2019-07-22 21:42:24 +02:00
Gerhard Schlager
271ddac467 FIX: Delete notifications users can't see after moving posts
No need to let notifications stay around when users can't access
a topic after it was converted into a PM or posts were moved
into a restricted topic.

Also makes sure that moving to a new topic correctly uses the
guardian for the first post by enqueuing jobs outside of a
transaction.
2019-07-22 19:02:21 +02:00
Gerhard Schlager
1235105c03 FIX: Old notifications didn't link to correct post after moving post 2019-07-22 17:38:45 +02:00
Robin Ward
8dd3cbfcb9
FEATURE: Allow choice of category when making a PM public (#7907)
* FEATURE: Allow choice of category when making a PM public

Previously it would default to uncategorized, which was not ideal on
some forums. This gives the staff member more choice about what they'd
like to do.

* Make the optional category more explicit

* Joffrey's feedback
2019-07-19 11:52:50 -04:00
Dan Ungureanu
30c491500a
FEATURE: Permit users who had no penalties in last 6 months to be TL3. (#7892)
Previously, users who had any penalties (were silenced or suspended)
were not allowed to promote to Trust Level 3.

There is also a more subtle change here: if users were silenced or
suspended and then the operation was reverted (user was un-silenced
or un-suspended), then it would have been like the user was never
penalized in the first place. This is no longer the case. To forgive a
user earlier, administrators can use "Clear Penalty History" feature.

Lastly, Jobs::UnsilenceUsers will automatically unsilence any users who
should no longer be silenced (silenced_till < now()). This made it so
silence_count - unsilence_count == 0 for any user who is not silenced,
which defeated the purpose of this TL3 requirement.
2019-07-19 12:46:10 +03:00
Arpit Jalan
eb9155f3fe
FEATURE: send max 200 emails every minute for bulk invites (#7875)
DEV: deprecate `invite.via_email` in favor of `invite.emailed_status`

This commit adds a new column `emailed_status` in `invites` table for
 tracking email sending status.
 0 - not required
 1 - pending
 2 - bulk pending
 3 - sending
 4 - sent

For normal email invites, invite record is created with emailed_status
 set to 'pending'.

When bulk invites are sent invite record is created with emailed_status
 set to 'bulk pending'.

For invites that generates link, invite record is created with
 emailed_status set to 'not required'.

When invite email is in queue emailed_status is updated to 'sending'

Once the email is sent via `InviteEmail` job the invite emailed_status
 is updated to 'sent'.
2019-07-19 11:29:12 +05:30
Vinoth Kannan
ad04ce9f43 FIX: remove post upload record creation inside 'find_missing_uploads' method. 2019-07-19 01:44:08 +05:30
Gerhard Schlager
7e69c5cc36 Revert "FEATURE: Use configured quotation marks in fancy topic title"
This reverts most of commit ce8e099639.

The rake task to update fancy topic titles is still there, because that's useful even without this feature.
2019-07-18 11:55:49 +02:00
David Taylor
ed5b31f427 FIX: Recompile extra_js theme assets when COMPILER_VERSION changes (#7897) 2019-07-16 16:34:33 +02:00
Joffrey JAFFEUX
b3eb67976d
DEV: Upgrades to Ember 3.10 (#7871)
Co-Authored-By: majakomel <maja.komel@gmail.com>
2019-07-16 12:45:15 +02:00
Bianca Nenciu
8e133de831
FIX: Ensure suppressed categories do not produce any featured topics. (#7863) 2019-07-15 17:32:03 +03:00
Gerhard Schlager
ce8e099639 FEATURE: Use configured quotation marks in fancy topic title 2019-07-12 21:10:10 +02:00
Gerhard Schlager
8f89254554 FIX: Recalculate settings when dependent settings change 2019-07-12 21:10:10 +02:00
Gerhard Schlager
7311eeed39 FIX: Use default locale for flag reasons 2019-07-12 12:04:23 +02:00
Robin Ward
1d38040579 SECURITY: SQL injection with default categories
This is a low severity security fix because it requires a logged in
admin user to update a site setting via the API directly to an invalid
value.

The fix adds validation for the affected site settings, as well as a
secondary fix to prevent injection in the event of bad data somehow
already exists.
2019-07-11 13:41:51 -04:00
Gerhard Schlager
a65a9a85d5 FEATURE: Remap uploads during restore when S3 or CDN changes
In order for this to work the Backuper stores a couple of site settings
in the new backup_metadata table, because the old setting values might
not be available on restore anymore.
2019-07-09 14:04:16 +02:00
Penar Musaraj
7b0517895e FEATURE: Add "Group owners" to posting options for groups
Context: https://meta.discourse.org/t/121589

This new setting option lets group owners message/mention large groups
without granting that privilege to all members.
2019-07-08 17:14:11 -04:00
Penar Musaraj
b690fc3d98
FEATURE: Add new group visibility option for "logged on users" (#7814)
Groups can now be marked as visible to "logged on users". All automatic groups (except `everyone`) are now visible to "logged on users", previously they were marked as public but suppressed in the group page for non-staff.
2019-07-08 15:09:50 -04:00
Robin Ward
3132a9007b FIX: Use correct timezone for manual SQL 2019-07-06 15:14:07 -04:00
Robin Ward
a075fd46fd FIX: Don't use exceptions to catch conflicts
If a database exception is raised ActiveRecord will always rollback
even if caught.

Instead we build the query in manual SQL and DO NOTHING when there's a
conflict. If we detect nothing was done, perform an update.
2019-07-06 14:43:56 -04:00
Ralph Rooding
1318e0b288 FEATURE: Rake themes installer (#7848)
* Delete remote_theme when deleting the theme

* Install themes and theme components through rake

* Removed unnecessary test
2019-07-04 14:33:05 -04:00
Robin Ward
72bac61c90 FIX: Upsert a custom field if a unique constraint fails 2019-07-04 13:26:25 -04:00
Joffrey JAFFEUX
71bf9ec1b2
FEATURE: opt-in guidance on topics for users without access (#7852)
Co-Authored-By: majakomel <maja.komel@gmail.com>
Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2019-07-04 10:12:39 +02:00
OsamaSayegh
426c8045de FIX: logs for enabling/disabling components should show up in the staff actions logs 2019-07-03 12:33:25 +00:00
Osama Sayegh
3d64532273 FEATURE: allow disabling theme components (#7812)
This allows you to temporarily disable components without having to remove them from a theme. 

This feature is very handy when doing quick fix engineering.
2019-07-03 18:18:11 +10:00
Joffrey JAFFEUX
9ee2c121c1
FIX: ensures emoji helper is working with custom emojis (#7843) 2019-07-03 09:23:40 +02:00
Gerhard Schlager
d513c28e3b FIX: Don't send notification email when user isn't allowed to see topic 2019-07-01 14:03:03 +02:00
Bianca Nenciu
b2eb0f4ad6 FEATURE: Export any type of report supporting table mode. (#7662) 2019-06-28 08:50:31 +02:00
Sam Saffron
8f7a387aa7 FEATURE: add support for tag group search
The behaviour of #TERM in search has been amended

1. We try category or subcategory slugs
2. We try tags
3. We try tag-groups

The term `hello #my-group` will search for all posts tagged with any of
the tags in the tag group `My Group`

Future work may be introducing a slug cache here or caching it in the table
but the assumption is that the number of tag groups will not be huge
2019-06-27 17:53:26 +10:00
Jeff Wong
88ef5e55fe
FEATURE: add ability to have multiple totp factors (#7626)
Adds a second factor landing page that centralizes a user's second factor configuration.

This contains both TOTP and Backup, and also allows multiple TOTP tokens to be registered and organized by a name. Access to this page is authenticated via password, and cached for 30 minutes via a secure session.
2019-06-26 16:58:06 -07:00
Robin Ward
817661aed9 UX: Require a confirmation if approving a post in a closed topic 2019-06-26 12:21:49 -04:00
Robin Ward
14a64e7c4c Move "Hide Post" option up to the top on the review interface
If you agree with the post you are likely to want to hide it.
2019-06-26 11:35:59 -04:00
Robin Ward
eedec7d79b FIX: Only include pending/agreed scores in the total score
This should prevent posts from being hidden if a previous flag was
rejected and a new one was added.
2019-06-26 11:21:35 -04:00
Joffrey JAFFEUX
dfde59f631
FIX: provides an emoji helper to replace codes by images (#7802) 2019-06-26 15:11:04 +02:00
Sam Saffron
333b5a19b2 FIX: do not include uncategorized_category_id in topic_create_allowed if posting in uncategorized is disabled
Previously users were still allowed to create topic via API even if
uncategorized was disabled.

Not 100% happy with all this special casing, but I guess we have to do
something.

This also splits up a mega spec now that we have fab! into a more easy to
understand structure (I hope)
2019-06-26 17:41:07 +10:00
Sam Saffron
f3e4e6941c DEV: Correct batch setting of categories
followup to #bc03c509

There were 2 problems

1. VALUES was not properly getting multiple results ... we need (1),(2),(3)
not (1,2,3)

2. changes was mistakenly lazy evaluated eg `changed ||=` which meant some
queries were not running
2019-06-25 13:05:43 +10:00
Daniel Waterworth
bc03c509ab FIX: CategoryUser#batch_set (#7787)
* Remove unused method

* Prefabricate user in category_user_spec.rb

* FIX: Remove notification_level from category_users unique indexes

* FIX: CategoryUser#batch_set wasn't updating pre-existing records

* Improve tests for CategoryUser#batch_set

* FIX: changed was being reported incorrectly

* DEV: Rewrote query to do a bulk insert

* DEV: remove unnecessary parentheses
2019-06-25 12:13:27 +10:00
Vinoth Kannan
b1ca64487a FIX: multisite upload urls must have either db name or the word 'short-url'. 2019-06-25 01:19:58 +05:30
Penar Musaraj
e51de4cc25
FEATURE: Add endpoint to individually update a theme setting (#7789)
- also adds a new staff action type called "change theme setting" for easier logging of staff changes to theme settings
2019-06-21 13:49:14 -04:00
Guo Xiang Tan
73a45048a0 FIX: Upload#short_url generates incorrect URL when extension is nil. 2019-06-19 09:10:50 +08:00
Sam Saffron
fe4f0a4369 FIX: staged users should not be included in TL groups
staged users should not be included in any automatic groups cause for all
purposes they do not exist.
2019-06-17 15:10:47 +10:00
Sam
fa2a5f6f56
FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
Maja Komel
0a1b16bb16 FIX: clean up cooked group bio when bio is removed (#7745) 2019-06-12 16:18:38 +10:00
Vinoth Kannan
788f995f30 FIX: skip external urls which has upload url in query string.
Add spec tests for post.each_upload_url method. e8fafbc123
2019-06-11 19:55:02 +05:30
Daniel Waterworth
d073a7d5a8 DEV: Added commit approved notification type for discourse-code-review
We need to reserve ids in core so plugins do not clash.
2019-06-11 11:17:23 +10:00
Sam Saffron
7b17eb06da FEATURE: ban any SSO attempts with invalid external id
We now treat any external_id of blank string (" " or "     " or "", etc) or a
invalid word (none, nil, blank, null) - case insensitive - as invalid.

In this case the client will see "please contact admin" the logs will explain
the reason clearly.
2019-06-11 10:04:26 +10:00
Robin Ward
ecebff5060 Only show deprecation warning if the webhook is active 2019-06-10 16:23:12 -04:00
Sam Saffron
ff3a1eae3a FIX: ensure consistency should handle cases where a topic trashed
Followup to c05b6170
2019-06-07 14:57:56 +10:00
Sam Saffron
c05b617067 FIX: ensure_consistency was able to create corrupt category topics
- Correct create_category_definition to skip validations and use a
transaction, no longer able to create corrupt topics

- ensure_consistency now clears topic_id if pointing at deleted or missing
topic_id

- Stop creating category definition topics for uncategorized
2019-06-07 11:20:13 +10:00
Bianca Nenciu
5377d1672f DEV: Fix linting issue. 2019-06-06 13:20:16 +03:00
Bianca Nenciu
f63b8bb79d FIX: Periodically ensure consistency of categories. (#7663) 2019-06-06 11:30:52 +02:00
Penar Musaraj
f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677)
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Bianca Nenciu
e0c821ebb0 FEATURE: Make staff action logs page support infinite loading 2019-06-06 13:02:53 +10:00
Roman Rizzi
c3a38d2304 DEV: Make groups/new extensible by plugins (#7642)
* Expose a new plugin outlet. Pass group model to the group-member-dropdown so it can be accessed by plugins

* Added controller tests for group custom fields. update custom fields when updating a group
2019-06-06 12:05:33 +10:00
Robin Ward
d902c4eb9f FEATURE: Can sort reviewable queue
Choices are Priority / Created At (and desc versions.)
2019-06-05 13:21:05 -04:00
Sam Saffron
19e3b3b1bc PERF: speed up topic poster lookups
During profiling looking up topic users popped up as a hot path, this
change more than halved the amount of work it does

It reduces object allocations and method calls and avoids repeate translation
of common terms
2019-06-05 18:28:36 +10:00
Arpit Jalan
e7fe7010b8
FIX: use hijack for processing bulk invites (#7679)
FIX: do not store bulk invite CSV file on server
2019-06-04 20:19:46 +05:30
Maja Komel
7da875f52a FIX: trigger user_updated webhook when avatar changes 2019-06-04 16:46:46 +08:00
Guo Xiang Tan
bb98785569 DEV: Fix broken Upload#base62_sha1. 2019-06-04 14:10:46 +08:00
Régis Hanol
b792db9d92 FIX: redirect to top was always redirecting to 'All' 2019-06-03 20:18:14 +02:00
David Taylor
81dcadf788
DEV: Stop compiling themes during DB migration. Recompile on cdn change. (#7676)
This is an improved implementation for bc8b7b13
2019-06-03 16:38:02 +01:00
David Taylor
7500eed4c0
FEATURE: Multi-file javascript support for themes (#7526)
You can now add javascript files under `/javascripts/*` in a theme, and they will be loaded as if they were included in core, or a plugin. If you give something the same name as a core/plugin file, it will be overridden. Support file extensions are `.js.es6`, `.hbs` and `.raw.hbs`.
2019-06-03 10:41:00 +01:00
Dan Ungureanu
c1e7a1b292 UX: Merge settings related to muted tags. (#7656) 2019-06-03 12:23:23 +10:00
Robin Ward
a2598632aa FIX: Histories should be reverse chronological 2019-05-31 14:59:26 -04:00
David Taylor
75fc126224
DEV: Change path for extra scss files in themes (#7664)
The preferred path is `/stylesheets/...`, to match core and plugins. The old `/scss/...` path will continue to function
2019-05-31 15:40:41 +01:00
David Taylor
03363d03e8 FEATURE: Load theme setting descriptions from theme locale files
Previously theme setting descriptions were defined in the `settings.yml` file like this:
```
setting_name:
  default: "My Default Value"
  description:
    en: "English description"
    fr: "French description"
```

This commit allows developers to store the localised descriptions in the theme locale files instead:
```
en:
  theme_metadata:
    description: Theme Description
    settings:
      setting_name: "The localised description for setting_name"
```
2019-05-31 14:49:59 +01:00
Sam Saffron
3b8819f0ab FEATURE: add support for defer topic functionality
This feature allows end users to "defer" topics by marking them unread

The functionality is default disabled. This also introduces the new site
setting default_other_enable_defer: to enable this by default on new user
accounts.
2019-05-31 15:44:35 +10:00
Robin Ward
2e0a40007b FIX: Category topics should not be deletable via review queue 2019-05-30 16:43:23 -04:00
Arpit Jalan
f54a865846 DEV: refactor cleaning up of user export topics 2019-05-30 10:24:45 +05:30
romanrizzi
2fa8df7cd2 New: can_see_groups? method for better perf 2019-05-30 08:45:20 +08:00
romanrizzi
e7ee556e87 Support multi-group user search 2019-05-30 08:45:20 +08:00
Bianca Nenciu
227c45107d FEATURE: Implement Onebox for posts including polls. (#7539) 2019-05-29 17:05:52 +02:00
Sam Saffron
04fdada96a annotate models 2019-05-29 14:26:24 +10:00
Sam Saffron
5c524ea8a4 FEATURE: introduce dedicated storage and DB constraints for anon users
Previously we used custom fields to denote a user was anonymous, this was
risky in that custom fields are prone to race conditions and are not
properly dedicated, missing constraints and so on.

The new table `anonymous_users` is properly protected. There is only one
possible shadow account per user, which is enforced using a constraint.

Every anonymous user will have a unique row in the new table.
2019-05-29 14:26:24 +10:00
Guo Xiang Tan
f0620e7118 FEATURE: Support [description|attachment](upload://<short-sha>) in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Guo Xiang Tan
63292cecd9 PERF: Prefer joins over IN () queries. 2019-05-29 09:00:15 +08:00
Penar Musaraj
7c9fb95c15 Temporarily revert "FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan
b1d3c678ca FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603) 2019-05-28 11:18:21 -04:00
Arpit Jalan
028121b95b
FIX: delete system generated message when user_export record is deleted (#7595)
FIX: system generated message for user export should be closed by default
2019-05-28 16:38:41 +05:30
Rafael dos Santos Silva
315a38e0e3 FEATURE: Allow running message_bus in a different redis instance (#7616)
Adds `DISCOURSE_MESSAGE_BUS_REDIS_ENABLED` env var, that when set
to true, will allow Discourse to connect to a different redis
instance for MessageBus needs.

When enabled you can configure the same env vars user for redis,
but prefixed by `MESSAGE_BUS`, eg:

`DISCOURSE_MESSAGE_BUS_REDIS_HOST`
2019-05-28 15:52:43 +10:00
Robin Ward
b85fa62882 FIX: Don't send replies to flags that are ignored by expiry
The `AutoQueueHandler` will ignore really old flags. In that case, don't
notify the user that the moderator is looking into it. They probably
never saw it because it didn't meet the reviewable minimum priority.
2019-05-27 17:24:36 -04:00
Robin Ward
03b4131933 FIX: When deleting a user via queued posts, block urls, ip and email 2019-05-27 15:54:01 -04:00
Penar Musaraj
dfcc2e7ad8 Revert "FEATURE: Send notification when member was accepted to group. (#7503)"
This reverts commit 42c82d544e.
2019-05-27 15:19:59 -04:00
Robin Ward
d95a68b837 FEATURE: When suspending a user, allow the Delete + Replies action
Previously you could only delete the post
2019-05-27 12:27:16 -04:00
Bianca Nenciu
42c82d544e
FEATURE: Send notification when member was accepted to group. (#7503) 2019-05-27 17:28:41 +03:00
Régis Hanol
d6aacadc4e make 🤖 happy 2019-05-25 00:32:29 +02:00
Robin Ward
89b84651c3 Migrate score settings to use sensitivities
We hide scores so these settings no longer made sense.
2019-05-24 15:44:24 -04:00
Robin Ward
fad5d9c49b Use an Enum class for the Reviewable Priority Site Setting 2019-05-24 15:44:24 -04:00
Osama Sayegh
e20c30987c
FEATURE: detect theme errors and catch them (#7589)
* FEATURE: detect theme errors and catch them

* Bump COMPILER_VERSION

* Feedback

* Override eslint no console for one line

* Can't use our ajax method

* remove emoji from translation file
2019-05-24 17:25:55 +03:00
Guo Xiang Tan
fd0eff6ce3 Revert "FIX: Add missing unique index on user_id for user_profiles."
Oops `user_id` is the primary key.

This reverts commit 5478aa63ab.
2019-05-24 12:58:36 +08:00
Guo Xiang Tan
5aed463649 Update annotation. 2019-05-24 12:58:26 +08:00
Guo Xiang Tan
5478aa63ab FIX: Add missing unique index on user_id for user_profiles. 2019-05-24 12:11:49 +08:00
Guo Xiang Tan
4f5c9bb8d3 DEV: Prefer create! over create.
We want to know when stuff fails.
2019-05-24 12:07:22 +08:00
Robin Ward
e74cd54fc6 REFACTOR: Replace score bonuses with low/med/high priorities
We removed score from the UX so it makes more sense to have sites set
priorities instead of score bonuses.
2019-05-23 11:54:45 -04:00
Vinoth Kannan
793915fe6a DEV: should ignore missing post uploads when a user export destroyed 2019-05-21 13:32:36 +05:30
Gerhard Schlager
b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
David Taylor
5666316285 FEATURE: Display 'last updated' on dashboard, improve release notes link (#7560) 2019-05-17 15:42:45 +10:00
Vinoth Kannan
0e677daaee FIX: include posts with data-orig-src attribute in have_uploads scope query. 2019-05-16 16:39:38 +05:30
Vinoth Kannan
56ada8374f DEV: wrap find_missing_uploads method in distributed mutex
And skip posts with deleted topics.\ne8fafbc123170dd1f7d2a8adea4e7810585d3e76
2019-05-16 15:17:53 +05:30
Vinoth Kannan
636b75fa16 REFACTOR: remove duplicate reject loop and implicit return
e8fafbc123
2019-05-16 10:04:04 +05:30
Bianca Nenciu
1d3375b176 FEATURE: Preserve notifications levels when splitting topics. (#7494) 2019-05-15 17:29:29 +10:00
Maja Komel
e0fe01925e FIX: make frozen string mutable in incoming_domain 2019-05-14 17:44:53 +02:00
Sam Saffron
624184560e PERF: improve performance of publish_notifications_state
User.publish_notifications_state is called every time a notification is
created, this can become a very critical code path.

On some heavy notification related sites this can be a major CPU user on PG

This index makes it much cheaper to publish notification state, cause a
simple index lookup does the trick.
2019-05-14 16:02:55 +10:00
Neil Lalonde
6f747c6b71 FIX: don't allow username to be changed to same as password
We were blocking user registrations with same username and password,
but allowing usernames to be changed to be same as password later.
Also disallow names to be the same as password.
2019-05-13 16:43:40 -04:00
David Taylor
750c125707 DEV: Include foreign keys in model annotations 2019-05-13 15:53:42 +01:00
David Taylor
1bff81419e DEV: Update annotations 2019-05-13 15:24:24 +01:00
Gerhard Schlager
53d2232731 FIX: Remove trailing slash from topic URL
this lead to duplicate slashes in concatenated URLs
2019-05-13 14:51:45 +02:00
Arpit Jalan
25b39b86ae FIX: reload the user record instead of fetching via email 2019-05-13 15:16:53 +05:30
Guo Xiang Tan
64c117519e Fix modifying frozen strings errors take 3. 2019-05-13 16:45:23 +08:00
Sam Saffron
d643294c9d FIX: delete duplicate invites earlier in the process
There was a race condition when 2 invites existed for 1 user where in some
cases data from both invites would be used for the redeem. Depending on DB
ordering.

Fix is to delete duplicate invites earlier in the process prior to
`redeem_from_email` being called.
2019-05-13 17:42:39 +10:00
Guo Xiang Tan
d369b84ced Fix modifying frozen strings error. 2019-05-13 14:31:20 +08:00
Guo Xiang Tan
8165ceb320 Make rubocop happy. 2019-05-13 09:55:44 +08:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Robin Ward
643c75c419 FIX: Don't increment version if the reviewable operation fails
In fact, roll back the entire operation in the database.
2019-05-10 09:53:25 -04:00
Sam Saffron
0ab6c6e24e PERF: speed up spec suite, avoid featuring topics
Before: 6:05
After: 5:42

Featuring topics for `list/categories` is a very expensive operation that
happened each time we created a topic. This introduces a test only bypass
2019-05-10 11:37:53 +10:00
Robin Ward
b380ed5282 FEATURE: Claim Reviewables by Topic
This is a feature that used to be present in discourse-assign but is
much easier to implement in core. It also allows a topic to be assigned
without it claiming for review and vice versa and allows it to work with
category group reviewers.
2019-05-09 13:40:36 -04:00
Bianca Nenciu
8dfb15a2e5
FIX: Do not show self edits in Post Edits report. (#7510) 2019-05-09 20:39:17 +03:00
Guo Xiang Tan
4e91839c97 FIX: Remove default val for colors step if a default theme has been set.
Running through the wizard after a default theme has been configured
will always revert the default theme to the light theme.
2019-05-09 17:25:09 +08:00
Vinoth Kannan
8c07c272f2 make rubocop happy. 2019-05-09 05:25:44 +05:30
Vinoth Kannan
87cd4701b8 FEATURE: option to skip posts with ignored missing uploads 2019-05-09 05:11:15 +05:30
Joffrey JAFFEUX
e6cfc584d0 Revert "FIX: frozen string exception (#7506)"
This reverts commit c4b7fb2754.
2019-05-08 17:27:09 +02:00
David Taylor
e84531a6a6 FIX: Correctly clear theme stylesheet cache when changing color scheme 2019-05-08 16:02:55 +01:00
Joffrey JAFFEUX
c4b7fb2754
FIX: frozen string exception (#7506)
Initial backtrace:

```
/var/www/discourse/app/models/incoming_domain.rb:29:in `to_url'
/var/www/discourse/app/models/incoming_link.rb:83:in `referer'
/var/www/discourse/app/models/incoming_link.rb:106:in `referer_valid'
```
2019-05-08 16:25:54 +02:00
Guo Xiang Tan
405ba00c08 FEATURE: Create notifications on wiki edits for watching users.
* Moves creation of notification into background job.
2019-05-08 15:31:10 +08:00
Robin Ward
5af7c90bab FEATURE: Hide Reviewable scores, change score filter to Priority
We found score hard to understand. It is still there behind the scenes
for sorting purposes, but it is no longer shown.

You can now filter by minimum priority (low, med, high) instead of
score.
2019-05-07 14:05:23 -04:00
Joffrey JAFFEUX
626534ad94
FIX: lighten instead of adding transparency to consolidated pageviews (#7495) 2019-05-07 18:14:13 +02:00
Sam Saffron
cac80cdc3b DEV: more send -> public_send changes
This is a work in progress commit for more send to public_send conversions

Also adds some comments for cases where we need to keep send
2019-05-07 11:58:06 +10:00
Guo Xiang Tan
61cc0f8c5f Follow up to 152238b4cf. 2019-05-07 09:57:27 +08:00
Guo Xiang Tan
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
Sam Saffron
9be70a22cd DEV: introduce new API to look up dynamic site setting
This removes all uses of both `send` and `public_send` from consumers of
SiteSetting and instead introduces a `get` helper for dynamic lookup

This leads to much cleaner and safer code long term as we are always explicit
to test that a site setting is really there before sending an arbitrary
string to the class

It also removes a couple of risky stubs from the auth provider test
2019-05-07 11:00:30 +10:00
Robin Ward
31e100530f FEATURE: Flag count in post menu
This change shows a notification number besides the flag icon in the
post menu if there is reviewable content associated with the post.
Additionally, if there is pending stuff to review, the icon has a red
background.

We have also removed the list of links below a post with the flag
status. A reviewer is meant to click the number beside the flag icon to
view the flags. As a consequence of losing those links, we've removed
the ability to undo or ignore flags below a post.
2019-05-06 16:13:31 -04:00
Dan Ungureanu
a40dcbde9b
FIX: Do not move hidden post actions. (#7424)
Hidden (staff-only) post actions are whisper posts with no content, that
are later transformed by the client into post actions (discourse-assign
uses this).
2019-05-06 16:21:42 +03:00
Sam Saffron
f8eddd40ad PERF: remove avg_time calculations and regular jobs from posts and topics
After careful analysis of large data-sets it became apparent that avg_time
had no impact whatsoever on "best of" topic scoring. Calculating avg_time
was a very costly operation especially on large databases.

We have some longer term plans of introducing other weighting that is read
time based into our scoring for "best of" and "top" topics, but in the
interim to stop a large amount of work that is not achieving any value we
are removing the jobs.

Column removal will follow once we decide on a new replacement metric.
2019-05-06 15:59:01 +10:00
Vinoth Kannan
e8fafbc123 List and restore missing post uploads from S3 inventory. 2019-05-04 01:16:20 +05:30
Penar Musaraj
b948d97c8f
FEATURE: Show diff of local changes before updating remote theme (#7443) 2019-05-02 21:43:54 -04:00
Sam Saffron
3dc4ab905c DEV: annotate models
(also looks like renaming of bigint(8) -> bigint in annotate
which seems fine)
2019-05-03 08:34:12 +10:00
Guo Xiang Tan
24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
Sam Saffron
1be01f8dd4 DEV: Add support for Rails 6
Minor fixes to add Rails 6 support to Discourse, we now will boot
with RAILS_MASTER=1, all specs pass

Only one tiny deprecation left

Largest change was the way ActiveModel:Errors changed interface a
bit but there is a simple backwards compat way of working it
2019-05-02 16:23:25 +10:00
Robin Ward
885f1e7e5f FIX: Allow category group reviewers to edit queued posts
They can edit title, body and tags. Category is disabled for now as it
could lead to some odd security issues.
2019-05-01 14:48:49 -04:00
Robin Ward
0f01c9d25f FIX: Posts in the approval queue were sometimes dropping category 2019-05-01 14:40:38 -04:00
David Taylor
0e303c7f5d
FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
Robin Ward
404b35bd04 FEATURE: Category Reviewable by Group
Allow a group to review content in a particular category.
2019-04-30 15:23:06 -04:00
Robin Ward
812905cbb6 FIX: Recursively delete nested replies
Previously we were only deleting top level replies
2019-04-30 12:54:53 -04:00
Guo Xiang Tan
587cfd4545 DEV: Use frozen_string_literal: true on models/theme.rb. 2019-04-30 16:02:55 +08:00
Rafael dos Santos Silva
1fdeec564b PERF: Move where clause up to speed up CalculateAvgTime daily job (#7462)
Cuts down affected posts earlier in the query, so the generated plan
deals with less rows, and runs faster.

https://meta.discourse.org/t/post-calculate-avg-time-taking-up-a-long-time/49750/13?u=falco
2019-04-30 13:34:46 +10:00
Rafael dos Santos Silva
526e76ced2 FIX: Use PostgreSQL 'ON CONFLICT' to deal with race condition
On busy sites, concurrent requests to insert into post_timings can
occur, which was dealt with using Ruby exceptions.

This moves the handling to PostgreSQL which makes it a bit faster,
and prevents a spam of ERROR in the database logs.
2019-04-29 16:34:42 -03:00
Neil Lalonde
f8f7091e57 FIX: staff-only tags visible on /tags page when restricted to a category
If a tag group is set to only be visible to staff, and is restricted
to a category that is visible by everyone, the tags in the group were
being shown on the /tags page. They weren't visible anywhere else.
This commit fixes it so they don't show on the /tags page.
2019-04-29 12:38:28 -04:00
Gerhard Schlager
7a27c93c03 FIX: URL encode username and first letter in avatar URL
And it adds specs as a follow-up to aadb7da7
2019-04-29 15:08:14 +02:00
Sam Saffron
45285f1477 DEV: remove update_attributes which is deprecated in Rails 6
See: https://github.com/rails/rails/pull/31998

update_attributes is a relic of the past, it should no longer be used.
2019-04-29 17:32:25 +10:00
Joffrey JAFFEUX
de79313f62
FIX: ensures file extension doesnt have a leading point (#7449) 2019-04-27 14:00:51 +02:00
Sam Saffron
b8c65cc6cf PERF: add index on user auth tokens (for user lookups)
This speeds up user destroy cause we cascade looking up all the tokens
for a user
2019-04-26 22:38:54 +10:00
Sam Saffron
378fe1c1cf PERF: allow looking up incoming email by user_id
This speeds up the ability to remove users from the system
2019-04-26 22:33:01 +10:00
Sam Saffron
26b4dbde41 PERF: use delete_all to remove notifications and topic_users
Previously we used destroy here which could be extremely expensive
2019-04-26 22:26:48 +10:00
Joffrey JAFFEUX
bcca2b5d73
FEATURE: initial implementation of generic filters for reports 2019-04-26 12:17:10 +02:00
Sam Saffron
c617e512ad annotate new indexes 2019-04-26 18:23:27 +10:00
Sam Saffron
c009a6ae36 PERF: speed up user deletion logic
Previously any user deletion would scan a very large number of tables

This avoids scans on 6 tables/indexes on delete
2019-04-26 18:11:39 +10:00
Guo Xiang Tan
abecb3aea8 Update annotations. 2019-04-26 10:36:21 +08:00
Guo Xiang Tan
761e1e25f7 Remove tables incorrectly excluded in Upload.migrate_to_new_scheme.
Follow up to 4a03b21f80.
2019-04-25 20:27:52 +08:00
Maja Komel
422237391e FIX: fix notifications for flag PMs and show topics with moderator posts in inbox (#7331)
Some old moderator posts that were not correctly processed before this change and 993d8f34 are not listed under /u/username/messages
2019-04-25 11:15:13 +02:00
Maja Komel
996e5e5dfa FIX: category list order using category featured topics (#7283) 2019-04-25 11:07:19 +02:00
Vinoth Kannan
84cfe30c1f Use update_all instead of save to select and update particular columns
9ec6ef85b6
2019-04-25 11:56:57 +05:30
Guo Xiang Tan
4a03b21f80 Exclude large tables when remapping in Upload.migrate_to_new_scheme.
Those tables do not carry any information about uploads.
2019-04-25 11:55:48 +08:00
Gerhard Schlager
aadb7da7d3 FIX: Use first grapheme cluster for letter avatar 2019-04-24 23:20:46 +02:00
Guo Xiang Tan
be88ac8b7e Additional old Upload#url format for Upload.migrate_to_new_scheme. 2019-04-24 18:19:25 +08:00
Guo Xiang Tan
9ec6ef85b6 Don't validate post when saving in Upload.migrate_to_new_scheme. 2019-04-24 17:17:31 +08:00
Guo Xiang Tan
3c9495b989 DEV: Put a mutex around Upload.migrate_to_new_scheme.
This ensures that only one migration is running at any given point in
time across the instances.
2019-04-24 17:07:10 +08:00
Guo Xiang Tan
55f406bb79 Follow up to 845b68f9ab. 2019-04-24 16:57:45 +08:00
Guo Xiang Tan
845b68f9ab Update possible upload url format for Upload.migrate_to_new_scheme. 2019-04-24 16:26:54 +08:00
Guo Xiang Tan
c811e59898 Avoid hardcoding db name in Upload.migrate_to_new_scheme. 2019-04-24 15:25:24 +08:00
Guo Xiang Tan
573ddcbbd0 Disable migrate_to_new_scheme once there is nothing to migrate. 2019-04-24 14:00:00 +08:00
Guo Xiang Tan
3094a603b7 Follow up to 149411ec90. 2019-04-24 12:20:53 +08:00
Guo Xiang Tan
149411ec90 PERF: Speed up Upload.migrate_to_new_scheme by limiting remap scope.
Doing a `LIKE` on `Post#raw` and `Post#cooked` takes forever on large
sites.
2019-04-24 11:56:48 +08:00
Guo Xiang Tan
a8baa6d9e7 FIX: Old Upload#url scheme support for Upload.get_from_url.
This is for backwards compatibility purposes. Even if `Upload#url` has a
format that we don't recognize, we should still return the upload object
as long as the upload record is present.
2019-04-24 10:15:47 +08:00
Robin Ward
646cdfa449 FIX: Approved posts were not enqueueing alerts 2019-04-23 17:30:12 -04:00
Robin Ward
fca3f53e9c FIX: Safer scoring with concurrency 2019-04-23 15:53:37 -04:00
Robin Ward
6f56fba016 UX: Update post actions to "Approve Post" and "Reject Post"
This should be more clear.
2019-04-23 12:19:11 -04:00
David Taylor
a2ddb6cf2d DEV: Increase max length of theme_field name column
Now that we have custom SCSS files with arbitrary names, it's easy to exceed 30 characters
2019-04-23 12:34:32 +01:00
Gerhard Schlager
a7bc1ecbae FEATURE: Add support for Unicode usernames and group names
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-04-23 13:00:27 +02:00
Guo Xiang Tan
d07605d885 DEV: Fix failing spec. 2019-04-23 18:25:28 +08:00
Dan Ungureanu
57d1dea8a2
FEATURE: Let staff add custom post notices. (#7377) 2019-04-19 17:53:58 +03:00
Vinoth Kannan
5de483a1b9 PERF: Very post upload's existence by preloaded upload sha1s array 2019-04-19 17:27:16 +05:30
Tarek Khalil
6e46197bc8
FIX: Disable webhooks on 410 and 404 HTTP responses (#7392)
FIX: Disable webhooks on 410 and 404 HTTP responses (#7392)
2019-04-18 12:36:37 +01:00
Robin Ward
14f9d40e48 FEATURE: Clarify Reviewable User Actions
"Approve" is now "Approve User" and "Delete" is a dropdown with a choice
that allows you to block.
2019-04-17 11:44:17 -04:00
Robin Ward
ba6d4b2a8d FIX: Better handling for toggling must_approve_users
If you turn it on now, default all users to approved since they were
previously. Also support approving a user that doesn't have a reviewable
record (it will be created first.)

This also includes a refactor to move class method calls to
`DiscourseEvent` into an initializer. Otherwise the load order of
classes makes a difference in the test environment and some settings
might be triggered and others not, randomly.
2019-04-16 15:56:35 -04:00
Guo Xiang Tan
784940bea0 Fix the build. 2019-04-16 15:47:16 +08:00
Guo Xiang Tan
1056dd16d3 FIX: Publish web hooks when topic archetype is converted. 2019-04-16 15:18:39 +08:00
Robin Ward
3ceff0a92a FEATURE: Send the user a notification when their post is approved 2019-04-15 16:20:16 -04:00
David Taylor
428536788c FIX: Do not error when importing a theme with empty files 2019-04-15 15:46:47 +01:00
Arpit Jalan
594674703c
FIX: properly log webhook errors in UI on rescue (#7376) 2019-04-15 12:19:48 +05:30
Vinoth Kannan
8e40c35eb8 FIX: 'have_uploads' scope should include all uploads without multisite 'upload_path' prefix 2019-04-15 01:54:55 +05:30
Arpit Jalan
95ada3f190 FIX: save registration IP address for invited users 2019-04-13 13:04:25 +05:30
Robin Ward
534d75310f FIX: Linting 2019-04-12 14:34:14 -04:00
Robin Ward
a5d9afe397 FEATURE: Include a user's pending posts in the topic view
Also includes a refactor to TopicView's serializer which was not
building our attributes using serializers properly.
2019-04-12 14:25:12 -04:00
David Taylor
268d4d4c82
FEATURE: Multiple SCSS file support for themes (#7351)
Theme developers can include any number of scss files within the /scss/ directory of a theme. These can then be imported from the main common/desktop/mobile scss.
2019-04-12 11:36:08 +01:00
James Haggerty
dcec256b9d Make 1gb warning a little less than 1gb (#7368)
When you have '1gb' RAM, this might be a little less than 1,000,000.
Let's not warn unless it's well under 1gb.
2019-04-11 16:44:29 -07:00
David Taylor
24ef4f7b2b Remove support for disable_jump_reply user setting (#7359) 2019-04-12 09:03:06 +10:00
David Taylor
dc703adad7 FEATURE: Add user preference for title counter mode (#7364) 2019-04-12 09:02:18 +10:00
Robin Ward
6f7b6a3333 FIX: Disagree and Restore should not revert edits 2019-04-11 16:21:18 -04:00
Robin Ward
331a809738 FEATURE: Display the reason for many reviewable items
Queued Posts and Users will now display a reason why they are in the
review queue.
2019-04-11 12:04:45 -04:00
Guo Xiang Tan
d705dd8bb8 Update annotations. 2019-04-11 12:37:24 +08:00
Robin Ward
cab83596ec FIX: Sites with SSO that required approval were not creating reviewables 2019-04-10 12:53:50 -04:00
Robin Ward
5d99346740 FIX: Always allow us to reject users, even if they are deleted 2019-04-10 11:00:39 -04:00
Bianca Nenciu
3d545d66df FEATURE: Send user activation reminders. (#7280) 2019-04-10 16:53:52 +02:00
Vinoth Kannan
b0600e52b6 FIX: use 'freeze' method again to fix 'cant modify frozen string' error
8d5c900142
2019-04-10 18:30:59 +05:30
Roman Rizzi
536b90e0ef
FEATURE: Support searching custom staff actions (#7346) 2019-04-10 09:53:17 -03:00
Vinoth Kannan
8d5c900142 DEV: add unique missing uploads index in post custom fields
https://review.discourse.org/t/feature-send-missing-post-uploads-stat-to-prometheus/2609/6?u=vinothkannans
2019-04-10 18:09:35 +05:30
Vinoth Kannan
d0fe42e2ef FIX: should look through posts for image markdown
Downloaded onebox images only included in the cooked HTML content. So we have to check 'post.cooked' instead of 'raw'. bfdd0fe64c
2019-04-10 13:52:35 +05:30
Vinoth Kannan
73663c2fdb FIX: should find records by sha1 only in Upload model 2019-04-09 18:48:49 +05:30
Joffrey JAFFEUX
df7f37f10a
FIX: web_crawlers had been mistakenly removed (#7342) 2019-04-09 09:26:22 +02:00
Guo Xiang Tan
c82a929025 PERF: Add index_for_rebake_old to posts.
The index becomes smaller over time and is much faster.

Follow up to 4791d992dc.
2019-04-09 13:59:46 +08:00
Guo Xiang Tan
28d117898f Update annotations. 2019-04-09 13:27:32 +08:00
Guo Xiang Tan
f0bb492d24 Fix regression due to 914ada1c74. 2019-04-09 09:47:11 +08:00
Vinoth Kannan
f5ac01253a FIX: use new changed method name and remove whitespaces 2019-04-09 02:25:26 +05:30
Vinoth Kannan
914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Vinoth Kannan
2ae87a27fa make rubocop happy 2019-04-08 17:03:26 +05:30
Guo Xiang Tan
f0f3deb32b PERF: Simplify query of UserStat#update_topic_reply_count.
For a user with alot of posts, we get a 25% speed up.
2019-04-08 17:48:19 +08:00
Guo Xiang Tan
6525613b89 PERF: Use joins for Post.for_mailing_list instead of NOT IN.
Joining on the topics table and then filtering the topics against a large set of
ids is so much slower than doing a join on a sub query.
2019-04-06 07:58:55 +08:00
Guo Xiang Tan
a7a5f90e20 Annotate models. 2019-04-05 17:13:12 +08:00
Sam Saffron
81a1bafe54 PERF: Speed up home page unread+new query
restructure query so it avoids ORs

It appears postgres is picking suboptimal indexes if too many ORs exist
despite how trivial the condition is.

This bypasses conditional in the query and evals them upfront.

On meta for my user this made a 10x perf difference.

This boils down to either having `OR u.admin` or not having `OR u.admin` in
the query.
2019-04-05 15:25:19 +11:00
Sam Saffron
5b837f620b correct unread resetting to handle nulls
Note, to avoid race conditions we are setting last_unread to 10 minutes ago
if there is nothing unread.

This is safer in case of in progress transactions
we don't want to lose unread for any window of time.
2019-04-05 13:42:41 +11:00
Sam Saffron
5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Robin Ward
8af7237092 FIX: Only perform logic on create commits 2019-04-04 17:01:47 -04:00
Robin Ward
df48c657fd FIX: Reviewable counts were not updating properly
Sometimes sidekiq is so fast that it starts jobs before transactions
have comitted. This patch moves the message bus stuff until after things
have comitted.
2019-04-04 16:07:34 -04:00
Robin Ward
2055804e95 FIX: The option to delete replies was missing from the new review queue 2019-04-04 15:51:36 -04:00
Guo Xiang Tan
6815f777f9 DEV: Remove unused method. 2019-04-04 14:19:39 +08:00
Vinoth Kannan
b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Robin Ward
111a502231 FIX: Deleting Users should work nicely with Reviewable Users
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.

Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
2019-04-03 16:42:39 -04:00
Robin Ward
828cdf3fea FIX: Protected method called, I'll fire myself now 2019-04-03 16:10:36 -04:00
Robin Ward
82bddcbe51 FIX: Don't create two reviewable scores for a user 2019-04-03 16:03:32 -04:00
Robin Ward
c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Arpit Jalan
7b194743d7
FIX: clean up user export csv upload records in scheduled job (#7309) 2019-04-03 13:31:19 +05:30
Sam Saffron
b2dc65f953 FIX: on rebakes should not recreate quoted_posts records
Previously every rebake would remove and recreate records in this table
This caused created_at and updated_at to keep changing

Yes, I know the SQL is somewhat complex, but this makes quote extraction
more efficient cause we do everything in 2 round trips.

This also removes some concurrency protection we should no longer need
2019-04-03 16:17:15 +11:00
Guo Xiang Tan
52c4711e80 DEV: Fix method that was incorrectly made private. 2019-04-03 12:37:50 +08:00
Guo Xiang Tan
4037a2fd0b FIX: Restrict scope of old scheme upload migration.
Some sites have external URLs that don't even match `%/uploads/%' and
some sites surprise me with URLs that contains the default path when it
is a site in a multisite cluster. We can't do anything about those.
2019-04-03 11:56:41 +08:00
Guo Xiang Tan
feb731bffd FIX: Regenerate optimized images instead of migrating from old scheme.
`OptimizedImage.migrate_to_new_scheme` was optimizing optimized images
which we don't need to do. Regnerating the optimized image is way easier.
2019-04-03 09:45:02 +08:00
Guo Xiang Tan
d85240335b Annotate models. 2019-04-02 13:19:14 +08:00
Robin Ward
d5a61ab167 FIX: Don't log a second pending action
If two people flagged the same post you'd get a "Pending" history event
logged which looked bad.
2019-04-01 17:07:47 -04:00
Penar Musaraj
fdf4145d4b
FEATURE: Delegated authentication via user api keys (#7272) 2019-04-01 13:18:53 -04:00
David Taylor
b375dcb14a DEV: Introduce parallel rspec testing
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:

```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```

This brings the test suite from 12m20s to 3m11s on my macOS machine
2019-04-01 11:06:47 -04:00
Régis Hanol
a6596662dc FIX: don't enqueue TruncateUserFlagStats job when not needed 2019-04-01 15:53:55 +02:00
Joffrey JAFFEUX
6e0e36f790
fix spec (#7298) 2019-04-01 14:35:09 +02:00
Joffrey JAFFEUX
e986e96227
DEV: removes old dashboard (#7295) 2019-04-01 12:39:49 +02:00
Guo Xiang Tan
8794d940d3 Revert update_columns -> update! when rebaking/post-processing post.
`update!` goes through validation which means old posts that doesn't
adhere to the existing validations will raise an error.
2019-04-01 16:29:00 +08:00
Maja Komel
88128f1ced UX: show which groups are missing permissions for parent category (#7252) 2019-04-01 15:34:52 +11:00
Guo Xiang Tan
cfd507822f
PERF: Improve quality of PostSearchData#raw_data. (#7275)
This commit fixes the follow quality issue with `PostSearchData#raw_data`:

1. URLs are being tokenized and links with similar href and characters
are being duplicated in the raw data.

`Post#cooked`:

```
<p><a href=\"https://meta.discourse.org/some.png\" class=\"onebox\" target=\"_blank\" rel=\"nofollow noopener\">https://meta.discourse.org/some.png</a></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png discourse org/some png https://meta.discourse.org/some.png discourse org/some png
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized https://meta.discourse.org/some.png meta discourse org
```

2. Ligthbox being included in search pollutes the
`PostSearchData#raw_data` unncessarily.

From 28 March 2018 to 28 March 2019, searches for the term `image` on
`meta.discourse.org` had a click through rate of 2.1%. Non-lightboxed images are not included in indexing for search yet we were indexing content within a lightbox. Also, search for terms like `image` was affected we were using `Pasted image` as the filename for
uploads that were pasted.

`Post#cooked`

```
<p>Let me see how I can fix this image<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://meta.discourse.org/some.png\" title=\"some.png\" rel=\"nofollow noopener\"><img src=\"https://meta.discourse.org/some.png\" width=\"275\" height=\"299\"><div class=\"meta\">\n<svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#far-image\"></use></svg><span class=\"filename\">some.png</span><span class=\"informations\">1750×2000</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use xlink:href=\"#discourse-expand\"></use></svg>\n</div></a></div></p>
```

`PostSearchData#raw_data` Before:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image some.png png https://meta.discourse.org/some.png discourse org/some png some.png png 1750×2000
```

`PostSearchData#raw_data` After:

```
This is a test topic 0 Uncategorized Let me see how I can fix this image
```

In terms of indexing performance, we now have to parse the given HTML
through nokogiri twice. However performance is not a huge worry here since a string length of 194170 takes only 30ms
to scrub plus the indexing takes place in a background job.
2019-04-01 10:14:29 +08:00
Guo Xiang Tan
7ac76fe935 DEV: Remove warning.
Library has already been loaded in application.rb.
2019-04-01 10:11:08 +08:00
Robin Ward
370355d754 FIX: Allow users with posts to be rejected 2019-03-29 13:53:46 -04:00
Maja Komel
4a3daacb1b FIX: reset embedding settings when no embeddable host, log host changes (#7264) 2019-03-29 17:05:51 +01:00
Maja Komel
9b00ca30ed FIX: check if user is already allowed before adding to topic allowed users (#7242) 2019-03-29 17:03:33 +01:00
Joffrey JAFFEUX
961fb2c70e
DEV: moves each report in its own file (#7277) 2019-03-29 10:54:56 +01:00
Guo Xiang Tan
1b3165d217 Make rubocop happy. 2019-03-29 12:15:39 +08:00
Guo Xiang Tan
f458cba4cb FIX: Admin search logs should filter by date instead of timestamp.
The client side filters by date so it is confusion when the data changes as each second passes.
2019-03-29 11:50:25 +08:00
Guo Xiang Tan
d8faf5f79e FIX: SearcLog.term_details generating incorrect data because of case.
Also match on equality rather than "LIKE ?" which is quite strange.
2019-03-29 11:31:01 +08:00
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Guo Xiang Tan
d0a1922ad9 FIX: Upload.migrate_to_new_scheme undefined error when external image fails to download. 2019-03-28 16:00:13 +08:00
Tim Lange
12181599db FIX: Staff action records now also accepts action_name as filter (#7256) 2019-03-27 21:29:15 +01:00
Bianca Nenciu
a9798f0c47
FEATURE: Add page for all group membership requests. (#6909) 2019-03-27 13:30:59 +02:00
Guo Xiang Tan
161a3c3870 DEV: Don't optimize image when migrating to new scheme.
The image has already been uploaded, the migrate to new scheme job's is
just to correct the scheme and not the content of the upload.
2019-03-26 15:12:10 +08:00
Guo Xiang Tan
0ac6a58b09 FIX: Skip validation when saving in Upload#migrate_to_new_scheme. 2019-03-26 14:28:39 +08:00
Bianca Nenciu
f072da1bfe
FEATURE: Add every month / 6 months options to email digest. (#7247) 2019-03-25 12:45:01 +02:00
Guo Xiang Tan
ac661e856a
FEATURE: Allow categories to be prioritized/deprioritized in search. (#7209) 2019-03-25 10:59:55 +08:00
Guo Xiang Tan
839a54b97b FIX: Destroy OptimizedImage record even if Upload record is invalid. 2019-03-22 16:47:06 +08:00