Commit Graph

7986 Commits

Author SHA1 Message Date
Roman Rizzi
671f882fa3
FIX: We don't create a Post object if the queued post gets rejected. We need to count review items directly. (#9856) 2020-05-22 11:50:28 -03:00
Arpit Jalan
61d8955fcb DEV: add spec for tag with unicode name 2020-05-22 14:42:22 +05:30
Martin Brennan
f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Robin Ward
a55d5bd1b0 SECURITY: ERB execution in custom Email Style 2020-05-21 14:49:06 -04:00
Robin Ward
d11c462104 SECURITY: ERB execution in custom Email Style 2020-05-21 14:48:30 -04:00
Guo Xiang Tan
f4b82f1dc0
DEV: Fix randomly failing spec.
If a user is created with an id of 999, a `upload.user_id ==
user_avatar.user_id` will return true. This fix increases the id of the
upload to something that we will not hit in the foreseeable future.
2020-05-21 11:41:07 +08:00
Martin Brennan
df68d11c38
FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
Guo Xiang Tan
68db5deaec
FIX: Badge granter was disabled by default. 2020-05-21 06:55:22 +08:00
Robin Ward
096eca0ee8 FIX: Requests were not being logged correctly
`enable` was defaulting to `nil` which is not what we wanted.
2020-05-20 15:09:17 -04:00
Mark VanLandingham
e1bd57007b
DEV: Specs for AboutController can_see_about_stats? impact (#9843) 2020-05-20 11:30:27 -05:00
Osama Sayegh
02f44def56
FIX: Don't blow up when trying to parse invalid or non-ASCII URLs (#9838)
* FIX: Don't blow up when trying to parseinvalid or non-ASCII URLs

Follow-up to 72f139191e
2020-05-20 12:46:27 +03:00
Bianca Nenciu
fb15da43da
Remove old web hooks in favor of 'reviewable' web hook (#9776)
* FIX: Emit web hooks for flags

* FEATURE: Remove 'flag' web hook in favor of 'reviewable' web hook

* FEATURE: Remove 'queued post' web hook in favor of 'reviewable' web hook

* FIX: Do not set a default value for web hooks with no events
2020-05-20 12:07:48 +03:00
Martin Brennan
72f139191e
FIX: S3 store has_been_uploaded? was not taking into account s3 bucket path (#9810)
In some cases, between Discourse forums the hostname of a URL could match if they are hosting S3 files on the same bucket but the S3 bucket path might not. So e.g. https://testbucket.somesite.com/testpath/some/file/url.png vs https://testbucket.somesite.com/prodpath/some/file/url.png. So has_been_uploaded? was returning true for the second URL, even though it may have been uploaded on a different Discourse forum.

This is a very rare case but must be accounted for, because this impacts UrlHelper.is_local which mistakenly thinks the file has already been downloaded and thus allows the URL to be cooked, where we want to return the full URL to be downloaded using PullHotlinkedImages.
2020-05-20 10:40:38 +10:00
David Taylor
e9f579bd3f
DEV: Clean up thumbnail tests correctly 2020-05-19 11:04:02 +01:00
David Taylor
725e38f9d7
DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
Guo Xiang Tan
96c02caba7
DEV: Change use of Redis flushall to flushdb.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
Krzysztof Kotlarek
f99f6ca111
FIX: randomize file name when created from fixtures (#9731)
* FIX: randomize file name when created from fixtures

When a temporary file is created from fixtures it should have a unique name.
It is to prevent a collision in parallel specs evaluation

* FIX: use /tmp/pid folder to keep fixture files
2020-05-19 09:09:36 +10:00
Gerhard Schlager
6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Gerhard Schlager
27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
David Taylor
96848b7649
UX: Allow secure media URLs to be cached for a short period of time
Signed S3 URLs are valid for 15 seconds, so we can safely allow the browser to cache them for 10 seconds. This should help with large numbers of requests when composing a post with many images.
2020-05-18 15:00:41 +01:00
Joffrey JAFFEUX
918bd75909
FIX: allows to remove a topic timer with delete_replies as type (#9800) 2020-05-18 11:47:08 +02:00
Guo Xiang Tan
d01c336899
DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
Sam Saffron
862773ec83
FIX: do not remove stop words when using English locale
PG already handles English stop words, the list in cppjieba is
bigger than the list PG uses, which in turn causes confusion cause
words such as "volume" are stripped using cppijieba stop word list

We will follow up with another commit here to apply the Chinese
word stopwords, but for now to eliminate the confusion we are
skipping applying the stopword list when the dictionary in PG is
in English.
2020-05-18 10:54:56 +10:00
Krzysztof Kotlarek
ea63fa7de7
FIX: extract reset_last_seen_cache! in user_spec (#9808)
Helper method to avoid code duplication
2020-05-18 09:13:37 +10:00
David Taylor
461b4e5cc6
DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
David Taylor
0495a748d0
FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
Guo Xiang Tan
858735b5da
DEV: Fix broken test due to 9ab5801a1b. 2020-05-15 15:00:04 +08:00
Guo Xiang Tan
9ab5801a1b
FIX: Allow post migrations using #change to carry out unsafe migration 2020-05-15 14:23:27 +08:00
Guo Xiang Tan
165fa13c9d
FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
Blake Erickson
1a2b9435b0
DEV: Standardize table sorting verbiage (#9757)
* DEV: Standardize table sorting verbiage

This commit creates a common component that tables can use to make their
headers sortable. This commit also standardizes on using `desc` as the
default and passing in the `asc=true` flag to adjust the sorting
direction.

* Add deprecation warnings

Adds deprecation warnings if using previous params and maintains
backwards compatibility. Set the default sort value for group members to
be asc.

* switch group requests to use common table-header-toggle

* update fixture
2020-05-14 20:10:59 -06:00
Krzysztof Kotlarek
65a3fdbc57
FIX: randomly failing user_spec (#9754)
* FIX: randomly falling user_spec

When we evaluate `update_last_seen!` we relay on Redis to not run that code too often

https://github.com/discourse/discourse/blob/master/app/models/user.rb#L753

The problem is that not all specs which are running `update_last_seen!` are not cleaning after themselves

For examples specs in that block https://github.com/discourse/discourse/blob/master/spec/models/user_spec.rb#L901

So it can be replicated when you run a few times
`bundle exec rspec  ./spec/models/user_spec.rb -e "should not update the first seen value if it doesn't exist" -e "should have 0 for days_visited"`

We should delete Redis key after each spec which is evaluating `update_last_seen!`
2020-05-15 09:24:07 +10:00
Kane York
869f9b20a2
PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
Penar Musaraj
5ff2a235f6 DEV: Allow 3-digit HEX color code in single icon route
Followup to aee8e62
2020-05-14 16:37:45 -04:00
Penar Musaraj
aee8e62e21
FEATURE: Add endpoint for individual SVG icons (#9765) 2020-05-14 14:17:19 -04:00
Mark VanLandingham
a047004c9a
FIX: Specific email error for replies to digest emails (#9770) 2020-05-14 09:04:58 -05:00
Dan Ungureanu
3ed6a0e904
FIX: Detect Wayback Machine using user agent (#9777) 2020-05-14 21:10:07 +10:00
David Taylor
d4717f5d1e
DEV: Reduce repeated code in discourse_plugin_registry (#9705)
Use a helper method to simplify creating a new register. Previously this would require creating lots of different methods manually, and adding every register to the clear/reset functions
2020-05-13 12:25:34 +01:00
Sam Saffron
7f841dc21f
PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
Martin Brennan
a64cf265fd
FIX: Change /bookmarks URL back to topic list and add bookmark poster avatars (#9759)
* Changes the /bookmarks URL to show the original "topics filtered by bookmark" list instead of redirecting to user activity bookmarks (see https://meta.discourse.org/t/domain-com-bookmarks-is-showing-domain-com-u-user-activity-bookmarks-with-reminders/149252/12)
* Add the user avatar for the user who made the post that is bookmarked
2020-05-13 14:03:24 +10:00
Blake Erickson
4078b22887 FIX: Handle missing provider return sso url
This commit prevents a 500 error from occurring if someone is trying to
setup their discourse instance as a sso provider and they don't pass in
a `return_sso_url` in their payload.
2020-05-12 18:16:50 -06:00
Martin Brennan
9981fa4466
FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
Robin Ward
033644ea82 DEV: Adds an integrity spec for JS constants 2020-05-12 12:23:36 -04:00
Rafael dos Santos Silva
2298e14d38
FEATURE: Support for App Shortcuts Menu (#9749)
* FEATURE: Support for App Shortcuts Menu

This adds a list of shortcuts to a installed Discourse instance.

It can be accessed by right clicks or long press on the app icon.

See https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/Shortcuts/explainer.md

List of possible follow ups include:

- Making it admin customizable
- Making it user customizable
- Using SVG icons from the site icon sprite
- Picking an accent color for icons

* FIX: Add type to shortcut menu icons
2020-05-12 12:24:33 -03:00
Joshua Rosenfeld
c94e6a9a66
FEATURE: add noindex header to tags pages (#9748) 2020-05-12 10:44:46 -04:00
David Taylor
6230f5c554
FEATURE: Allow parameter authentication for UserApiKeys (#9742)
This refactors default_current_user_provider in a few ways:
- Introduce a generic `api_parameter_allowed?` method which checks for whitelisted routes/formats
- Only read the api_key parameter on allowed routes. It is now completely ignored on other routes (previously it would raise a 403)
- Start reading user_api_key parameter on allowed routes
- Refactor tests as end-end integration tests

A plugin API for PARAMETER_API_PATTERNS will be added soon
2020-05-12 13:35:36 +01:00
Sam Saffron
a29ae17d3a
FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
Guo Xiang Tan
d301af39bd
PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
Guo Xiang Tan
3c423faafd
DEV: Avoid repeating draft key logic in multiple places. 2020-05-12 11:14:03 +08:00
Krzysztof Kotlarek
4c8bece104
FEATURE: default canonical URL (#9738)
For pages that do not specify canonical URL we will default to `https://SITENAME/PATH`. 

This ensures that if a URL is crawled on the CDN the search ranking will transfer to the main site.

Additionally we whitelist the `?page` param
2020-05-12 09:13:20 +10:00
Blake Erickson
ab919332dc
DEV: api documentation updates (#9612)
* DEV: api documentation updates

- Created a script to convert json responses to rswag
- Documented several api endpoints
- Switched rswag to use header based auth

* Update script, fix some schema missmatches
2020-05-11 13:06:49 -06:00
David Taylor
5fc51ed49c
DEV: Remove unused DiscoursePlugin class (#9715) 2020-05-11 15:46:54 +01:00
Joshua Rosenfeld
07b728c5e5
FEATURE: add noindex header to badges, groups, and /my pages (#9736) 2020-05-11 15:05:42 +10:00
Sam Saffron
bb4e8899c4
FEATURE: let Google index pages so it can remove them
Google insists on indexing pages so it can figure out if they
can be removed from the index.

see: https://support.google.com/webmasters/answer/6332384?hl=en

This change ensures the we have special behavior for Googlebot
where we allow indexing, but block the actual indexing via
X-Robots-Tag
2020-05-11 12:15:18 +10:00
Sam Saffron
676be3a853
FEATURE: add no_index header if robots is disabled
Expand SiteSetting.allow_index_in_robots_txt so it also adds a
noindex header if set to false.

This makes sure that nothing is indexed even if it somehow reaches
Google.
2020-05-11 11:07:21 +10:00
Bianca Nenciu
8149bfbaf1
FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
Sam Saffron
d8d54a92f1
FEATURE: tighten rate limiting rules for forgot password
1. Total 6 attempts per day per user
2. Total of 5 per unique email/login that is not found per hour
3. If an admin blocks an IP that IP can not request a reset
2020-05-08 13:30:51 +10:00
Martin Brennan
22dffe6f69 FIX: Do not allow null options for bookmark manager 2020-05-08 15:24:59 +00:00
Robin Ward
f9608c0af5 DEV: Remove INLINE_ONEBOX_* constants
There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and
`INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they
were DRYing up: `inline-onebox-loading` and `inline-onebox`

I normally appreciate constants, but in this case it meant that we had
a lot of JS imports resulting in many more lines of code (and CPU cycles
spent figuring them out.)

It also meant we had an `.erb` file and had to invoke Ruby to create the
JS file, which meant the app was harder to port to Ember CLI.

I removed the constants. It's less DRY but faster and simpler, and
arguably the loss of DRYness is not significant as you can still search
for the `inline-onebox-loading` and `inline-onebox` strings easily if
you are refactoring.
2020-05-07 16:14:38 -04:00
Vinoth Kannan
744bbf6904 FEATURE: exclude muted categories from the "top" topics list. 2020-05-08 00:34:53 +05:30
Jarek Radosz
781e3f5e10
DEV: Use response.parsed_body in specs (#9615)
Most of it was autofixed with rubocop-discourse 2.1.1.
2020-05-07 17:04:12 +02:00
Martin Brennan
37be2958f7
DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
Martin Brennan
6fb0f36ce1
FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
tshenry
0d394e6b76
FEATURE: Include category position when exporting categories (#9658) 2020-05-07 12:17:15 +10:00
Krzysztof Kotlarek
05799538e6
FIX: correct edit notification username for PMs (#9649)
When the tag is added to PM by admin, notification has an incorrect username (post author username instead of admin username)
2020-05-07 07:52:21 +10:00
Gerhard Schlager
ec2f3169ff FIX: Using the default_locale in locale fallbacks caused problems
Locale files get precompiled after deployment and they contained translations from the `default_locale`. That's especially bad in multisites, because the initial `default_locale` is `en_US`. Sites where the `default_locale` isn't `en_US` could see missing translations. The same thing could happen when users are allowed to chose a different locale.

This change simplifies the logic by not using the `default_locale` in the locale chain. It always falls back to `en` in case of missing translations.
2020-05-06 22:59:07 +02:00
Roman Rizzi
70bf1669be
DEV: Use a copy of the fixture file instead of the original one (#9645) 2020-05-06 11:54:08 -03:00
Sam Saffron
f8c18ac888
DEV: remove fragile spec
The failover spec is very fragile and tests specific implementation
vs actual behavior

We rely on a different script during the build process to test
failover operates correctly
2020-05-06 16:21:48 +10:00
Martin Brennan
fa572d3a7a
DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
Robin Ward
612284cef3
DEV: Remove Discourse.RAW_TEMPLATES (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
David Taylor
03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek
9bff0882c3
FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
David Taylor
5901717531
DEV: Allow DB.after_commit to be used outside of a transaction
In this case, it will execute the given block immediately
2020-05-04 09:42:41 +01:00
Sam Saffron
3877ef2cfa
FEATURE: Unconditionally notify on edits at least once a day
This ensures that at a minimum you are notified once a day of
repeat edits by the same user.

Long term we may consider winding this down to say 1 hour or
making it configurable.
2020-05-04 17:55:00 +10:00
Sam Saffron
3d0ccf8642
FIX: missing edit notifications in some rare cases
Due to a refactor in e90f9e5cc4 we stopped notifying on edits if
a user liked a post and then edited.

The like could have happened a long time ago so this gets extra
confusing.

This change makes the suppression more deliberate. We only want
to suppress quite/link/mention if the user already got a reply
notification.

We can expand this suppression if it is not enough.
2020-05-04 17:40:56 +10:00
Kane York
784bf2a173
FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
Martin Brennan
bcc9ad6f57
FIX: Bookmark UI tweaks (#9604)
* When hovering over the bookmark icon for a post, show the name of the bookmark at the end of the tooltip _if_ it has been set.
* Order bookmarks by `updated_at DESC` in the user list and show that instead of created at.
2020-05-01 16:14:20 +10:00
Martin Brennan
867bc3b48e
FIX: Change base importer to create new Bookmark records (#9603)
Also add a spec and fixture with a mock importer that we can use to test the create_X methods of the base importer
2020-05-01 11:34:55 +10:00
Krzysztof Kotlarek
37e93914fc
FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
Vinoth Kannan
71241a50f7 DEV: improve code readability & add tests for user guardian.
a511bea4cc
2020-04-30 20:59:33 +05:30
Régis Hanol
501b19b6e0
FIX: server-side HtmlToMarkdown improvements (#9586)
TLDR; this commit vastly improves how whitespaces are handled when converting from HTML to Markdown.
It also adds support for converting HTML <tables> to markdown tables.

The previous 'remove_whitespaces!' method was traversing the whole HTML tree and used a heuristic to remove
leading and trailing whitespaces whenever it was appropriate (ie. mostly before and after HTML block elements)

It was a good idea, but it was very limited and leaded to bad conversion when the html had leading whitespaces on several lines for example.
One such example can be found [here](https://meta.discourse.org/t/86782).

For various reasons, most of the whitespaces in a HTML file is ignored when the page is being displayed in a browser.
The rules that the browsers follow are the [CSS' White Space Processing Rules](https://www.w3.org/TR/css-text-3/#white-space-rules).
They can be quite complicated when you take into account RTL languages and other various tidbits but they boils down to the following:

- Collapse whitespaces down to one space (0x20) inside an inline context (ie. nodes/tags that are being displaying on the same line)
- Remove any leading/trailing whitespaces inside an inline context

One quick & dirty way of getting this 90% solved would be to do 'HTML.gsub!(/[[:space:]]+/, " ")'.
We would also need to hoist <pre> elements in order to not mess with their whitespaces.
Unfortunately, this solution let some whitespaces creep around HTML tags which leads to more '.strip!' calls than I can bear.

I decided to "emulate" the browser's handling of whitespaces and came up with a solution in 4 parts

1. remove_not_allowed!

The HtmlToMarkdown library is recursively "visiting" all the nodes in the HTML in order to convert them to Markdown.
All the nodes that aren't handled by the library (eg. <script>, <style> or any non-textual HTML tags) are "swallowed".
In order to reduce the number of nodes visited, the method 'remove_not_allowed!' will automatically delete all the nodes
that have no "visitor" (eg. a 'visit_<tag>' method) defined.

2. remove_hidden!

Similar purpose as the previous method (eg. reducing number of nodes visited), there's no point trying to convert something that is hidden.
The 'remove_hidden!' method removes any nodes that was hidden using the "hidden" HTML attribute, some CSS or with a width or height equal to 0.

3. hoist_line_breaks!

The 'hoist_line_breaks!' method is there to handle <br> tags. I know those tiny <br> don't do much but they can be quite annoying.
The <br> tags are inline elements but they visually work like a block element (ie. they create a new line).
If you have the following HTML "<i>Foo<br>Bar</i>", it ends up visually similar to "<i>Foo</i><br><i>Bar</i>".
The latter being much more easy to process than the former, so that's what this method is doing.
The "hoist_line_breaks" will hoist <br> tags out of inline tags until their parent is a block element.

4. remove_whitespaces!

The "remove_whitespaces!" is where all the whitespace removal is happening. It's broken down into 4 methods as well

- remove_whitespaces!
- is_inline?
- collapse_spaces!
- remove_trailing_space!

The 'remove_whitespace!' method is recursively walking the HTML tree (skipping <pre> tags).
If a node has any children, they will be chunked into groups of inline elements vs block elements.
For each chunks of inline elements, it will call the "collapse_space!" and "remove_trailing_space!" methods.
For each chunks of block elements, it will call "remote_whitespace!" to keep walking the HTML tree recursively.

The "is_inline?" method determines whether a node is part of a inline context.
A node is inline iif it's a text node or it's an inline tag, but not <br>, and all its children are also inline.

The "collapse_spaces!" method will collapse any kind of (white) space into a single space (" ") character, even accros tags.
For example, if we have "  Foo \n<i> Bar </i>\t42", it will return "Foo <i>Bar </i>42".

Finally, the "remove_trailing_space!" method is there to remove any trailing space that might creep in at the end of the inline chunk.

This solution is not 100% bullet-proof.
It does not support RTL languages at all and has some caveats that I felt were not worth the work to get properly fixed.

FIX: better detection of hidden elements when converting HTML to Markdown
FIX: take into account the 'allowed_href_schemes' site setting when converting HTML <a> to Markdown
FIX: added support for 'mailto:' scheme when converting <a> from HTML to Markdown
FIX: added support for <img> dimensions when converting from HTML to Markdown
FIX: added support for <dl>, <dd> and <dt> when converting from HTML to Markdown
FIX: added support for multilines emphases, strongs and strikes when converting from HTML to Markdown
FIX: added support for <acronym> when converting from HTML to Markdown
DEV: remove unused 'sanitize' gem

Wow, did you just read all that?! Congratz, here's a cookie: 🍪.
2020-04-30 12:21:25 +02:00
Sam Saffron
d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
David Taylor
367cbf5d2b
FEATURE: Allow user creation with admin api when local logins disabled (#9587) 2020-04-30 11:39:24 +10:00
Martin Brennan
ca539fdccf
FIX: Rename all instances of bookmarkWithReminder to just bookmark (#9579)
* Rename all instances of bookmarkWithReminder and bookmark_with_reminder to just bookmark
* Delete old bookmark code at the same time
* Add migration to remove the bookmarkWithReminder post menu item if people have it set in site settings
2020-04-30 10:09:22 +10:00
Dan Ungureanu
0a85a7aef9
FEATURE: Add user_profile to user_archive CSV export (#9571) 2020-04-29 12:09:50 +03:00
Neil Lalonde
9d781d4959
FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
Joffrey JAFFEUX
a6f986b50f
FEATURE: allows to to style published page with themes/plugins (#9570) 2020-04-28 18:24:24 +02:00
Penar Musaraj
b19dcac272
FIX: Include default notification level in category serializer (#9572)
Fixes an issue where the notification level state goes missing when user edits a category in the UI.
2020-04-28 12:05:53 -04:00
David Taylor
ba616ffb50
DEV: Use a tmp directory for storing uploads in tests (#9554)
This avoids development-mode upload files from polluting the test environment
2020-04-28 14:03:04 +01:00
Martin Brennan
5108cf8ddf
FIX: Topic user bookmarked column logic was not correct (#9563)
Make sure the topic_user.bookmarked column is set correctly when user bookmarks/unbookmarks any post in a topic. For example, you bookmarked a post in the topic that was not the OP, the bookmark icon in the topic list would not be shown. Same if deleting a bookmark for the last bookmarked post in a topic, the bookmark icon in the topic list would not be removed.

Previously this was only setting it to true if bookmarking the OP/topic, which was not correct -- we want to show the icon on the topic list if any post is bookmarked.
Also set to false if unbookmarking the last bookmarked post in the topic.
Also in this PR is a migration to correct any out of sync topic_user.bookmarked columns, based on the new logic.
2020-04-28 16:19:25 +10:00
mentalstring
67f3fe14aa
FEATURE: support SSO website and location overrides
Add location and website + the ability to override using SSO using the `sso_overrides_location` and `sso_overrides_website` site settings.
2020-04-28 16:06:35 +10:00
Blake Erickson
a93ef2926d
DEV: Add rswag to aid in api documention (#9546)
Adding in rswag will allow us to write spec files to document and test
our api.
2020-04-27 16:40:07 -06:00
Joffrey JAFFEUX
01f2819dde
DEV: enables and fixes multisite-spec (#9557) 2020-04-27 20:55:36 +02:00
Roman Rizzi
394babcae3
FIX: Only show the review page to users that can see it. Do not publish the reviewable count update message to everyone. (#9556) 2020-04-27 14:51:25 -03:00
David Taylor
f95609ae23
FIX: Add additional checks for automatic theme script CSP
- Skip absolute URLs
- Skip CDN URLs
- Skip invalid URLs
- Properly format protocol-less URLs
2020-04-27 15:56:29 +01:00
Benno
6e01acb3cb
FIX: Apply category priority for empty query (#9516) 2020-04-27 10:35:27 -04:00
Sam Saffron
8c1e008c59
DEV: Skip erratic spec for now
Spec fails intermittently due to CDN state.
2020-04-25 13:20:04 +10:00
David Taylor
8a112b7464
DEV: Automatically extend CSP when themes link to external scripts (#9531) 2020-04-24 09:47:01 +01:00
Sam Saffron
cbceadf48b
FEATURE: when blocking emails prefer blocking canonical
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.

This commit improves the situation by always preferring to block (and check)
canonical emails.

This means that if:

`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`

This means that `samevil+2@domain.com` (ad infinitum) will be blocked
2020-04-24 14:09:51 +10:00
Sam Saffron
6a18c9aa0b
Revert "FEATURE: enforce_canonical_emails site setting"
This reverts commit 6f9177e2ed.

We decided on a completely different approach to the problem.

Instead we will let blocked emails be treated as canonical.
2020-04-24 13:52:06 +10:00
Martin Brennan
cd1c7d7560
FIX: Copying image markdown for secure media loading full image (#9488)
* When copying the markdown for an image between posts, we were not adding the srcset and data-small-image attributes which are done by calling optimize_image! in cooked post processor
* Refactored the code which was confusing in its current state (the consider_for_reuse method was super confusing) and fixed the issue
2020-04-24 10:29:02 +10:00
Penar Musaraj
585e7bcfe8 DEV: update specs followup to 67e96f6 2020-04-23 16:11:17 -04:00
Arpit Jalan
39be639c37 FIX: update GitHub screen_name on login via GitHub 2020-04-23 20:54:26 +05:30
Krzysztof Kotlarek
52c1d7337e
FEATURE: don't display new/unread notification for muted topics (#9482)
* FEATURE: don't display new/unread notification for muted topics

Currently, even if user mute topic, when a new reply to that topic arrives, the user will get "See 1 new or updated topic" message. After clicking on that link, nothing is visible (because the topic is muted)

To solve that problem, we will send background message to all users who recently muted that topic that update is coming and they can ignore the next message about that topic.
2020-04-23 14:57:35 +10:00
Blake Erickson
9cbbaf4237 FIX: Don't throw 500 for invalid website url input
It's possible to cause a 500 error by putting in weird characters in the
input field for updating a users website on their profile.

Normal invalid input like not including the domain extension is already
handled by the user_profile model validation. This fix ensures a server
error doesn't occur for weird input characters.
2020-04-22 13:57:56 -06:00
Jarek Radosz
8adccaf98c
DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
Vinoth Kannan
df0c386f8a
UX: drop the automatic_membership_retroactive column from groups model. (#9430) 2020-04-22 22:07:39 +05:30
Krzysztof Kotlarek
81c216e04c
FIX: composer problem on the categories page (#9508) 2020-04-22 09:24:54 -07:00
Robin Ward
13f2723dcb FIX: Don't log an error to logster if a topic could not be updated.
If for some reason an update did not go through (for example,
concurrently updating the same topic twice), we were logging something
like:

```
create_errors_json called with unrecognized type: #<Topic
```

This happened because we knew an error occurred but the active record
object had no errors attached.

This patch fixes the issue by attaching a proper error message in the
event that this happens.
2020-04-22 11:56:04 -04:00
Dan Ungureanu
4e5f9d4cd1
DEV: Drop 'key' column from user_api_keys (#9388) 2020-04-22 12:13:19 +03:00
Dan Ungureanu
e733701887
FEATURE: Make report filters reusable (#9444)
This commit also adds 'include subcategories' report filter
2020-04-22 11:52:50 +03:00
Vinoth Kannan
a511bea4cc
FEATURE: admin UI to merge two users. (#9509) 2020-04-22 14:07:51 +05:30
Martin Brennan
628ba9d1e2
FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
Martin Brennan
e18aeb799e
FIX: Make sure reminder_type is parsed on bookmark update (#9503)
Otherwise we are trying to update the reminder type with a string which often evaluates to 0 (At Desktop) which causes reminders to come through early.
2020-04-22 10:44:04 +10:00
Jarek Radosz
c08753dc34
DEV: Fix topic_timer_spec (#9483)
File it under "how did it even work".
2020-04-21 14:38:35 +10:00
Jarek Radosz
a781ef7662
FIX: Reject invalid Category slugs (#9473)
Previously it would sanitize given slug and then save the resulting empty slug.
2020-04-21 03:50:50 +02:00
Jarek Radosz
17cf300b71
DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
Jarek Radosz
ab52bed014
DEV: Remove the return value of disable_if_low_on_disk_space (#9469)
It was used only in specs.
2020-04-21 03:48:33 +02:00
Jarek Radosz
5a81e3999c
DEV: Remove bypass_bump from CookedPostProcessor (#9468)
It was only passing it along to `PullHotlinkedImages` and that class have not used that arg since April 2014 (c52ee665b4)
2020-04-21 03:48:19 +02:00
Jarek Radosz
9a6e4b1fa1
FIX: Make InlineUploads handle more URL formats (#9467)
It previously failed to match URLs with characters other than `[a-zA-z0-9\.\/:-]`. This meant that `PullHotlinkedImages` would sometimes download an external image and then never use it in any posts.
2020-04-21 03:47:48 +02:00
Mark VanLandingham
06796bec8b
FIX: Set category description to first posts cooked value (#9478) 2020-04-20 14:51:49 -05:00
Robin Ward
25bed4f643 FIX: Concurrency issues with making topic embedded posts visible 2020-04-20 15:11:59 -04:00
Robin Ward
56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
Robin Ward
8f5314bf98 FIX: An opts hash was not, in fact, optional :) 2020-04-20 14:17:13 -04:00
Daniel Waterworth
7876ee2d67 DEV: upgrade Rails
Latest version of Rails contains compatibility fixes for Ruby 2.7 and some
minor security fixes we would like to have

It also broke some of the multisite tests.

Rails tries to use the same connection for reading from a replica as writing
to the leader during tests, because, with everything happening in a
transaction, changes to the DB wouldn't otherwise be reflected in the
replica connection.

The difference now is that Rails tries to do this for connections opened
after the test has started which affected rails multisite connections.

The upshot of this is that, as things stand, you are likely to
experience problems if you try to connect to a different multisite DB in
a test when the `current_db` is not 'default'.
2020-04-20 12:55:53 +01:00
Sam Saffron
ee36382640
FEATURE: improve rendering of RSS feeds
- Eliminate superfluous "author wrote" block
- Eliminate block-quote for all posts
- Move participant count and reply count to 1 line
- Prioritize name over username if forum requests
- Use fabrication in list controller spec to speed up spec
2020-04-20 16:08:24 +10:00
Martin Brennan
344ef5226c
FEATURE: Edit bookmark reminders from post and explicit delete button (#9455)
There is now an explicit "Delete Bookmark" button in the edit modal. A confirmation is shown before deleting.

Along with this, when the bookmarked post icon is clicked the modal is now shown instead of just deleting the bookmark. Also, the "Delete Bookmark" button from the user bookmark list now confirms the action.

Add a `d d` shortcut in the modal to delete the bookmark.
2020-04-20 13:30:04 +10:00
tshenry
e41f1e6ec2
FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
Mark VanLandingham
0e74dd7d7c
FIX: Set user timezone on password reset login (#9461) 2020-04-17 13:39:04 -05:00
Roman Rizzi
dce46086f4
FIX: Do not raise an error if the post action type is nil (#9458) 2020-04-17 14:23:33 -03:00
David Taylor
0e4497b6be
DEV: Update tl3 spec to remove hard-coded primary keys
In some test runs, the hard-coded ids could correspond to real topics created earlier in the tests, and cause random test failures
2020-04-17 17:24:14 +01:00
Martin Brennan
8f0544137a
FEATURE: Allow editing bookmark reminders (#9437)
Users can now edit the bookmark name and reminder time from their list of bookmarks.

We use "Custom" for the date and time in the modal because if the user set a reminder for "tomorrow" then edit the reminder "tomorrow", the definition of what "tomorrow" is has changed.
2020-04-17 11:08:07 +10:00
Krzysztof Kotlarek
e8e764c064
FIX: flaky groups_controller_spec (#9439)
Sometimes spec which is testing order groups by user count is failing.

My theory is that cause is the randomness of Postgres when the order value is the same for 2 rows.

In spec, we got three groups
`moderator_group` - 0 users
`group` - 1 user
`other_group` - 1 user

And we are expecting that controller will return them in ascending order [moderator, group, other_group]

Because `group` and `other_group` contain the same amount of users, we are dealing with luck

Therefore, I believe that adding one more user to other_group should make that query reliable.

It was not crashing on my local machine, so I am not 100% sure.
2020-04-17 09:04:10 +10:00
David Taylor
b6c19cba20
FIX: Abort emit_web_hook_event job cleanly if web hook was deleted (#9445)
Raising an error causes the job to be retried, and causes a lot of noise in the logs
2020-04-16 21:24:09 +01:00
Neil Lalonde
074509fd95
FIX: don't demote users to TL2 when default trust level is 3
Within 24 hours of signing up, new users were losing their
default trust level of 3. With this fix, demotions from
trust level 3 won't happen when the "default trust level"
setting is 3 or 4.
2020-04-16 12:28:16 -04:00
Martin Brennan
51672b9121
FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
Vinoth Kannan
884eea7a83 FEATURE: add support for upload format in theme settings. 2020-04-15 18:34:02 +05:30
Krzysztof Kotlarek
c21df2286c
FIX: add short_path to upload_serializer (#9417)
What problem I am trying to solve?

When an encrypted message is crafted and the image is added - discourse needs a hard refresh to display that image.

What is happening?

Everything starts here - when the upload is finished we add serialized object to the cache https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/components/composer-editor.js#L748:L757

Then, `discourse-encrypt` is trying to get an image from the cache and use `short_path` property
https://github.com/discourse/discourse-encrypt/blob/master/assets/javascripts/discourse/initializers/hook-decrypt-post.js.es6#L142:L143

Why is it working after a hard refresh?

After refresh, we populate cache once again using that function: https://github.com/discourse/discourse/blob/master/app/assets/javascripts/pretty-text/upload-short-url.js#L11:L17

And lookup_urls method from backend is returning `short_path`
https://github.com/discourse/discourse/blob/master/app/controllers/uploads_controller.rb#L55:L64

TL;DR We should expose short path in upload serializer. I ensured that this serializer is used only when attachments are uploaded so it should not affect performance.
2020-04-15 09:19:59 +10:00
Blake Erickson
fec68d3d25 DEV: Confirm email when creating users via the API
This commit is for a frequently requested task on meta so that only 1
API call is needed instead of 3!

In order to create a user via the api and not have them receive an
activation email you can pass in the `active=true` flag. This prevents
sending an email, but it is only half of the solution and puts the db in
a weird state where it has an active user with an unconfirmed email.

This commit fixes that and ensures that if the `active=true` flag is set
the user's email is also marked as confirmed.

This change only applies to admins using the API.

Related topics on meta:

 - https://meta.discourse.org/t/-/68663
 - https://meta.discourse.org/t/-/33133
 - https://meta.discourse.org/t/-/36133
2020-04-14 14:56:32 -06:00
Mark VanLandingham
40531fc85e
DEV: themes_install_task find existing themes consistently (#9424) 2020-04-14 14:20:21 -05:00
Jeff Wong
e3590d4ead
FEATURE: add user_session_refreshed trigger (#9412)
Trigger an event for plugins to consume when a user session is refreshed.

This allows external auth to be notified about account activity, and be
able to take action such as use oauth refresh tokens to keep oauth
tokens valid.
2020-04-14 09:32:24 -07:00
Mark VanLandingham
f07c4a781c
DEV: Add options to theme install rake task - more options (#9394) 2020-04-14 09:17:00 -05:00
Sam Saffron
6f9177e2ed
FEATURE: enforce_canonical_emails site setting
The new `enforce_canonical_emails` site setting ensures that emails in the
canonical form are unique.

This mean that if `s.a.m+1@gmail.com` is registered `sam@gmail.com` will
not be allowed.

The commit contains a blanket "tag strip" (stripping everything after +)
it also contains special handling of a "dot strip" for googlemail and gmail.

The setting only impacts new registrations after `enforce_canonical_emails`

The setting is default false so it will not impact any existing installs.
2020-04-14 14:16:30 +10:00
Robin Ward
b6b92a562c
FEATURE: New site setting embed_unlisted (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
Justin DiRose
5471c065cd
FIX: Missing timezone guess on email session login (#9404)
Timezone is guessed by moment.js if unset upon a normal login, but was not when
logging in via an email link. This adds logic to update a guessed
timezone upon email login so timezones don't end up blank.
2020-04-10 13:19:39 -05:00
Jeff Wong
5ccbc5f3ee
FEATURE: add after-reviewable-post-user plugin outlet (#9397)
FEATURE: add after-reviewable-post-user plugin outlet

Add a plugin outlet after reviewable post user

Add a basic user serializer that includes custom fields.
Allows review queue serializer to include custom fields for its users
2020-04-09 11:32:39 -07:00
Kane York
138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
Robin Ward
e1f8014acd
FEATURE: Support for publishing topics as pages (#9364)
If the feature is enabled, staff members can construct a URL and publish a
topic for others to browse without the regular Discourse chrome.

This is useful if you want to use Discourse like a CMS and publish
topics as articles, which can then be embedded into other systems.
2020-04-08 12:52:36 -04:00
Roman Rizzi
8e1bdc9458
FIX: Prevent low score flags from auto-closing a topic if the reviewable default visibility is higher than low (#9375) 2020-04-08 10:44:31 -03:00
Jarek Radosz
874999941a
DEV: Fix some more flaky tests (#9384)
Includes:
* DEV: Don't overwrite a computed property in test
2020-04-08 12:46:43 +02:00
Sam Saffron
b824898f61
FIX: respect automatic group membership when sso changes email
Previously we were only updating group membership when a user record was
first created in an SSO setting.

This corrects it so we also update it if SSO changes the email
2020-04-08 16:33:50 +10:00
Sam Saffron
0375a5ac0b
DEV: reduce logging when no external id is specified
Previously we were returning an unknown sso error and logging a message
when external id was blank. This noise is not needed.
2020-04-08 12:42:28 +10:00
Neil Lalonde
c45574c9e7
FIX: exclude private messages from TL3 requirements
It wasn't intended that people should be able to earn trust level
3 without participating in public topics. When counting topic
views and likes given/received, don't count private topics.
2020-04-07 17:36:07 -04:00
Krzysztof Kotlarek
a811976023
FIX: reset gravatar cache by adding random param to URL (#9370)
Users noticed that sometimes, avatar from Gravatar is not correctly updated - https://meta.discourse.org/t/updated-image-on-gravatar-not-seeing-it-update-on-site/54357

A potential reason for that is that even if you update your avatar in Gravatar, URL stays the same and if the cache is involved, service is still receiving the old photo.

For example. In my case, when I click the button to refresh avatar the
new Upload record is created with `origin` URL to new avatar, and `url` to
old one

I made some tests in the rails console and adding random param to Gravatar URL is deceiving cache and correct, the newest avatar is downloaded
2020-04-08 07:35:42 +10:00
Rafael dos Santos Silva
3053c6d95e
FEATURE: Enable offline browsing and fullscreen PWA in iOS by default 2020-04-07 16:58:21 -03:00
Robin Ward
ce663d67e1 FIX: CSV Exports were throwing errors with invalid dates
This fix will consider any invalid dates to be non-existant.
2020-04-07 14:05:27 -04:00
Roman Rizzi
e52e7f6e75
FIX: Include pending queued users regardless of their score (#9372) 2020-04-07 11:42:12 -03:00
Dan Ungureanu
0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
Blake Erickson
d04ba4b3b2
DEPRECATION: Remove support for api creds in query params (#9106)
* DEPRECATION: Remove support for api creds in query params

This commit removes support for api credentials in query params except
for a few whitelisted routes like rss/json feeds and the handle_mail
route.

Several tests were written to valid these changes, but the bulk of the
spec changes are just switching them over to use header based auth so
that they will pass without changing what they were actually testing.

Original commit that notified admins this change was coming was created
over 3 months ago: 2db2003187

* fix tests

* Also allow iCalendar feeds

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2020-04-06 16:55:44 -06:00
Joffrey JAFFEUX
b248c553c2
DEV: ensures CustomEmoji cache is cleared after spec (#9361) 2020-04-06 19:41:59 +02:00
David Taylor
3814ca06a8
DEV: Allow using .js extension for javascript modules in themes (#9358) 2020-04-06 17:24:59 +01:00
Vinoth Kannan
fd39c85c1a FIX: add category hashtags support for sub-sub categories.
Hashtags will include last two levels only (ex: "parent:child").
2020-04-06 20:43:38 +05:30
Krzysztof Kotlarek
ce00da3bcd
FIX: guardian always got user but sometimes it is anonymous (#9342)
* FIX: guardian always got user but sometimes it is anonymous

```
  def initialize(user = nil, request = nil)
    @user = user.presence || AnonymousUser.new
    @request = request
  end
```

AnonymouseUser defines `blank?` method
```
  class AnonymousUser
    def blank?
      true
    end
    ...
  end
```
so if we would use @user.present? it would be correct, however, just @user is always true
2020-04-06 09:56:47 +10:00
Dan Ungureanu
b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
David Taylor
3cfe086a94
FIX: Limit personal message participants when converting from topic (#9343)
Previously all topic posters would be added which could lead to major performance issues. Now if there are too many posters, only the acting user will be added as a participant.
2020-04-03 16:42:01 +01:00
Vinoth Kannan
82201fa466 FIX: jobs/delete_replies: Add Time+Duration, not Time+Time #9314
Co-authored-by: Kane York <kanepyork@gmail.com>
2020-04-03 09:23:40 +05:30
Vinoth Kannan
2f9879a9a1 FIX: move total rows count & load more URL inside meta.
We're fetching "total rows count" from root attributes only if meta object not found. 2b78bd01ab/app/assets/javascripts/discourse/models/store.js (L236)
2020-04-03 07:32:50 +05:30
Daniel Waterworth
76610acb6f FIX: Default to light theme in wizard so that previews are displayed
Previously, without a theme selection, the previews wouldn't show.
2020-04-02 18:37:45 +01:00
Penar Musaraj
724d2e99de
DEV: Only include "report-sample" CSP directive when reporting is enabled (#9337) 2020-04-02 11:16:38 -04:00
Kane York
cdaa60b56b FEATURE: Allow admins to disable self-service account deletion
https://meta.discourse.org/t/-/146276
2020-04-01 15:16:07 -07:00
Neil Lalonde
90fcede832
FIX: replace default welcome topic post with new value from wizard
Previously the text entered in the wizard would be prepended onto the
default first paragraph.
2020-04-01 15:42:45 -04:00
Mark VanLandingham
689c61b462
DEV: Allow plugins to add wizard steps after specific steps (#9315) 2020-04-01 08:36:50 -05:00
Roman Rizzi
b2f30aa0b5
FIX: FlagSockpuppets should not flag a post if a post of that user was already rejected by staff (#9328)
* FIX: FlagSockpuppets should not flag a post if a post of that user was already rejected by staff

* Update spec/services/flag_sockpuppets_spec.rb

Co-Authored-By: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-04-01 09:20:45 -03:00
Martin Brennan
c07dd0d22a
FEATURE: Add lazy loading to user bookmarks list (#9317)
This is so users with huge amount of bookmarks do not have to wait a long time to see results.

* Add a bookmark list and list serializer to server-side to be able to handle paging and load more URL
* Use load-more component to load more bookmark items, 20 at a time in user activity
* Change the way current user is loaded for bookmark ember models because it was breaking/losing resolvedTimezone when loading more items
2020-04-01 14:09:07 +10:00
Blake Erickson
6b20d52338 FIX: Staged users getting user_linked and user_quoted emails
This fix ensures that if a staged user is linked to or quoted they won't
be emailed about it.

A staged user could email into a category, and another user could quote
them inside of a completely different category and we don't want a
staged user to receive an email for this.

Bug report:

https://meta.discourse.org/t/-/145202/9
2020-03-31 21:00:56 -06:00
Blake Erickson
04b431b6a4 FIX: Staged users getting user_linked and user_quoted emails
This fix ensures that if a staged user is linked to or quoted they won't
be emailed about it.

A staged user could email into a category, and another user could quote
them inside of a completely different category and we don't want a
staged user to receive an email for this.

Bug report:

https://meta.discourse.org/t/-/145202/9
2020-03-31 20:17:56 -06:00
Martin Brennan
b79ea986ac
FEATURE: High priority bookmark reminder notifications (#9290)
Introduce the concept of "high priority notifications" which include PM and bookmark reminder notifications. Now bookmark reminder notifications act in the same way as PM notifications (float to top of recent list, show in the green bubble) and most instances of unread_private_messages in the UI have been replaced with unread_high_priority_notifications.

The user email digest is changed to just have a section about unread high priority notifications, the unread PM section has been removed.

A high_priority boolean column has been added to the Notification table and relevant indices added to account for it.

unread_private_messages has been kept on the User model purely for backwards compat, but now just returns unread_high_priority_notifications count so this may cause some inconsistencies in the UI.
2020-04-01 09:09:20 +10:00
Arpit Jalan
b2a0d34bb7
FEATURE: add setting auto_approve_email_domains to auto approve users (#9323)
* FEATURE: add setting `auto_approve_email_domains` to auto approve users

This commit adds a new site setting `auto_approve_email_domains` to
auto approve users based on their email address domain.

Note that if a domain already exists in `email_domains_whitelist` then
`auto_approve_email_domains` needs to be duplicated there as well,
since users won’t be able to register with email address that is
not allowed in `email_domains_whitelist`.

* Update config/locales/server.en.yml

Co-Authored-By: Robin Ward <robin.ward@gmail.com>
2020-03-31 23:59:15 +05:30
Krzysztof Kotlarek
a0f59a55cc
FIX: when a post is moved copy notifications level (#9311)
This is a revert of 2c011252f1

More information on meta: https://meta.discourse.org/t/when-a-reply-is-moved-to-a-new-topic-the-followers-of-the-previous-topic-are-automatically-follower-of-the-new-topic-as-well/130025
2020-03-31 16:19:47 +11:00
Neil Lalonde
29e733832d
FIX: error when changing a topic's category and creating a tag 2020-03-30 16:43:01 -04:00
Robin Ward
db35baba26 FEATURE: Display "Last Updated At" on user directory 2020-03-30 14:34:48 -04:00
Joffrey JAFFEUX
0996c3b7b3
FEATURE: allows multiple custom emoji groups (#9308)
Note: DBHelper would fail with a sql syntax error on columns like "group".

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2020-03-30 20:16:10 +02:00
Joffrey JAFFEUX
aeaea3c154
FIX: correctly take category/group filters into csv export (#9300) 2020-03-30 19:08:47 +02:00
Joffrey JAFFEUX
96e841a635
DEV: trigger :username_changed when a user is renamed (#9301) 2020-03-30 18:02:34 +02:00
Jarek Radosz
104b647ca4 DEV: Fix another time-sensitive flaky spec 2020-03-27 20:39:25 +01:00
Michael Brown
9026c55fe4
FIX: custom SQL with a trailing comment might break BadgeGranter SQL (#9285)
For example given a custom badge with SQL:
```
SELECT 1
-- I am a comment
```

You end up with

```
FROM (SELECT 1
-- I am a comment) q
```

This fix adds newlines so you end up with the now-valid:

```
FROM (
  SELECT 1
-- I am a comment
) q
```
2020-03-27 14:16:14 -04:00
Jarek Radosz
7ff889574d
DEV: Add rubocop-rspec (#9288)
This adds rubocop-rspec, and enables some cops that were either already passing or are passing now, after fixing them in this commit.

Some new cops are disabled for now, with annotation: "TODO" or "To be decided". Those either need to be discussed first, or require manual changes, or the number of found and fixed offenses is too large to bundle them up in a single PR.

Includes:

* DEV: Update rubocop's `TargetRubyVersion` to 2.6
* DEV: Enable RSpec/VoidExpect
* DEV: Enable RSpec/SharedContext
* DEV: Enable RSpec/EmptyExampleGroup (Removed an obsolete empty spec file)
* DEV: Enable RSpec/ItBehavesLike
* DEV: Remove RSpec/ScatteredLet (It's too strict, as it doesn't recognize fab! as a let-like)
* DEV: Remove RSpec/MultipleExpectations
2020-03-27 17:35:40 +01:00
Sam Saffron
40b6e278a0
FIX: ensure wiki editor is assigned consistently
`DISTINCT ON` was not providing stability, this ensures the backfill will
be stable on a specific post avoiding risk around double grants.
2020-03-27 12:41:06 +11:00
Jarek Radosz
d21d80198c
DEV: Update rubocop-discourse (#9270)
Includes:
* DEV: Use `eq_time` matcher
2020-03-26 16:32:41 +01:00
Bianca Nenciu
7952cbb9a2
FIX: Perform crop using user-specified image sizes (#9224)
* FIX: Perform crop using user-specified image sizes

It used to resize the images to max width and height first and then
perform the crop operation. This is wrong because it ignored the user
specified image sizes from the Markdown.

* DEV: Use real images in test
2020-03-26 16:40:00 +02:00
Sam Saffron
25f1f23288
FEATURE: Stricter rules for user presence
Previously we would consider a user "present" and "last seen" if the
browser window was visible.

This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.

Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility

This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
2020-03-26 17:36:52 +11:00
Martin Brennan
efd5fb665b
DEV: Fix flaky time sensitive uploads.rake specs (#9283)
Also fix issues in spec where certain uploads were not considered secure
2020-03-26 13:31:39 +10:00
Sam Saffron
ecbccab159
FEATURE: invite_code is case-insensitive
Previously we required exact casing for invite code, this can cause a lot
of confusion. Relax the requirement.
2020-03-26 13:44:02 +11:00
Jeff Wong
7817d8b2ff FEATURE: Unassign the review queue topic when a flag is handled 2020-03-25 14:17:08 -07:00
Martin Brennan
097851c135
FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
Mark VanLandingham
c14f6d4ced
FEATURE: Allow plugins to exclude wizard steps (#9275) 2020-03-25 11:36:42 -05:00
Jeff Wong
7282435e17 FEATURE: Unassign the review queue topic when a flag is handled 2020-03-24 14:57:44 -07:00
Bianca Nenciu
fe15082d44 SECURITY: Ensure user can see group and group members 2020-03-24 12:23:11 +02:00
Bianca Nenciu
61c1af0124 SECURITY: Ensure user can see group and group members 2020-03-24 11:59:41 +02:00
Bianca Nenciu
d8640fd042
DEV: Move requested_group_id custom field from post to topic (#9127)
Follow-up-to accbbded15
2020-03-24 11:12:52 +02:00
Sam Saffron
46a9622246
FIX: prevent scheduled publishing to deleted category
We missed a dependency which left timers firing on missing categories.

Co-authored-by: tshenry
2020-03-24 16:59:42 +11:00