Commit Graph

155 Commits

Author SHA1 Message Date
David Taylor
03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Joffrey JAFFEUX
01f2819dde
DEV: enables and fixes multisite-spec (#9557) 2020-04-27 20:55:36 +02:00
Dan Ungureanu
0653750fbf
FEATURE: Hash user API keys in the database (#9344)
The 'key' column will be dropped in a future commit.
2020-04-07 16:42:52 +03:00
Blake Erickson
d04ba4b3b2
DEPRECATION: Remove support for api creds in query params (#9106)
* DEPRECATION: Remove support for api creds in query params

This commit removes support for api credentials in query params except
for a few whitelisted routes like rss/json feeds and the handle_mail
route.

Several tests were written to valid these changes, but the bulk of the
spec changes are just switching them over to use header based auth so
that they will pass without changing what they were actually testing.

Original commit that notified admins this change was coming was created
over 3 months ago: 2db2003187

* fix tests

* Also allow iCalendar feeds

Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2020-04-06 16:55:44 -06:00
David Taylor
19814c5e81
FIX: Allow CSP to work correctly for non-default hostnames/schemes (#9180)
- Define the CSP based on the requested domain / scheme (respecting force_https)
- Update EnforceHostname middleware to allow secondary domains, add specs
- Add URL scheme to anon cache key so that CSP headers are cached correctly
2020-03-19 19:54:42 +00:00
Jarek Radosz
29b35aa64c
DEV: Improve flaky time-sensitive specs (#9141) 2020-03-10 22:13:17 +01:00
Bianca Nenciu
3ec2081059
UX: Include public groups in mentionable groups set (#8516) 2019-12-12 13:13:40 +02:00
Neil Lalonde
875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
David Taylor
a6aada16bd DEV: Refactor API key specs to avoid hard-coding keys
By hard-coding keys, we are not testing the API key system end to end. This change also makes the specs more resilient to upcoming API key changes
2019-11-29 15:16:22 +00:00
Neil Lalonde
228c4814be FIX: errors when using tags with colons in their name 2019-11-18 13:20:37 -05:00
Neil Lalonde
7711df40e6
REFACTOR: redo DiscourseTagging.filter_allowed_tags (#8328)
This method had grown into a monster. Its query had bugs
that I couldn't fix, and new features would be hard to add.
Also I don't understand how it all works anymore...
Replace it with common table expressions that can be queried
to generate the results we need, instead of subtracting
results using lots of "NOT IN" clauses.

Fixed are bugs with tag schemas that use combinations of
tag groups, parent tags, and one-tag-per-topic restrictions.
For example: https://meta.discourse.org/t/130991/6
2019-11-12 14:28:44 -05:00
Neil Lalonde
b6d140e4bd UX: tag input suggests required tags if none have been selected
This is a follow-up to the new feature that allows a category to
require a certain number of tags from a tag group. The tag input will
shows results from the required group if none have been chosen yet.
Once a require tag is selected, the tag input will include other
results as usual. Staff users can ignore this restriction, so the input
behaviour is unchanged for them.
2019-11-04 16:51:54 -05:00
Neil Lalonde
cfe26eb301 FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
Neil Lalonde
f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Neil Lalonde
5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
David Taylor
92f2202e4a SECURITY: Restrict message-bus access on login_required sites 2019-08-14 09:43:12 +01:00
Sam Saffron
1f47ed1ea3 PERF: message_bus will be deferred by server when flooded
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)

This back-off is ultra cheap and happens very early in the middleware.

It corrects a situation where a flood to message bus could cause the app
to become unresponsive

MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.

Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
2019-08-09 17:48:01 +10:00
Neil Lalonde
9656a21fdb
FEATURE: customization of html emails (#7934)
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.

Customizations can be added and edited in Admin > Customize > Email Style.

Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.

As part of this work, RTL locales are now rendered correctly for all emails.
2019-07-30 15:05:08 -04:00
Gerhard Schlager
abe6202af9 DEV: Fix heisentest 2019-07-23 18:04:53 +02:00
Osama Sayegh
f14c6d81f4
FEATURE: Watched words improvements (#7899)
This commit contains 3 features:

- FEATURE: Allow downloading watched words
This introduces a button that allows admins to download watched words per action in a `.txt` file.

- FEATURE: Allow clearing watched words in bulk
This adds a "Clear All" button that clears all deleted words per action (e.g. block, flag etc.)

- FEATURE: List all blocked words contained in the post when it's blocked
When a post is rejected because it contains one or more blocked words, the error message now lists all the blocked words contained in the post.

-------

This also changes the format of the file for importing watched words from `.csv` to `.txt` so it becomes inconsistent with the extension of the file when watched words are exported.
2019-07-22 14:59:56 +03:00
Guo Xiang Tan
a4234e9be0 DEV: Minor tweaks to Admin::WatchedWordsController. 2019-07-15 10:22:46 +08:00
Sam Saffron
bb8dfb124e DEV: change testing cache clearing so it is more deliberate
This test is flaky, this changes it so we always start tests with a clean
cache (concurrency and redis delays pending)
2019-06-03 10:21:38 +10:00
Robin Ward
89b84651c3 Migrate score settings to use sensitivities
We hide scores so these settings no longer made sense.
2019-05-24 15:44:24 -04:00
David Taylor
b5dd4478e5
FIX: Blocked watched words should apply to staff (#7547)
Having different behavior for staff and regular users can make it confusing for admins to understand how their configuration changes affect regular users
2019-05-16 15:19:41 +01:00
Sam Saffron
88650a1259 PERF: avoid checking for consecutive replies in test
This check can issue up to 2 queries per post created, we have specific
tests for it so we can avoid.

This also rolls back #4da6ca4d
2019-05-09 13:28:53 +10:00
Daniel Waterworth
4da6ca4d9f PERF: Skip post validation by default when fabricating posts (#7508)
This speeds up tests by 10%
2019-05-09 10:15:38 +10:00
Daniel Waterworth
e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Sam Saffron
45285f1477 DEV: remove update_attributes which is deprecated in Rails 6
See: https://github.com/rails/rails/pull/31998

update_attributes is a relic of the past, it should no longer be used.
2019-04-29 17:32:25 +10:00
Daniel Waterworth
ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Neil Lalonde
c2a8a2bc97 FIX: if mandatory parent tag is missing, add it
Previous behaviour was to silently remove tags that
belonged to a group with a parent tag that was missing.

The "required parent tag" feature is meant to guide people
to use the correct tags and show scoped results in the tag
input field, and to help create topic lists of related
tags. It isn't meant to be a strict requirement in the
composer that should trigger errors or restrictions.
2019-04-26 14:46:11 -04:00
Robin Ward
87de5d9e8f Record the reason that a post was put into the queue 2019-04-10 17:42:49 -04:00
Neil Lalonde
a0f7953f0c remove extra lines 2019-04-04 17:03:57 -04:00
Neil Lalonde
47eefb9f37 remove wip comments 2019-04-04 16:47:02 -04:00
Neil Lalonde
83996fc8ea FEATURE: ability to restrict some tags to a category while allowing all others
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
2019-04-04 16:40:15 -04:00
Robin Ward
c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron
9ebabc1de8 FEATURE: unconditionally update Topic updated_at when posts change in topic
Previously we would bypass touching `Topic.updated_at` for whispers and post
recovery / deletions.

This meant that certain types of caching can not be done where we rely on
this information for cache accuracy.

For example if we know we have zero unread topics as of yesterday and whisper
is made I need to bump this date so the cache remains accurate

This is only half of a larger change but provides the groundwork.

Confirmed none of our serializers leak out Topic.updated_at so this is safe
spot for this info

At the moment edits still do not change this but it is not relevant for the
unread cache.

This commit also cleans up some specs to use the new `eq_time` matcher for
millisecond fidelity comparison of times

Previously `freeze_time` would fudge this which is not that clean.
2019-03-28 17:28:01 +11:00
Neil Lalonde
1812a38f0a FIX: upload watched words should use UTF-8 2019-03-21 13:46:16 -04:00
Robin Ward
fa5a158683 REFACTOR: Move queue_jobs out of SiteSetting
It is not a setting, and only relevant in specs. The new API is:

```
Jobs.run_later!        # jobs will be thrown on the queue
Jobs.run_immediately!  # jobs will run right away, avoid the queue
```
2019-03-14 10:47:38 -04:00
Robin Ward
d1d9a4f128 Add new run_jobs_synchronously! helper for tests
Previously if you wanted to have jobs execute in test mode, you'd have
to do `SiteSetting.queue_jobs = false`, because the opposite of queue
is to execute.

I found this very confusing, so I created a test helper called
`run_jobs_synchronously!` which is much more clear about what it does.
2019-03-11 16:58:35 -04:00
Robin Ward
bc3efab816 FIX: When disagreeing with a flag that silenced a user, unsilence them
Previously it would unhide their post but leave them silenced.

This fix also cleans up some of the helper classes to make it easier
to pass extra data to the silencing code (for example, a link to the
post that caused the user to be silenced.)

This patch also refactors the auto_silence specs to avoid using
stubs.
2019-02-08 08:50:50 -05:00
Bianca Nenciu
860c1c3dcd FEATURE: Automatically expire keys if not used for a configurable amount of time. (#6264) 2018-08-20 17:36:14 +02:00
Régis Hanol
93201d8dbe FIX: don't trigger 'flag_reviewed' when no flags were reviewed 2018-08-16 18:11:29 +02:00
Guo Xiang Tan
7fc8a36529 DEV: Take 2 Queue jobs in tests by default.
On my machine this cuts the time taken to run our test suite
from ~11mins to ~9mins.
2018-05-31 16:23:23 +08:00
Guo Xiang Tan
56e9ff6853 Revert "DEV: Queue jobs in tests by default."
Too risky for now

This reverts commit be28154d3b.
2018-05-31 15:34:46 +08:00
Guo Xiang Tan
be28154d3b DEV: Queue jobs in tests by default. 2018-05-31 14:45:47 +08:00
Neil Lalonde
6ca71e1319 FIX: when creating a topic in a category that only allows tags from a tag group, don't allow creation of new tags 2018-03-19 11:42:10 -04:00
Neil Lalonde
58508e553d FIX: tag input should not include tags you've already chosen in the search results 2018-03-13 17:17:16 -04:00