Commit Graph

7353 Commits

Author SHA1 Message Date
Guo Xiang Tan
d8cd912769
DEV: Switch to db config to disable advisory locks. 2020-06-15 14:33:41 +08:00
Guo Xiang Tan
0ff86b00cb
DEV: Upgrade Redis to 4.2.1. 2020-06-15 10:05:22 +08:00
Gerhard Schlager
36a3675e0a Update translations 2020-06-14 23:39:33 +02:00
Joffrey JAFFEUX
4b793a1072
FIX: allows PM owner to remove any user if >= TL2 (#10036) 2020-06-12 12:54:28 +02:00
Andrew Schleifer
b2c94cc8ea FIX: do not migrate backups in the new prefix 2020-06-12 02:56:07 +00:00
Guo Xiang Tan
78b5ab746c
DEV: No longer need to clear anon cache when toggling readonly mode. 2020-06-12 09:58:17 +08:00
Roman Rizzi
a4bfa35420
DEV: Pass the is_image flag when triggering the before_upload_creation event (#10031) 2020-06-11 11:06:48 -03:00
Jarek Radosz
3d55f2e3b7
FIX: Improvements and fixes to the image downsizing script (#9950)
Fixed bugs, added specs, extracted the upload downsizing code to a class, added support for non-S3 setups, changed it so that images aren't downloaded twice.

This code has been tested on production and successfully resized ~180k uploads.

Includes:

* DEV: Extract upload downsizing logic
* DEV: Add support for non-S3 uploads
* DEV: Process only images uploaded by users
* FIX: Incorrect usage of `count` and `exist?` typo
* DEV: Spec S3 image downsizing
* DEV: Avoid downloading images twice
* DEV: Update filesizes earlier in the process
* DEV: Return false on invalid upload
* FIX: Download images that currently above the limit (If the image size limit is decreased, then there was no way to resize those images that now fall outside the allowed size range)
* Update script/downsize_uploads.rb (Co-authored-by: Régis Hanol <regis@hanol.fr>)
2020-06-11 14:47:59 +02:00
Dan Ungureanu
b7e70850e4
FIX: Allow users to add emails which were deleted before 2020-06-11 14:54:11 +03:00
Guo Xiang Tan
2e8075bac3
FIX: Ignore Redis readonly errors in RateLimiter#rollback!.
This is similar to what we're doing in `RateLimiter#performed!`.
2020-06-11 15:13:11 +08:00
Guo Xiang Tan
90dc6c8ce5
DEV: Fix deprecation warning when using Redis#close with redis namespace. 2020-06-11 14:09:19 +08:00
Guo Xiang Tan
531eca985e
DEV: Log error backtrace in Discourse.warn_exception without logster. 2020-06-11 10:49:46 +08:00
Kane York
5f3fd23fa8
DEV: Add stub flush_sw task to prevent rebuild errors (#10021) 2020-06-10 13:07:37 -07:00
romanrizzi
ac555610d9 DEV: Trigger an event before creating an upload 2020-06-10 16:30:53 -03:00
Neil Lalonde
81a265cecd
Version bump to v2.5.0.beta7 2020-06-10 13:38:18 -04:00
Dan Ungureanu
5bfe1ee4f1
FEATURE: Improve UX support for multiple email addresses (#9691) 2020-06-10 19:11:49 +03:00
Guo Xiang Tan
a3dfd553a1
Revert "Bump redis to 4.2.0."
This reverts commit 98bc28cea2.
2020-06-10 14:52:05 +08:00
Guo Xiang Tan
98bc28cea2
Bump redis to 4.2.0. 2020-06-10 14:28:56 +08:00
Jeff Wong
70a88111dd
FIX: prevent re-flagging when we have reviewed flags before (#10010)
FIX: prevent re-flagging when we have reviewed flags before

Fixes an edge case where a review can be reflagged when:
User flags as inappropriate.
Moderator rejects the flag.
Another user re-flags the post as spam.

Before, anyone was able to re-flag as inappropriate despite it being flagged
previously. With this, users are unable to re-flag for the same reason
regardless of reviewable status.
2020-06-09 15:26:10 -07:00
Arpit Jalan
3094459cd9
FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
Guo Xiang Tan
c35649ccf1
DEV: Remove Redis noop logging.
When Redis drops into readonly mode, it floods the log. The log isn't
actionable and we have other means of knowing that an app is in readonly
mode.
2020-06-09 16:45:34 +08:00
Guo Xiang Tan
a1c13eb3c6
DEV: Redis failover should only clear redis recently readonly. 2020-06-09 16:36:31 +08:00
Guo Xiang Tan
4065cd198a
DEV: Print backtrace of error when plugin fails to initialize. 2020-06-09 10:25:43 +08:00
Joffrey JAFFEUX
d4caf69ed7
DEV: makes SKIP_INSTALL_PLUGINS called last and global (#9990) 2020-06-05 17:59:23 +02:00
Sam Saffron
1c48853ede
DEV: allow skipping of a single plugin when installing
SKIP_INSTALL_PLUGINS can be used to tell plugin:install_all_official
to skip a plugin.

Comma seperated list
2020-06-05 18:26:25 +10:00
Guo Xiang Tan
e82d4d8a75
DEV: Update rails_failover to avoid monkey patching Rails config. 2020-06-05 09:05:19 +08:00
Martin Brennan
36841210fb Move UploadSecurity public types into PUBLIC_TYPES constant
* this is so plugins can extend the public types if required
2020-06-05 10:56:57 +10:00
Penar Musaraj
2d880b42a3
UX: Add simple-list setting type (#9970) 2020-06-04 10:44:54 -04:00
Joshua Rosenfeld
76af25f753
FEATURE: Support append when bulk tagging via rake (#9978)
New argument option for the `tags:bulk_tag_category` rake task.
Backwards compatible - defaults to false (no append) if no argument given.
2020-06-04 09:33:48 -04:00
Guo Xiang Tan
248db310d3 DEV: Fix lint. 2020-06-04 17:25:00 +08:00
Guo Xiang Tan
aaece34e8b DEV: Update rails_failover so that we can move middleware up the stack. 2020-06-04 17:14:13 +08:00
Guo Xiang Tan
8e1681d356 Bump rails_failover. 2020-06-04 15:22:35 +08:00
Arpit Jalan
07774894a6 DEV: notify when the rake ask is completed 2020-06-04 12:00:06 +05:30
Arpit Jalan
a93d24501c FIX: base import script was not updating first_post_created_at column
FEATURE: new rake task to update first_post_created_at column

The not-equal operator (`<>`) in PostgreSQL does not compare values
with NULL. We should instead use `IS DISTINCT FROM` when comparing
values with NULL.
2020-06-04 11:26:40 +05:30
Guo Xiang Tan
2188ccccd5 DEV: Remove initiating_user keyword arg from EmailUpdater.
The guardian contains the acting user.
2020-06-04 13:21:56 +08:00
Guo Xiang Tan
d3c972c30c FIX: Rescue from readonly errors when looking up auth tokens.
Since this is rare, we don't want to check for
`Discourse.pg_readonly_mode?` on every request since we have to reach
for Redis. Instead, just rescue the error here.
2020-06-04 09:12:49 +08:00
Michael K Johnson
81e6bc7a0f
FEATURE: Add uploads:batch_migrate_from_s3 task to limit total posts migrated at once (#9933)
Allow limiting the number of migrations to do at once, both to do migrations that
have impact limited to multiple off-peak usage hours to reduce user impact from
a migration, and to allow tests that do only a very small number for test
purposes. ("Give me a ping, Vasili. One ping only, please.")
2020-06-04 09:48:11 +10:00
Blake Erickson
a89574ccb9 FIX: Inline error when converting html to markdown
Looks like some html elements like `aside` and `section` will throw an error
when checking if they are inline or not. The commit simply handles

```
Job exception: undefined method `inline?' for nil:NilClass
```

and adds a test for it.
2020-06-03 15:59:19 -06:00
Gerhard Schlager
f683c5d0e0 DEV: Check English locale for errors in CI
Moves the most important checks into a linter. It gets executed by Lefthook as well as the docker rake task and Github actions. Doing those checks in rspec takes too long and it produces errors when the discourse:test Docker image contains old, invalid locale files.
2020-06-03 21:54:58 +02:00
Robin Ward
ba00cc8ec4
DEV: Move Discourse.getURL and related functions to a module (#9966)
* DEV: Move `Discourse.getURL` and related functions to a module

* DEV: Remove `Discourse.getURL` and `Discourse.getURLWithCDN`

* FIX: `get-url` is required for server side code

* DEV: Deprecate `BaseUri` too.
2020-06-03 12:45:26 -04:00
Guo Xiang Tan
76ce814a62
DEV: Use safe_each_connection patch when preloading Rails.
This is to ensure we still boot even if there is a problem with any of
the databases.
2020-06-03 15:36:50 +08:00
Sam Saffron
57a3d4e0d2
FEATURE: whitelist theme repo mode (experimental)
In some restricted setups all JS payloads need tight control.

This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.

There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.

Use an example like this to enable:

`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`

By default this feature is not enabled and no changes are made.

One exception is that default theme id was missing a security check
this was added for correctness.
2020-06-03 13:19:57 +10:00
Guo Xiang Tan
062db10c52
FIX: EmailValidator needs to validate format of email. 2020-06-03 10:34:37 +08:00
Robin Ward
4e3a84c687 FIX: If creating a flag for a watched word, include the reason 2020-06-02 11:49:02 -04:00
Guo Xiang Tan
439db7ca1e
DEV: Add REDIS_RAILS_FAILOVER env to test our new redis failover. 2020-06-02 17:24:14 +08:00
Aman Gupta Karmani
6358e79f0f
FIX: catch NoMethodError during precompile_css too (#9940)
Ran into this on heroku:

    NoMethodError: undefined method `enabled?' for #<Theme:0x0000558f069e7718>
    vendor/bundle/ruby/2.6.0/gems/activemodel-6.0.1/lib/active_model/attribute_methods.rb:431:in `method_missing'
    app/models/theme.rb:155:in `block (2 levels) in transform_ids'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/delegation.rb:85:in `each'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/delegation.rb:85:in `each'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/query_methods.rb:260:in `select'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/query_methods.rb:260:in `select'
    app/models/theme.rb:155:in `block in transform_ids'
    app/models/theme.rb:105:in `get_set_cache'
    app/models/theme.rb:140:in `transform_ids'
    app/models/theme.rb:321:in `list_baked_fields'
    app/models/theme.rb:317:in `resolve_baked_field'
    lib/stylesheet/manager.rb:289:in `theme_digest'
2020-06-02 15:18:03 +10:00
Guo Xiang Tan
fe9c82994d
DEV: Actually disconnect from Redis connections after fork. 2020-06-02 11:40:16 +08:00
Neil Lalonde
09dc5eb5ea
Version bump to v2.5.0.beta6 2020-06-01 14:13:48 -04:00
Neil Lalonde
53c7d0200a
Version bump to v2.4.5 2020-06-01 14:05:48 -04:00
Guo Xiang Tan
389bdcf5ab
DEV: Fix implementation for DiscourseRedis#exists. 2020-06-01 13:11:32 +08:00
Jarek Radosz
27ad562ff5 DEV: Rubocop fix 2020-06-01 06:07:07 +02:00
Jarek Radosz
7df688d108 FIX: Handle files removed between glob and mtime 2020-06-01 05:50:50 +02:00
Guo Xiang Tan
df62407f35
DEV: Implement multiple keys support for DiscourseRedis#exists. 2020-06-01 11:20:26 +08:00
Guo Xiang Tan
3894555b2e
DEV: Prefer Discourse.redis.reconnct. 2020-06-01 10:55:53 +08:00
Robin Ward
2b2434b82d
Start Discourse in an initializer (#9930)
* DEV: To be pedantic, there is more than EMBER in there now

* DEV: Use less globals. Have `Discourse` start in an initializer

* DEV: Remove another global
2020-05-29 14:37:02 -04:00
David Taylor
e159fb06df
FEATURE: Download remote images even for old posts (#9925)
When a post is rebaked, the admins expect it to work the same regardless of how old the post is.
2020-05-29 17:13:55 +01:00
David Taylor
28f46c171c
FIX: Pull hotlinked images even when edited by system users (#9890)
Previously the pull hotlinked images job was skipped after system edits. This ensured that we never had an infinite loop of system-edit/pull-hotlinked/system-edit/pull-hotlinked etc.

A side effect was that edits made by system for any other reason (e.g. API, removing full quotes) would prevent pulling hotlinked images. This commit removes the system edit check, and replaces it with another method to avoid an infinite job scheduling loop.
2020-05-29 13:07:47 +01:00
Joffrey JAFFEUX
7635c18a14
DEV: ensures highlightjs is correctly tested (#9923) 2020-05-29 13:05:44 +02:00
Guo Xiang Tan
3b311f6b15
DEV: Bump rails_failover. 2020-05-29 09:02:34 +08:00
Vinoth Kannan
ce1491e830
UX: remove in:unpinned filter from advanced search page. (#9911) 2020-05-29 00:47:28 +05:30
Andrew Schleifer
74d28a43d1
new S3 backup layout (#9830)
* DEV: new S3 backup layout

Currently, with $S3_BACKUP_BUCKET of "bucket/backups", multisite backups
end up in "bucket/backups/backups/dbname/" and single-site will be in
"bucket/backups/".

Both _should_ be in "bucket/backups/dbname/"

- remove MULTISITE_PREFIX,
- always include dbname,
- method to move to the new prefix
- job to call the method

* SPEC: add tests for `VacateLegacyPrefixBackups` onceoff job.

Co-authored-by: Vinoth Kannan <vinothkannan@vinkas.com>
2020-05-29 00:28:23 +05:30
Guo Xiang Tan
6491db579b
DEV: Add discourse-staff-alias to official plugin list. 2020-05-28 16:43:15 +08:00
Blake Erickson
6548cd1a96 FIX: sending messages to groups with non-lowercase names
Fixes a regression in

e8fb9d4066

which caused a bug where you couldn't send a message to a group that
contained an Uppercase letter. Added a test case for this.

Bug report: https://meta.discourse.org/t/-/152999
2020-05-27 14:52:08 -06:00
Joffrey JAFFEUX
77801aa9be
FIX: allows to have custom emoji translation without static file (#9893) 2020-05-27 20:11:52 +02:00
Joffrey JAFFEUX
1d685c22af
REVERT: removes translate_emoji (#9889)
This API is actually used in some plugins.
2020-05-27 12:08:24 +02:00
Penar Musaraj
b1c726be0d
Remove support for FontAwesome 4.7 icon names (#9871) 2020-05-26 14:53:32 -04:00
Gerhard Schlager
69ee94b526 FIX: XML files could be detected as SVG files 2020-05-26 18:18:20 +02:00
Jeff Wong
4dc6504234 DEV: Mark fontawesome-pro plugin as official 2020-05-26 08:50:55 -07:00
Neil Lalonde
788b8becde
Version bump to v2.5.0.beta5 2020-05-26 11:13:05 -04:00
Neil Lalonde
fe275c97c1
Version bump to v2.4.4 2020-05-26 10:31:32 -04:00
Roman Rizzi
b61a291cf3
FIX: returns false if the upload url is an invalid mailto link (#9877) 2020-05-26 10:32:48 -03:00
Guo Xiang Tan
878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
Sam Saffron
fc97f7e0e7
FIX: properly ban non human users from draft system
Previously we had a partial fix in place where non human users
were not allowed draft sequences, this left edges around where non
human users asked for drafts yet had none.

For example system could already have a few drafts in place.

This also removes and extensibility point we added that is not in use
2020-05-26 10:07:09 +10:00
Rafael dos Santos Silva
b48299f81c
FEATURE: Add setting to disable automatic CORS rule install in S3 buckets (#9872) 2020-05-25 17:09:34 -03:00
Arpit Jalan
e8fb9d4066 FIX: when creating new PM username/groupname should be case-insensitive
(take 2)

https://meta.discourse.org/t/case-sensitivity-in-links-to-groupname/147596
https://meta.discourse.org/t/remove-case-sensitive-in-adding-users-to-a-message/151275
2020-05-25 21:34:05 +05:30
Arpit Jalan
302b37c805 Revert "FIX: when creating new PM username/groupname should be case-insensitive"
This reverts commit 2be79d94f5.

This is affecting multiple code path. Investigating.
2020-05-25 20:10:14 +05:30
Arpit Jalan
30849c8b37 FIX: no need for downcasing second time 2020-05-25 19:20:15 +05:30
Arpit Jalan
2be79d94f5 FIX: when creating new PM username/groupname should be case-insensitive
https://meta.discourse.org/t/case-sensitivity-in-links-to-groupname/147596
https://meta.discourse.org/t/remove-case-sensitive-in-adding-users-to-a-message/151275
2020-05-25 19:04:59 +05:30
Vinoth Kannan
8e56197728
UX: use "icon-picker" & "image-uploader" fields to set group flair. (#9779) 2020-05-25 11:08:47 +05:30
Michael Brown
d9a02d1336
Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eee.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
Jeff Atwood
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6f, reversing
changes made to 2660c2e21d.
2020-05-22 20:25:56 -07:00
Blake Erickson
7f6a321fec SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 15:15:44 -06:00
Blake Erickson
03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
Martin Brennan
f9d55b4941
FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
Mark VanLandingham
bd57ae83f1
DEV: Guardian for hiding about stats (#9841) 2020-05-20 10:26:20 -05:00
Arpit Jalan
4a533ec594
FIX: do not parameterize tag_id (#9839)
Parameterizing tag_id was breaking tags with non-ascii characters or emoji.

Bug report: https://meta.discourse.org/t/unable-to-see-pm-lists-for-non-ascii-tag/151723/4
2020-05-20 19:22:20 +05:30
Osama Sayegh
02f44def56
FIX: Don't blow up when trying to parse invalid or non-ASCII URLs (#9838)
* FIX: Don't blow up when trying to parseinvalid or non-ASCII URLs

Follow-up to 72f139191e
2020-05-20 12:46:27 +03:00
Sam Saffron
b031e3220a
FIX: migrations should not fail when db is part migrated
When part migrated the translate overrides may have missing columns
in that case just silently ignore overrides
2020-05-20 15:28:01 +10:00
Martin Brennan
72f139191e
FIX: S3 store has_been_uploaded? was not taking into account s3 bucket path (#9810)
In some cases, between Discourse forums the hostname of a URL could match if they are hosting S3 files on the same bucket but the S3 bucket path might not. So e.g. https://testbucket.somesite.com/testpath/some/file/url.png vs https://testbucket.somesite.com/prodpath/some/file/url.png. So has_been_uploaded? was returning true for the second URL, even though it may have been uploaded on a different Discourse forum.

This is a very rare case but must be accounted for, because this impacts UrlHelper.is_local which mistakenly thinks the file has already been downloaded and thus allows the URL to be cooked, where we want to return the full URL to be downloaded using PullHotlinkedImages.
2020-05-20 10:40:38 +10:00
Gerhard Schlager
0a700d81fc FIX: Restoring backups could fail for database dumps > 8GiB
This is a temporary fix until we ship a new image with bsdtar.
2020-05-19 22:36:59 +02:00
Daniel Waterworth
38c05a4f07 FIX: Check backtrace and backtrace_locations before trying to print an error
... during plugin initialization.
2020-05-19 14:27:22 +01:00
Daniel Waterworth
b4c5ff17a3 FIX: Show a useful message when starting a plugin fails without a backtrace 2020-05-19 14:03:54 +01:00
David Taylor
725e38f9d7
DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
Martin Brennan
779dc30d2e FIX: Join bookmarks migration on users to avoid missing user records
* the post_actions table has no FK to users, so if a user has been
  deleted we may end up with dangling post_action records, which then
  interferes with the bookmarks migration because bookmarks DO have
  an FK to users
2020-05-19 16:31:05 +10:00
Guo Xiang Tan
05beaa7f45
DEV: Remove override of flushdb in DiscourseRedis. 2020-05-19 10:26:23 +08:00
Gerhard Schlager
6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
Sam Saffron
862773ec83
FIX: do not remove stop words when using English locale
PG already handles English stop words, the list in cppjieba is
bigger than the list PG uses, which in turn causes confusion cause
words such as "volume" are stripped using cppijieba stop word list

We will follow up with another commit here to apply the Chinese
word stopwords, but for now to eliminate the confusion we are
skipping applying the stopword list when the dictionary in PG is
in English.
2020-05-18 10:54:56 +10:00
Robin Ward
db8e872bda
DEV: Move select kit to an addon (#9797) 2020-05-15 16:07:35 -04:00
David Taylor
461b4e5cc6
DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
Guo Xiang Tan
9ab5801a1b
FIX: Allow post migrations using #change to carry out unsafe migration 2020-05-15 14:23:27 +08:00