Commit Graph

6651 Commits

Author SHA1 Message Date
Roman Rizzi
070a3dcf9b
FIX: When running the wizard and using a custom theme, fallback to the color_scheme name if the base_scheme_id is nil (#8236) 2019-10-25 09:29:51 -03:00
Gerhard Schlager
452d541b74 DEV: Add env variable to allow easier debugging of smoke tests
Set `DEBUG_NODE=1` when running `rake smoke:test` and use your favorite tool to debug the smoke tests. See https://nodejs.org/en/docs/guides/debugging-getting-started/ for more information.

The debugger will break at the beginning of the smoke tests when the env variable is set.
2019-10-24 17:48:15 +02:00
Krzysztof Kotlarek
f530378df3 FIX: Restore for non-multisite is not raising an error on reconnect step (#8237)
That commit introduced a bug to the system: f69dacf979

Restore works fine for multisite, however, stopped working for non-multisite.

Reason for that was that `establish_connection` method got a check if the multisite instance is available:
```
    def self.instance
      @instance
    end

    def self.establish_connection(opts)
      @instance.establish_connection(opts) if @instance
    end
```
However, the reload method don't have that check
```
    def self.reload
      @instance = new(instance.config_filename)
    end
```

To solve it, let's ensure we are in a multisite environment before call reload
2019-10-24 11:46:22 +11:00
Neil Lalonde
f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Vinoth Kannan
726bf37a12 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:50:10 +05:30
Vinoth Kannan
31577b2131 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:35:38 +05:30
Robin Ward
6287eccb35 REFACTOR: Remove Ember.Controller in favor of import 2019-10-23 13:06:54 -04:00
Krzysztof Kotlarek
f34a0141c7 FIX: Correct path to ImportExport module (#8227)
During the move from Classic autoloader to Zeitwerk import_export module was moved to correct file name convention.
427d54b2b0 (diff-d896ec33b95afb7fae9f8bfe73d0580b)

Problem is that export/import is still using old path to require that module

Meta: https://meta.discourse.org/t/topic-and-category-export-import/38930/40
2019-10-23 17:27:14 +11:00
Krzysztof Kotlarek
f69dacf979 FIX: Reconnect in restore process connects to correct DB (#8218)
Simplified flow of restore is like that
```
migrate_database
reconnect
extract_uploads
```

Problem with incorrect current database started with this fix https://github.com/discourse/discourse/commit/025d4ee91f4

Dump task is reconnecting to default database https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L429

And then, we are trying to reconnect to the original database with that code:
```
def reconnect_database
  log "Reconnecting to the database..."
  RailsMultisite::ConnectionManagement::establish_connection(db: @current_db)
end
```

This reconnect is not switching us back to correct database because of that check
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L181
Basically, it finds existing handler and it thinks that we are connected to correct DB and this step can be skipped.

To solve it, we can reload RailsMultisite::ConnectionManagement which creates a new instance of that class
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L38
2019-10-23 17:23:50 +11:00
Arpit Jalan
1e9d9d9346
FIX: respect tl3 links no follow setting (#8232) 2019-10-22 22:41:04 +05:30
Mark VanLandingham
e5311ab645 DEV: Import DiscourseRoute rather than Discourse.Route (#8225)
* DEV: Discourse.Route -> DiscourseRoute with import

* ran prettier on all routes

* prettiered one more file
2019-10-22 09:46:10 -04:00
Nacho Caballero
d5121e5ddb FIX: Add common HTML5 media extensions to onebox audio and video tags (#8216) 2019-10-21 12:10:40 -04:00
Vinoth Kannan
5e55e75aed FIX: 'only_hidden_tags_changed?' method returned 'true' even when tags are not changed.
While editing the first post it does't bumped the topic when the new post revision created. Because we wrongly assumed that the hidden tags are changed even when no tags are updated.
2019-10-21 17:57:31 +05:30
Daniel Waterworth
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Joffrey JAFFEUX
8f51445e9f
FIX: fixes plugin generator with zeitwerk (#8220) 2019-10-21 09:24:14 +02:00
Krzysztof Kotlarek
858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
tshenry
95114f87ae
Improve spam_hosts copy (#8203)
- Increase size of the reviewable's conversation excerpt to prevent truncation of the new copy
- Remove the `domain` parameter from the `flag_linked_posts_as_spam` method in the user model since it is no longer needed
- Remove the `domain` interpolation variable from all translation files
- Add "All posts from this user that include links should be reviewed." to server.en.yml for added clarity on why the posts entered the queue
2019-10-18 09:31:15 -07:00
Rafael dos Santos Silva
2304dcf993 FIX: Do not truncate encoded slugs
Trying to truncate encoded slugs will mean that we have to keep the URL
valid, which can be tricky as you have to be aware of multibyte
characters.

Since we already have upper bounds for the title, the slug won't grow
for more than title*6 in the worst case. The slug column in the topic
table can store that just fine.

Added a test to ensure that a generated slug is a valid URL too, so we
don't introduce regressions in the future.
2019-10-17 13:38:31 -03:00
David Taylor
815dcb25ba UX: Improve new GitHub onebox rendering in emails
Unfortunately SVGs are not supported in most email clients, so remove them
2019-10-17 10:54:04 +01:00
Sam Saffron
f5d1aff8dd FEATURE: experimental hidden setting for draft backups
Under exceptional situations the automatic draft feature can fail.

This new **hidden, default off** site setting
`backup_drafts_to_pm_length` will automatically backup any draft that is
saved by the system to a dedicated PM (originating from self)

The body of that PM will contain the text of the reply.

We can enable this feature strategically on sites exhibiting issues to
diagnose issues with the draft system and offer a recourse to users who
appear to lose drafts. We automatically checkpoint these drafts every 5
minutes forcing a new revision each 5 minutes so you can revert to old
content.

Longer term we are considering automatically enabling this kind of feature
for extremely long drafts where the risk is really high one could lose
days of writing.
2019-10-17 16:58:21 +11:00
Sam Saffron
ae2a56999e Revert "FIX: update Redis gem to version 4.1.3 (#8197)"
This reverts commit ab74a50d85.

We really want to upgrade redis, but discovered some edge cases
around failover we need to test.

Holding off on the upgrade till a bit more testing happens
2019-10-17 11:41:46 +11:00
Krzysztof Kotlarek
ab74a50d85 FIX: update Redis gem to version 4.1.3 (#8197)
* FIX: update Redis gem to version 4.1.3

I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

* FIX: get rid of redis freedom patch
2019-10-17 08:49:23 +11:00
Neil Lalonde
5ef49692e0 FIX: tag cannot be used if it belongs to two tag groups with parent tag
If two tag groups exist with a mandatory parent tag, and one tag is
added to both tag groups, then the tag couldn't be used on any topics.
2019-10-16 14:28:04 -04:00
Sam Saffron
d5d8db7fa8 FEATURE: improve honeypot and challenge logic
This feature amends it so instead of using one challenge and honeypot
statically per site we have a rotating honeypot and challenge value which
changes every hour.

This means you must grab a fresh copy of honeypot and challenge value once
an hour or account registration will be rejected.

We also now cycle the value of the challenge when after successful account
registration forcing an extra call to hp.json between account registrations

Client has been made aware of these changes.

Additionally this contains a JavaScript workaround for:
https://bugs.chromium.org/p/chromium/issues/detail?id=987293

This is client side code that is specific to Chrome user agent and swaps
a PASSWORD type honeypot with a TEXT type honeypot.
2019-10-16 16:53:44 +11:00
David Taylor
ac2cd07775 DEV: Add discourse-restricted-replies as official plugin 2019-10-15 22:49:46 +01:00
Vinoth Kannan
f92a6f7ac5 FIX: 'local_cdn_url' method should work for local relative urls too. 2019-10-14 11:39:16 +05:30
Vinoth Kannan
e4fe864c0b FIX: use upload's cdn url in composer preview if available. 2019-10-14 08:47:15 +05:30
Roman Rizzi
01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Rafael dos Santos Silva
76ab0350f1
FIX: Properly encoded slugs when configured to (#8158)
When an admin changes the site setting slug_generation_method to
encoded, we weren't really encoding the slug, but just allowing non-ascii
characters in the slug (unicode).

That brings problems when a user posts a link to topic without the slug, as
our topic controller tries to redirect the user to the correct URL that contains
the slug with unicode characters. Having unicode in the Location header in a
response is a RFC violation and some browsers end up in a redirection loop.

Bug report: https://meta.discourse.org/t/-/125371?u=falco

This commit also checks if a site uses encoded slugs and clear all saved slugs
in the db so they can be regenerated using an onceoff job.
2019-10-11 12:38:16 -03:00
Régis Hanol
9a81cb9e55 FIX: ensure we remove tempfiles from disk when creating an upload
Follow-up to 46d12c5ad3
2019-10-11 11:13:10 +02:00
David Taylor
67787799bb DEV: Allow plugins to mark user custom fields as editable only by staff
This adds a staff_only parameter to the register_editable_user_custom_field API. The default is false, to maintain backwards compatibility.
2019-10-11 09:57:55 +01:00
Nacho Caballero
7d2f5240d9 FIX: Show a correct diff when editing consecutive paragraphs (#8177) 2019-10-11 03:50:37 -04:00
Neil Lalonde
09021c35a3 Version bump to v2.4.0.beta6 2019-10-10 11:46:28 -04:00
romanrizzi
fa66290b61 FIX: Don't fail when there's no directory to strip 2019-10-10 11:21:36 -03:00
Joffrey JAFFEUX
c5326682d6 DEV: improves speed of live reload css in core and plugins (#8161)
This corrects live refresh of CSS when working on plugins (and core) in many cases live refresh was not working.
2019-10-10 16:10:23 +11:00
Sam Saffron
5aaf7e3316 FIX: during concurrent emails generation renderer should not be reused
Our instance used for template rendering needs a lock to ensure there is
no race condition where rendering happens on 2 threads at the same time.

This can lead to local poisoning which can cause unexpected results in
emails
2019-10-10 08:50:48 +11:00
Roman Rizzi
5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166)
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
2019-10-09 11:41:16 -03:00
Mark VanLandingham
9b4aba0d39
DEV: support --fail-fast in bin/turbo_rspec (#8170)
* [WIP] - default turbo spec env to test

* FEATURE: support for --fast-fail in bin/turbo_rspec

* fast-fail -> fail_fast to match rspec

* Moved thread killing outside of fail-fast check

* Removed failure_count incrementation from fast_fail_met
2019-10-09 09:40:06 -05:00
Roman Rizzi
10478cb395
DEV: Bump uglifyjs (#7834)
* Rewrite uglifyjs command to work with 3.x

* Use ES5 syntax in plain JS files

* Use the older command if uglifyJS V2.x is installed
2019-10-09 10:02:49 -03:00
Régis Hanol
349c1cd085 FIX: remove site setting 'shadowed-by-global' option (#8061) 2019-10-08 12:43:26 -04:00
Dan Ungureanu
fdb1d3404c
FEATURE: Add site setting to show more detailed 404 errors. (#8014)
If the setting is turned on, then the user will receive information
about the subject: if it was deleted or requires some special access to
a group (only if the group is public). Otherwise, the user will receive
a generic #404 error message. For now, this change affects only the
topics and categories controller.

This commit also tries to refactor some of the code related to error
handling. To make error pages more consistent (design-wise), the actual
error page will be rendered server-side.
2019-10-08 14:15:08 +03:00
David Taylor
d2bceff133
FEATURE: Use full page redirection for all external auth methods (#8092)
Using popups is becoming increasingly rare. Full page redirects are already used on mobile, and for some providers. This commit removes all logic related to popup authentication, leaving only the full page redirect method.

For more info, see https://meta.discourse.org/t/do-we-need-popups-for-login/127988
2019-10-08 12:10:43 +01:00
Krzysztof Kotlarek
3f9673f23c FIX: solution for pending migrations for bin/turbo_rspec
Currently, if you try to run `./bin/turbo_rspec` you will got that error `There are pending migrations, run rake parallel:migrate`

Reason for that is that command is running in `development` mode which includes plugins migration files in ActiveRecord::Migrator.migrations_paths:
```
["db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-details/db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-details/db/post_migrate",
 "/home/lis2/projects/discourse/plugins/discourse-local-dates/db/migrate",
 "/home/lis2/projects/discourse/plugins/discourse-local-dates/db/post_migrate",
...
]
```

A workaround solution would be to run the command with the TEST environment like `RAILS_ENV=test ./bin/turbo_rspec`

I want to propose in this PR to override migration_paths to check only Discourse core migrations.
2019-10-08 10:32:40 +00:00
Sam Saffron
586dfcc795 PERF: avoid regex for hot path
Camelize is called quite a lot in zeitwerk, avoid using a regex here which
is far slower than using ends_with?
2019-10-08 13:43:54 +11:00
Sam Saffron
8d5f47dded PREF: optimise preloading application
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)

This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes

1. We attempt to localize a string on each site, ensuring we warmup
the i18n

2. We preload all our templates (compiling .erb to class)

3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
2019-10-07 00:33:37 -04:00
Sam Saffron
71ea4ad7fc PERF: reuse renderer when rendering email templates
Previous to this fix we were leaking methods on the internal action view
template class per render.

This caused email generation to be very low and a steady memory leak in the
application in sidekiq when sending out emails

The behavior change is new to Rails 6 so this fix does not need to be
backported into stable.
2019-10-06 23:57:03 -04:00
Jarek Radosz
bfe0bc0cbd
FIX: Check for category conflicts in SiteSetting validations (#8137)
It was possible to add a category to more than one default group, e.g. "default categories muted" and "default categories watching first post".

The bug was caused by category validations inadvertently comparing strings and numbers.
2019-10-06 20:50:07 +02:00
Jarek Radosz
87be6fe98c
DEV: Add a plugin incompatibility message (#8151)
* DEV: Add a plugin incompatibility message

* Extract the plugin_initialization_guard
2019-10-06 20:47:33 +02:00
Joffrey JAFFEUX
e23b9051ff
DEV: restart watcher after css change notifier crashed (#8157) 2019-10-04 13:20:51 -04:00
Régis Hanol
46d12c5ad3 FIX: properly downsize image on upload
Overwriting the same file with 'convert' is not always working as expected.
Adding a temporary file as the destination of the downsize makes this operation much more reliable.

Also switched to using (the more aggressive) 50% resize instead of halving the number of pixels.
2019-10-04 16:22:57 +02:00