* Chinese segmenetation will continue to rely on cppjieba
* Japanese segmentation will use our port of TinySegmenter
* Korean currently does not rely on segmentation which was dropped in c677877e4f
* SiteSetting.search_tokenize_chinese_japanese_korean has been split
into SiteSetting.search_tokenize_chinese and
SiteSetting.search_tokenize_japanese respectively
Job arguments go via JSON, and so DateTime objects will appear as strings in the Job's `#execute` method. The latest version of Sidekiq has started warning about this to reduce developer confusion.
Sometimes, the 'message' portion of an exception isn't enough to work out what's happening. In these cases, including the exception class name can help with debugging.
Job arguments go via JSON, and so symbols will appear as strings in the Job's `#execute` method. The latest version of Sidekiq has started warning about this to reduce developer confusion.
The latest version of Sidekiq introduced a warning when jobs are queued with arguments which 'do not stringify to JSON safely'. In the vast majority of cases, this is because a hash is passed with symbols as keys. When those args are passed to the job, the keys will be stringified.
Our job wrapper already takes care of this issue by calling '.with_indifferent_access' on the args before passing them to `#execute`, so we don't need to change anything about our use. All we need to do is satisfy Sidekiq's warning system by 'stringifying' all the keys before enqueuing the job.
Most of our logging goes through Rails.logger, and therefore appears in Logster at `/logs` on a site. The Sidekiq logger was bypassing this and writing directly to STDERR.
Unfortunately it's not possible to do `Sidekiq.logger = Rails.logger` because `Sidekiq#logger=` applies a number of patches to the logger instance, causing our whole logging system to break.
Instead, this commit adds a dedicated Logger instance with no output, which is then patched to forward all messages directly to `Rails.logger`
When uploading an image, we change the uploading placeholder several times. Every time, we correct the position of the cursor after replacing. But we schedule repositioning of cursor to the afterRender queue in Ember Run Loop. As a result, sometimes we replace the placeholder several times but correct the cursor position only once at the end.
It just cannot work correctly with scheduling, we'll always be dealing with cumulative error. Removing scheduling fixes the problem.
Sadly, I cannot make the test work, I skipped it for now, going to give it another try later.
This makes a small improvement to 'cold cache' ember-cli build times, and a large improvement to 'warm cache' build times
The ember-auto-import update means that vendor is now split into multiple files for efficiency. These are named `chunk.*`, and should be included immediately after the `vendor.js` file. This commit also updates the rails app to render script tags for these chunks
This commit adds a requestCustomMarkdownCookFunction function
to the `helper` that is provided to custom markdown rules
via their `setup` function.
The way this works is that once the default markdown engine that
we use for cooking posts has been set up, we loop through all
of the callbacks registered by `requestCustomMarkdownCookFunction`
and call `_buildCustomMarkdownCookFunction`. This creates
a new markdown engine using many of the same settings as the
default one, but will allow for the following options to be
changed by the markdown rule requesting the custom function:
* featuresOverride - The markdown-it features to allow for the engine
* markdownItRules - The markdown-it rules to allow for the engine
After this engine is set up a render function which renders + sanitizes
the output is returned for use by the markdown rule.
The use case for this API is mainly for block BBCode markdown rules
which want to render their content with a limited subset of the
markdown features/rules. Our initial use case for this is chat message
quoting.
This commit also does some minor refactoring of discourse-markdown-it
to accommodate this new engine building.
When changing to uppy for file uploads we forgot to add
these conditions to the paste event from 9c96511ec4
Basically, if you are pasting more than just a file (e.g. text,
html, rtf), then we should not handle the file and upload it, and
instead just paste in the text. This causes issues with spreadsheet
tools, that will copy the text representation and also an image
representation of cells to the user's clipboard.
This also moves the paste event for composer-upload-uppy to the
element found by the `editorClass` property, so it shares the paste
event with d-editor (via TextareaTextManipulation), which makes testing
this possible as the ember paste bindings are not picked up unless both
paste events are on the same element.
Adds up and down buttons next to the inputs of value lists when there is more than 1 item present. This helps to re-order the items in the value lists if necessary.
- Limit bulk re-invite to 1 time per day
- Move bulk invite by csv behind a site setting (hidden by default)
- Bump invite expiry from 30 -> 90 days
## Updates to rate_limiter
When limiting reinvites I found that **staff** are never limited in any way. So I updated the **rate_limiter** model to allow for a few things:
- add an optional param of `staff_limit`, which (when included and passed values, and the user passes `.staff?`) will override the default `max` & `secs` values and apply them to the user.
- in the case you **do** pass values to `staff_limit` but the user **does not** pass `staff?` the standard `max` & `secs` values will be applied to the user.
This should give us enough flexibility to
1. continue to apply a strict rate limit to a standard user
2. but also apply a secondary (less strict) limit to staff
In our legacy environment, Ember RFC176 shims are included in `discourse-loader.js` which is part of the `vendor.js` bundle. This meant that the module shims were available as soon as the vendor.js asset was loaded.
Under Ember CLI, we were defining these shims in `discourse-boot.js`. This is loaded by the browser much later, and meant that the shims were not available to themes/plugins that call `require()` before Discourse has booted. This was causing errors under some circumstances.
This commit refactors the Ember CLI implementation so that the shims are included in the vendor.js bundle. This is done via an addon which leans on the ember-rfc176-data NPM package. This will ensure we have all the definitions, without the need for manual copy/paste.
In our legacy environment, Ember RFC176 shims are included in `discourse-loader.js` which is part of the `vendor.js` bundle. This meant that the module shims were available as soon as the vendor.js asset was loaded.
Under Ember CLI, we were defining these shims in `discourse-boot.js`. This is loaded by the browser much later, and meant that the shims were not available to themes/plugins that call `require()` before Discourse has booted. This was causing errors under some circumstances.
This commit refactors the Ember CLI implementation so that the shims are included in the vendor.js bundle. This is done via an addon which leans on the ember-rfc176-data NPM package. This will ensure we have all the definitions, without the need for manual copy/paste.
Previously, `resetSite()` would immediately generate a new `Site` instance, and run all the initialization logic within the model. This included initializing Category objects.
This was problematic because `resetSite()` is called before any initializers have been run. That means that any modifications to the Site or Category classes would not have any effect on the already-initialized Site/Category instances.
This commit makes two main changes so so that the test environment is more production-like:
1. Update `resetSite` so that it simply stores the new data in the PreloadStore, and destroys the old Site instance. Initialization of a new site instance happens 'just in time' (normally during the `inject-discourse-objects` initializer)
2. Update the `helperContext` in tests to use getters. This avoids the need to look up `Site.current()` before initializers have run
It also makes a minor adjustment to one test which was relying on a side-effect of the previous behavior.
This should resolve the failing tests for discourse-category-expert under Ember-CLI: https://github.com/discourse/discourse-category-experts/pull/69
This adapter ensures that MiniSql locks the ActiveRecord mutex before using the raw PG connection. This ensures that multiple threads will not attempt to use the same connection simultaneously.
This commit also removes the schema_cache_concurrency freedom-patch, which is no longer required now that cross-thread connection use is controlled by the mutex.
For the original root cause of this issue, see https://github.com/rails/rails/pull/38577
