This commit contains a couple of improvements for this
rake task.
* We no longer limit the uploads to only ones with Post
upload references, it doesn't matter what the secure uploads
are linked to, they should all be un-secured
* We now only get distinct uploads from the initial query,
multiple upload references on the same upload caused
double ups and confusing counts for the task
* We now also disable the secure_uploads_pm_only site
setting at the same time
Rake allows env variables to specified in arguments, so we need to use
the list of top_level_tasks which excludes those env
Followup to c8718a64dd7a26165efe91706ce5507e6999044a.
By design, db:create initializes the Rails app with SKIP_DB=true. That
means that SiteSettings get set up with the LocalProcessProvider instead
of the DBProvider. In other words: any calls to site settings will
return the default, rather then the actual value in the database.
Running db:migrate in the same rake invocation means that rails will not
be re-initialized, and so skip_db will remain true. Site settings
accessed during migrations and fixtures will therefore return incorrect
values.
One example of this is that running bin/rake db:create db:migrate
repeatedly in a development environment will cause the FAQ topic to be
seeded repeatedly, because the seed logic does not have access to the
site setting which stores the already-seeded topic id.
This commit will automatically re-exec the Rake command if any tasks are
specified after `db:create`
This commit updates the `uploads:sync_s3_acls` rake task to accept a
`sync` argument that would run the rake task in a synchronous manner
so that the outcome of running the rake task can easily be determined.
This reverts commit b9f8a77d9ba5222bbf55172092f0b235b92d1c85.
Reverting while we work on adding error handling. At the moment,
failures are logged, but the process still exits with status=0.
This commit converts the `AdminReport` component, which is quite
high complexity, to gjs. After this initial round, ideally this
component would be broken up into smaller components because it is
getting quite big now.
Also in this commit:
* Add an option to display the report description in a tooltip, which
was
the main way the description was shown until recently. We want to use
this on the dashboard view mostly.
* Move admin report "mode" definitions to the server-side Report model,
inside a `Report::MODES` constant, collecting the modes defined in
various
places in the UI into one place
* Refactor report code to refer to mode definitions
* Add a `REPORT_MODES` constant in JS via javascript.rake and refactor
JS to refer to the modes
* Delete old admin report components that are no longer used
(trust-level-counts, counts, per-day-counts) which were replaced
by admin-report-counters a while ago
* Add a new `registerReportModeComponent` plugin API, some plugins
introduce their own modes (like AI's `emotion`) and components and
we need a way to render them
This is a revert of 92793c5b73871ba84b024c2ce50055a0776f1ba6.
Following on from discussions after the previous commit, it became evident that it was only a small step towards solving the larger problem of finding site settings in a reliable fashion across multiple languages.
This is going to take more thought and discussion, and since the changes introduced in the previous commit are effectively non functional without additional work, I'm going to revert it for now.
This commit introduces a new rake task that can be used in situations where a community receives a large number of flags/reports and needs a quick way to handle all of those pending reports. Usage instructions are included in the rake task source code.
Internal topic: t/145475.
The chat emoji picker is renamed emoji-picker, and the old emoji-picker is removed.
This commit doesn't attempt to fully rework a new emoji-picker but instead tries to migrate everything to one picker (the chat one) and add small changes.
Other notable changes:
- all the favorite emojis code has been mixed into one service which is able to store one state per context, favorites emojis will be stored for all topics, and for each chat channel. Meaning that if you always use a specific emoji in a channel, it will only show as favorite emoji in this channel.
- a lot of static code has been removed which should improve initial load perf of discourse. Initially this code was around to improve the performance of the emoji picker rendering.
- the emojis are now stored, once the full list has been loaded, if you close and reopen the picker it won't have to load them again.
List of components:
- `<EmojiPicker />` will render a button which will open a dropdown
- `<EmojiPickerContent />` represents the content of the dropdown alone, it's useful when you want to render a picker from an action which is not the default picker button
- `<EmojiPickerDetached />` just a simple wrapper over `<EmojiPickerContent />` to make it easier to use it with `this.menu.show(...)`
---------
Co-authored-by: Renato Atilio <renatoat@gmail.com>
Adds a new reviewables:populate rake task that works in a similar fashion to the existing *:populate rake tasks. The rake task creates pending reviewable of all core types, with possibility for plugins to extend the task to populate their own reviewable types.
This commit replaces the `full_name_required` setting with a new `full_name_requirement` setting to allow more flexibility with the name field in the signup form. The new setting has 2 options, "Required at signup" and "Optional at signup", which are equivalent to the true/false possibilities of the old setting, and a third option "Hidden at signup" that hides the name field from the signup form, making it effectively optional too.
New sites will have the "Hidden at signup" option as the default option, and existing site will continue to use the option that maps to their current configuration.
Internal topic: t/136746.
We can't delete the file from disk as some of the assets are still
served by the app instead of going through the S3 bucket. It is a bug we
need to fix but it also means this ENV is unsafe now. Just drop the env
until we ensure all assets requested by the app are requested from the
S3 bucket directly.
The `max_compress?` logic is totally broken at least when used for
brotli compression because we are only seeing 4 assets subjected to the
max compression level in production. Instead of fixing the broken logic,
we should just drop this unnecessary complexity cause things are easier
to reason about when we only have one compression level to deal with
across all assets.
Now that we run the `upload` method in different threads, we need to
synchronize writes to `STDOUT` which we can do so by using a `Logger`.
Follow-up to 49e835395966fbddb9ad2988961d8375e501151f
Following a recent commit (cb4b8146a30cd94b04db1b662fbebdb8021a7771),
the benchmark script wasn’t working anymore (and the related rake task).
This patch fixes it. It also adds some information about Ruby YJIT being
enabled or not.
It splits the hide_profile_and_presence user option and the default_hide_profile_and_presence site setting for more granular control. It keeps the option to hide the profile under /u/username/preferences/interface and adds the presence toggle in the quick user menu.
Co-authored-by: Régis Hanol <regis@hanol.fr>
This commit changes the uploads:secure_upload_analyse_and_update
and uploads:disable_secure_uploads to no longer rebake affected
posts inline. This just took way too long, and if the task stalled
you couldn't be sure if the rest of it completed.
Instead, we can update the baked_version of affected posts and
utilize our PeriodicalUpdates job to gradually rebake them. I added
warnings about increasing the site setting rebake_old_posts_count and
the global setting max_old_rebakes_per_15_minutes before doing this
as well.
For good measure, the affected post IDs are written to a JSON file too.
The `categories_only_optimized` category page style has been introduced
in commit d37a0d401ceca044002fd640be3745de643a3f2a. This commit makes
sure that style is enforced for users who can see over 1000 categories
in order to keep `/categories` page functional.
Followup 30fdd7738ec942126bb055c9a6a8d69ddffba2da
Adds a new site setting and corresponding user preference
to disable smart lists. By default they are enabled, because
this is a better experience for most users. A small number of
users would prefer to not have this enabled.
Smart lists automatically append new items to each
list started in the composer when enter is pressed. If
enter is pressed on an empty list item, it is cleared.
This setting will be removed when the new composer is complete.
- Add concurrency when running on multisite clusters (default 10, configurable via THEME_UPDATE_CONCURRENCY env)
- Add a version cache for the duration of the rake task. This avoids duplicating work when many sites in the cluster have the same theme installed, and it is already up-to-date
- Updates output to be more concurrent friendly (all `puts`, no `print`)
- limits security key deletes to second factor keys
- also deletes backup codes (lingering backup codes break login flow entirely)
* Add spec for rake task to disable 2FA for a user
Remove emoji.clear cache calls as data.js.es6.erb hasn't existed in a while.
Emoji data is now compiled separately via javascript rake tasks.
Skip db and redis precompilation when no db is present
We were running into errors running `ember build` on machines with high
CPU counts. It was then noted that `thread-loader`, which embroider uses, defaults to spinning
up x workers where x is number of physical CPU cores - 1. That is
probably too much so we set out to find out an optimial count to set for
the `JOBS` env which embroider will use to set the number of
`thread-loader` workers.
I first built an image using the following Dockerfile.
```
FROM discourse/base:release
RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake plugin:install_all_official
RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake assets:precompile:prereqs
```
I then ran the following command on my M3 Max Macbook Pro that has 14
phyisal CPU cores.
```
for j in 1 2 4 8 14; do echo "JOBS=$j"; time docker run --rm -it -e JOBS=$j test:latest /bin/bash -c "su discourse -c 'cd /var/www/discourse && bundle exec rake assets:precompile:build'"; done
```
These are the results I got:
```
JOBS=1 0.04s user 0.03s system 0% cpu 1:01.92 total
JOBS=2 0.04s user 0.02s system 0% cpu 42.605 total
JOBS=4 0.04s user 0.02s system 0% cpu 37.012 total
JOBS=8 0.04s user 0.02s system 0% cpu 35.199 total
JOBs=14 0.04s user 0.02s system 0% cpu 37.941 total
```
We think JOBS=2 is a good default when the `JOBS` env has not been set.
Anything above just consumes more resources for little benefit.
The recorded runtime log was specified when running the plugin specs,
but it failed to match the entries from the runtime log and the passed
as command line arguments because the runtime log entries were not
prefixed with `./`.
This will bring significant improvements to install speed & storage requirements. For information on how it may affect you, see https://meta.discourse.org/t/324521
This commit:
- removes the `yarn.lock` and replaces with `pnpm-lock.yaml`
- updates workspaces to pnpm format
- adjusts package dependencies to work with pnpm's stricter resolution strategy
- updates Rails app to load modules from more specific node_modules directories
- adds a `.pnpmfile` which automatically cleans up old yarn-managed `node_modules` directories
- updates various scripts to call `pnpm` instead of `yarn`
- updates patches to use pnpm's native patch system instead of patch-package
- adds a patch for licensee to support pnpm
`ember exam` points all browsers at a single ember-cli server process. With very high parallelism, we've started seeing `ChunkLoadError` exceptions, which may indicate the server being overloaded. Capping to 8 browsers to avoid that.
We were writing theme-transpiler JS files to the filesystem on a per-process basis, and then immediately reading them back in. Plus, there was no cleanup mechanism, so the tmp directory would grow indefinitely.
This commit refactors things so that the `build.js` script outputs the theme-transpiler source to stdout. That way, we can read it directly into the process, and then into mini-racer, without needing to go via the filesystem. No cleanup required!
In production, the theme-transpiler is still cached in a file during `assets:precompile`
We have been seeing `ZLib::BufError` when running the `assets:precompile` rake
task.
```
I, [2024-07-30T05:19:58.807019 #1059] INFO -- : Writing /var/www/discourse/public/assets/scripts/discourse-test-listen-boot-9b14a0fc65c689577e6a428dcfd680205516fe211700a71c7adb5cbcf4df2cc5.js
rake aborted!
Zlib::BufError: buffer error (Zlib::BufError)
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sprockets-3.7.3/lib/sprockets/cache/file_store.rb💯in `<<'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sprockets-3.7.3/lib/sprockets/cache/file_store.rb💯in `set'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sprockets-3.7.3/lib/sprockets/cache.rb:212:in `set'
```
The hypothesis here is that some thread unsafe issue is causing the
problem since we download the Maxmind databases in a thread and run
decompression operations once the gzip file is downloaded.
In the near term, we plan to move downloading of Maxmind databases out
of the Rake task into a scheduled job so this patch should be considered
a temporary solution.
The trade-off here is that build time will slightly increase since we
are not longer downloading Maxmind databases while precompiling assets
at the same time.
We have a dedicated admin page (`/admin/customize/email_templates`) that lets admins customize all emails that Discourse sends to users. The way this page works is that it lists all translations strings that are used for emails, and the list of translation strings is currently hardcoded and hasn't been updated in years. We've had a number of new emails that Discourse sends, so we should add those templates to the list to let admins easily customize those templates.
Meta topic: https://meta.discourse.org/t/3-2-x-still-ignores-some-custom-email-templates/308203.
