Commit Graph

1292 Commits

Author SHA1 Message Date
David Taylor
3f8a85ed49
DEV: Write ember exam execution file for plugin qunit in CI (#23877) 2023-10-10 16:29:28 +01:00
David Taylor
99e9e3c75b
DEV: Enable USE_TURBO flag for plugin specs in docker.rake (#23761)
We run plugin specs in parallel in GitHub actions, so it makes sense to (optionally) do the same in the docker-based tests
2023-10-03 17:45:35 +01:00
arturo-seijas
d7b64b121b
DEV: Add task to anonymize user data (#20522) 2023-10-03 16:59:43 +08:00
Jarek Radosz
5a904949b2
DEV: Add gjs support for themes (#23473) 2023-10-02 12:36:06 +02:00
Jarek Radosz
0fb2812414
DEV: Rescue another http error in qunit task (#23651)
Should take care of a flake issue. Also removes an extraneous `/` character in the used URL.
2023-09-25 19:27:49 +02:00
David Taylor
e0daacf3ef
DEV: Drop /theme-qunit from smoke test (#23562)
We will soon be dropping support for `/theme-qunit` in production, so this will start failing if we don't remove it. Plus, we now have system specs which verify the end-to-end functionality of the Theme QUnit system.

This was the last thing which was using the legacy `run-qunit` script, so that can also be dropped.
2023-09-13 16:14:27 +01:00
David Taylor
8b51a89919
DEV: Do not squash commits in version_bump:stage_security_fixes (#23547)
Sometimes fixes will deliberately keep commits separate, and we don't want to undo that
2023-09-12 18:00:42 +01:00
Loïc Guitaut
b7d7099d08 DEV: Add link to PR when generating release notes 2023-09-12 09:26:46 +02:00
Alan Guo Xiang Tan
07c29f3066
Revert "DEV: Run core system tests by default in docker test image (#23517)" (#23525)
This reverts commit 40acb9a111.

Reverting because test runs are breaking due to this change
2023-09-12 11:45:49 +10:00
Daniel Waterworth
40acb9a111
DEV: Run core system tests by default in docker test image (#23517) 2023-09-11 16:04:33 -05:00
David Taylor
b07445ced8
DEV: Disable Webpack parallelization for low-memory environments (#23487)
This reduces memory usage for Embroider-based builds on low-memory servers (e.g. entry-level Digital Ocean droplets)
2023-09-11 09:32:37 +01:00
David Taylor
9667485951
DEV: Stop building test assets in production under Embroider (#23388)
Until now, we have allowed testing themes in production environments via `/theme-qunit`. This was made possible by hacking the ember-cli build so that it would create the `tests.js` bundle in production. However, this is fundamentally problematic because a number of test-specific things are still optimized out of the Ember build in production mode. It also makes asset compilation significantly slower, and makes it more difficult for us to update our build pipeline (e.g. to introduce Embroider).

This commit removes the ability to run qunit tests in production builds of the JS app when the Embdroider flag is enabled. If a production instance of Discourse exists exclusively for the development of themes (e.g. discourse.theme-creator.io) then they can add `EMBER_ENV: development` to their `app.yml` file. This will build the entire app in development mode, and has a significant performance impact. This must not be used for real production sites.

This commit also refactors many of the request specs into system specs. This means that the tests are guaranteed to have Ember assets built, and is also a better end-to-end test than simply checking for the presence of certain `<script>` tags in the HTML.
2023-09-11 09:12:37 +01:00
Alan Guo Xiang Tan
dc76d82f24
DEV: Fix broken conditional in docker:test Rake task (#23477)
Broke in ef73d20832
2023-09-08 12:16:02 +08:00
Alan Guo Xiang Tan
ef73d20832
DEV: Fix docker:test:setup Rake task that was missing the DB (#23474)
This is a follow up to 9caba30d5c

In that commit, we were migrating the database but we didn't actually
ensure that the database was created and that plugins were updated
before the databases were migrated.
2023-09-08 10:16:23 +08:00
David Taylor
75ce01a69b
DEV: Ensure Embroider sourcemaps are collected by Sprockets (#23468)
Names of sourcemaps are not necessarily equal to the js file names. Instead, we can check the `sourceMappingURL` comment to find the map's filename.
2023-09-07 22:20:52 +01:00
Alan Guo Xiang Tan
1f0a78fb82
DEV: Remove accidentally hardcoded Redis port in docker.rake (#23455)
Follow up to 9caba30d5c
2023-09-07 14:22:14 +08:00
Alan Guo Xiang Tan
9caba30d5c
DEV: Add docker:test:setup Rake task (#23430)
## What is the context here?

The `docker.rake` Rakefile contains Rake tasks that are meant to be run
in the `discourse/discourse_test:release` Docker image. For example, we
have the `docker:test` Rake task that makes it easier to run the test
suite for a particular Discourse commit.

Why are we introducing a `docker:test:setup` Rake task?

While we have the `docker:test` Rake task, it is very limited in the
test commands that can be executed. It is very useful for automated
testing but not very useful for running tests in the development
environment. Therefore, we are introducing a `docker:test:setup` rake
task that can be used to set up the test environment for running tests.

The envisioned example usage is something like this:

```
docker run -d --name=discourse_test --entrypoint=/sbin/boot discourse/discourse_test:release
docker exec -u discourse:discourse discourse_test ruby script/docker_test.rb --no-tests
docker exec -u discourse:discourse discourse_test bundle exec rake docker:test:setup
docker exec -u discourse:discourse discourse_test bundle exec rspec <path to file>
```
2023-09-07 13:46:23 +08:00
Alan Guo Xiang Tan
7cc5501cfd
DEV: Remove outdated ENV in docker.rake that is no longer being used (#23428)
We now have `USE_TURBO` which relis on `turbo_rspec`.
2023-09-06 09:07:57 +08:00
Martin Brennan
c532f6eb3d
FEATURE: Secure uploads in PMs only (#23398)
This adds a new secure_uploads_pm_only site setting. When secure_uploads
is true with this setting, only uploads created in PMs will be marked
secure; no uploads in secure categories will be marked as secure, and
the login_required site setting has no bearing on upload security
either.

This is meant to be a stopgap solution to prevent secure uploads
in a single place (private messages) for sensitive admin data exports.
Ideally we would want a more comprehensive way of saying that certain
upload types get secured which is a hybrid/mixed mode secure uploads,
but for now this will do the trick.
2023-09-06 09:39:09 +10:00
Alan Guo Xiang Tan
738343d4d2
DEV: Remove unused line of comment (#23401)
Wizard tests have been merged into core so the env isn't being used
anymore.
2023-09-05 14:28:23 +08:00
David Taylor
082ccdbd66
DEV: Reduce theme-qunit smoke test timeout (#23394)
The theme tests we use for the smoke-test typically take 3-4 seconds to complete. This commit reduces the timeout from 10 minutes to 20 seconds, so that failures are detected more quickl
2023-09-04 23:10:40 +01:00
Blake Erickson
5d438f805c
DEV: Add rake command to help detect dead settings (#23300)
* DEV: Add rake command to help detect dead settings

Some Site Settings may still exist but are no longer being used in the
core discourse code or in related plugins. This rake task will help
identify any unused (aka: dead) settings by using the `rg` command to
search for them.

You can execute the rake task by using this command:

`LOAD_PLUGINS=1 bin/rails "site_settings:find_dead"`

* Add env variable, apply feedback
2023-08-29 09:42:52 -06:00
Jarek Radosz
3ce3c2ff81
DEV: Add a spec for assets:precompile:js_processor (#23220) 2023-08-25 11:44:30 +02:00
Jarek Radosz
e60d227c8f
DEV: Avoid constant redefinition warnings in specs (#23241)
Specs sometimes do `Discourse::Application.load_tasks` which re-loads rake task files, causing constant redefinition.
2023-08-24 23:16:32 +02:00
Jarek Radosz
1e59e18ad2
FIX: Compile js-processor before db:migrate (#23229)
In production env it's possible to have migrations run before js-processor is available.
2023-08-24 19:24:43 +02:00
Jarek Radosz
70f1cc5552
DEV: Use esbuild to make DiscourseJsProcessor (#23223)
Reverts e2705df and re-lands #23187 and #23219.

The issue was incorrect order of execution of Rails' `assets:precompile` task in our own precompilation stack.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 16:36:22 +02:00
David Taylor
e2705df0f4
Revert "DEV: Use esbuild to make DiscourseJsProcessor (#23187)" (#23221)
This reverts commit 4dfe25d062 and 4fdeb6281e. We are investigating an issue related to asset compilation and S3 assets
2023-08-24 13:25:44 +01:00
Jarek Radosz
4fdeb6281e
FIX: assets:precompile:js_processor task issue (#23219)
* Fix the reference
* Use mutex in non-prod only (…and don't try to build the processor in runtime in prod)
2023-08-24 13:19:57 +02:00
Jarek Radosz
4dfe25d062
DEV: Use esbuild to make DiscourseJsProcessor (#23187)
Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 12:43:59 +02:00
Jarek Radosz
9b63ac473b
FIX: next vs return in maxmind task (#23196)
(and added the dependence on `environment` for trying the task independently of `assets:precompile`)
2023-08-22 23:00:02 +02:00
Jarek Radosz
c9de84c63d
DEV: Extract maxmind refresh logic to its own task (#23195) 2023-08-22 22:27:16 +02:00
Sam
e654edf844
FEATURE: do not bump topics when retroactively closing (#23115)
The category feature that automatically closes topics does it silently

This amends it so `rake topics:apply_autoclose` which does retroactive
closing will also do so silently.
2023-08-16 11:20:47 +10:00
Jarek Radosz
8bebd8fd99
DEV: Format .thor files (#23059) 2023-08-10 13:59:13 +02:00
Jarek Radosz
94649565ce
DEV: Correct Style/RedundantReturn rubocop issues (#23052) 2023-08-10 02:03:38 +02:00
Gerhard Schlager
eabea3e8fd DEV: Create missing user profiles in "import:ensure_consistency" rake task 2023-08-09 20:56:14 +02:00
Gerhard Schlager
b2fee68b3f DEV: Add rake task for generating avatars from SSO 2023-08-09 20:56:14 +02:00
David Taylor
13b13a758c
Remove discourse-canned-replies from official list (#22932)
This plugin is no longer supported, and so we no longer need to run its tests in CI

(removing the comment and the 'Canned Replies' value from the array caused syntax_tree to change to the `%w` syntax)
2023-08-02 12:48:20 +01:00
David Taylor
8a0d00b866
DEV: Introduce version_bump rake tasks (#22817)
This commit introduces five rake tasks to help us with version bump procedures:

- `version_bump:beta` and `version_bump:minor_stable` are for our minor releases
- `version_bump:major_stable_prepare` and `version_bump:major_stable_merge` are for our major release process
- `version_bump:stage_security_fixes` is to collate multiple security fixes from private branches into a single branch for release

The scripts will stage the necessary commits in a branch and prompt you to create a PR for review. No changes to release branches or tags will be made without the PR being approved, and explicit confirmation of prompts in the scripts.

To avoid polluting the operator's primary working tree, the scripts create a temporary git worktree in a temporary directory and perform all checkouts/commits there.
2023-07-31 16:05:13 +01:00
Jarek Radosz
3a11c82547
DEV: Update minitest to 5.19.0 (#22821) 2023-07-27 12:18:40 +02:00
Isaac Janzen
37942cb8bb
DEV: Convert admin-incoming-email modal to component-based API (#22701)
- Convert `admin-incoming-email` modal to component-based API
- Testing that the modal was working in local development was extremely challenging due to the need for `rejected` and `bounced` emails. Something that is not easy to stub in a local dev environment. To make this process more smooth for future developers I have added a new rake task:

```
desc "Creates sample email logs"
task "email_logs:populate" => ["db:load_config"] do |_, args|
  DiscourseDev::EmailLog.populate!
end
```

That will generate fully functional email logs in development to be toyed with.

<img width="787" alt="Screenshot 2023-07-20 at 3 27 04 PM" src="https://github.com/discourse/discourse/assets/50783505/47b3fe34-cd7e-49a5-8fe6-768c0fbd1aa2">
2023-07-20 16:31:20 -05:00
David Taylor
9c915345ea
DEV: Simplify qunit target selection (#22591)
Previously we had three query parameters to control which tests would be run. The default was to run all core/plugin tests together, which would almost always lead to errors and does not match the way we run tests in CI.

This commit removes the three old parameters (skip_core, skip_plugins and single_plugin), and introduces a new 'target' parameter. This can have a value of 'core', 'plugins', 'all', or a specific plugin name. The default is 'core'. Attempting to use the old parameters will raise an error.
2023-07-13 14:20:00 +01:00
Matt Palmer
bd9c919e06
FIX: don't use etags for post-upload verification (#21923)
They don't work for server-side encryption with customer keys, and so instead we just use Content-MD5 to ensure there was no corruption in transit, which is the best we can do.

See also: https://meta.discourse.org/t/s3-uploads-incompatible-with-server-side-encryption/266853
2023-07-07 09:53:49 +02:00
Osama Sayegh
d2d6d727de
FEATURE: Add default site settings to control the defaults of navigation menu preferences (#22485)
Follow-up to b27e12445d

This commit adds 2 new site settings `default_sidebar_link_to_filtered_list` and `default_sidebar_show_count_of_new_items` to control the default values for the navigation menu preferences that were added in the linked commit (`sidebar_link_to_filtered_list` and `sidebar_show_count_of_new_items` respectively).
2023-07-07 04:52:10 +03:00
Jarek Radosz
6b4762e450
DEV: Delete uglify asset codepath (#22346)
We no longer process any thrid-party assets with uglify
2023-06-29 12:23:46 +02:00
Neil Lalonde
d6a82f1034
DEV: Don't print plugin changes in details blocks (#22328) 2023-06-28 11:28:08 -04:00
Sam
0de3b279ce
FEATURE: add db:resize:notification_id task for growing table (#20505)
Under exceptional cases people may need to resize the notification table.
This only happens on forums with a total of more than 2.5 billion notifications.

This rake task can be used to convert all the notification columns to
bigint to make more room.
2023-06-21 09:57:16 -04:00
Emmanuel Ferdman
722180edba
DEV: Typo in an output message of uploads.rake (#22209)
Signed-off-by: emmanuel-ferdman <emmanuelferdman@gmail.com>
2023-06-21 12:00:26 +08:00
David Taylor
9c926ce645
PERF: Improve workbox loading strategy (#22019)
Previously workbox JS was vendored into our git repository, and would be loaded from the `public/javascripts` directory with a 1 day cache lifetime. The main aim of this commit is to add 'cachebuster' to the workbox URL so that the cache lifetime can be increased.

- Remove vendored copies of workbox.
- Use ember-cli/broccoli to collect workbox files from node_modules into assets/workbox-{digest}
- Add assets to sprockets manifest so that they're collected from the ember-cli output directory (and uploaded to s3 when configured)

Some of the sprockets-related changes in this commit are not ideal, but we hope to remove sprockets in the not-too-distant future.
2023-06-09 11:14:11 +01:00
Matt Palmer
a98d2a8086
FEATURE: allow S3 ACLs to be disabled (#21769)
AWS recommends running buckets without ACLs, and to use resource policies to manage access control instead.
This is not a bad idea, because S3 ACLs are whack, and while resource policies are also whack, they're a more constrained form of whack.
Further, some compliance regimes get antsy if you don't go with the vendor's recommended settings, and arguing that you need to enable ACLs on a bucket just to store images in there is more hassle than it's worth.
The new site setting (s3_use_acls) cannot be disabled when secure
uploads is enabled -- the latter relies on private ACLs for security
at this point in time. We may want to reexamine this in future.
2023-06-06 15:47:40 +10:00
David Taylor
0330f51d75
DEV: Ensure maxminddb:fetch works as standalone task (#21893)
`DiscourseIpInfo` expects zeitwerk auto-loading to be available, so we need to ensure the rake task loads the full rails environment. Normally we run this task as part of assets:precompile, so the app is already initialized. This commit only affects the case where the maxmind task is run directly.
2023-06-01 21:46:49 +01:00