Commit Graph

289 Commits

Author SHA1 Message Date
Gerhard Schlager
bce7c486d8 DEV: Skip tests when unrelated GitHub config files are changed 2024-04-15 16:22:11 +02:00
Gerhard Schlager
1a9f4ec32b DEV: Skip tests when migrations-tests workflow is changed 2024-04-15 16:22:11 +02:00
David Taylor
8098876bfa
DEV: Restore RSpec 'documentation' output, but collapse in CI (#26485)
This reverts commit ef895f1c32 and 57df0d526e, but adds GitHub actions line grouping to the verbose output to reduce scrolling.
2024-04-03 10:54:55 +01:00
David Taylor
1e42e86601
DEV: Restore qunit 'tap' reporter, but collapse verbose output in CI (#26484)
This reverts commit 5adfb299ac, but uses GitHub actions grouping to achieve the same result: less scrolling to see failures.
2024-04-03 10:22:20 +01:00
Alan Guo Xiang Tan
337edc2f21
DEV: Update actions cache keys for bundler cache take 2 (#26411)
This is a follow up to 69c132723f which
wasn't right and broke our bundler github actions cache key
2024-03-28 10:25:58 +08:00
Alan Guo Xiang Tan
69c132723f
DEV: Update actions cache keys for bundler and app cache (#26398)
Why this change?

Bundle cache should be keyed on ruby version as well as the debian
release name. Changes to the debian release can affect the way gems are
installed since gems may link to different versions of binaries.
2024-03-28 09:20:26 +08:00
Loïc Guitaut
e911362864 DEV: Use Chrome in system specs
The version of Chromium we have in our images (120) is relatively
unstable and our system specs break regularly.

This patch makes sure Chrome is used instead for system specs.
2024-03-27 12:19:55 +01:00
Alan Guo Xiang Tan
0df50a7e5d
DEV: Use the right number of cores for bundle install in tests workflow (#26389)
Why this change?

We run on different runners depending on the scenario. We should use the
right number of parallel jobs for bundle install based on the number of
CPU cores the runner has.
2024-03-27 10:19:09 +08:00
Martin Brennan
5adfb299ac
DEV: Change QUnit reporters to dot (#26360)
This is so the CI output on GitHub actions isn't showing
tons and tons of unnecessary log data every time you want
to see the important thing, which is the actual test failure.
2024-03-26 16:54:22 +10:00
Alan Guo Xiang Tan
ef895f1c32
DEV: Drop documentation format on CI (#26356)
Why this change?

The output is too verbose and prevents us from quickly identifying tests
failures. Now that our tests are way more stable and less flaky, we can
drop the documentation format since we do not need it for debugging
purposes that often anymore
2024-03-26 12:03:55 +08:00
Loïc Guitaut
99921120a1 DEV: Switch to Chromium in CI
This patch switches to using Chromium in our CI jobs instead of Chrome.
2024-03-25 21:24:26 +01:00
Martin Brennan
57df0d526e
DEV: Change plugin rspec format to progress (#26098)
Having format documentation for the plugin specs just
makes a huge output that must be scrolled in order to
see the spec failures.
2024-03-08 16:12:15 +10:00
David Taylor
9f523af19c
DEV: Improve output of release-notes task to include start/end info (#26079) 2024-03-07 09:58:00 +00:00
David Taylor
a9371a2a87
DEV: Fix input definition for release-notes workflow (#26077) 2024-03-07 09:17:59 +00:00
David Taylor
99b6068ede
DEV: Add release-notes GitHub workflow (#26060) 2024-03-06 17:45:38 +00:00
Jarek Radosz
299b323d9a
DEV: Update ember-cli to 5.6.0 (#25886) 2024-02-27 10:48:30 +01:00
David Taylor
588a79c80c
DEV: Merge root JS packages (#25857)
Before this commit, we had a yarn package set up in the root directory and also in `app/assets/javascripts`. That meant two `yarn install` calls and two `node_modules` directories. This commit merges them both into the root location, and updates references to node_modules.

A previous attempt can be found at https://github.com/discourse/discourse/pull/21172. This commit re-uses that script to merge the `yarn.lock` files.

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-02-26 13:45:58 +00:00
David Taylor
542cb22fd4 DEV: Drop Ember 3 feature flag 2024-02-26 12:22:05 +00:00
Alan Guo Xiang Tan
c08d6b0f93
DEV: Only enable turbo_rspec retry on main branch (#25827)
Why this change?

Our tests are more stable these days and there is little to no need for
us to be retrying on PRs which helps to increase confidence in our test
suite since flaky tests are raised earlier.
2024-02-26 12:35:19 +08:00
Alan Guo Xiang Tan
ad0824b7e3
DEV: Fix connections timeout in system test (#25835)
Why this change?

This regressed in 6e9fbb5bab because we
had a `request.xhr?` check before we decide to block requests. However,
there could not none-xhr requests which we need to block as well at the
end of each system test when `@@block_requests` is true.

This also reverts commit 6437f27f90.
2024-02-23 16:03:46 +08:00
Alan Guo Xiang Tan
6437f27f90
DEV: Debug AR connection pool queue on CI (#25828)
Why this change?

On CI, we have been seeing flaky system tests because ActiveRecord is
unable to checkout a connection. This patch is meant to help us debug
which thread is not returning the connection to the queue.

Example of timeout issue: https://github.com/discourse/discourse/actions/runs/8012541636/job/21888013082
2024-02-23 13:37:37 +08:00
Alan Guo Xiang Tan
ad900ef9dd
Revert "DEV: Debug AR connection pool queue on CI (#25687)" (#25714)
This reverts commit 796af077c5.

We have not seen checkout timeout errors since c30aeafd9d
2024-02-16 10:11:46 +08:00
Alan Guo Xiang Tan
796af077c5
DEV: Debug AR connection pool queue on CI (#25687)
Why this change?

On CI, we have been seeing flaky system tests because ActiveRecord is
unable to checkout a connection. This patch is meant to help us debug
which thread is not returning the connection to the queue.
2024-02-15 14:00:30 +08:00
Alan Guo Xiang Tan
96ae5c395f
DEV: Set database checkout timeout to 10 on CI (#25604)
Why this change?

We have been seeing checkout timeouts happening on CI when using the
default of 5 seconds. This can happen in system tests when the server
has to process many requests using the same database connection.
Therefore, we will double the timeout for now and monitor if stuff
continues to timeout.
2024-02-08 10:35:55 +08:00
Alan Guo Xiang Tan
f9789e974b
DEV: Reduce pool size for test environment to 1 (#25584)
Why this change?

I have been investigating transaction related issues with our system
tests and I have a hard time figuring out what is causing the problem.
To help simplify our environment further, we will set the pool size in
the test environment to 1 so that it is impossible for us to be fetching
a different connection between the threads since they all share the
connection pool.

Also set `reaping_frequency` to `0` to ensure we don't reap any
connection ensuring the same connection is always used.
2024-02-07 11:10:17 +08:00
Alan Guo Xiang Tan
18d652c8f0
DEV: Create unlogged tables by default in the test environment (#25451)
Why this change?

In https://www.postgresql.org/docs/current/non-durability.html, it is
recommended to create unlogged tables to avoid WAL writes which can help
speed at performance at the expense of durability. In the CI env, there is no need for durability at all.
Therefore, we are going to be creating unlogged tables by default.

Co-authored-by: Ted Johansson <ted@discourse.org>
Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
2024-01-29 09:57:58 +08:00
Selase Krakani
93264da206
DEV: Add script to generate schema for intermediate DB (#24484)
This will be used by migration scripts.
2024-01-22 18:45:52 +01:00
Jarek Radosz
ae2d9de164
DEV: Remove "(Ember 5)" suffix from CI job names (#25347) 2024-01-22 12:10:17 +01:00
dependabot[bot]
d1be9310ed
Build(deps): Bump actions/cache from 3 to 4 (#25359)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-22 11:50:56 +01:00
Alan Guo Xiang Tan
54e6c1d823
DEV: Allow test-prof to be disabled completely with PREFABRICATION env (#25294)
Why this change?

We have been looking into a flaky system tests in one of our plugins
where the DB transaction flow can be messed up from time to time. Our
debugging effort is complicated by that fact that `test-prof` starts a
DB transaction in a `before(:all)` block which makes it hard to properly
log information. By allowing test-prof to be disabled completely, we
believe it will make it easier for us to isolate the problem we are
investigating.

What does this change do?

1. Avoid loading test-prof files if `PREFABRICATION` env has been set to
   `0`.

2. Set `PREFABRICATION=0` for plugin system tests in Github actions
2024-01-17 11:00:32 +08:00
Gerhard Schlager
1f81e8d857 DEV: Add 'i18n' label to PRs which include changes related to i18n 2024-01-16 15:00:19 +01:00
Gerhard Schlager
49916b0a0c DEV: Don't run 'labeler' GitHub Action for translator-bot
It's not needed and confusing when PRs created by discourse-translator-bot have the "chat" label.
2024-01-16 15:00:19 +01:00
Gerhard Schlager
50a997a53a
DEV: labeler GitHub action was broken after upgrade to v5 (#25243)
Follow-up to 5140caf0e4
2024-01-12 16:31:18 +01:00
David Taylor
7a8cbf8422
DEV: Switch default Ember version to 5 (#25203)
https://meta.discourse.org/t/287211
2024-01-10 12:12:36 +00:00
Gerhard Schlager
ce57969016
DEV: Add "migrations-tooling" label to PRs for import scripts (#25062)
Also, PRs for import scripts should automatically be assigned to the migrations-tooling group.
2023-12-28 21:26:05 +01:00
Alan Guo Xiang Tan
bf3e121323
DEV: Set config.eager_load = true on CI (#25032)
Why this change?

When running system tests on our CI, we have been occasionally seeing
server errors like:

```
Error encountered while proccessing /stylesheets/desktop_e58cf7f686aab173f9b778797f241913c2833c39.css
  NoMethodError: undefined method `+' for nil:NilClass
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/path/pattern.rb:139:in `[]'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:127:in `block (2 levels) in find_routes'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:126:in `each'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:126:in `each_with_index'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:126:in `block in find_routes'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:123:in `map!'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:123:in `find_routes'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/journey/router.rb:32:in `serve'
    /__w/discourse/discourse/vendor/bundle/ruby/3.2.0/gems/actionpack-7.0.7/lib/action_dispatch/routing/route_set.rb:852:in `call'
```

While looking through various Rails issues related to the error above, I
came across https://github.com/rails/rails/pull/27647 which is a fix to
fully initialize routes before the first request is handled. However,
the routes are only fully initialize only if `config.eager_load` is set
to `true`. There is no reason why `config.eager_load` shouldn't be `true` in the
CI environment and this is what a new Rails 7.1 app is generated with.

What does this change do?

Enable `config.eager_load` when `env["CI"]` is present
2023-12-26 13:05:55 +08:00
David Taylor
2f40d9b07b
DEV: Correct ember-5 lockfile generation (#24983)
The regen_ember_5_lockfile script was actually just duplicating the ember3 lockfile without changes 🤦‍♂️. This commit fixes that, and updates the ember-version-enforcement workflow to detect lockfile issues in future.
2023-12-20 11:45:01 +00:00
Alan Guo Xiang Tan
b364de7387
DEV: Revert upload-artifacts to v3 for flaky test related actions (#24960)
Why this change?

`upload-artifacts` v4 introduces a breaking change where uploading
multiple uploads to the same name artifact is no longer supported.
However, we have external services reading these artifacts and the
change in the artifact name is breaking those services.
2023-12-19 14:20:21 +08:00
Jarek Radosz
157a2fce1b
DEV: Update actions/upload-artifact to v4 (#24943) 2023-12-18 11:47:22 +01:00
Alan Guo Xiang Tan
fc8075c169
DEV: Fix flaky tests report artifacts not using the right job_id (#24939)
Why this change?

`github.job` returns the `job_id` per the docs but it doesn't actually
return the id of the job but instead returns the job's name strangely.

Per https://github.com/orgs/community/discussions/8945, there is no way
to get the `job_id` from the existing contexts in the actions run.
Therefore, we have to hit Github's API to fetch it. Not ideal but no
way around this.
2023-12-18 15:59:41 +08:00
Alan Guo Xiang Tan
a390dc0360
DEV: Fix path for flaky test report when uploading artifact (#24933)
This regressed in b2c27a8c60
2023-12-18 08:06:00 +08:00
David Taylor
15fc2a289a
DEV: Restore chat system spec job in GitHub CI (#24918)
Accidentally removed in c9cf3f5a52
2023-12-15 14:33:04 +00:00
Alan Guo Xiang Tan
b2c27a8c60
DEV: Flaky test report should be differentiated between job runs (#24895)
Why this change?

The `tests` workflow runs many jobs. Each job when ran is given a unique
id. Since a job can be re-run, we do not want the test reports to
override each other so we differentiate it further by the `job_id` given
by `${{ github.job }}`.
2023-12-14 12:12:14 +08:00
Alan Guo Xiang Tan
2a1952d9ba
DEV: Only retry and log flaky tests on the main branch (#24889)
Why this change?

Pull requests can introduce flaky tests into the mix and we do not want
to be hiing that during the pull request process. While this does mean
builds for PR will be less stable than the `main` branch without
retries, we do not foresee this to be a problem long term since the
monitoring of flaky tests on the `main` branch will mean that the number
of flaky tests will eventually be reduced.

What does this change do?

1. Introduce the `DISCOURSE_TURBO_RSPEC_RETRY_AND_LOG_FLAKY_TESTS` env
   variable which will initialize `TurboTest::Runner` with the `retry_and_log_flaky_tests`
   kwarg set to true when set.

2. Change the tests workflow run to set `DISCOURSE_TURBO_RSPEC_RETRY_AND_LOG_FLAKY_TESTS` only when
   the build type is `backend` or `system` and the `github.ref_name` is
   `main`.
2023-12-14 09:41:30 +08:00
David Taylor
c9cf3f5a52
DEV: Remove most Ember 3 jobs from CI (#24872)
It's very unlikely that something will be introduced which works under Ember 5 and not Ember 3. To reduce GitHub actions costs, flakiness, and visual noise, let's cut down the matrix so we're only using Ember 3 for the 'core frontend' job. All others can run under Ember 5.
2023-12-13 15:07:07 +00:00
Alan Guo Xiang Tan
39da9106ba
DEV: Introduce automatic reruns to RSpec tests on Github actions (#24811)
What motivated this change?

Our builds on Github actions have been extremely flaky mostly due to system tests. This has led to a drop in confidence
in our test suite where our developers tend to assume that a failed job is due to a flaky system test. As a result, we
have had occurrences where changes that resulted in legitimate test failures are merged into the `main` branch because developers
assumed it was a flaky test.

What does this change do?

This change seeks to reduce the flakiness of our builds on Github Actions by automatically re-running RSpec tests once when
they fail. If a failed test passes subsequently in the re-run, we mark the test as flaky by logging it into a file on disk
which is then uploaded as an artifact of the Github workflow run. We understand that automatically re-runs will lead to 
lower accuracy of our tests but we accept this as an acceptable trade-off since a fragile build has a much greater impact
on our developers' time. Internally, the Discourse development team will be running a service to fetch the flaky tests 
which have been logged for internal monitoring.

How is the change implemented?

1. A `--retry-and-log-flaky-tests` CLI flag is added to the `bin/turbo_rspec` CLI which will then initialize `TurboTests::Runner` 
with the `retry_and_log_flaky_tests` kwarg set to `true`. 

2. When the `retry_and_log_flaky_tests` kwarg is set to `true` for `TurboTests::Runner`, we will register an additional 
formatter `Flaky::FailuresLoggerFormatter` to the `TurboTests::Reporter` in the `TurboTests::Runner#run` method. 
The `Flaky::FailuresLoggerFormatter` has a simple job of logging all failed examples to a file on disk when running all the 
tests. The details of the failed example which are logged can be found in `TurboTests::Flaky::FailedExample.to_h`.

3. Once all the tests have been run once, we check the result for any failed examples and if there are, we read the file on
disk to fetch the `location_rerun_location` of the failed examples which is then used to run the tests in a new RSpec process.
In the rerun, we configure a `TurboTests::Flaky::FlakyDetectorFormatter` with RSpec which removes all failed examples from the log file on disk since those examples are not flaky tests. Note that if there are too many failed examples on the first run, we will deem the failures to likely not be due to flaky tests and not re-run the test failures. As of writing, the threshold of failed examples is set to 10. If there are more than 10 failed examples, we will not re-run the failures.
2023-12-13 07:18:27 +08:00
Jarek Radosz
5140caf0e4
DEV: Update actions/labeler to v5 (#24835) 2023-12-12 09:27:58 +08:00
David Taylor
a1cf426e27
DEV: Add ember-5 to test matrix for official plugins and themes (#24571)
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2023-12-11 22:58:36 +01:00
dependabot[bot]
7392071c28
Build(deps): Bump actions/stale from 8 to 9 (#24812)
Bumps [actions/stale](https://github.com/actions/stale) from 8 to 9.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v8...v9)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-11 09:44:52 +01:00
Alan Guo Xiang Tan
edf074bf78
DEV: Only enable minio for core system tests (#24642)
Why this change?

Let's us not run stuff when we don't need it.
2023-12-07 13:46:14 +10:00
David Taylor
f7c514dc99
DEV: Add ember-5 to test matrix for core/chat system specs (#24721) 2023-12-06 12:12:04 +00:00
David Taylor
bcb7e86c24
DEV: Update stale-pr-closer configuration (#24626) 2023-11-30 00:08:36 +00:00
David Taylor
75efeb395e
DEV: Add stale PR workflow (#24625)
- Mark stale after 60 days
- Close 14 days later
2023-11-29 10:37:11 +00:00
dependabot[bot]
1eaf774f47
Build(deps): Bump actions/setup-node from 3 to 4 (#24594) 2023-11-28 10:57:28 +01:00
David Taylor
cbc5694b77
DEV: Remove RAILS_MASTER flag from Gemfile (#24574)
We don't use this flag, and it prevents Dependabot from being able to automatically create PRs for Rails gem updates
2023-11-28 09:36:23 +00:00
David Taylor
cda86e63bb
DEV: Run ember enforcement on merge-result ref (#24577)
Checking out the raw PR branch was only needed for the dependabot-helper workflow, but then I copy/pasted it here 🤦‍♂️

Followup to 16b6e86932
2023-11-27 20:27:32 +00:00
David Taylor
16b6e86932 DEV: Introduce feature-flag for Ember 5 upgrade
This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production.

Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace.

To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable.

When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs.

When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3.

Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 16:40:22 +00:00
Alan Guo Xiang Tan
585e010af9
DEV: Cache plugin gems when running plugin test jobs in Github CI (#24535)
Why this change?

Plugin gems for official plugins are being installed over and over again
each time we run RSpec and QUnit tests for plugins. In particular, the
rugged gem installed by the discourse-code-review plugin takes
approximately 50-60 seconds to install because it is compiling libgit2.
2023-11-27 07:22:32 +08:00
Alan Guo Xiang Tan
ebf7553ad7
DEV: Drop Ruby version from Github job name (#24475)
Why this change?

Right now, the job names are `core system 3.2`, `core frontend 3.2` etc.
The problem here is that 3.2 is very vague. I thought about making the
job names something like `core system (Ruby 3.2)` but then wondered if
there is even value in including that when we are only running with one
ruby version in the matrix all the time. Therefore, I decided to drop
`3.2` from the job names.
2023-11-21 17:40:23 +08:00
Alan Guo Xiang Tan
e0ef88abca
DEV: Run QUnit tests for official Discourse themes (#24405)
Why this change?

As the number of themes which the Discourse team supports officially
grows, we want to ensure that changes made to Discourse core do not
break the plugins. As such, we are adding a step to our Github actions
test job to run the QUnit tests for all official themes.

What does this change do?

This change adds a new job to our tests Github actions workflow to run the QUnit
tests for all official plugins. This is achieved with the following
changes:

1. Update `testem.js` to rely on the `THEME_TEST_PAGES` env variable to set the
   `test_page` option when running theme QUnit tests with testem. The
   `test_page` option [allows an array to be specified](https://github.com/testem/testem#multiple-test-pages) such that tests for
   multiple pages can be run at the same time. We are relying on a ENV variable
   because  the `testem` CLI does not support passing a list of pages
   to the `--test_page` option.

2. Support a `/testem-theme-qunit/:testem_id/theme-qunit` Rails route in the development environment. This
   is done because testem prefixes the path with a unique ID to the configured `test_page` URL.
   This is problematic for us because we proxy all testem requests to the
   Rails server and testem's proxy configuration option does not allow us
   to easily rewrite the URL to remove the prefix. Therefore, we configure a proxy in testem to prefix `theme-qunit` requests with
  `/testem-theme-qunit` which can then be easily identified by the Rails server and routed accordingly. 

3. Update `qunit:test` to support a `THEME_IDS` environment variable
   which will allow it to run QUnit tests for multiple themes at the
   same time.

4. Support `bin/rake themes:qunit[ids,"<theme_id>|<theme_id>"]` to run
   the QUnit tests for multiple themes at the same time.

5. Adds a `themes:qunit_all_official` Rake task which runs the QUnit
   tests for all the official themes.
2023-11-17 07:17:32 +08:00
David Taylor
9449a0e0ed
DEV: Silence successful database migration output in github actions (#24416)
The output of db:migrate for a new database is 20k+ lines. We only need the output when an error occurs.
2023-11-16 15:55:41 +00:00
Alan Guo Xiang Tan
6ce55e5347
DEV: Run system tests for official themes (#24378)
Why this change?

As the number of themes which the Discourse team supports officially
grows, we want to ensure that changes made to Discourse core do not
break the plugins. As such, we are adding a step to our Github actions
test job to run the system tests for all official themes.

What does this change do?

This change adds a step to our Github actions test job to run the system
tests for all official plugins. This is achieved by the introduction of
the `themes:install_all_official` Rake task which installs all the
themes that are officially supported by the Discourse team.
2023-11-16 07:11:35 +08:00
Blake Erickson
bed4b3ee91
DEV: Remove .github/workflows/check-branches.yml (#24331)
We no longer need this workflow.
2023-11-10 08:49:38 -07:00
David Taylor
de8c8f1d28
DEV: Limit re-use of bundler/yarn caches in CI (#24256)
Using restore-keys means we will always use an old cache, and then add more dependencies to it. This leads to the cache growing over time and becoming increasingly slow. Instead, we should rebuild the cache from scratch each time our dependencies change.
2023-11-10 10:03:15 +00:00
Blake Erickson
179abfca1a
DEV: Update gh workflow check-branches.yml (#24298)
Without this change the resulting comparison looks like

```
if [ tests-passed == "tests-passed" ]; then
```

and so it was always failing. This way the resulting base branch name will also be in quotes for the comparison.

Follow up to: #24273
2023-11-08 11:56:49 -07:00
Blake Erickson
670692cfa9
DEV: Adds a GitHub workflow to check target branch (#24273)
* DEV: Adds a GitHub workflow to check target branch

Adds a GitHub workflow to check that the target branch for PRs in the
discourse-private-mirror repo aren't set to the tests-passed branch.

* Rename workflow
2023-11-08 10:34:20 -07:00
David Taylor
a546dcb0cc
DEV: Split chat system tests into separate GitHub actions job (#24096)
The 'plugins system' job is currently our longest-running job. Therefore, splitting it up will reduce the overall workflow runtime.
2023-10-25 12:58:36 +01:00
Martin Brennan
d5e8bd790b
DEV: Re-enable minio specs by pre-installing minio binary before test run (#24050)
This fixes a similar issue to 8b3eca0 where an Errno::ETXTBSY error was raised because the minio_runner gem was trying to install the binary across multiple processes in rspec. If we just make sure the latest version is installed before the tests run, this shouldn't happen, since MinioRunner.start will not do any further attempts at installation if the latest version is installed.
2023-10-24 12:43:14 +10:00
David Taylor
8c01947c45
DEV: Remove USE_EMBROIDER flag (#23971)
Embroider has been the default since b72ed3cb38. This commit removes the ability to set `USE_EMBROIDER=0` and go back to the classic build.
2023-10-19 10:38:25 +01:00
Jarek Radosz
dca23d3a53
DEV: Group @types/* updates (#23992) 2023-10-19 00:01:16 +02:00
David Taylor
c8c38bea7e
DEV: Write execution file for test failures (#23879)
Followup to 3f8a85ed49
2023-10-16 11:12:13 +01:00
Gerhard Schlager
ec8ae3fc65 DEV: Add GitHub Actions workflow for testing migrations
This workflow runs only for code underneath the `migrations/` directory. The usual test workflow is skipped for migrations because running frontend and backend tests is a waste of time and resources when only migrations are changed.
2023-10-13 16:03:55 +02:00
Gerhard Schlager
0907c0deb7 DEV: Update labeler for migration related code
This activates the `sync-labels` flag which causes the removal of labels when matching files are reverted or no longer changed by the PR.
2023-10-13 16:03:55 +02:00
David Taylor
3f8a85ed49
DEV: Write ember exam execution file for plugin qunit in CI (#23877) 2023-10-10 16:29:28 +01:00
Godfrey Chan
6c836af532
DEV: test the primary (embroider) build on Firefox (#23869)
Previously, classic was the default so we opted to test Embroider
only on Chrome. Now that Embroider is the default, we should flip
this around.
2023-10-10 14:03:17 +01:00
David Taylor
7f851175db
DEV: Group embroider updates in dependabot (#23727) 2023-09-29 17:49:47 +01:00
dependabot[bot]
3a8c0d0408
Build(deps): Bump actions/checkout from 3 to 4 (#23500)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 11:42:02 +02:00
Godfrey Chan
e1373c3e84
DEV: introduce Embroider behind a flag, and start testing in CI (#23005)
Discourse core now builds and runs with Embroider! This commit adds
the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start
testing it on CI.

The new pipeline uses Embroider's compat mode + webpack bundler to
build discourse code, and leave everything else (admin, wizard,
markdown-it, plugins, etc) exactly the same using the existing
Broccoli-based build as external bundles (<script> tags), passed
to the build as `extraPublicTress` (which just means they get
placed in the `/public` folder).

At runtime, these "external" bundles are glued back together with
`loader.js`. Specifically, the external bundles are compiled as
AMD modules (just as they were before) and registered with the
global `loader.js` instance. They expect their `import`s (outside
of whatever is included in the bundle) to be already available in
the `loader.js` runtime registry.

In the classic build, _every_ module gets compiled into AMD and
gets added to the `loader.js` runtime registry. In Embroider,
the goal is to do this as little as possible, to give the bundler
more flexibility to optimize modules, or omit them entirely if it
is confident that the module is unused (i.e. tree-shaking).

Even in the most compatible mode, there are cases where Embroider
is confident enough to omit modules in the runtime `loader.js`
registry (notably, "auto-imported" non-addon NPM packages). So we
have to be mindful of that an manage those dependencies ourselves,
as seen in #22703.

In the longer term, we will look into using modern features (such
as `import()`) to express these inter-dependencies.

This will only be behind a flag for a short period of time while we
perform some final testing. Within the next few weeks, we intend
to enable by default and remove the flag.

---------

Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 13:15:43 +01:00
David Taylor
31563e6b29
DEV: Reduce psql output when restoring db in github actions (#23391)
This output is around 30k lines long, and serves no real purpose. In the unlikely event of an error, it will still be shown.
2023-09-06 22:19:59 +01:00
Jarek Radosz
7b78895930
DEV: Group babel updates together (#23322)
Grouped Dependabot updates are now available (https://github.blog/changelog/2023-08-24-grouped-version-updates-for-dependabot-are-generally-available/) so we can finally make it create a single PR for both `@babel/core` and `@babel/standalone` 😌
2023-08-30 10:38:26 +02:00
Martin Brennan
cf42466dea
DEV: Add S3 upload system specs using minio (#22975)
This commit adds some system specs to test uploads with
direct to S3 single and multipart uploads via uppy. This
is done with minio as a local S3 replacement. We are doing
this to catch regressions when uppy dependencies need to
be upgraded or we change uppy upload code, since before
this there was no way to know outside manual testing whether
these changes would cause regressions.

Minio's server lifecycle and the installed binaries are managed
by the https://github.com/discourse/minio_runner gem, though the
binaries are already installed on the discourse_test image we run
GitHub CI from.

These tests will only run in CI unless you specifically use the
CI=1 or RUN_S3_SYSTEM_SPECS=1 env vars.

For a history of experimentation here see https://github.com/discourse/discourse/pull/22381

Related PRs:

* https://github.com/discourse/minio_runner/pull/1
* https://github.com/discourse/minio_runner/pull/2
* https://github.com/discourse/minio_runner/pull/3
2023-08-23 11:18:33 +10:00
Mark VanLandingham
fbf7b106cc
DEV: Bump selenium-webdriver version to fix system spec running (#23117)
We can no long user Webdriver - SeleniumHQ/selenium#11066. Bumping selenium-webdriver did the trick, as well as manually setting the user_agent for mobile system specs. Unsure what changed to make this necessary, but it is necessary to get the app to boot in mobile view.
2023-08-16 15:07:03 -05:00
Jarek Radosz
8bebd8fd99
DEV: Format .thor files (#23059) 2023-08-10 13:59:13 +02:00
Krzysztof Kotlarek
77b4e42f61
FIX: specify chrome version (#22681) 2023-07-19 15:06:56 +10:00
Alan Guo Xiang Tan
433cb7092d
DEV: Use documentation format for core specs on CI (#22531)
The documentation format makes it easier to link a failing test to the
process it was launched in.
2023-07-11 10:12:51 +08:00
Alan Guo Xiang Tan
7b40ec859f
DEV: Reduce parallel test processors for system tests to 4 (#22456)
Why this change?

This is abit of a trial and error but we're starting to see selenium
session not created errors on CI. One of the reason for this is that the
system has run out of resources to create a new tab.

This commit reduces the number of parallel test processors in an attempt
to increase the amount of resources available to each test process and
hopefully lead to more stable CI system tests.
2023-07-06 10:34:44 +08:00
Alan Guo Xiang Tan
b764c53cc0
Revert "DEV: Reduce number of parallel test processors to 3 for system tests (#22423)" (#22453)
This reverts commit 865f7a9852.

The flakiness that we have been seeing and fixing on CI were not related
to system resource problems. Therefore, we can bump this up back to 5.
2023-07-06 08:47:43 +08:00
Alan Guo Xiang Tan
858cc6aff2
DEV: Log server errors encountered when running system tests on CI (#22427)
Why is this change required?

We've been seeing flaky tests due to server errors on CI but are unable
to debug it because we do not log any of the errors. This change gives
us a fighting chance the next time we encounter a server error during
system test runs.

See
https://github.com/discourse/discourse/actions/runs/5459248864/jobs/9935049920?pr=22424
for an example of server errors encountered during system tests.
2023-07-05 09:46:48 +08:00
Alan Guo Xiang Tan
865f7a9852
DEV: Reduce number of parallel test processors to 3 for system tests (#22423)
This is an experiment to see if not saturating resources on a runner
with 8 vCPU leads to more stability on our CI.
2023-07-05 08:08:46 +08:00
Jarek Radosz
876ff17cc2
DEV: Update eslint/prettier (#22226) 2023-06-21 20:59:03 +02:00
David Taylor
251d6f0627
DEV: Introduce Zeitwerk reloading check in CI (#22151)
This should help us to catch the most catastrophic cases where plugins fail to auto-reload in development mode.
2023-06-16 14:33:14 +01:00
Alan Guo Xiang Tan
5897709a90
DEV: Use runtime info to split test files for parallel testing (#22060)
Using the runtime information, we will be able to more efficiently group
the test files across the test processes hence leading to better
utilization of resources.
2023-06-12 09:07:17 +08:00
Daniel Waterworth
67afd85aae
Revert "DEV: Use runtime info to split test files for parallel testing (#21896)" (#22016)
This reverts commit 14ed971db6.

This prevented the core backend tests from running in GitHub CI
2023-06-08 15:13:26 -05:00
Alan Guo Xiang Tan
14ed971db6
DEV: Use runtime info to split test files for parallel testing (#21896)
Using the runtime information, we will be able to more efficiently group
the test files across the test processes hence leading to better
utilization of resources.
2023-06-05 08:01:41 +08:00
Alan Guo Xiang Tan
ba251dec6b
DEV: Increase number of parallel process for system test to 5 on CI (#21878)
4dd053a69c addressed most of the
instability we were seeing with system tests on CI and locally. Let's
try pushing the number of parallel processes up to squeeze as much time
savings as possible from the runner.
2023-06-01 10:41:11 +08:00
Alan Guo Xiang Tan
2c1c7d1e4f DEV: Use --profile when running system tests in CI 2023-05-30 13:46:14 +09:00
Alan Guo Xiang Tan
123a77a2bc
DEV: Correct Capybara default max wait time setting in CI (#21801)
We were not setting capybara's default max wait time correctly in CI due
to a spelling error.

This regressd in fc17045876
2023-05-29 10:41:24 +08:00
Alan Guo Xiang Tan
038318bffc
DEV: Bump CAPYBARA_DEFAULT_MAX_WAIT_TIME to 10 seconds on CI (#21711)
We're running on pretty crappy hardware on Github's CI and this has an
impact on the stability of our system tests on CI. Therefore, we are
bumping `CABPYARA_DEFAULT_MAX_WAIT_TIME` to 10 seconds to account for
the less than ideal hardware we're running the system tests on.

This change trades off speed for stability but speed is already bad on
CI so stability is more important for our case.
2023-05-25 09:25:08 +08:00
Jarek Radosz
fc17045876
DEV: Clean up workflow files (#21526) 2023-05-12 14:00:04 +02:00
Jarek Radosz
19ac90536f
DEV: Restore the documentation format in system tests (#21471) 2023-05-12 11:13:52 +02:00