discourse/.github/workflows/tests.yml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

327 lines
12 KiB
YAML
Raw Normal View History

name: Tests
on:
pull_request:
push:
branches:
- main
- beta
- stable
concurrency:
group: tests-${{ format('{0}-{1}', github.head_ref || github.run_number, github.job) }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
2023-05-12 20:00:04 +08:00
if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
name: ${{ matrix.target }} ${{ matrix.build_type }} ${{ matrix.ruby }}${{(matrix.embroider == '1') && ' (Embroider)' || ''}}
runs-on: ${{ (matrix.build_type == 'annotations') && 'ubuntu-latest' || 'ubuntu-20.04-8core' }}
container: discourse/discourse_test:slim${{ (matrix.build_type == 'frontend' || matrix.build_type == 'system') && '-browsers' || '' }}${{ (matrix.ruby == '3.1') && '-ruby-3.1.0' || '' }}
timeout-minutes: 20
env:
DISCOURSE_HOSTNAME: www.example.com
RAILS_ENV: test
PGUSER: discourse
PGPASSWORD: discourse
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
USE_EMBROIDER: ${{ matrix.embroider }}
USES_PARALLEL_DATABASES: ${{ matrix.build_type == 'backend' || matrix.build_type == 'system' }}
CAPYBARA_DEFAULT_MAX_WAIT_TIME: 10
MINIO_RUNNER_LOG_LEVEL: DEBUG
strategy:
fail-fast: false
matrix:
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
build_type: [backend, frontend, system, annotations]
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
embroider: ["0", "1"]
target: [core, plugins]
2023-05-12 20:00:04 +08:00
ruby: ["3.2"]
exclude:
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
- build_type: annotations
embroider: "1"
- build_type: annotations
target: plugins
- build_type: frontend
target: core # Handled by core_frontend_tests job (below)
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
- build_type: backend
embroider: "1"
steps:
- name: Set working directory owner
run: chown root:root .
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Git
run: |
git config --global user.email "ci@ci.invalid"
git config --global user.name "Discourse CI"
- name: Start redis
run: |
redis-server /etc/redis/redis.conf &
- name: Start Postgres
run: |
chown -R postgres /var/run/postgresql
sudo -E -u postgres script/start_test_db.rb
sudo -u postgres psql -c "CREATE ROLE $PGUSER LOGIN SUPERUSER PASSWORD '$PGPASSWORD';"
- name: Bundler cache
uses: actions/cache@v3
with:
path: vendor/bundle
key: ${{ runner.os }}-${{ matrix.ruby }}-gem-${{ hashFiles('**/Gemfile.lock') }}
2023-05-12 20:00:04 +08:00
restore-keys: ${{ runner.os }}-${{ matrix.ruby }}-gem-
- name: Setup gems
run: |
gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
bundle config --local path vendor/bundle
bundle config --local deployment true
bundle config --local without development
bundle install --jobs 4
bundle clean
- name: Get yarn cache directory
id: yarn-cache-dir
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- name: Yarn cache
uses: actions/cache@v3
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
2023-05-12 20:00:04 +08:00
restore-keys: ${{ runner.os }}-yarn-
- name: Yarn install
run: yarn install --frozen-lockfile
- name: Checkout official plugins
if: matrix.target == 'plugins'
run: bin/rake plugin:install_all_official
- name: Pull compatible versions of plugins
if: matrix.target == 'plugins'
run: bin/rake plugin:pull_compatible_all
- name: Add hosts to /etc/hosts, otherwise Chrome cannot reach minio
run: |
echo "127.0.0.1 minio.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 discoursetest.minio.local" | sudo tee -a /etc/hosts
- name: Fetch app state cache
uses: actions/cache@v3
id: app-cache
with:
path: tmp/app-cache
2023-05-12 20:00:04 +08:00
key: >-
${{ runner.os }}-
${{ hashFiles('.github/workflows/tests.yml') }}-
${{ hashFiles('db/**/*', 'plugins/**/db/**/*') }}-
${{ env.USES_PARALLEL_DATABASES }}
- name: Restore database from cache
if: steps.app-cache.outputs.cache-hit == 'true'
run: psql --quiet -o /dev/null -f tmp/app-cache/cache.sql postgres
- name: Restore uploads from cache
if: steps.app-cache.outputs.cache-hit == 'true'
run: rm -rf public/uploads && cp -r tmp/app-cache/uploads public/uploads
- name: Create and migrate database
if: steps.app-cache.outputs.cache-hit != 'true'
run: |
bin/rake db:create
bin/rake db:migrate
- name: Create and migrate parallel databases
if: >-
env.USES_PARALLEL_DATABASES == 'true' &&
steps.app-cache.outputs.cache-hit != 'true'
run: |
bin/rake parallel:create
bin/rake parallel:migrate
- name: Dump database for cache
if: steps.app-cache.outputs.cache-hit != 'true'
run: mkdir -p tmp/app-cache && pg_dumpall > tmp/app-cache/cache.sql
- name: Dump uploads for cache
if: steps.app-cache.outputs.cache-hit != 'true'
run: rm -rf tmp/app-cache/uploads && cp -r public/uploads tmp/app-cache/uploads
- name: Fetch turbo_rspec_runtime.log cache
uses: actions/cache@v3
id: test-runtime-cache
if: matrix.build_type == 'backend' || matrix.build_type == 'system'
with:
path: tmp/turbo_rspec_runtime.log
key: rspec-runtime-${{ matrix.build_type }}-${{ matrix.target }}-${{ github.run_id }}
restore-keys: rspec-runtime-${{ matrix.build_type }}-${{ matrix.target }}-
- name: Check Zeitwerk eager_load
if: matrix.build_type == 'backend'
env:
LOAD_PLUGINS: ${{ (matrix.target == 'plugins') && '1' || '0' }}
run: |
if ! bin/rails zeitwerk:check --trace; then
echo
echo "---------------------------------------------"
echo
echo "::error::'bin/rails zeitwerk:check' failed - the app will fail to boot with 'eager_load=true' (e.g. in production)."
echo "To reproduce locally, run 'bin/rails zeitwerk:check'."
echo "Alternatively, you can run your local server/tests with the 'DISCOURSE_ZEITWERK_EAGER_LOAD=1' environment variable."
echo
exit 1
fi
- name: Check Zeitwerk reloading
if: matrix.build_type == 'backend'
env:
LOAD_PLUGINS: ${{ (matrix.target == 'plugins') && '1' || '0' }}
run: |
if ! bin/rails runner 'Rails.application.reloader.reload!'; then
echo
echo "---------------------------------------------"
echo
echo "::error::Zeitwerk reload failed - the app will not be able to reload properly in development."
echo "To reproduce locally, run \`bin/rails runner 'Rails.application.reloader.reload!'\`."
echo
exit 1
fi
- name: Core RSpec
if: matrix.build_type == 'backend' && matrix.target == 'core'
run: bin/turbo_rspec --use-runtime-info --verbose --format documentation
- name: Plugin RSpec
if: matrix.build_type == 'backend' && matrix.target == 'plugins'
run: bin/rake plugin:turbo_spec['*','--verbose --format documentation --use-runtime-info']
- name: Plugin QUnit
if: matrix.build_type == 'frontend' && matrix.target == 'plugins'
run: QUNIT_WRITE_EXECUTION_FILE=1 QUNIT_PARALLEL=3 bin/rake plugin:qunit['*','1200000']
timeout-minutes: 30
- uses: actions/upload-artifact@v3
if: matrix.build_type == 'frontend' && matrix.target == 'plugins'
with:
name: ember-exam-execution-plugins-frontend-${{(matrix.embroider == '1') && 'embroider' || 'classic'}}
path: ./app/assets/javascripts/discourse/test-execution-*.json
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Ember Build for System Tests
if: matrix.build_type == 'system'
run: bin/ember-cli --build
- name: Core System Tests
if: matrix.build_type == 'system' && matrix.target == 'core'
run: RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --use-runtime-info --profile=50 --verbose --format documentation spec/system
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Plugin System Tests
if: matrix.build_type == 'system' && matrix.target == 'plugins'
run: LOAD_PLUGINS=1 RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --use-runtime-info --profile=50 --verbose --format documentation plugins/*/spec/system
timeout-minutes: 30
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Upload failed system test screenshots
uses: actions/upload-artifact@v3
if: matrix.build_type == 'system' && failure()
with:
name: failed-system-test-screenshots
path: tmp/capybara/*.png
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Check Annotations
if: matrix.build_type == 'annotations'
run: |
bin/rake annotate:ensure_all_indexes
bin/annotate --models --model-dir app/models
if [ ! -z "$(git status --porcelain app/models/)" ]; then
echo "Core annotations are not up to date. To resolve, run:"
echo " bin/rake annotate:clean"
echo
echo "Or manually apply the diff printed below:"
echo "---------------------------------------------"
git -c color.ui=always diff app/models/
exit 1
fi
timeout-minutes: 30
core_frontend_tests:
2023-05-12 20:00:04 +08:00
if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
name: core frontend (${{ matrix.browser }})${{(matrix.embroider == '1') && ' (Embroider)' || ''}}
runs-on: ubuntu-20.04-8core
container:
image: discourse/discourse_test:slim-browsers
options: --user discourse
timeout-minutes: 35
strategy:
fail-fast: false
matrix:
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
embroider: ["1", "0"]
browser: ["Chrome", "Firefox ESR", "Firefox Evergreen"]
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
exclude:
# Testing the classic build on one browser is good enough
- embroider: "0"
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
browser: Firefox ESR
- embroider: "0"
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
browser: Firefox Evergreen
env:
DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 20:15:43 +08:00
USE_EMBROIDER: ${{ matrix.embroider }}
TESTEM_BROWSER: ${{ (startsWith(matrix.browser, 'Firefox') && 'Firefox') || matrix.browser }}
TESTEM_FIREFOX_PATH: ${{ (matrix.browser == 'Firefox Evergreen') && '/opt/firefox-evergreen/firefox' }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Git
run: |
git config --global user.email "ci@ci.invalid"
git config --global user.name "Discourse CI"
- name: Get yarn cache directory
id: yarn-cache-dir
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- name: Yarn cache
uses: actions/cache@v3
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
2023-05-12 20:00:04 +08:00
restore-keys: ${{ runner.os }}-yarn-
- name: Yarn install
working-directory: ./app/assets/javascripts/discourse
run: yarn install --frozen-lockfile
- name: Ember Build
working-directory: ./app/assets/javascripts/discourse
run: |
mkdir /tmp/emberbuild
2023-05-12 20:00:04 +08:00
yarn ember build --environment=test -o /tmp/emberbuild
- name: Core QUnit
working-directory: ./app/assets/javascripts/discourse
2023-05-12 20:00:04 +08:00
run: yarn ember exam --path /tmp/emberbuild --load-balance --parallel=5 --launch "${{ env.TESTEM_BROWSER }}" --write-execution-file --random
timeout-minutes: 15
- uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
2023-05-12 20:00:04 +08:00
name: ember-exam-execution-${{ matrix.browser }}
path: ./app/assets/javascripts/discourse/test-execution-*.json