discourse/.github/workflows/tests.yml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

377 lines
14 KiB
YAML
Raw Normal View History

name: Tests
on:
pull_request:
paths-ignore:
- "migrations/**"
push:
branches:
- main
- beta
- stable
paths-ignore:
- "migrations/**"
concurrency:
group: tests-${{ format('{0}-{1}', github.head_ref || github.run_number, github.job) }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
2023-05-12 20:00:04 +08:00
if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
name: ${{ matrix.target }} ${{ matrix.build_type }}${{ matrix.updated_ember && ' (Ember 5)' || '' }}
runs-on: ${{ (matrix.build_type == 'annotations') && 'ubuntu-latest' || 'ubuntu-20.04-8core' }}
container: discourse/discourse_test:slim${{ (matrix.build_type == 'frontend' || matrix.build_type == 'system') && '-browsers' || '' }}${{ (matrix.ruby == '3.1') && '-ruby-3.1.0' || '' }}
timeout-minutes: 20
env:
DISCOURSE_HOSTNAME: www.example.com
RAILS_ENV: test
PGUSER: discourse
PGPASSWORD: discourse
USES_PARALLEL_DATABASES: ${{ matrix.build_type == 'backend' || matrix.build_type == 'system' }}
CAPYBARA_DEFAULT_MAX_WAIT_TIME: 10
MINIO_RUNNER_LOG_LEVEL: DEBUG
EMBER_VERSION: ${{ (matrix.updated_ember && '5') || '3' }}
strategy:
fail-fast: false
matrix:
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
build_type: [backend, frontend, system, annotations]
target: [core, plugins, themes]
2023-05-12 20:00:04 +08:00
ruby: ["3.2"]
updated_ember: [false]
exclude:
- build_type: annotations
target: plugins
- build_type: annotations
target: themes
- build_type: backend
target: themes
- build_type: frontend
target: core # Handled by core_frontend_tests job (below)
include:
- build_type: system
target: chat
ruby: "3.2"
- build_type: system
target: chat
updated_ember: true
- build_type: system
target: core
updated_ember: true
steps:
- name: Set working directory owner
run: chown root:root .
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Git
run: |
git config --global user.email "ci@ci.invalid"
git config --global user.name "Discourse CI"
- name: Start redis
run: |
redis-server /etc/redis/redis.conf &
- name: Start Postgres
run: |
chown -R postgres /var/run/postgresql
sudo -E -u postgres script/start_test_db.rb
sudo -u postgres psql -c "CREATE ROLE $PGUSER LOGIN SUPERUSER PASSWORD '$PGPASSWORD';"
- name: Bundler cache
uses: actions/cache@v3
with:
path: vendor/bundle
key: ${{ runner.os }}-${{ matrix.ruby }}-gem-${{ hashFiles('**/Gemfile.lock') }}-cachev2
- name: Setup gems
run: |
gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
bundle config --local path vendor/bundle
bundle config --local deployment true
bundle config --local without development
bundle install --jobs 4
bundle clean
- name: Get yarn cache directory
id: yarn-cache-dir
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- name: Yarn cache
uses: actions/cache@v3
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}-cachev2
- name: Upgrade Ember
if: matrix.updated_ember == true
run: script/switch_ember_version 5
- name: Yarn install
run: yarn install --frozen-lockfile
- name: Checkout official plugins
if: matrix.target == 'plugins'
run: bin/rake plugin:install_all_official
- name: Pull compatible versions of plugins
if: matrix.target == 'plugins'
run: bin/rake plugin:pull_compatible_all
- name: Plugin gems cache
uses: actions/cache@v3
with:
path: plugins/*/gems
key: ${{ runner.os }}-plugin-gems-${{matrix.ruby}}-${{ hashFiles('plugins/*/plugin.rb') }}
- name: Checkout official themes
DEV: Run QUnit tests for official Discourse themes (#24405) Why this change? As the number of themes which the Discourse team supports officially grows, we want to ensure that changes made to Discourse core do not break the plugins. As such, we are adding a step to our Github actions test job to run the QUnit tests for all official themes. What does this change do? This change adds a new job to our tests Github actions workflow to run the QUnit tests for all official plugins. This is achieved with the following changes: 1. Update `testem.js` to rely on the `THEME_TEST_PAGES` env variable to set the `test_page` option when running theme QUnit tests with testem. The `test_page` option [allows an array to be specified](https://github.com/testem/testem#multiple-test-pages) such that tests for multiple pages can be run at the same time. We are relying on a ENV variable because the `testem` CLI does not support passing a list of pages to the `--test_page` option. 2. Support a `/testem-theme-qunit/:testem_id/theme-qunit` Rails route in the development environment. This is done because testem prefixes the path with a unique ID to the configured `test_page` URL. This is problematic for us because we proxy all testem requests to the Rails server and testem's proxy configuration option does not allow us to easily rewrite the URL to remove the prefix. Therefore, we configure a proxy in testem to prefix `theme-qunit` requests with `/testem-theme-qunit` which can then be easily identified by the Rails server and routed accordingly. 3. Update `qunit:test` to support a `THEME_IDS` environment variable which will allow it to run QUnit tests for multiple themes at the same time. 4. Support `bin/rake themes:qunit[ids,"<theme_id>|<theme_id>"]` to run the QUnit tests for multiple themes at the same time. 5. Adds a `themes:qunit_all_official` Rake task which runs the QUnit tests for all the official themes.
2023-11-17 07:17:32 +08:00
if: matrix.target == 'themes' && matrix.build_type == 'system'
run: bin/rake themes:clone_all_official
- name: Add hosts to /etc/hosts, otherwise Chrome cannot reach minio
if: matrix.build_type == 'system' && matrix.target == 'core'
run: |
echo "127.0.0.1 minio.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 discoursetest.minio.local" | sudo tee -a /etc/hosts
- name: Fetch app state cache
uses: actions/cache@v3
id: app-cache
with:
path: tmp/app-cache
2023-05-12 20:00:04 +08:00
key: >-
${{ runner.os }}-
${{ hashFiles('.github/workflows/tests.yml') }}-
${{ hashFiles('db/**/*', 'plugins/**/db/**/*') }}-
${{ env.USES_PARALLEL_DATABASES }}
- name: Restore database from cache
if: steps.app-cache.outputs.cache-hit == 'true'
run: script/silence_successful_output psql --quiet -o /dev/null -f tmp/app-cache/cache.sql postgres
- name: Restore uploads from cache
if: steps.app-cache.outputs.cache-hit == 'true'
run: rm -rf public/uploads && cp -r tmp/app-cache/uploads public/uploads
- name: Create and migrate database
if: steps.app-cache.outputs.cache-hit != 'true'
run: |
bin/rake db:create
script/silence_successful_output bin/rake db:migrate
- name: Create and migrate parallel databases
if: >-
env.USES_PARALLEL_DATABASES == 'true' &&
steps.app-cache.outputs.cache-hit != 'true'
run: |
bin/rake parallel:create
script/silence_successful_output bin/rake parallel:migrate
- name: Dump database for cache
if: steps.app-cache.outputs.cache-hit != 'true'
run: mkdir -p tmp/app-cache && pg_dumpall > tmp/app-cache/cache.sql
- name: Dump uploads for cache
if: steps.app-cache.outputs.cache-hit != 'true'
run: rm -rf tmp/app-cache/uploads && cp -r public/uploads tmp/app-cache/uploads
- name: Fetch turbo_rspec_runtime.log cache
uses: actions/cache@v3
id: test-runtime-cache
if: matrix.build_type == 'backend' || matrix.build_type == 'system'
with:
path: tmp/turbo_rspec_runtime.log
key: rspec-runtime-${{ matrix.build_type }}-${{ matrix.target }}-${{ github.run_id }}
restore-keys: rspec-runtime-${{ matrix.build_type }}-${{ matrix.target }}-
- name: Check Zeitwerk eager_load
if: matrix.build_type == 'backend'
env:
LOAD_PLUGINS: ${{ (matrix.target == 'plugins') && '1' || '0' }}
run: |
if ! bin/rails zeitwerk:check --trace; then
echo
echo "---------------------------------------------"
echo
echo "::error::'bin/rails zeitwerk:check' failed - the app will fail to boot with 'eager_load=true' (e.g. in production)."
echo "To reproduce locally, run 'bin/rails zeitwerk:check'."
echo "Alternatively, you can run your local server/tests with the 'DISCOURSE_ZEITWERK_EAGER_LOAD=1' environment variable."
echo
exit 1
fi
- name: Check Zeitwerk reloading
if: matrix.build_type == 'backend'
env:
LOAD_PLUGINS: ${{ (matrix.target == 'plugins') && '1' || '0' }}
run: |
if ! bin/rails runner 'Rails.application.reloader.reload!'; then
echo
echo "---------------------------------------------"
echo
echo "::error::Zeitwerk reload failed - the app will not be able to reload properly in development."
echo "To reproduce locally, run \`bin/rails runner 'Rails.application.reloader.reload!'\`."
echo
exit 1
fi
- name: Core RSpec
if: matrix.build_type == 'backend' && matrix.target == 'core'
run: bin/turbo_rspec --use-runtime-info --verbose --format documentation
- name: Plugin RSpec
if: matrix.build_type == 'backend' && matrix.target == 'plugins'
run: bin/rake plugin:turbo_spec['*','--verbose --format documentation --use-runtime-info']
- name: Plugin QUnit
if: matrix.build_type == 'frontend' && matrix.target == 'plugins'
run: QUNIT_WRITE_EXECUTION_FILE=1 QUNIT_PARALLEL=3 bin/rake plugin:qunit['*','1200000']
timeout-minutes: 30
DEV: Run QUnit tests for official Discourse themes (#24405) Why this change? As the number of themes which the Discourse team supports officially grows, we want to ensure that changes made to Discourse core do not break the plugins. As such, we are adding a step to our Github actions test job to run the QUnit tests for all official themes. What does this change do? This change adds a new job to our tests Github actions workflow to run the QUnit tests for all official plugins. This is achieved with the following changes: 1. Update `testem.js` to rely on the `THEME_TEST_PAGES` env variable to set the `test_page` option when running theme QUnit tests with testem. The `test_page` option [allows an array to be specified](https://github.com/testem/testem#multiple-test-pages) such that tests for multiple pages can be run at the same time. We are relying on a ENV variable because the `testem` CLI does not support passing a list of pages to the `--test_page` option. 2. Support a `/testem-theme-qunit/:testem_id/theme-qunit` Rails route in the development environment. This is done because testem prefixes the path with a unique ID to the configured `test_page` URL. This is problematic for us because we proxy all testem requests to the Rails server and testem's proxy configuration option does not allow us to easily rewrite the URL to remove the prefix. Therefore, we configure a proxy in testem to prefix `theme-qunit` requests with `/testem-theme-qunit` which can then be easily identified by the Rails server and routed accordingly. 3. Update `qunit:test` to support a `THEME_IDS` environment variable which will allow it to run QUnit tests for multiple themes at the same time. 4. Support `bin/rake themes:qunit[ids,"<theme_id>|<theme_id>"]` to run the QUnit tests for multiple themes at the same time. 5. Adds a `themes:qunit_all_official` Rake task which runs the QUnit tests for all the official themes.
2023-11-17 07:17:32 +08:00
- name: Theme QUnit
if: matrix.build_type == 'frontend' && matrix.target == 'themes'
run: DISCOURSE_DEV_DB=discourse_test QUNIT_PARALLEL=3 bin/rake themes:qunit_all_official
timeout-minutes: 15
- uses: actions/upload-artifact@v3
if: always() && matrix.build_type == 'frontend' && matrix.target == 'plugins'
with:
name: ember-exam-execution-plugins-frontend
path: ./app/assets/javascripts/discourse/test-execution-*.json
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Ember Build for System Tests
if: matrix.build_type == 'system'
run: bin/ember-cli --build
- name: Ensure latest minio binary installed for Core System Tests
if: matrix.build_type == 'system' && matrix.target == 'core'
run: bundle exec ruby script/install_minio_binaries.rb
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Core System Tests
if: matrix.build_type == 'system' && matrix.target == 'core'
run: RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --use-runtime-info --profile=50 --verbose --format documentation spec/system
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Plugin System Tests
if: matrix.build_type == 'system' && matrix.target == 'plugins'
run: |
GLOBIGNORE="plugins/chat/*";
LOAD_PLUGINS=1 RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --use-runtime-info --profile=50 --verbose --format documentation plugins/*/spec/system
shell: bash
timeout-minutes: 30
- name: Chat System Tests
if: matrix.build_type == 'system' && matrix.target == 'chat'
run: LOAD_PLUGINS=1 RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --use-runtime-info --profile=50 --verbose --format documentation plugins/chat/spec/system
timeout-minutes: 30
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Theme System Tests
if: matrix.build_type == 'system' && matrix.target == 'themes'
run: |
RAILS_ENABLE_TEST_LOG=1 RAILS_TEST_LOG_LEVEL=error PARALLEL_TEST_PROCESSORS=4 bin/turbo_rspec --profile=50 --verbose --format documentation tmp/themes/*/spec/system
shell: bash
timeout-minutes: 30
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Upload failed system test screenshots
uses: actions/upload-artifact@v3
if: matrix.build_type == 'system' && failure()
with:
name: failed-system-test-screenshots
path: tmp/capybara/*.png
DEV: Minimal first pass of rails system test setup (#16311) This commit introduces rails system tests run with chromedriver, selenium, and headless chrome to our testing toolbox. We use the `webdrivers` gem and `selenium-webdriver` which is what the latest Rails uses so the tests run locally and in CI out of the box. You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra verbose logs of what selenium is doing to communicate with the system tests. By default JS logs are verbose so errors from JS are shown when running system tests, you can disable this with `SELENIUM_DISABLE_VERBOSE_JS_LOGS=1` You can use `SELENIUM_HEADLESS=0` to run the system tests inside a chrome browser instead of headless, which can be useful to debug things and see what the spec sees. See note above about `bin/ember-cli` to avoid surprises. I have modified `bin/turbo_rspec` to exclude `spec/system` by default, support for parallel system specs is a little shaky right now and we don't want them slowing down the turbo by default either. ### PageObjects and System Tests To make querying and inspecting parts of the page easier and more reusable inbetween system tests, we are using the concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in our system tests. A "Page" here is generally corresponds to an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`, and this contains logic for querying components within the topic such as "Posts". I have also split "Modals" into their own entity. Further down the line we may want to explore creating independent "Component" contexts. Capybara DSL should be included in each PageObject class, reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl For system tests, since they are so slow, we want to focus on the "happy path" and not do every different possible context and branch check using them. They are meant to be overarching tests that check a number of things are correct using the full stack from JS and ember to rails to ruby and then the database. ### CI Setup Whenever a system spec fails, a screenshot is taken and a build artifact is produced _after the entire CI run is complete_, which can be downloaded from the Actions UI in the repo. Most importantly, a step to build the Ember app using Ember CLI is needed, otherwise the JS assets cannot be found by capybara: ``` - name: Build Ember CLI run: bin/ember-cli --build ``` A new `--build` argument has been added to `bin/ember-cli` for this case, which is not needed locally if you already have the discourse rails server running via `bin/ember-cli -u` since the whole server is built and set up by default. Co-authored-by: David Taylor <david@taylorhq.com>
2022-09-28 09:48:16 +08:00
- name: Check Annotations
if: matrix.build_type == 'annotations'
run: |
bin/rake annotate:ensure_all_indexes
bin/annotate --models --model-dir app/models
if [ ! -z "$(git status --porcelain app/models/)" ]; then
echo "Core annotations are not up to date. To resolve, run:"
echo " bin/rake annotate:clean"
echo
echo "Or manually apply the diff printed below:"
echo "---------------------------------------------"
git -c color.ui=always diff app/models/
exit 1
fi
timeout-minutes: 30
core_frontend_tests:
2023-05-12 20:00:04 +08:00
if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
DEV: Introduce feature-flag for Ember 5 upgrade This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production. Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace. To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable. When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs. When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3. Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 23:52:17 +08:00
name: core frontend (${{ matrix.browser }})${{ matrix.updated_ember && ' (Ember 5)' || '' }}
runs-on: ubuntu-20.04-8core
container:
image: discourse/discourse_test:slim-browsers
options: --user discourse
timeout-minutes: 35
strategy:
fail-fast: false
matrix:
browser: ["Chrome", "Firefox ESR", "Firefox Evergreen"]
DEV: Introduce feature-flag for Ember 5 upgrade This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production. Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace. To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable. When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs. When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3. Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 23:52:17 +08:00
updated_ember: [false]
include:
- browser: Chrome
updated_ember: true
env:
TESTEM_BROWSER: ${{ (startsWith(matrix.browser, 'Firefox') && 'Firefox') || matrix.browser }}
TESTEM_FIREFOX_PATH: ${{ (matrix.browser == 'Firefox Evergreen') && '/opt/firefox-evergreen/firefox' }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Git
run: |
git config --global user.email "ci@ci.invalid"
git config --global user.name "Discourse CI"
- name: Get yarn cache directory
id: yarn-cache-dir
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- name: Yarn cache
uses: actions/cache@v3
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}-cachev2
DEV: Introduce feature-flag for Ember 5 upgrade This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production. Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace. To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable. When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs. When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3. Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 23:52:17 +08:00
- name: Upgrade Ember
if: matrix.updated_ember == true
run: script/switch_ember_version 5
- name: Yarn install
working-directory: ./app/assets/javascripts/discourse
run: yarn install --frozen-lockfile
- name: Ember Build
working-directory: ./app/assets/javascripts/discourse
run: |
mkdir /tmp/emberbuild
2023-05-12 20:00:04 +08:00
yarn ember build --environment=test -o /tmp/emberbuild
- name: Core QUnit
working-directory: ./app/assets/javascripts/discourse
2023-05-12 20:00:04 +08:00
run: yarn ember exam --path /tmp/emberbuild --load-balance --parallel=5 --launch "${{ env.TESTEM_BROWSER }}" --write-execution-file --random
timeout-minutes: 15
- uses: actions/upload-artifact@v3
if: ${{ always() }}
with:
2023-05-12 20:00:04 +08:00
name: ember-exam-execution-${{ matrix.browser }}
path: ./app/assets/javascripts/discourse/test-execution-*.json