DEV: Introduce feature-flag for Ember 5 upgrade

This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production.

Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace.

To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable.

When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs.

When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3.

Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.

.github/workflows/ember-version-enforcement.yml

@@ -0,0 +1,57 @@
+# This workflow is designed to stop us accidently committing the lockfile/packagejson symlink changes
+# which would cause the default Ember version to change.
+
+name: Ember Version Enforcement
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: write
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.head_ref }}
+      - name: Setup Node
+        uses: actions/setup-node@v3
+        with:
+          node-version: 20
+          cache: yarn
+      - name: Yarn install
+        run: |
+          cd app/assets/javascripts/discourse
+          yarn install
+      - name: "Check default ember version"
+        working-directory: app/assets/javascripts/discourse
+        run: |
+          node -e '
+            const version = require("ember-source/package.json").version;
+            console.log("Ember version is", version);
+            if(version.split(".")[0] !== "3"){
+              console.log(`Ember has unexpectedly been upgraded to version ${version}. If this is intentional, remove this github workflow.`);
+              process.exit(1);
+            }
+          '
+      - name: Upgrade ember
+        run: |
+          script/switch_ember_version 5
+          cd app/assets/javascripts/discourse
+          yarn install
+      - name: "Check upgraded version"
+        working-directory: app/assets/javascripts/discourse
+        run: |
+          node -e '
+            const version = require("ember-source/package.json").version;
+            console.log("Ember version is", version);
+            if(version.split(".")[0] !== "5"){
+              console.log(`Expected Ember 5, but found ${version}`);
+              process.exit(1);
+            }
+          '

.github/workflows/ember-version-lockfiles.yml

@@ -0,0 +1,38 @@
+# This workflow will run on dependabot pull requests to ensure that they update both the ember3 and ember5 lockfiles
+
+name: Ember Version Lockfiles
+
+on:
+  - pull_request
+
+permissions:
+  contents: write
+
+jobs:
+  help_dependabot:
+    runs-on: ubuntu-latest
+    if: ${{ github.actor == 'dependabot[bot]' }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.head_ref }}
+      - name: Move lockfile to correct location
+        run: |
+          # Dependabot gets confused by the symlinks and dumps the updated lockfile in the root of the repo.
+          # Let's move it back to the correct location.
+          if [[ -f yarn-ember3.lock ]]; then
+            mv yarn-ember3.lock app/assets/javascripts/yarn-ember3.lock
+          fi
+      - name: Update ember5 lockfile
+        run: script/regen_ember_5_lockfile
+      - name: Push changes
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          if [ ! -z "$(git status --porcelain)" ]; then
+            git config --global user.email "build@discourse.org"
+            git config --global user.name "discoursebuild"
+            git add .
+            git commit -m "Update lockfiles for ember version flag"
+            git push
+          fi

.github/workflows/tests.yml

@@ -295,7 +295,7 @@ jobs:
 
   core_frontend_tests:
     if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
-    name: core frontend (${{ matrix.browser }})
+    name: core frontend (${{ matrix.browser }})${{ matrix.updated_ember && ' (Ember 5)' || '' }}
     runs-on: ubuntu-20.04-8core
     container:
       image: discourse/discourse_test:slim-browsers
@@ -307,6 +307,10 @@ jobs:
       fail-fast: false
       matrix:
         browser: ["Chrome", "Firefox ESR", "Firefox Evergreen"]
+        updated_ember: [false]
+        include:
+          - browser: Chrome
+            updated_ember: true
 
     env:
       TESTEM_BROWSER: ${{ (startsWith(matrix.browser, 'Firefox') && 'Firefox') || matrix.browser }}
@@ -333,6 +337,10 @@ jobs:
           path: ${{ steps.yarn-cache-dir.outputs.dir }}
           key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}-cachev2
 
+      - name: Upgrade Ember
+        if: matrix.updated_ember == true
+        run: script/switch_ember_version 5
+
       - name: Yarn install
         working-directory: ./app/assets/javascripts/discourse
         run: yarn install --frozen-lockfile

app/assets/javascripts/package-ember5.json

@@ -0,0 +1,35 @@
+{
+  "private": true,
+  "scripts": {
+    "postinstall": "./run-patch-package"
+  },
+  "workspaces": [
+    "admin",
+    "bootstrap-json",
+    "deprecation-silencer",
+    "dialog-holder",
+    "discourse",
+    "discourse-common",
+    "discourse-hbr",
+    "discourse-i18n",
+    "discourse-markdown-it",
+    "discourse-plugins",
+    "discourse-widget-hbs",
+    "ember-cli-progress-ci",
+    "ember-production-deprecations",
+    "float-kit",
+    "pretty-text",
+    "select-kit",
+    "theme-transpiler",
+    "truth-helpers",
+    "wizard"
+  ],
+  "resolutions": {
+    "**/unset-value": "2.0.1",
+    "**/ember-source": "5.4.0"
+  },
+  "devDependencies": {
+    "patch-package": "^8.0.0",
+    "postinstall-postinstall": "^2.1.0"
+  }
+}

app/assets/javascripts/package.json

@@ -0,0 +1 @@
+package-ember3.json

app/assets/javascripts/yarn.lock

@@ -0,0 +1 @@
+yarn-ember3.lock

bin/ember-cli

@@ -56,6 +56,17 @@ if !args.include?("test") && !args.include?("build") && !args.include?("--proxy"
   args << PROXY
 end
 
+if !["3", "5", nil].include?(ENV["EMBER_VERSION"])
+  raise "Unknown ember version #{ENV["EMBER_VERSION"]}"
+end
+
+system(
+  "script/switch_ember_version",
+  ENV["EMBER_VERSION"] || "3",
+  exception: true,
+  chdir: RAILS_ROOT,
+)
+
 # Running yarn install in the root directory will also run it for YARN_DIR via a post-install hook
 exit 1 if !system "yarn", "-s", "install", "--cwd", RAILS_ROOT
 

lib/tasks/assets.rake

@@ -8,6 +8,16 @@ end
 
 task "assets:precompile:build" do
   if ENV["SKIP_EMBER_CLI_COMPILE"] != "1"
+    ember_version = ENV["EMBER_VERSION"] || "3"
+
+    raise "Unknown ember version '#{ember_version}'" if !%w[3 5].include?(ember_version)
+
+    if ENV["EMBER_VERSION"] == "5"
+      puts "Upgrading to Ember 5..."
+      system("script/switch_ember_version", ember_version, exception: true, chdir: Rails.root)
+      system("yarn install", exception: true, chdir: "app/assets/javascripts/discourse")
+    end
+
     compile_command = "yarn --cwd app/assets/javascripts/discourse run ember build"
 
     heap_size_limit = check_node_heap_size_limit

script/regen_ember_5_lockfile

@@ -0,0 +1,21 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "fileutils"
+
+# rubocop:disable Discourse/NoChdir
+Dir.chdir("#{__dir__}/../app/assets/javascripts") do
+  FileUtils.mv("yarn.lock", "yarn.lock-tmp")
+  FileUtils.mv("package.json", "package.json-tmp")
+
+  File.symlink("package-ember5.json", "package.json")
+  File.symlink("yarn-ember5.lock", "yarn.lock")
+
+  system "yarn install", exception: true
+
+  FileUtils.rm("yarn-ember5.lock")
+  FileUtils.cp("yarn-ember3.lock", "yarn-ember5.lock")
+
+  FileUtils.mv("yarn.lock-tmp", "yarn.lock")
+  FileUtils.mv("package.json-tmp", "package.json")
+end

script/switch_ember_version

@@ -0,0 +1,17 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "fileutils"
+
+v = ARGV[0]
+
+raise "Unexpected version #{v}" if !%w[3 5].include?(v)
+
+# rubocop:disable Discourse/NoChdir
+Dir.chdir("#{__dir__}/../app/assets/javascripts") do
+  FileUtils.rm("package.json")
+  FileUtils.rm("yarn.lock")
+
+  File.symlink("package-ember#{v}.json", "package.json")
+  File.symlink("yarn-ember#{v}.lock", "yarn.lock")
+end