discourse/script/check_reproducible_assets.rb
Godfrey Chan e1373c3e84
DEV: introduce Embroider behind a flag, and start testing in CI (#23005)
Discourse core now builds and runs with Embroider! This commit adds
the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start
testing it on CI.

The new pipeline uses Embroider's compat mode + webpack bundler to
build discourse code, and leave everything else (admin, wizard,
markdown-it, plugins, etc) exactly the same using the existing
Broccoli-based build as external bundles (<script> tags), passed
to the build as `extraPublicTress` (which just means they get
placed in the `/public` folder).

At runtime, these "external" bundles are glued back together with
`loader.js`. Specifically, the external bundles are compiled as
AMD modules (just as they were before) and registered with the
global `loader.js` instance. They expect their `import`s (outside
of whatever is included in the bundle) to be already available in
the `loader.js` runtime registry.

In the classic build, _every_ module gets compiled into AMD and
gets added to the `loader.js` runtime registry. In Embroider,
the goal is to do this as little as possible, to give the bundler
more flexibility to optimize modules, or omit them entirely if it
is confident that the module is unused (i.e. tree-shaking).

Even in the most compatible mode, there are cases where Embroider
is confident enough to omit modules in the runtime `loader.js`
registry (notably, "auto-imported" non-addon NPM packages). So we
have to be mindful of that an manage those dependencies ourselves,
as seen in #22703.

In the longer term, we will look into using modern features (such
as `import()`) to express these inter-dependencies.

This will only be behind a flag for a short period of time while we
perform some final testing. Within the next few weeks, we intend
to enable by default and remove the flag.

---------

Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 13:15:43 +01:00

60 lines
1.8 KiB
Ruby
Executable File

#! /usr/bin/env ruby
# frozen_string_literal: true
# It's important that our JS asset builds are reproducible so that users aren't forced to re-download
# assets after every deploy. This script runs two builds and compares the output to ensure that they
# are identical.
require "digest"
DIST_DIR = File.expand_path("#{__dir__}/../app/assets/javascripts/discourse/dist")
def collect_asset_info
files =
Dir.glob("**/*", base: DIST_DIR).reject { |path| File.directory? "#{DIST_DIR}/#{path}" }.sort
puts "Found #{files.length} files"
raise "No files found" if files.empty?
digests = files.map { |file| Digest::MD5.file("#{DIST_DIR}/#{file}").hexdigest }
{ files: files, digests: digests }
end
puts "Running first build..."
system "#{__dir__}/../bin/yarn-app ember build -prod", exception: true
first_build_info = collect_asset_info
puts "Running second build..."
system "#{__dir__}/../bin/yarn-app ember build -prod", exception: true
second_build_info = collect_asset_info
puts nil, nil, "Comparing builds...", nil, nil
if first_build_info[:files] != second_build_info[:files]
puts "Set of files is different"
new_assets = first_build_info[:files].difference(second_build_info[:files])
puts "Second build had additional assets: #{new_assets.inspect}"
missing_assets = second_build_info[:files].difference(first_build_info[:files])
puts "Second build was missing assets: #{missing_assets.inspect}"
exit 1
else
puts "Both builds produced the same file names"
end
if first_build_info[:digests] != second_build_info[:digests]
puts "File digests are different"
first_build_info[:files].each_with_index do |file, index|
if first_build_info[:digests][index] != second_build_info[:digests][index]
puts "File #{file} has different digest"
end
end
exit 1
else
puts "Files in both builds had identical digests"
end
puts nil, "Success!"