Commit Graph

164 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan
22614ca85b
DEV: Compile theme migrations javascript files when running theme qunit (#25219)
Why this change?

Currently, is it hard to iteratively write a theme settings migrations
because our theme migrations system does not rollback. Therefore, we
want to allow theme developers to be able to write QUnit tests for their
theme migrations files enabling them to iteratively write their theme
migrations.

What does this change do?

1. Update `Theme#baked_js_tests_with_digest` to include all `ThemeField`
records of `ThemeField#target` equal to `migrations`. Note that we do
not include the `settings` and `themePrefix` variables for migration files.

2. Don't minify JavaScript test files becasue it makes debugging in
   development hard.
2024-01-16 09:50:44 +08:00
Alan Guo Xiang Tan
59839e428f
DEV: Add skip_migrations param when importing remote theme (#25218)
Why this change?

Importing theme with the `bundle` params is used mainly by
`discourse_theme` CLI in the development environment. However, we do not
want migrations to automatically run in the development environment
and instead want the developer to be intentional about running theme
migrations. As such, this commit adds support for a
`skip_migrations` param when importing a theme with the `bundle` params.

This commit also adds a `migrated` attribute for migrations theme fields
to indicate whether a migrations theme field has been migrated or not.
2024-01-11 14:04:02 +08:00
Sam
b09422428d
DEV: update syntax tree to latest (#24623)
update format to latest syntax tree
2023-11-29 16:38:07 +11:00
David Taylor
eda79186ee
FIX: Recompile theme translations when fallback data changes (#24371)
Previously we would only recompile a theme locale when its own data changes. However, the output also includes fallback data from other locales, so we need to invalidate all locales when fallback locale data is changed. Building a list of dependent locales is tricky, so let's just invalidate them all.
2023-11-14 19:53:27 +00:00
Osama Sayegh
3cadd6769e
FEATURE: Theme settings migrations (#24071)
This commit introduces a new feature that allows theme developers to manage the transformation of theme settings over time. Similar to Rails migrations, the theme settings migration system enables developers to write and execute migrations for theme settings, ensuring a smooth transition when changes are required in the format or structure of setting values.

Example use cases for the theme settings migration system:

1. Renaming a theme setting.

2. Changing the data type of a theme setting (e.g., transforming a string setting containing comma-separated values into a proper list setting).

3. Altering the format of data stored in a theme setting.

All of these use cases and more are now possible while preserving theme setting values for sites that have already modified their theme settings.

Usage:

1. Create a top-level directory called `migrations` in your theme/component, and then within the `migrations` directory create another directory called `settings`.

2. Inside the `migrations/settings` directory, create a JavaScript file using the format `XXXX-some-name.js`, where `XXXX` is a unique 4-digit number, and `some-name` is a descriptor of your choice that describes the migration.

3. Within the JavaScript file, define and export (as the default) a function called `migrate`. This function will receive a `Map` object and must also return a `Map` object (it's acceptable to return the same `Map` object that the function received).

4. The `Map` object received by the `migrate` function will include settings that have been overridden or changed by site administrators. Settings that have never been changed from the default will not be included.

5. The keys and values contained in the `Map` object that the `migrate` function returns will replace all the currently changed settings of the theme.

6. Migrations are executed in numerical order based on the XXXX segment in the migration filenames. For instance, `0001-some-migration.js` will be executed before `0002-another-migration.js`.

Here's a complete example migration script that renames a setting from `setting_with_old_name` to `setting_with_new_name`:

```js
// File name: 0001-rename-setting.js

export default function migrate(settings) {
  if (settings.has("setting_with_old_name")) {
    settings.set("setting_with_new_name", settings.get("setting_with_old_name"));
  }
  return settings;
}
```

Internal topic: t/109980
2023-11-02 08:10:15 +03:00
Alan Guo Xiang Tan
f2a90afa4c
DEV: Introduce Theme#get_setting (#24032)
Why this change?

Currently, we do not have a method to easily retrieve a theme setting's
value on the server side. Such a method can be useful in the test
environment where we need to retrieve the theme's setting and use its
value in assertions.

What does this change do?

This change introduces the `Theme#get_setting` instance method.
2023-10-23 07:41:40 +08:00
David Taylor
525cfcbe0e
FIX: Ensure nested ember components can be used with mustache syntax (#23912)
We run the ember-this-fallback transformation on plugin and theme code so that they can continue omitting `this.` in `.hbs` templates. A bug in the implementation meant that it was incorrectly transforming things like `{{dir/some-component}}` into `<DirSomeComponent />` (rather than `<Dir::SomeComponent />`).

This commit uses patch-package to apply the fix from https://github.com/tildeio/ember-this-fallback/pull/56
2023-10-12 11:08:57 +01:00
Jarek Radosz
5a904949b2
DEV: Add gjs support for themes (#23473) 2023-10-02 12:36:06 +02:00
tshenry
c163634ff9
DEV: Bump theme compiler version for max svg sprite size change (#23567)
This should have been included in f6326d0
2023-09-13 10:02:08 -07:00
Joffrey JAFFEUX
f1d8cd529e
Revert "Revert "PERF: Cache each theme field value once (#23192)" (#23354)" (#23356)
This reverts commit 9821ca9413.
2023-08-31 14:12:03 -05:00
Joffrey JAFFEUX
9821ca9413
Revert "PERF: Cache each theme field value once (#23192)" (#23354)
This reverts commit 82a56334a3.
2023-08-31 19:04:43 +02:00
Daniel Waterworth
82a56334a3
PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-08-31 11:24:02 -05:00
Jarek Radosz
70f1cc5552
DEV: Use esbuild to make DiscourseJsProcessor (#23223)
Reverts e2705df and re-lands #23187 and #23219.

The issue was incorrect order of execution of Rails' `assets:precompile` task in our own precompilation stack.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 16:36:22 +02:00
David Taylor
e2705df0f4
Revert "DEV: Use esbuild to make DiscourseJsProcessor (#23187)" (#23221)
This reverts commit 4dfe25d062 and 4fdeb6281e. We are investigating an issue related to asset compilation and S3 assets
2023-08-24 13:25:44 +01:00
Jarek Radosz
4dfe25d062
DEV: Use esbuild to make DiscourseJsProcessor (#23187)
Co-authored-by: David Taylor <david@taylorhq.com>
2023-08-24 12:43:59 +02:00
David Taylor
82b16f4f47
DEV: Do not manipulate theme module paths at build-time (#23148)
Manipulating theme module paths means that the paths you author are not the ones used at runtime. This can lead to some very unexpected behavior and potential module name clashes. It also meant that the refactor in 16c6ab8661 was unable to correctly match up theme connector js/templates.

While this could technically be a breaking change, I think it is reasonably safe because:

1. Themes are already forced to use relative paths when referencing their own modules (since they're namespaced based on the site-specific id). The only time this might be problematic is when theme tests reference modules in the theme's main `javascripts` directory

2. For things like components/services/controllers/etc. our custom Ember resolver works backwards from the end of the path, so adding `discourse/` in the middle will not affect resolution.
2023-08-18 18:15:23 +01:00
Daniel Waterworth
91f560a521
DEV: Make every DistributedCache lazily instantiated (#23147) 2023-08-18 10:59:11 -05:00
David Taylor
16c6ab8661
DEV: Allow plugin outlets to be defined using gjs (#23142)
Previously we were discovering plugin outlets by checking first for dedicated template files, and then looking for classes to match them. This doesn't work for components which are entirely defined in JS (e.g. those authored with gjs, or those which are re-exports of a colocated component).

This commit refactors our detection logic to look for both class and template modules in a single pass. It also refactors things so that the modules themselves are required lazily when needd, rather than all being loaded during app boot.
2023-08-18 12:07:10 +01:00
David Taylor
e76e0ad592
DEV: In development, refresh client when theme changes are made (#22978)
This brings the theme development experience (via the discourse_theme cli) closer to the experience of making javascript changes in Discourse core/plugins via Ember CLI. Whenever a change is made to a non-css theme field, all clients will be instructed to immediately refresh via message-bus.
2023-08-04 11:02:26 +01:00
Daniel Waterworth
b7404373cf
FIX: Always clear caches after committing the current transaction (#22550)
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.

The only exception to this is caches that aren't caching things from
postgres.

This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.

Reapplying this - it now doesn't use after_commit if skip_db is set
2023-07-12 09:49:28 -05:00
Daniel Waterworth
cb794275a7
Revert "FIX: Always clear caches after committing the current transaction" (#22493)
This reverts commit 8310c7842c.

This was breaking precompilation.
2023-07-07 18:05:38 -05:00
Daniel Waterworth
9dd01ca2ef
FIX: Always clear caches after committing the current transaction (#22491)
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.

The only exception to this is caches that aren't caching things from
postgres.

This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.
2023-07-07 14:24:56 -05:00
Daniel Waterworth
6f8056572e
DEV: Refactor updating parent/child relationships between themes (#22244)
The prior add_relative_themes! method was confusing and cleared the
cache repeatedly instead of once for the whole operation.
2023-06-22 13:57:39 -05:00
Blake Erickson
56b74e6042
SECURITY: Limit ThemeField value length to prevent DoS (#22087)
Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
2023-06-13 12:07:47 -05:00
Daniel Waterworth
3d3e8c8fbd
FIX: json_schemer no longer allows nil keys (#22015) 2023-06-08 15:12:47 -05:00
Daniel Waterworth
ae63d9ea09
PERF: Remove unnecessary queries from Theme#set_field (#21281) 2023-04-27 13:04:58 -05:00
Daniel Waterworth
0728fa055b
DEV: Make theme cache work like other caches (#20813) 2023-03-24 12:37:16 -05:00
Daniel Waterworth
84f590ab83
DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
David Taylor
41f3bb8b50
FIX: Restore class-property babel transform for themes (#19883)
This seems to be required for decorators to work on class properties. Followup to 624f4a7de9
2023-01-16 18:06:46 +00:00
David Taylor
624f4a7de9
Drop support for iOS < 15.7 (#19847)
https://meta.discourse.org/t/224747
2023-01-16 17:28:59 +00:00
David Taylor
5a003715d3
DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
Rafael dos Santos Silva
0434de6cee
DEV: Compatibility with Ruby 3.2 (#19303) 2022-12-13 16:03:53 -03:00
Kris
07e4b7550d
DEV: relocate :root CSS custom properties (#19389) 2022-12-09 14:33:15 -05:00
Isaac Janzen
aea492df5e
DEV: Add link rel preload to theme-javascripts (#19231)
* DEV: Add link rel preload to theme-javascripts
2022-11-30 12:43:01 -06:00
David Taylor
695b44269b
DEV: Do not remove debugger statements from themes (#18814) 2022-11-01 13:56:33 +00:00
Osama Sayegh
ce53152e53
DEV: Include theme_uploads and theme_uploads_local objects in theme tests (#18645)
Our theme system injects a magical `settings` object at the top of themes JS modules to allow theme authors to access the settings as configured by admins in the UI. Within this `settings` object, there are a couple of special objects `theme_uploads` and `theme_uploads_local` that contain URLs for all the assets/uploads that the theme has.

For test modules/files, the theme system also injects a `settings` object at the top of tests modules, but it's not the same object as the object that's injected in non-test files. The difference is that in tests we want the settings to have their default values as opposed to any custom values that may exist in the site's database. This ensures that test results are consistent no matter the site that runs them.

However, the `settings` object in tests files currently doesn't have the special objects `theme_uploads` and `theme_uploads_local` which means that if a theme includes an asset that's lazy-loaded, it's not possible to write tests for anything that depends on the lazy-loaded asset because the theme will not be able to load the asset during the tests since `theme_uploads_local` and `theme_uploads` don't exist. This PR adds these special objects inside the `settings` object for test files.

Internal topic: t/71825/52.
2022-10-20 08:00:29 +03:00
David Taylor
8d3fe3ddc5
DEV: Bump theme compiler version for connector fix (#18665)
This should have been included in c185043590
2022-10-19 10:52:39 +01:00
David Taylor
be3d6a56ce
DEV: Introduce minification and source maps for Theme JS (#18646)
Theme javascript is now minified using Terser, just like our core/plugin JS bundles. This reduces the amount of data sent over the network.

This commit also introduces sourcemaps for theme JS. Browser developer tools will now be able show each source file separately when browsing, and also in backtraces.

For theme test JS, the sourcemap is inlined for simplicity. Network load is not a concern for tests.
2022-10-18 18:20:10 +01:00
David Taylor
cb87067c77 DEV: Introduce support for template colocation in themes 2022-10-18 10:10:49 +01:00
David Taylor
65a5c84a92 DEV: perform theme extra_js compilation all together
Previously, compiling theme 'extra_js' was done with a number of steps. Each theme_field would be compiled into its own value_baked column, and then the JavascriptCache content would be built by concatenating all of those compiled values.

This commit streamlines things by removing the value_baked step. The raw value of all extra_js theme_fields are passed directly to the ThemeJavascriptCompiler, and then the result is stored in the JavascriptCache.

In itself, this commit should not cause any behavior change. It is designed to open the door to more advanced compilation features which have interdependencies between different source files (e.g. template colocation, sourcemaps).
2022-10-18 10:10:49 +01:00
David Taylor
f4e1d0c546
FIX: Ensure <script> handlebars templates are namespaced correctly (#18178)
This regressed in 7e74dd0afe
2022-09-04 13:01:10 +02:00
Gerhard Schlager
fd6109a6e1
FIX: Theme components should work with empty locale files (#18167)
Theme components didn't work with empty locale files (just the locale key without any translations) that are created by translator-bot.
2022-09-02 18:28:18 +02:00
David Taylor
7e74dd0afe
DEV: Use DiscourseJsProcessor for theme template compilation (#18135)
Previously we were relying on a highly-customized version of the unmaintained Barber gem for theme template compilation. This commit switches us to use our own DiscourseJsProcessor, which makes use of more modern patterns and will be easier to maintain going forward.

In summary:
- Refactors DiscourseJsProcessor to move multiline JS heredocs into a companion `discourse-js-processor.js` file
- Use MiniRacer's `.call` method to avoid manually escaping JS strings
- Move Theme template AST transformers into DiscourseJsProcessor, and formalise interface for extending RawHandlebars AST transformations
- Update Ember template compilation to use a babel-based approach, just like Ember CLI. This gives each template its own ES6 module rather than directly assigning `Ember.TEMPLATES` values
- Improve testing of template compilation (and move some tests from `theme_javascript_compiler_spec.rb` to `discourse_js_processor_spec.rb`
2022-09-01 11:50:46 +01:00
David Taylor
6570fed175
DEV: Fix Ember version lookup (#18136)
The source-of-truth for our ember version is now the installed node_module. The `ember_source` gem carries an old version of Ember and so the constant is no longer useful. We'll be dropping the gem soon.
2022-08-30 16:27:14 +01:00
David Taylor
e16c8ea2e7
DEV: Support inline-hbs compilation in themes (#18112)
This commit makes a number of improvements to the DiscourseJsProcessor:

1. Remove dependence on the out-of-date Ember template compiler from the ember-rails gem; switch to modern template compiler
2. Refactor to make use of a proper module system with `define`/`require`
3. Introduce `babel-plugin-ember-template-compilation` to enable inline hbs compilation

The `mini-loader` is upgraded to support relative lookup and `require.has`, so that these new JS packages work correctly.
2022-08-29 19:53:42 +01:00
David Taylor
d6bba1ea9d
DEV: Invalidate theme cache when S3 configuration changes (#17872)
Compiled themes often include upload URLs which will need to be re-calculated
2022-08-11 11:03:57 +01:00
David Taylor
47824bc69f
DEV: Bump asset compiler versions (#17843)
This will trigger recompilation of theme/plugin assets following 7f9c2c0b
2022-08-09 12:06:15 +01:00
Osama Sayegh
306dca3a48
DEV: Add a helper method to clear every possible theme cache (#17585)
Our theme system is very complex and it can take a while to figure out how to invalidate the various types of caches that are used throughout the theme system. So, having a single helper method that invalidates everything can be useful in emergency situations where there is no time to read through the code and figure out how to clear the various caches.

Internal ticket: t64732.
2022-07-22 09:46:52 +03:00
David Taylor
a4c4d3cb50 DEV: Patch Ruby Ember template compiler to work with recent ember 2022-07-19 10:00:59 +01:00
David Taylor
fab1c00c8f
DEV: Drop the deprecated themeSettings.blah syntax (#17394)
This syntax has been printing deprecation messages since 880311dd4d
2022-07-18 10:10:23 +01:00