discourse/app/models
Osama Sayegh 7ed899fed9
FIX: Ensure theme JavaScript cache get consistent SHA1 digest (#15933)
There is a couple of layers of caching for theme JavaScript in Discourse:

The first layer is the `javascript_caches` table in the database. When a theme
with JavaScript files is installed, Discourse stores each one of the JavaScript
files in the `theme_fields` table, and then concatenates the files, compiles
them, computes a SHA1 digest of the compiled JavaScript and store the results
along with the SHA1 digest in the `javascript_caches` table.

Now when a request comes in, we need to render `<script>` tags for the
activated theme(s) of the site. To do this, we retrieve the `javascript_caches`
records of the activated themes and generate a `<script>` tag for each record.
The `src` attribute of these tags is a path to the `/theme-javascripts/:digest`
route which simply responds with the compiled JavaScript that has the requested
digest.

The second layer is a distributed cache whose purpose is to make rendering
`<script>` a lot more efficient. Without this cache, we'd have to query the
`javascript_caches` table to retrieve the SHA1 digests for every single
request. So we use this cache to store the `<script>` tags themselves so that
we only have to retrieve the `javascript_caches` records of the activated
themes for the first request and future requests simply get the cached
`<script>` tags.

What this commit does it ensures that the SHA1 digest in the
`javascript_caches` table stay the same across compilations by adding an order
by id clause to the query that loads the `theme_fields` records. Currently, we
specify no order when retrieving the `theme_fields` records so the order in
which they're retrieved can change across compilations and therefore cause the
SHA1 to change even though the individual records have not changed at all.

An inconsistent SHA1 digest across compilations can cause the database cache
and the distributed cache to have different digests and that causes the
JavaScript to fail to load (and if the theme heavily customizes the site, it
gives the impression that the site is broken) until the cache is cleared.

This can happen in busy sites when 2 concurrent requests recompile the
JavaScript files of a theme at the same time (this can happen when deploying a
new Discourse version) and request A updates the database cache after request B
did, and request B updates the distributed cache after request A did.

Internal ticket: t60783.

Co-authored-by: David Taylor <david@taylorhq.com>
2022-02-14 16:23:06 +03:00
..
concerns DEV: Clear custom field preload proxy on preload_custom_fields (#15671) 2022-01-21 14:29:51 +10:00
about.rb FEATURE: adds last day to about page stats (#12663) 2021-04-12 12:50:33 +10:00
admin_dashboard_data.rb FEATURE: Scheduled group email credential problem check (#15396) 2022-01-04 10:14:33 +10:00
admin_dashboard_general_data.rb FIX: Allow dashboard to load even when git version cannot be found 2019-08-28 12:37:42 +01:00
admin_dashboard_index_data.rb
allowed_pm_user.rb FEATURE: Allow List for PMs (#10270) 2020-07-20 15:23:49 -06:00
anonymous_user.rb
api_key_scope.rb FIX: Allowed URLs for API scopes added by plugins (#15662) 2022-01-20 22:29:03 +03:00
api_key.rb REFACTOR: Introduce RouteMatcher class 2020-10-19 10:40:55 +01:00
application_request.rb Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse"" 2020-05-23 00:56:13 -04:00
associated_group.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
auto_track_duration_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
backup_draft_post.rb FEATURE: experimental hidden setting for draft backups 2019-10-17 16:58:21 +11:00
backup_draft_topic.rb FEATURE: experimental hidden setting for draft backups 2019-10-17 16:58:21 +11:00
backup_file.rb
backup_location_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
backup_metadata.rb FIX: Prevent "uploads are missing in S3" alerts after restoring a backup 2020-09-10 21:37:48 +02:00
badge_grouping.rb
badge_type.rb
badge.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
base_font_setting.rb DEV: Do not translate font names (#10723) 2020-09-23 12:00:07 +03:00
bookmark.rb DEV: Ignore dropped Bookmark columns. 2022-01-10 16:04:22 +08:00
category_and_topic_lists.rb
category_custom_field.rb
category_featured_topic.rb FIX: Use Discourse.system_user when we need a placeholder admin (#9781) 2020-06-24 15:51:30 +10:00
category_group.rb PERF: Add index on group to category_groups (#8231) 2019-10-23 10:30:43 +01:00
category_list.rb FIX: exclude topics from muted tag in category featured list. (#14925) 2021-11-16 12:10:50 +05:30
category_page_style.rb
category_search_data.rb
category_tag_group.rb PERF: Cache categories in Site model take 3. 2021-06-24 13:30:51 +08:00
category_tag_stat.rb
category_tag.rb PERF: Cache categories in Site model take 3. 2021-06-24 13:30:51 +08:00
category_user.rb FIX: Don't allow NULL values for notification_level in category_users (#15407) 2021-12-29 09:19:39 +11:00
category.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
child_theme.rb
color_scheme_color.rb FIX: Wizard could not send custom color schemes to the client correctly (#10484) 2020-08-20 17:10:33 -07:00
color_scheme_setting.rb Add site setting to pick dark mode color scheme (#10390) 2020-08-07 08:52:47 -04:00
color_scheme.rb DEV: Instantiate relation early to save a query (#14766) 2021-10-29 03:03:22 +02:00
custom_emoji.rb DEV: annotate models 2020-04-28 15:59:39 +05:30
developer.rb DEV: Update annotations 2019-11-29 15:49:08 +00:00
digest_email_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
directory_column.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
directory_item.rb DEV: Plugin API to add directory columns (#13440) 2021-06-22 13:00:04 -05:00
discourse_connect.rb DEV: rename single_sign_on classes to discourse_connect (#15332) 2022-01-06 16:28:46 +04:00
discourse_version_check.rb
dismissed_topic_user.rb FEATURE: New way to dismiss new topics (#11927) 2021-02-04 11:27:34 +11:00
do_not_disturb_timing.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
draft_sequence.rb FIX: Update draft count when sequence is increased (#13940) 2021-08-04 13:30:37 +03:00
draft.rb FEATURE: Cook drafts excerpt in user activity (#14315) 2021-09-14 15:18:01 +03:00
email_change_request.rb DEV: Hash tokens stored from email_tokens (#14493) 2021-11-25 09:34:39 +02:00
email_level_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
email_log.rb FIX: Improve participant display in group SMTP emails (#13539) 2021-06-28 10:42:06 +10:00
email_style.rb FIX: backwards compatibility for uncompiled email style css 2019-10-23 19:22:33 -04:00
email_token.rb DEV: Update :critical_user_email calls to use strings (#15827) 2022-02-04 23:43:53 +00:00
embeddable_host.rb FIX: Handle addressable error when parsing an invalid URL. (#15836) 2022-02-07 11:25:42 +08:00
embedding.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
emoji_set_site_setting.rb DEV: Remove JoyPixels emoji option (#12197) 2021-02-26 07:44:52 -05:00
emoji.rb DEV: add native lazy loading for emojis (#15830) 2022-02-09 12:18:59 +01:00
external_upload_stub.rb DEV: Do not destroy external upload stub on error in debug mode (#14139) 2021-08-25 11:11:19 +10:00
given_daily_like.rb
global_setting.rb FIX: LOAD_PLUGINS=0 in dev/prod, warn in plugin:pull_compatible_all (#15537) 2022-01-11 12:30:22 +00:00
group_archived_message.rb FIX: Don't publish PM archive events to acting user. (#14291) 2021-09-10 09:20:50 +08:00
group_associated_group.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
group_category_notification_default.rb FEATURE: set notification levels when added to a group (#10378) 2020-08-06 12:27:27 -04:00
group_custom_field.rb
group_history.rb
group_manager.rb
group_mention.rb
group_request.rb
group_tag_notification_default.rb FEATURE: set notification levels when added to a group (#10378) 2020-08-06 12:27:27 -04:00
group_user.rb FIX: use active record update_attribute instead of mini sql. (#14367) 2021-09-21 09:29:12 +08:00
group.rb FEATURE: Allow sending group SMTP emails with from alias (#15687) 2022-02-07 13:52:01 +10:00
ignored_user.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
imap_sync_log.rb DEV: IMAP debugging improvements (#11784) 2021-01-21 11:37:47 +10:00
incoming_domain.rb
incoming_email.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
incoming_link.rb DEV: pluck_first 2019-10-21 12:08:20 +01:00
incoming_links_report.rb FEATURE: Make report filters reusable (#9444) 2020-04-22 11:52:50 +03:00
incoming_referer.rb
invite_redeemer.rb FEATURE: Restrict link invites to email domain (#15211) 2021-12-08 17:06:57 +02:00
invite.rb FIX: Ensure that login does not fail for users with invite records (#15647) 2022-01-20 10:54:38 +00:00
invited_group.rb FEATURE: Various improvements to invite system (#12314) 2021-03-09 00:15:14 +02:00
invited_user.rb FEATURE: Various improvements to invite system (#12298) 2021-03-06 13:29:35 +02:00
javascript_cache.rb Replace base_uri with base_path (#10879) 2020-10-09 12:51:24 +01:00
like_notification_frequency_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
linked_topic.rb FEATURE: new setting to create a linked topic on autoclosing mega topics (#11001) 2020-11-02 12:18:48 +05:30
locale_site_setting.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
mailing_list_mode_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
muted_user.rb
new_topic_duration_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
notification_level_when_replying_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
notification.rb DEV: Add chat_group_mention notification type (#15616) 2022-01-18 08:26:27 -06:00
oauth2_user_info.rb DEV: Move OAuth2UserInfo deprecation to after_save (#15704) 2022-01-25 10:29:31 +00:00
onceoff_log.rb
optimized_image.rb DEV: Add more verbose logging for image uploads (#13270) 2021-06-04 15:13:58 +03:00
permalink.rb FIX: Show error messages when adding permalinks in the admin UI (#12545) 2021-03-29 13:36:59 -05:00
plugin_store_row.rb
plugin_store.rb
post_action_type.rb FIX: Clear post action types application serializer fragment cache. 2021-06-04 09:14:49 +08:00
post_action.rb FIX: Liking whispers should not contribute to Topic#like_count. (#15703) 2022-02-03 16:24:33 +11:00
post_analyzer.rb FIX: Improve anchor links (#12683) 2021-04-14 10:27:07 +03:00
post_custom_field.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
post_detail.rb
post_mover.rb FIX: post mover validation color and message (#15688) 2022-02-02 16:22:52 +11:00
post_reply_key.rb
post_reply.rb DEV: Remove stale ignored_columns. (#11160) 2020-11-09 15:57:59 +11:00
post_revision.rb FEATURE: Add post edits count to user activity (#13495) 2021-08-02 10:15:53 -04:00
post_search_data.rb DEV: annotate models (#11047) 2020-10-27 23:42:33 +05:30
post_stat.rb
post_timing.rb PERF: Remove redundant post_timings_summary index (#14164) 2021-08-26 10:50:34 -05:00
post_upload.rb
post.rb FIX: Update user stat counts when post/topic visibility changes. (#15883) 2022-02-11 09:00:58 +08:00
previous_replies_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
private_message_topic_tracking_state.rb FIX: Do not publish post for PM topic tracking if not new for user. (#14469) 2021-09-29 13:54:24 +08:00
published_page.rb DEV: Upgrade Rails to 6.1.3.1 (#12688) 2021-04-21 12:36:32 +03:00
push_subscription.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
quoted_post.rb FEATURE: Nokogumbo (#9577) 2020-05-05 13:46:57 +10:00
remote_theme.rb DEV: In themes:update, only update themes which are out-of-date (#15676) 2022-01-21 18:23:26 +00:00
remove_muted_tags_from_latest_site_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
report.rb Add three reports (#14338) 2021-12-02 22:41:55 +05:30
reviewable_claimed_topic.rb FIX: Don't log a claimed topic database error during tests 2020-01-09 12:32:05 -05:00
reviewable_flagged_post.rb DEV: Make :send_system_message arguments strings (#15851) 2022-02-07 20:18:17 +00:00
reviewable_history.rb FEATURE: Add logging when claiming and unclaiming reviewable flagged posts (#8920) 2020-02-10 15:40:01 -08:00
reviewable_post.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
reviewable_priority_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
reviewable_queued_post.rb FEATURE: Display pending posts on user’s page 2021-11-29 10:26:33 +01:00
reviewable_score.rb DEV: Stop polluting all Ruby classes 2021-12-08 11:32:25 +01:00
reviewable_sensitivity_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
reviewable_user.rb DEV: Update :critical_user_email calls to use strings (#15827) 2022-02-04 23:43:53 +00:00
reviewable.rb DEV: Don't polute all ActiveRecord classes (#15103) 2021-11-26 10:17:10 +08:00
s3_region_site_setting.rb Added S3 region eu-south-1 Milanù 2021-09-16 12:33:43 -04:00
screened_email.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
screened_ip_address.rb FEATURE: Search screened IP address in blocks (#15461) 2022-01-11 09:16:51 +02:00
screened_url.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
search_log.rb FEATURE: show recent searches in quick search panel (#15024) 2021-11-25 15:44:15 -05:00
shared_draft.rb
shelved_notification.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
single_sign_on_record.rb
site_setting.rb FIX: LOAD_PLUGINS=0 in dev/prod, warn in plugin:pull_compatible_all (#15537) 2022-01-11 12:30:22 +00:00
site.rb DEV: Add markdown_additional_options to Site (#15738) 2022-01-28 13:02:02 +10:00
skipped_email_log.rb FIX: Handle edge cases for group SMTP email job (#13631) 2021-07-05 14:56:32 +10:00
slug_setting.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
stylesheet_cache.rb PERF: Add scheduled job to delete old stylesheet cache rows (#13747) 2021-07-16 10:58:01 -04:00
tag_group_membership.rb
tag_group_permission.rb
tag_group.rb FIX: remove parent tag from tag group 2020-03-13 12:25:58 -04:00
tag_search_data.rb
tag_user.rb DEV: Do not use hard-coded everyone group id (#15679) 2022-01-21 15:56:45 -07:00
tag.rb FEATURE: ability to add description to tags (#15125) 2021-12-01 09:18:56 +11:00
theme_field.rb DEV: Improve theme error handling UX 2022-02-14 10:11:19 +00:00
theme_modifier_set.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
theme_setting.rb FEATURE: add support for upload format in theme settings. 2020-04-15 18:34:02 +05:30
theme_translation_override.rb FIX: Clear in-process theme cache after clearing DB cache (#11517) 2020-12-17 11:02:41 +11:00
theme.rb FIX: Ensure theme JavaScript cache get consistent SHA1 digest (#15933) 2022-02-14 16:23:06 +03:00
top_lists.rb
top_menu_item.rb
top_topic.rb FIX: Better and more secure validation of periods for TopicQuery 2021-07-23 14:24:44 -04:00
topic_allowed_group.rb
topic_allowed_user.rb
topic_converter.rb FIX: Update user stat counts when post/topic visibility changes. (#15883) 2022-02-11 09:00:58 +08:00
topic_custom_field.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
topic_embed.rb FIX: Handle malformed URLs in TopicEmbed.absolutize_urls. 2022-01-21 11:18:54 +08:00
topic_featured_users.rb
topic_group.rb Changed CONFLICT to SQL for multiline strings 2019-12-13 11:51:40 -05:00
topic_invite.rb
topic_link_click.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
topic_link.rb FIX: Hide links to muted topics and in categories list (#14761) 2021-10-29 17:52:23 +03:00
topic_list.rb FIX: Add plugin event to topic list user lookup (#14116) 2021-08-25 13:16:08 +03:00
topic_notifier.rb
topic_participants_summary.rb PERF: Combine avatar_lookup and primary_group_lookup into user_lookup (#10253) 2020-07-17 10:48:08 +01:00
topic_poster.rb Fix i18n issues reported on Crowdin (#11747) 2021-02-02 10:50:04 +01:00
topic_posters_summary.rb FIX: Add plugin event to topic list user lookup (#14116) 2021-08-25 13:16:08 +03:00
topic_search_data.rb
topic_tag.rb
topic_thumbnail.rb DEV: followup to prev commit 2020-05-26 16:19:05 +10:00
topic_timer.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
topic_tracking_state.rb FIX: topic_tracking_state not erroring when missing user_stat (#14559) 2021-10-11 13:20:55 +11:00
topic_user.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
topic_view_item.rb FIX: exclude private messages from TL3 requirements 2020-04-07 17:36:07 -04:00
topic.rb FIX: Clear drafts only when post is created by real user (#15720) 2022-02-09 10:37:38 +02:00
translation_override.rb DEV: Sanitize HTML admin inputs (#14681) 2021-10-27 11:33:07 -03:00
trust_level_and_staff_and_disabled_setting.rb FEATURE: Make allow_uploaded_avatars accept TL (#14091) 2021-08-24 10:46:28 +03:00
trust_level_and_staff_setting.rb FIX: Don't store translated trust level names in anonymous cache (#13224) 2021-06-01 22:11:48 +02:00
trust_level_setting.rb FIX: Don't store translated trust level names in anonymous cache (#13224) 2021-06-01 22:11:48 +02:00
trust_level3_requirements.rb PERF: Faster TL3 promotion replies needed calculation (#10416) 2020-08-12 11:28:34 -03:00
unsubscribe_key.rb
upload.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
user_action.rb FIX: rename action_code_href to action_code_path (#14834) 2021-11-08 14:32:17 +11:00
user_api_key_scope.rb FIX: Restore users#topic_tracking_state route to api session_info scope (#10992) 2020-10-21 19:44:34 +01:00
user_api_key.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
user_archived_message.rb FIX: Don't publish PM archive events to acting user. (#14291) 2021-09-10 09:20:50 +08:00
user_associated_account.rb
user_associated_group.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
user_auth_token_log.rb
user_auth_token.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
user_avatar.rb DEV: Add more verbose logging for image uploads (#13270) 2021-06-04 15:13:58 +03:00
user_badge.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
user_badges.rb
user_bookmark_list.rb FIX: Add bookmark limits (#11725) 2021-01-19 08:53:49 +10:00
user_custom_field.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
user_email.rb FEATURE: Add email normalization rules setting (#14593) 2021-11-24 11:30:06 +02:00
user_export.rb REVERT: DEV: should ignore missing post uploads when a user export destroyed 2019-07-25 19:41:25 +05:30
user_field_option.rb FIX: Validate value of custom dropdown user fields - dropdowns and multiple selects (#13890) 2021-07-30 13:50:47 -04:00
user_field.rb DEV: Sanitize HTML admin inputs (#14681) 2021-10-27 11:33:07 -03:00
user_history.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
user_ip_address_history.rb DEV: annotate models (#11047) 2020-10-27 23:42:33 +05:30
user_notification_schedule.rb DEV: Update core model annotations 2021-07-06 10:11:06 +01:00
user_open_id.rb DEV: Drop legacy OpenID 2.0 support (#8894) 2020-02-07 17:32:35 +00:00
user_option.rb DEV: Use string timestamp when enqueuing update_top_redirection (#15826) 2022-02-04 23:14:35 +00:00
user_profile_view.rb DEV: s/\$redis/Discourse\.redis (#8431) 2019-12-03 10:05:53 +01:00
user_profile.rb FEATURE: Pull hotlinked images in user bios (#14726) 2021-10-29 17:58:05 +03:00
user_search_data.rb
user_search.rb FIX: when composing a message do not suggest deleted posts users (#15488) 2022-01-07 17:06:11 +05:30
user_second_factor.rb SECURITY: Improve second factor auth logic 2020-01-10 10:45:56 +10:00
user_security_key.rb DEV: annotate models 2019-10-17 16:58:22 +11:00
user_stat.rb Revert "DEV: Remove stale ignored_columns from models." 2022-01-11 11:00:58 +08:00
user_summary.rb FIX: Improve top links section from user summary (#15675) 2022-01-24 11:33:23 +11:00
user_upload.rb
user_visit.rb
user_warning.rb
user.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
username_validator.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
watched_word.rb FEATURE: add staff action logs for watched words (#13574) 2021-06-30 11:22:46 +05:30
web_crawler_request.rb DEV: s/\$redis/Discourse\.redis (#8431) 2019-12-03 10:05:53 +01:00
web_hook_event_type.rb FEATURE: add support for like webhooks (#12917) 2021-04-30 17:08:38 -07:00
web_hook_event.rb
web_hook.rb FEATURE: add support for like webhooks (#12917) 2021-04-30 17:08:38 -07:00