github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-30 20:34:32 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
e01f75cb32
discourse/spec/models
History
Alan Guo Xiang Tan 898b71da88
PERF: Add indexes to speed up notifications queries by user menu (#26048) 
Why this change?

There are two problematic queries in question here when loading
notifications in various tabs in the user menu:

```
SELECT "notifications".*
FROM "notifications"
LEFT JOIN topics ON notifications.topic_id = topics.id
WHERE "notifications"."user_id" = 1338 AND (topics.id IS NULL OR topics.deleted_at IS NULL)
ORDER BY notifications.high_priority AND NOT notifications.read DESC,
  NOT notifications.read AND notifications.notification_type NOT IN (5,19,25) DESC,
  notifications.created_at DESC
LIMIT 30;
```

and

```
EXPLAIN ANALYZE SELECT "notifications".*
FROM "notifications"
LEFT JOIN topics ON notifications.topic_id = topics.id
WHERE "notifications"."user_id" = 1338
AND (topics.id IS NULL OR topics.deleted_at IS NULL)
AND "notifications"."notification_type" IN (5, 19, 25)
ORDER BY notifications.high_priority AND NOT notifications.read DESC, NOT notifications.read DESC, notifications.created_at DESC LIMIT 30;
```

For a particular user, the queries takes about 40ms and 26ms
respectively on one of our production instance where the user has 10K notifications while the site has 600K notifications in total.

What does this change do?

1. Adds the `index_notifications_user_menu_ordering` index to the `notifications` table which is
   indexed on `(user_id, (high_priority AND NOT read) DESC, (NOT read)
DESC, created_at DESC)`.

1. Adds a second index `index_notifications_user_menu_ordering_deprioritized_likes` to the `notifications`
   table which is indexed on `(user_id, (high_priority AND NOT read) DESC, (NOT read AND notification_type NOT IN (5,19,25)) DESC, created_at DESC)`. Note that we have to hardcode the like typed notifications type here as it is being used in an ordering clause.

With the two indexes above, both queries complete in roughly 0.2ms. While I acknowledge that there will be some overhead in insert,update or delete operations. I believe this trade-off is worth it since viewing notifications in the user menu is something that is at the core of using a Discourse forum so we should optimise this experience as much as possible.
2024-03-06 16:52:19 +08:00
..
about_spec.rb DEV: Ability to collect stats without exposing them via API (#23933) 2023-11-10 00:44:05 +04:00
admin_dashboard_data_spec.rb DEV: Introduce a problem checks API (#25783) 2024-02-23 11:20:32 +08:00
api_key_scope_spec.rb FIX: Logs api scope not working (#25215) 2024-01-10 19:30:10 -07:00
api_key_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
application_request_spec.rb
associated_group_spec.rb
badge_grouping_spec.rb
badge_spec.rb FEATURE: reduce avatar sizes to 6 from 20 (#21319) 2023-06-01 10:00:01 +10:00
badge_type_spec.rb
bookmark_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
category_featured_topic_spec.rb DEV: Convert min_trust_to_create_topic to groups (#24740) 2023-12-13 14:50:13 +11:00
category_group_spec.rb
category_list_spec.rb PERF: Reduce ActiveRecord allocations in CategoryList#find_relevant_topics (#25950) 2024-02-29 12:19:04 +08:00
category_setting_spec.rb
category_spec.rb DEV: Async category search for sidebar modal (#25686) 2024-02-20 11:24:30 -06:00
category_tag_stat_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
category_user_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
child_theme_spec.rb
color_scheme_color_spec.rb
color_scheme_spec.rb
developer_spec.rb
digest_email_site_setting_spec.rb
directory_item_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
discourse_connect_spec.rb
do_not_disturb_timing_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
draft_sequence_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
draft_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
email_change_request_spec.rb
email_log_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
email_token_spec.rb
embeddable_host_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
emoji_spec.rb DEV: Fix various spec linting issues (#24672) 2023-12-04 13:45:19 +01:00
form_template_spec.rb FEATURE: support to initial values for form templates through /new-topic (#23313) 2023-08-29 18:41:33 -03:00
given_daily_like_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
global_setting_spec.rb
group_archived_message_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
group_associated_group_spec.rb
group_history_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
group_request_spec.rb
group_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
group_user_spec.rb FIX: down downgrade trust level if all requirements are met. (#25953) 2024-03-04 09:30:30 +11:00
incoming_link_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
incoming_links_report_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
invite_redeemer_spec.rb UX: Improve invite error message when a user uses an email that has already redeemed (#25695) 2024-02-27 18:24:20 +08:00
invite_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
javascript_cache_spec.rb
locale_site_setting_spec.rb
mailing_list_mode_site_setting_spec.rb
notification_spec.rb PERF: Add indexes to speed up notifications queries by user menu (#26048) 2024-03-06 16:52:19 +08:00
optimized_image_spec.rb
permalink_spec.rb FEATURE: Permalinks for users (#25552) 2024-02-05 17:31:31 +01:00
plugin_store_spec.rb
post_action_spec.rb FEATURE: Add new 'illegal' flag reason (#25498) 2024-02-07 10:12:22 +08:00
post_action_type_spec.rb
post_analyzer_spec.rb
post_detail_spec.rb
post_mover_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
post_reply_key_spec.rb
post_reply_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
post_revision_spec.rb
post_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
post_stripper_spec.rb FIX: user got notified about a mention inside a chat message quote (#24229) 2023-11-08 23:13:25 +04:00
post_timing_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
private_message_topic_tracking_state_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
problem_check_tracker_spec.rb DEV: Add DB backed problem checks to support perform_every config (#25834) 2024-02-27 11:17:39 +08:00
published_page_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
quoted_post_spec.rb DEV: Fix a flaky quote post spec (#22891) 2023-08-01 00:48:40 +02:00
remote_theme_spec.rb DEV: Refactor Theme#settings to return a hash instead of array (#25516) 2024-02-01 10:26:56 +08:00
report_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
reviewable_claimed_topic_spec.rb
reviewable_flagged_post_spec.rb FIX: Allow the flags to be cleaned up (#25085) 2024-01-02 18:32:50 +08:00
reviewable_history_spec.rb DEV: Convert min_trust_to_flag_posts setting to groups (#24864) 2023-12-13 17:18:42 +08:00
reviewable_post_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
reviewable_queued_post_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
reviewable_score_spec.rb DEV: Automatically update groups for test users with explicit TL (#25415) 2024-01-29 17:52:02 +08:00
reviewable_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
reviewable_user_spec.rb DEV: reviewable_user spec should assert on delete_user_block instead of delete_user (#24692) 2023-12-04 12:44:32 -03:00
s3_region_site_setting_spec.rb
screened_email_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
screened_ip_address_spec.rb DEV: find_each in CSV exports (#22573) 2023-08-17 12:33:52 +10:00
screened_url_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
search_log_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
sidebar_section_link_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
sidebar_section_spec.rb FEATURE: Initial admin sidebar navigation (#24789) 2023-12-18 11:48:25 +10:00
sidebar_url_spec.rb FIX: increase sidebar URL limit to 1000 (#23120) 2023-08-17 14:46:24 +10:00
site_setting_spec.rb DEV: Add auto map from TL -> group site settings in DeprecatedSettings (#24959) 2023-12-26 14:39:18 +08:00
site_spec.rb FIX: Preload parent categories for sidebar (#25726) 2024-02-16 16:39:18 +02:00
sitemap_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
skipped_email_log_spec.rb
stylesheet_cache_spec.rb
tag_group_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
tag_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
tag_user_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
theme_field_spec.rb DEV: Validate default value for type: objects theme settings (#25833) 2024-02-27 09:16:37 +08:00
theme_modifier_set_spec.rb
theme_setting_spec.rb DEV: Set a bytesize limit for ThemeSetting#json_value (#25761) 2024-02-21 08:09:37 +08:00
theme_settings_migration_spec.rb FEATURE: Theme settings migrations (#24071) 2023-11-02 08:10:15 +03:00
theme_spec.rb DEV: Update rubocop-discourse to latest version 2024-03-04 15:08:35 +01:00
theme_svg_sprite_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
top_menu_item_spec.rb
top_topic_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_allowed_user_spec.rb
topic_converter_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_embed_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
topic_featured_users_spec.rb
topic_group_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_hot_scores_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
topic_invite_spec.rb
topic_link_click_spec.rb DEV: Convert min_trust_to_post_links to groups (#25298) 2024-01-18 14:08:40 +08:00
topic_link_spec.rb FEATURE: support silent internal links (#25472) 2024-01-30 17:03:58 +11:00
topic_list_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_participant_groups_summary_spec.rb FEATURE: display PM participant group names in the topics list. (#21677) 2023-05-31 19:32:06 +05:30
topic_participants_summary_spec.rb
topic_posters_summary_spec.rb
topic_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
topic_tag_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_thumbnail_spec.rb DEV: Fix random typos (#22078) 2023-06-13 22:02:21 +02:00
topic_timer_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
topic_tracking_state_spec.rb DEV: Convert min_trust_level_to_tag_topics to groups (#25273) 2024-01-26 13:25:03 +08:00
topic_user_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
topic_view_item_spec.rb
translation_override_spec.rb FIX: Don't show admin warnings about deleted translation overrides (#22614) 2023-07-14 16:52:39 +08:00
trust_level_and_staff_setting_spec.rb
trust_level_setting_spec.rb DEV: Skip flaky specs (#25111) 2024-01-03 12:32:26 +01:00
trust_level3_requirements_spec.rb DEV: Remove full group refreshes from tests (#25414) 2024-01-25 14:28:26 +08:00
unsubscribe_key_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
upload_reference_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
upload_spec.rb FIX: Change max_image_megapixels logic (#25625) 2024-02-12 09:56:43 +10:00
user_action_spec.rb DEV: Add a plugin modifier for user_action_stream_builder (#25691) 2024-02-16 10:24:39 +10:00
user_api_key_spec.rb
user_archived_message_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_associated_group_spec.rb
user_auth_token_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_avatar_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_badge_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_bookmark_list_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_email_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_export_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_field_spec.rb DEV: Update the rubocop-discourse gem 2023-06-26 11:41:52 +02:00
user_history_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_notification_schedule_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_option_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_profile_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_profile_view_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_search_spec.rb DEV: Update rubocop-discourse to latest version 2024-03-04 15:08:35 +01:00
user_second_factor_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_security_key_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_spec.rb DEV: Use freeze_time_safe in more places (#25949) 2024-03-01 10:07:35 +10:00
user_stat_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_status_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
user_summary_spec.rb
user_visit_spec.rb DEV: Allow fab! without block (#24314) 2023-11-09 16:47:59 -06:00
username_validator_spec.rb DEV: Make sure max_username_length is within MAX_USERNAME_LENGTH_RANGE (#23104) 2023-08-15 12:12:22 -03:00
watched_word_spec.rb
web_crawler_request_spec.rb
web_hook_event_spec.rb
web_hook_event_type_spec.rb FIX: active webhook types exclude inactive plugins (#26022) 2024-03-05 12:47:04 +11:00
web_hook_spec.rb DEV: Convert min_trust_to_create_topic to groups (#24740) 2023-12-13 14:50:13 +11:00