Commit Graph

50587 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan
135e88cb12
PERF: Improve performance of queries when loading a topic list (#22949)
What is the context of this change?

Before 7c6a8f1c74, we were using
`preload(:tags)` on the topics relation but that was accidentally
removed in the refactor. This was discovered and fixed in
5bec894a8c but insteadl of using
`preload(:tags)` we ended up using `includes(:tags)`. The problem here
is that `includes(:tags)` can either result in `preload(:tags)` or
`eager_load(:tags)` but for some reason ActiveRecord is deciding to
`eager_load(:tags)` resulting in a joins to the `topic_tags` and `tags`
table which is not necessarily and leads to more inefficient queries.

When `includes(:tags)` is used, listing the latest topics ended up
generating the following sample queries to fetch the list of topics to display.

```
SELECT DISTINCT "topics"."pinned_at" AS alias_0, "topics"."id" FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) ORDER BY "topics"."pinned_at" DESC LIMIT 30

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54, "tags"."id" AS t2_r0, "tags"."name" AS t2_r1, "tags"."created_at" AS t2_r2, "tags"."updated_at" AS t2_r3, "tags"."pm_topic_count" AS t2_r4, "tags"."target_tag_id" AS t2_r5, "tags"."description" AS t2_r6, "tags"."public_topic_count" AS t2_r7, "tags"."staff_topic_count" AS t2_r8 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) AND "topics"."id" = 7 ORDER BY "topics"."pinned_at" DESC

SELECT DISTINCT topics.bumped_at AS alias_0, "topics"."id" FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) ORDER BY topics.bumped_at DESC LIMIT 30

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54, "tags"."id" AS t2_r0, "tags"."name" AS t2_r1, "tags"."created_at" AS t2_r2, "tags"."updated_at" AS t2_r3, "tags"."pm_topic_count" AS t2_r4, "tags"."target_tag_id" AS t2_r5, "tags"."description" AS t2_r6, "tags"."public_topic_count" AS t2_r7, "tags"."staff_topic_count" AS t2_r8 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN "topic_tags" ON "topic_tags"."topic_id" = "topics"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "topic_tags"."tag_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) AND "topics"."id" IN (477, 481, 480, 479, 478, 467, 466, 230, 209, 183, 173, 179, 168, 139, 102, 144, 150, 118, 126, 88, 63, 46, 117, 171, 45, 77, 154, 158, 43, 79) ORDER BY topics.bumped_at DESC
```

Note how there are two extra queries which has to select `DISTINCT
topics.pinned_at` and `DISTINCT topics.bumped_at` because of the
unnecessary left joins to the `topic_tags` and `tags` table result in
duplicated rows in the topic tables. As a result, PG is not able to
use our indexes to effectively execute the query.

Comparing this to the queries being executed when `preload(:tags)` is
used.

```
SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL)) ORDER BY "topics"."pinned_at" DESC LIMIT 30

SELECT "topic_tags".* FROM "topic_tags" WHERE "topic_tags"."topic_id" = 7

SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, "topics"."created_at" AS t0_r3, "topics"."updated_at" AS t0_r4, "topics"."views" AS t0_r5, "topics"."posts_count" AS t0_r6, "topics"."user_id" AS t0_r7, "topics"."last_post_user_id" AS t0_r8, "topics"."reply_count" AS t0_r9, "topics"."featured_user1_id" AS t0_r10, "topics"."featured_user2_id" AS t0_r11, "topics"."featured_user3_id" AS t0_r12, "topics"."deleted_at" AS t0_r13, "topics"."highest_post_number" AS t0_r14, "topics"."like_count" AS t0_r15, "topics"."incoming_link_count" AS t0_r16, "topics"."category_id" AS t0_r17, "topics"."visible" AS t0_r18, "topics"."moderator_posts_count" AS t0_r19, "topics"."closed" AS t0_r20, "topics"."archived" AS t0_r21, "topics"."bumped_at" AS t0_r22, "topics"."has_summary" AS t0_r23, "topics"."archetype" AS t0_r24, "topics"."featured_user4_id" AS t0_r25, "topics"."notify_moderators_count" AS t0_r26, "topics"."spam_count" AS t0_r27, "topics"."pinned_at" AS t0_r28, "topics"."score" AS t0_r29, "topics"."percent_rank" AS t0_r30, "topics"."subtype" AS t0_r31, "topics"."slug" AS t0_r32, "topics"."deleted_by_id" AS t0_r33, "topics"."participant_count" AS t0_r34, "topics"."word_count" AS t0_r35, "topics"."excerpt" AS t0_r36, "topics"."pinned_globally" AS t0_r37, "topics"."pinned_until" AS t0_r38, "topics"."fancy_title" AS t0_r39, "topics"."highest_staff_post_number" AS t0_r40, "topics"."featured_link" AS t0_r41, "topics"."reviewable_score" AS t0_r42, "topics"."image_upload_id" AS t0_r43, "topics"."slow_mode_seconds" AS t0_r44, "topics"."bannered_until" AS t0_r45, "topics"."external_id" AS t0_r46, "categories"."id" AS t1_r0, "categories"."name" AS t1_r1, "categories"."color" AS t1_r2, "categories"."topic_id" AS t1_r3, "categories"."topic_count" AS t1_r4, "categories"."created_at" AS t1_r5, "categories"."updated_at" AS t1_r6, "categories"."user_id" AS t1_r7, "categories"."topics_year" AS t1_r8, "categories"."topics_month" AS t1_r9, "categories"."topics_week" AS t1_r10, "categories"."slug" AS t1_r11, "categories"."description" AS t1_r12, "categories"."text_color" AS t1_r13, "categories"."read_restricted" AS t1_r14, "categories"."auto_close_hours" AS t1_r15, "categories"."post_count" AS t1_r16, "categories"."latest_post_id" AS t1_r17, "categories"."latest_topic_id" AS t1_r18, "categories"."position" AS t1_r19, "categories"."parent_category_id" AS t1_r20, "categories"."posts_year" AS t1_r21, "categories"."posts_month" AS t1_r22, "categories"."posts_week" AS t1_r23, "categories"."email_in" AS t1_r24, "categories"."email_in_allow_strangers" AS t1_r25, "categories"."topics_day" AS t1_r26, "categories"."posts_day" AS t1_r27, "categories"."allow_badges" AS t1_r28, "categories"."name_lower" AS t1_r29, "categories"."auto_close_based_on_last_post" AS t1_r30, "categories"."topic_template" AS t1_r31, "categories"."contains_messages" AS t1_r32, "categories"."sort_order" AS t1_r33, "categories"."sort_ascending" AS t1_r34, "categories"."uploaded_logo_id" AS t1_r35, "categories"."uploaded_background_id" AS t1_r36, "categories"."topic_featured_link_allowed" AS t1_r37, "categories"."all_topics_wiki" AS t1_r38, "categories"."show_subcategory_list" AS t1_r39, "categories"."num_featured_topics" AS t1_r40, "categories"."default_view" AS t1_r41, "categories"."subcategory_list_style" AS t1_r42, "categories"."default_top_period" AS t1_r43, "categories"."mailinglist_mirror" AS t1_r44, "categories"."minimum_required_tags" AS t1_r45, "categories"."navigate_to_first_post_after_read" AS t1_r46, "categories"."search_priority" AS t1_r47, "categories"."allow_global_tags" AS t1_r48, "categories"."reviewable_by_group_id" AS t1_r49, "categories"."read_only_banner" AS t1_r50, "categories"."default_list_filter" AS t1_r51, "categories"."allow_unlimited_owner_edits_on_first_post" AS t1_r52, "categories"."default_slow_mode_seconds" AS t1_r53, "categories"."uploaded_logo_dark_id" AS t1_r54 FROM "topics" LEFT OUTER JOIN "categories" ON "categories"."id" = "topics"."category_id" LEFT OUTER JOIN topic_users AS tu ON (topics.id = tu.topic_id AND tu.user_id = 29) LEFT JOIN category_users ON category_users.category_id = topics.category_id AND category_users.user_id = 29 WHERE "topics"."deleted_at" IS NULL AND (topics.archetype <> 'private_message') AND (COALESCE(categories.topic_id, 0) <> topics.id) AND (COALESCE(tu.notification_level,1) > 0) AND (topics.category_id = -1
                OR
                (COALESCE(category_users.notification_level, 1) <> 0 AND (topics.category_id IS NULL OR topics.category_id NOT IN(-1)))

                OR tu.notification_level > 1) AND (NOT ( pinned_globally AND  pinned_at IS NOT NULL  AND (topics.pinned_at > tu.cleared_pinned_at OR tu.cleared_pinned_at IS NULL) )) ORDER BY topics.bumped_at DESC LIMIT 30

SELECT "topic_tags".* FROM "topic_tags" WHERE "topic_tags"."topic_id" IN (477, 481, 480, 479, 478, 467, 466, 230, 209, 183, 173, 179, 168, 139, 102, 144, 150, 118, 126, 88, 63, 46, 117, 171, 45, 77, 154, 158, 43, 79)
SELECT "tags"."id", "tags"."name", "tags"."created_at", "tags"."updated_at", "tags"."pm_topic_count", "tags"."target_tag_id", "tags"."description", "tags"."public_topic_count", "tags"."staff_topic_count" FROM "tags" WHERE "tags"."id" IN (10, 20, 26, 7, 27, 28, 30, 19, 9, 4, 15, 29, 14, 18, 11, 25, 1, 21, 8, 22, 5, 32)
```

We end up with queries that are much more efficient as those queries can
effectively use the indexes.
2023-08-03 10:48:41 +08:00
dependabot[bot]
e88dd30ab9
Build(deps): Bump zeitwerk from 2.6.10 to 2.6.11 (#22945)
Bumps [zeitwerk](https://github.com/fxn/zeitwerk) from 2.6.10 to 2.6.11.
- [Changelog](https://github.com/fxn/zeitwerk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/fxn/zeitwerk/compare/v2.6.10...v2.6.11)

---
updated-dependencies:
- dependency-name: zeitwerk
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-03 08:51:28 +08:00
Krzysztof Kotlarek
db4e2f41c2
DEV: experimental outlet for navigation filter (#22897)
Temporary outlet for navigation filter for /filter page. In addition, copy button and reset button were added to core.
2023-08-03 09:22:16 +10:00
Sérgio Saquetim
03690ccccf
DEV: Add :push_notification event and deprecate :post_notification_alert (#22917)
This commit introduces the :push_notification event and deprecates :post_notification_alert.

The old :post_notification_alert event was not triggered when pushing chat notifications and did not respect when the user was in "do not disturb" mode.

The new event fixes these issues.
2023-08-02 18:44:19 -03:00
Jarek Radosz
f9b4cfe67e
DEV: Use async/await in checklist (#22943) 2023-08-02 23:24:20 +02:00
Jarek Radosz
7405aae85a
DEV: Remove checklist syntax migrator (#22942)
Added in https://github.com/discourse/discourse-checklist/pull/23

Those who wanted to convert most likely did in the last three years :]
2023-08-02 22:31:56 +02:00
Canapin
b3c722f2f7
FIX: created:@ search keyword for uppercase usernames (#22878)
The filter wasn't working if the username had uppercase letters.
2023-08-02 15:28:17 -04:00
Faizaan Gagan
47593a9922
DEV: added a plugin outlet to reviewable-item (#22934)
* DEV: added a plugin outlet to reviewable-item

* fix linting
2023-08-02 23:18:18 +05:30
Jarek Radosz
c5c786b4ac
DEV: Fix array-related linting issues in templates (#22935)
and remove more linting exemptions
2023-08-02 19:43:37 +02:00
David Taylor
cc7dabe20d
DEV: Document additional parameters for admin users list API (#22940) 2023-08-02 17:46:48 +01:00
David Taylor
089dead654
DEV: Convert discovery controllers to native class syntax (#22938) 2023-08-02 17:46:27 +01:00
Penar Musaraj
5dc3a276c8
FIX: Compact tag picker input not focused in iOS (#22922)
Should fix an iOS regression in f5e8e73. iOS does not pull up the keyboard if the `.focus()` call is delayed by a rendering timeout or an asynchronous ajax call. This PR adds earlier `.focus()` calls if the input element is present.
2023-08-02 09:59:50 -04:00
Jarek Radosz
09502cf627
DEV: Reduce the number of IntersectionObservers (#22933)
…by adding the `chat/track-message` modifier only to the last thread rather than all of them

(plus: don't use Ember's `lastObject`)
2023-08-02 15:24:30 +02:00
David Taylor
13b13a758c
Remove discourse-canned-replies from official list (#22932)
This plugin is no longer supported, and so we no longer need to run its tests in CI

(removing the comment and the 'Canned Replies' value from the array caused syntax_tree to change to the `%w` syntax)
2023-08-02 12:48:20 +01:00
David Taylor
17d5e3ec23
DEV: Update composer service to inherit from Service, not Controller (#22929)
This was forgotten during the work in 22991bba44

This revealed two differences we were depending on: the merged `actions` hash (re-implemented on the service), and a couple of calls to `composer.send` (now removed)
2023-08-02 12:44:18 +01:00
TheJammiestDodger
e680437030
FIX: Add 'Ignored' flags to Moderator Activity report (#22041)
* FIX Add 'Ignored' flags to Moderator Activity report

The Moderator Activity query didn’t include the number of deferred flags in the Flags Reviewed totals. As this number is designed to reflect how many flags a moderator has seen, reviewed, and made a judgement on, the Ignored ones should also be included.

* Apply suggestions from code review

---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2023-08-02 12:27:31 +01:00
Jarek Radosz
407ff39fdf
FIX: Poll builder UI issues (#22931)
1. recent css regression related to modal upgrade
2. autofocus and on-enter regressions
3. array related linting issue (reliance on Ember's firstObject/lastObject)
2023-08-02 12:55:25 +02:00
David Taylor
2d4be458a5
FEATURE: Bundle discourse-checklist plugin into core (#22927)
Formerly https://github.com/discourse/discourse-checklist
2023-08-02 10:17:24 +01:00
David Battersby
906bfdebea
FIX: prevent event bubbling when closing modals with escape key (#22928)
This change prevents event bubbling for the Escape key on all modals. Currently when we close the modal using the Escape key, all other event listeners attached will also be triggered (such as closing the chat drawer if it's open).
2023-08-02 17:15:08 +08:00
Jarek Radosz
5ae0220638
DEV: De-jQ selectedText() (#22802)
And touch up `initializeDefaultHomepage()` while at it.
2023-08-02 10:57:58 +02:00
Rishabh
eebe3210bb
UX: Add tooltip about multiple emails to admin.groups.incoming_email field (#22926)
* UX: Add tooltip about multiple emails to admin.groups.incoming_email field
* prettier (linting)
2023-08-02 14:25:37 +05:30
Martin Brennan
d32e90a8d3
DEV: Remove unused chat translations (#22925)
Both placeholder_start_conversation and
placeholder_start_conversation_users are no
longer used.
2023-08-02 17:15:49 +10:00
David Battersby
d8bf926d2d
FIX: prevent event propagation when pressing escape key on lightbox (#22924)
When we use the escape key to exit lightbox for images within a chat channel, it also closes the chat drawer due to event bubbling (since both lightbox and chat use an event listener on the escape key).

This change prevents event bubbling when using the escape key within lightbox, which means that it will close the lightbox but won't close the chat drawer.
2023-08-02 13:59:10 +08:00
David Battersby
db7581ce49
FIX: intercept escape in lightbox and prevent default (#22923)
The escape key is used as a shortcut to escape the Discourse Lightbox. However, some browsers also use the escape key to exit fullscreen mode.

This change is to allow escaping the lightbox when browser is in fullscreen mode, while preventing any behavior associated with the Escape key (such as exiting fullscreen). This has to be done on the keydown event, as this means we can handle our logic and then preventDefault before the browser tries to exit fullscreen.
2023-08-02 12:48:47 +08:00
Alan Guo Xiang Tan
773b22e8d0
DEV: Seperate concerns of tracking GC stat from MethodProfiler (#22921)
Why this change?

This is a follow up to e8f7b62752.
Tracking of GC stats didn't really belong in the `MethodProfiler` class
so we want to extract that concern into its own class.

As part of this PR, the `track_gc_stat_per_request` site setting has
also been renamed to `instrument_gc_stat_per_request`.
2023-08-02 10:46:37 +08:00
Alan Guo Xiang Tan
e8f7b62752
DEV: Add site setting to allow collection of GC stats during requests (#22898)
What does this change do?

This change adds a hidden `track_gc_stat_per_request` site setting which
when enabled will track the time spent in GC, major GC count and minor
GC count during a request.

Why is this change needed?

We have plans to tune our GC in production but without any
instrumentation, we will not be able to know if our tuning is effective
or not. This commit takes the first step at instrumenting some basic GC
stats in core during a request which can then be consumed by the discourse-prometheus plugin.
2023-08-02 09:16:32 +08:00
dependabot[bot]
e88452a247
Build(deps): Bump handlebars in /app/assets/javascripts (#22919)
Bumps [handlebars](https://github.com/handlebars-lang/handlebars.js) from 4.7.7 to 4.7.8.
- [Release notes](https://github.com/handlebars-lang/handlebars.js/releases)
- [Changelog](https://github.com/handlebars-lang/handlebars.js/blob/v4.7.8/release-notes.md)
- [Commits](https://github.com/handlebars-lang/handlebars.js/compare/v4.7.7...v4.7.8)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-02 01:07:36 +02:00
dependabot[bot]
cd90deb4bd
Build(deps-dev): Bump sass in /app/assets/javascripts (#22918)
Bumps [sass](https://github.com/sass/dart-sass) from 1.64.1 to 1.64.2.
- [Release notes](https://github.com/sass/dart-sass/releases)
- [Changelog](https://github.com/sass/dart-sass/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sass/dart-sass/compare/1.64.1...1.64.2)

---
updated-dependencies:
- dependency-name: sass
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-02 01:06:41 +02:00
Isaac Janzen
ecd9e6d4b7
DEV: Convert watched-word-test modal to component-based API (#22911) 2023-08-01 16:08:56 -05:00
Isaac Janzen
7c8b0b9869
DEV: Convert delete-user-posts-progress modal to component-based API (#22916)
https://github.com/discourse/discourse/assets/50783505/414ffcc5-06e9-470f-b160-83b4c12bbb96
2023-08-01 15:49:26 -05:00
Roman Rizzi
514f5d25e6
FIX: Default to the first list if the preferred one is empty (#22915) 2023-08-01 17:01:00 -03:00
Isaac Janzen
da6ddb638f
DEV: Convert api-key-urls modal to component-based API (#22913)
<img width="474" alt="Screenshot 2023-08-01 at 1 29 26 PM" src="https://github.com/discourse/discourse/assets/50783505/1a46901f-8231-46fb-8c36-cb441b7fd883">
2023-08-01 14:51:18 -05:00
Daniel Waterworth
50d527b80c
DEV: Remove db_timeout setting (#22912)
It doesn't actually do anything.
2023-08-01 14:17:43 -05:00
David Taylor
945bb9161f
UX: Do not scroll-top for aborted transitions (#22744)
We only want to scroll to the top for successful transitions. If a transition is aborted (e.g. when clicking a chat link when chat is in drawer mode) then we should maintain the existing scroll location.
2023-08-01 18:44:25 +01:00
David Taylor
2c4faf48f9
DEV: Fix ember-cli proxy assets (#22674)
We were proxying all `/assets/*` requests through to the origin. In local development that was fine, because Rails was able to serve files from the `dist/` directory. But when proxying to a remote origin, we want the local ember-cli to serve its own JS assets
2023-08-01 18:43:33 +01:00
Roman Rizzi
a418dce1aa
FIX: Display similar topics for anons on mobile. (#22907)
We never propagated the preference change because of the early return, meaning lists listening to it never got to decide if they had to remain hidden.

Also, we don't want to track the preference when there's a single list, as the user didn't choose to see it.
2023-08-01 13:34:02 -03:00
Isaac Janzen
ed000be722
DEV: Convert merge-users modals to component-based API (#22727) 2023-08-01 10:35:36 -05:00
Rafael dos Santos Silva
5be658849b
UX: Fixes for new related topic styles (#22906) 2023-08-01 11:57:29 -03:00
Discourse Translator Bot
d5f4b8e02c
Update translations (#22904) 2023-08-01 16:05:44 +02:00
Isaac Janzen
542b3ffc32
DEV: Convert change-timestamp modal to component-based API (#22832) 2023-08-01 08:52:03 -05:00
Isaac Janzen
6b9e208612
DEV: Convert download-calendar modal to component-based API (#22837) 2023-08-01 08:50:52 -05:00
Isaac Janzen
6c8af90f5d
DEV: Convert start-backup modal to component-based API (#22812) 2023-08-01 08:40:25 -05:00
Isaac Janzen
106a0498f1
DEV: Convert convert-to-public-topic modal to component-based API (#22835) 2023-08-01 08:18:03 -05:00
Isaac Janzen
99e05b1280
DEV: Convert edit-slow-mode modal to component-based API (#22840)
<img width="772" alt="Screenshot 2023-07-28 at 1 41 18 PM" src="https://github.com/discourse/discourse/assets/50783505/9e97c4c8-af29-4e56-80dc-4f0244e2c4db">
2023-08-01 08:17:49 -05:00
dependabot[bot]
4191b631bf
Build(deps): Bump rack-mini-profiler from 3.1.0 to 3.1.1 (#22903)
Bumps [rack-mini-profiler](https://github.com/MiniProfiler/rack-mini-profiler) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/MiniProfiler/rack-mini-profiler/releases)
- [Changelog](https://github.com/MiniProfiler/rack-mini-profiler/blob/master/CHANGELOG.md)
- [Commits](https://github.com/MiniProfiler/rack-mini-profiler/compare/v3.1.0...v3.1.1)

---
updated-dependencies:
- dependency-name: rack-mini-profiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-01 13:38:01 +01:00
dependabot[bot]
00d99ab24d
Build(deps): Bump sass-embedded from 1.64.1 to 1.64.2 (#22902)
Bumps [sass-embedded](https://github.com/ntkme/sass-embedded-host-ruby) from 1.64.1 to 1.64.2.
- [Commits](https://github.com/ntkme/sass-embedded-host-ruby/compare/v1.64.1...v1.64.2)

---
updated-dependencies:
- dependency-name: sass-embedded
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-01 14:23:44 +02:00
Gerhard Schlager
ad0b8aed51
FIX: Use category hashtag instead of link in discourse_welcome_topic.body (#22875)
Linking to the #feedback category can break if the category gets renamed or a different site locale is used. By using the correct hashtag (at the time of seeding) this issues can be avoided.
2023-08-01 13:53:23 +02:00
David Taylor
20481abefc
Bump version to v3.2.0.beta1-dev 2023-08-01 09:50:46 +01:00
David Taylor
9b339bcd2c
Bump version to v3.1.0.beta8 2023-08-01 09:50:45 +01:00
David Taylor
bb217bbcc8
FIX: Ensure PresenceChannel does not raise error during readonly (#22899)
PresenceChannel configuration is cached using redis. That cache is used, and sometimes repopulated, during normal GET requests. When the primary redis server was readonly, that `redis.set` call would raise an error and cause the entire request to fail. Instead, we should ignore the failure and continue without populating the cache.
2023-08-01 09:34:57 +01:00