```
EXPLAIN ANALYZE SELECT "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
topics.id IN (
SELECT topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 13455 AND gu.group_id =
tg.group_id
WHERE gu.group_id IN (47)
)
)
AND (
topics.id IN (
SELECT ta.topic_id
FROM topic_allowed_users ta
WHERE ta.user_id IN (32852,-10)
)
OR
topics.id IN (
SELECT tg.topic_id
FROM topic_allowed_groups tg
WHERE tg.group_id IN (-10)
)
)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```
Planning time: 1.277 ms
Execution time: 71.577 ms
```
EXPLAIN ANALYZE SELECT "topics".* FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN (
SELECT * FROM topic_allowed_groups _tg
LEFT JOIN group_users gu
ON gu.user_id = 13455
AND gu.group_id = _tg.group_id
AND gu.group_id IN (47)
) tg ON topics.id = tg.topic_id
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (32852)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (tg.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL)
AND (topics.id NOT IN (69933,69995,69988,69984,69968,69973,69971,69952))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 3;
```
Planning time: 1.191 ms
Execution time: 0.129 ms
```
EXPLAIN ANALYZE SELECT "topics".*
FROM "topics" LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND
tu.user_id = 13455
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (
topics.id IN (
SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 13455
)
)
AND (
topics.id IN (
SELECT ta.topic_id
FROM topic_allowed_users ta
WHERE ta.user_id IN (2,1995,8307,17621,22980,-10)
)
OR
topics.id IN (
SELECT tg.topic_id
FROM topic_allowed_groups tg
WHERE tg.group_id IN (-10)
)
)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at
DESC LIMIT 5;
```
Planning time: 1.196 ms
Execution time: 21.176 ms
```
EXPLAIN ANALYZE SELECT "topics".*
FROM "topics"
LEFT JOIN topic_users tu ON topics.id = tu.topic_id AND tu.user_id =
13455
LEFT JOIN topic_allowed_users ta ON topics.id = ta.topic_id AND
ta.user_id = 13455
LEFT JOIN topic_allowed_users ta2 ON topics.id = ta2.topic_id AND
ta2.user_id IN (2,1995,8307,17621,22980,-10)
LEFT JOIN topic_allowed_groups tg ON topics.id = tg.topic_id AND
tg.group_id IN (-10)
WHERE ("topics"."deleted_at" IS NULL)
AND (topics.archetype = 'private_message')
AND (ta.topic_id IS NOT NULL)
AND (ta2.topic_id IS NOT NULL OR tg.topic_id IS NOT NULL)
AND (topics.id NOT IN (68559,60069,42145))
AND "topics"."visible" = 't'
ORDER BY topics.bumped_at DESC
LIMIT 5;
```
Planning time: 1.792 ms
Execution time: 2.546 ms
Figuring out what unread topics a user has is a very expensive
operation over time.
Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)
When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.
This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.
We use pg functions, instead of nested queries here to assist the
planner.
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
