Commit Graph

320 Commits

Author SHA1 Message Date
Sam
0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Sam
29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Guo Xiang Tan
604aaf8686 FIX: N+1 query in suggested topics list. 2017-05-22 18:05:38 +08:00
Vinoth Kannan
1241660c2c FIX: 'read' filter in top menu showing new topics 2017-04-29 14:37:45 +05:30
Sam
abc4dff0fe FEATURE: add bumped_before query param for topic list 2017-03-02 15:11:50 -05:00
Sam
4dac4c69a6 FEATURE: add before topic list filter 2017-03-02 14:54:33 -05:00
Sam
872d9eae45 FEATURE: add :all filter for messages 2017-03-02 14:54:33 -05:00
Sam Saffron
040e10a627 reduce duplication 2017-02-15 17:27:10 -05:00
Sam
74d4209d24 FEATURE: allow plugins to register custom topic list filters 2017-02-15 15:25:43 -05:00
Sam
c04d4171ff FIX: whisper no longer experimental
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
2016-12-02 17:03:31 +11:00
Neil Lalonde
9ef1688a76 FEATURE: per-category default topic list sort order 2016-11-01 12:18:41 -04:00
James Kiesel
386b8b8498 Don't join on tags unnecessarily when matching all tags 2016-08-19 10:37:32 -05:00
Régis Hanol
6d1d7b7c8f UX: new /categories layout 2016-08-17 23:23:16 +02:00
James Kiesel
037e9bb7b8 Support any number of tag intersections 2016-08-15 15:30:17 -04:00
James Kiesel
e14f3c802b Cleanup 2016-08-12 15:56:56 -04:00
James Kiesel
7e73b933c7 First pass 2016-08-12 15:28:46 -04:00
Neil Lalonde
f10c4682cd FIX: muted tags showing in latest topic list 2016-08-04 11:54:48 -04:00
Neil Lalonde
7c092b0fe0 FEATURE: add filter to show topics that have not been tagged 2016-07-20 16:21:51 -04:00
Sam
1c964bf730 PERF: cache category topic ids 2016-07-19 12:34:54 +10:00
Guo Xiang Tan
8fd0414cdf
WIP: Tags which are not allowed in a category showing in drop down. 2016-07-08 10:27:56 +08:00
Guo Xiang Tan
e221414935
PERF: Remove N+1 queries on user messages page. 2016-06-29 09:30:54 +08:00
Neil Lalonde
884779b5c1 FIX: N+1 query when tagging enabled and no tags in topic list query. Topic query ignored tags input when tagging is disabled. 2016-05-26 18:03:50 -04:00
Neil Lalonde
f13470b96b Use db schema for tags instead of plugin store and custom fields 2016-05-26 14:29:48 -04:00
Robin Ward
1714019cf7
FIX: OFFSET wasn't being applied correctly 2016-05-11 13:39:21 -04:00
Robin Ward
49a6d0b789
FIX: Don't bother with negative offsets 2016-05-09 16:33:55 -04:00
Neil Lalonde
e5918c7d00 FEATURE: Merge tagging plugin into core 2016-04-27 11:58:53 -04:00
Joe Buhlig
8d6579c729 Added ability to create topic sorts in plugins with custom_fields 2016-03-04 11:07:39 -06:00
Sam
f0e942f647 PERF: move 3 more option columns out of the user table 2016-02-18 16:57:22 +11:00
Sam Saffron
bc75010b20 FEATURE: suggested messages for messages
FEATURE: clicking envelope takes you to inbox

Suggested messages works somewhat like suggested topics.

- New show up first (in either group inbox or inbox)
- Then unread (in either group inbox or inbox)
- Finally "related" which are messages with same participants as the current pm.
2016-02-03 18:50:05 +11:00
Régis Hanol
e2744fc19f FIX: adding the 'contains_messages' column back 2016-01-12 12:06:51 +01:00
Sam
03ea0bfe22 FEATURE: allow users to archive messages
Messages are now in 3 buckets

- Inbox for all new messages
- Sent for all sent messages
- Archive for all messages you are done with

You can select messages from your Inbox or Sent and move them to your Archive,
you can move messages out of your Archive similarly

Similar concept applied to group messages, except that archiving and unarchiving
will apply to all group members
2015-12-23 11:09:30 +11:00
Neil Lalonde
b3198d7a6a callback in TopicQuery for plugins to add result set filters 2015-12-21 11:43:17 -05:00
Sam
8cf4d52cb6 FEATURE: split up group PMS on user page 2015-12-10 11:39:33 +11:00
Régis Hanol
a37d575d7d FEATURE: new 'My Groups' messages filter in user page 2015-12-07 18:37:03 +01:00
Sam
48ef609003 correct implementation add tests 2015-11-02 15:05:08 +11:00
Sam
3575012a47 correct implementation 2015-11-02 14:59:10 +11:00
Sam Saffron
606b10445e FEATURE: remove muted topics from suggested and latest 2015-11-02 09:20:22 +11:00
Sam Saffron
8a0a87812b FIX: navigating back to latest was not respecting pinned topics 2015-10-14 13:03:55 +11:00
Sam
a56e3ccea0 FIX: explicitly tracked topics will show up even if category is muted 2015-10-13 17:54:31 +11:00
Sam
a61765b9e4 PERF: improve perf of initial payload
also reduce  querying in topic query
2015-09-23 13:13:50 +10:00
Neil Lalonde
2ae032c9b0 FIX: categories page would sometimes show no topics, even if there are some visible topics to show 2015-09-21 18:13:14 -04:00
Régis Hanol
a501947d67 FEATURE: suppress categories from the homepage 2015-09-02 20:25:18 +02:00
Sam
2876725e1b REFACTOR: remove hacky search from discovery 2015-07-27 16:47:06 +10:00
Robin Ward
32a1610b5d FIX: Search had wrong preload store key 2015-07-14 11:58:42 -04:00
Sam
1408563a7f HACK: allow for multiple hits on same topic for search 2015-06-26 17:47:20 +10:00
Robin Ward
fc090b4568 FIX: With many stickies some latest posts got missed 2015-06-25 16:26:09 -04:00
Sam
41e427bd2e Work in progress, full page search 2015-06-22 18:09:08 +10:00
Neil Lalonde
611b5f996e FIX: unpinned topics shouldn't remain pinned on categories page 2015-06-10 14:36:55 -04:00
Sam
51d82fc25d PERF: Active Record #pluck is twice faster
PERF: SqlBuilder bypasses AR converters and uses PG directly
2015-05-13 16:46:17 +10:00
Sam
4dbd065026 work-in-progress full page search 2015-04-02 16:02:07 +11:00
Robin Ward
71860aab03 FIX: Missing avatars on Messages page
(We like private_messages more than private_messags)
2015-03-31 17:29:38 -04:00
Robin Ward
ff3e1e1dd7 FIX: User's topic lists weren't consistent WRT visibility 2015-03-23 18:13:06 -04:00
Sam
c4c5c10ea0 FEATURE: stop pinning child categories on to parent 2015-03-12 10:42:26 +11:00
Sam
f432b9f5b7 correct spec failure, only test code was affected 2015-03-03 10:21:01 +11:00
Sam
c02f1b026a add extra safety so offset can not be negative 2015-02-26 14:48:56 +11:00
Sam
b760d22460 PERF: avoid expensive order by random for suggested topics 2015-02-25 17:19:12 +11:00
Sam
7b6e85cb6c remove unused api call
correct spec
2015-02-25 14:39:50 +11:00
Sam
3658c21fc5 REGRESSION: incorrect ordering when featuring topics 2015-02-25 14:24:25 +11:00
Sam
738f2cb72c REGRESSION: pms not ordering on bumped_at descending 2015-02-24 11:28:17 +11:00
Sam
8370b26cba PERF: optimise pinned handling on home page
Old query used to scan the full topics table, on home page

Instead we now perform 2 queries, one for pinned and one for unpinned and merge
results in a 10x improvement on a 1 million topic DB
2015-02-23 16:51:21 +11:00
Robin Ward
e207ca36ee Easier helper for filtering secured categories 2015-02-12 11:52:59 -05:00
Sam
ba186b25a7 FEATURE: optional top level bookmarks tab 2015-01-25 15:53:11 +11:00
Jeff Atwood
8e6f7521ec change in/visible search querystring to un/listed 2015-01-12 01:00:45 -08:00
Robin Ward
74051a2df4 Allow plugins to build topic lists 2015-01-08 16:44:27 -05:00
Sam
ea87f5fd8a FEATURE: support for filter=bookmarked and filter=liked in topic list 2015-01-07 18:20:10 +11:00
Sam
95f9788a77 FEATURE: add ?bookmarked=true for topic lists 2015-01-07 13:58:34 +11:00
Sam
efc717c14a FEATURE: remove star concept from Discourse 2015-01-07 13:43:27 +11:00
Sam
0396fd66f6 FEATURE: sorting by op likes shows the op likes count 2015-01-05 17:39:49 +11:00
Robin Ward
c15f78f9c9 FIX: Should be 30 topics on desktop, 15 on android 2014-12-19 13:18:26 -05:00
Sam
4cd9243888 FEATURE: topics per page should not be configurable
Can lead to severe performance issues
2014-12-15 11:24:16 +11:00
Robin Ward
b1bc4741b1 FEATURE: Load fewer topics in the topic list on slow platforms (Android) 2014-12-15 11:54:26 -05:00
Sam
228f7542da FEATURE: Re-prioritize related so unread is always first 2014-12-04 10:04:23 +11:00
cpradio
0d6e5470d4 FEATURE: Add ?status=deleted querystring
Add tests around the ?status=deleted querystring
2014-11-20 10:32:33 -05:00
Robin Ward
ae20369733 FIX: unread was being tagged as "new" in the PreloadStore 2014-10-09 15:45:06 -04:00
Robin Ward
1f26a79899 FIX: Category latest pages were not preloading properly, causing weird
refreshes when clicking the home logo.
2014-10-08 12:45:18 -04:00
Sam
cc23bb84d9 FEATURE: ninja op_likes sort for topic lists 2014-10-03 13:16:53 +10:00
Sam
f275d93d6c Merge pull request #2787 from cpradio/add-visible-status
Add visible/invisible status for Advanced Users
2014-09-12 10:02:08 +10:00
cpradio
1485538434 FEATURE: Add querystring value for Invisible
Allow advanced functionality to only show invisible topics on the latest, unread, new pages.
2014-09-11 19:21:25 -04:00
cpradio
dfda41567c FEATURE: Add querystring value for Visible
Allow advanced functionality to only show visible topics on the latest, unread, new pages.
2014-09-11 19:17:16 -04:00
riking
056249cc93 PERF: Tame eager-loading of category definition topic URL
With this change, the EXPLAIN on the topic query is reduced in cost, and
script/bench.rb results are improved.

Explain topic query, cost at root level:

cost=641.84..641.92 --> cost=570.03..570.11

script/bench.rb result, 50th percentile:

BEFORE
---

categories_admin: 95
home_admin: 80
topic_admin: 33
categories: 71
home: 51
topic: 15

AFTER
---

categories_admin: 78
home_admin: 58
topic_admin: 49
categories: 40
home: 34
topic: 30
2014-09-10 19:55:10 -07:00
Robin Ward
4f6b9815ae FEATURE: List moderator warnings on admin dashboard 2014-09-08 13:25:02 -04:00
Robin Ward
334e21a03a Revert "Revert "FEATURE: Can create warnings for users via PM""
This reverts commit 1c7559380c.
2014-09-08 11:11:56 -04:00
Robin Ward
1c7559380c Revert "FEATURE: Can create warnings for users via PM"
This reverts commit b0bfc1f93f.
2014-09-08 10:38:59 -04:00
Robin Ward
b0bfc1f93f FEATURE: Can create warnings for users via PM 2014-09-08 10:27:06 -04:00
Sam
59d04c0695 Internal renaming of elder,leader,regular,basic to numbers
Changed internals so trust levels are referred to with

TrustLevel[1], TrustLevel[2] etc.

This gives us much better flexibility naming trust levels, these names
are meant to be controlled by various communities.
2014-09-05 15:20:52 +10:00
Robin Ward
f10d6ed88a FIX: RSS feeds should use created_at not bumped_at 2014-08-27 12:42:54 -04:00
Robin Ward
ecf3a2a510 REVERT: Previous fix was broken, needs more work 2014-08-18 16:46:33 -04:00
Robin Ward
f1486c7728 FIX: Inconsistency between unread counter and unread filter 2014-08-18 16:15:26 -04:00
Robin Ward
629bfb73f5 FIX: Sort order on activity column is incorrect 2014-08-18 13:26:27 -04:00
Sam
8c68309b5a FIX: remove muted categories from suggested 2014-07-29 14:34:54 +10:00
Sam
189b1c729a FEATURE: query param for topic state 2014-07-17 09:29:09 +10:00
Robin Ward
0f52f26587 TWEAK: Don't show subcategory topic definitions when viewing a category list. 2014-06-30 15:22:40 -04:00
Sam
90d7718103 add search filter 2014-06-30 17:18:08 +10:00
Sam
dc0266cc22 FEATURE: correct muted category implementation
- Don't change tracking state on muted categories
- Exclude muted sub categories from parent
2014-06-18 11:23:31 +10:00
Benjamin Kampmann
6b3d5c7348 add min_posts, max_posts to filter the topic list queries
superseeds #2412
2014-06-05 15:30:24 +02:00
Régis Hanol
9125453628 FEATURE: add the first 3 participants in a private message 2014-05-12 09:32:49 +02:00
Régis Hanol
21e8ae0eca BUGFIX: the /top page now shows the first non-empty period 2014-05-07 19:04:39 +02:00
Régis Hanol
4642218662 FEATURE: add private messages counters in user activity feed 2014-05-02 22:53:58 +02:00
Robin Ward
b3ed8b6a32 Use query params for sortable table headings 2014-04-17 10:52:57 -04:00
riking
86fe11b277 Fix the 'posted' view
Now is actually topics that you have posted in - previously, it was all topics you had tracked, starred, posted, read, ....
No clue how that came about.
2014-03-30 14:21:35 -07:00
Neil Lalonde
20ced1a55e FIX: ORDER BY for new topics within current category was second in list, needs to be first 2014-03-14 18:13:45 -04:00
Robin Ward
b94abc9c8e FIX: Deprecation warnings on Rails with some topic queries 2014-02-26 11:09:02 -05:00
Robin Ward
1aa27ade17 FEATURE: If you don't select any topics to "Dissmiss Read" it does all
by filter.
2014-02-21 15:18:45 -05:00
Sam
22af3fe085 FEATURE: muted categories should not be missing from drill down 2014-02-12 17:01:13 +11:00
Sam
820fe572ca BUGFIX: phantom new and unread for category definition topics 2014-02-11 10:22:15 +11:00
Robin Ward
350eb82223 Don't include category definitions on Categories page 2014-02-07 17:01:31 -05:00
Régis Hanol
cef2d8dc82 BUGFIX: make sure moderators can't view topics they aren't allowed to 2014-02-07 07:08:56 -08:00
Robin Ward
659546c4e4 CHANGE: Hide category definition topics unless you are viewing that
category.
2014-02-04 15:55:30 -05:00
Neil Lalonde
9601684405 Suggested topics includes closed and archived in new and unread, but not in random results 2014-02-04 12:26:49 -05:00
Sam
d0a6dd88df FEATURE: exclude muted categories from the latest and new tabs 2014-02-03 16:05:49 +11:00
Neil Lalonde
947d896a8c FIX: when filling suggested topic list, don't try to add topics that have already been added, and topics that are category descriptions 2014-01-28 18:15:46 -05:00
Régis Hanol
b49e448556 FEATURE: show pinned topics for TL0 on top/year page
BUGFIX: word-wrap on topic-excerpt
2014-01-18 19:27:26 +01:00
Régis Hanol
8d2e5041bc BUGFIX: proper handling of /none subcategory 2014-01-18 19:27:25 +01:00
Robin Ward
4981525047 REFACTOR: Fixes poor class hierarchy for listing topics
- Upgrades Ember to latest
- Fixes a bunch of bugs with page titles and missing "active" states
2014-01-18 19:26:24 +01:00
Régis Hanol
3a6bffa05d FEATURE: better top pages 2014-01-14 01:09:12 +01:00
Sam
2c75e15049 FEATURE: query param for closed / archived / open
?status=closed for closed topics
?status=open for open topics
?status=archived for archived topics
2014-01-13 14:40:21 +11:00
Neil Lalonde
52580f09af Rename favorite to starred everywhere 2014-01-10 14:54:19 -05:00
Régis Hanol
28bf78d801 BUGFIX: make sure we only show topics that have a useful score in the Top page 2014-01-03 09:15:10 +01:00
Régis Hanol
9c8e50351d first stab at calculating the score of a topic for the top tab 2013-12-27 18:10:35 +01:00
Régis Hanol
cb6d83943a only show topics that have activity in top page 2013-12-24 15:26:03 +01:00
Régis Hanol
567d2bd23c add top page 2013-12-24 00:50:36 +01:00
Godfrey Chan
e41eecb91b Added some missing references, see https://github.com/rails/rails/issues/12982 2013-12-18 15:06:19 -08:00
Robin Ward
acf262b631 Support for "no subcategories" 2013-12-16 13:06:35 -05:00
Robin Ward
8f0a0059cc Sorting by activity now sorts on created_at rather than bumped_at 2013-11-19 12:28:50 -05:00
Robin Ward
639394e38a FIX: Pinned topics within a category weren't working without a refresh 2013-11-18 10:52:01 -05:00
Robin Ward
968c0672ef Enable column sorting main topic list 2013-11-14 15:50:36 -05:00
Robin Ward
bda5a72c6a Server side support for custom ordering of topics 2013-11-13 14:17:06 -05:00
Robin Ward
7207cef7aa TopicQuery cleanup in advance of custom sorting:
- Move SQL method constants into a module
- Removed unused count methods
- Moved methods that don't return a TopicList into Topic
- Replaced some confusing method signatures
2013-11-13 12:26:32 -05:00
Robin Ward
df568df9dc Controls for sorting topic columns 2013-11-13 11:48:38 -05:00
Robin Ward
be5de4244c FIX: Pinned posts were not displaying at the top of categories. 2013-11-08 15:05:14 -05:00
Robin Ward
744cd93f28 FIX: Non-English category name regression 2013-10-31 16:25:26 -04:00
Sam
b7d3b52e4f basic implementation of per-category new/latest etc filters 2013-10-24 21:05:06 +11:00
Sam
666264879c change it so all topics MUST include a category, we store a special uncategorized category to compensate
this cleans up a bunch of internals and removes some settings
2013-10-24 12:08:02 +11:00
Sam
af467a1675 404 page had horrible perf, it was using "take" instead of limit.
in such cases EVERY row is materialized and only after the limit is applied.

So ... imagine what happens on a forum with 2,000,000 topics
2013-10-02 13:05:03 +10:00
Alexander
e7b5774bc6 make category's RSS feed actually order by topic.created_at 2013-09-20 14:36:19 -07:00
Sam
36f8c9c45b improve logic and performance on front page to avoid massive query 2013-09-10 16:02:54 +10:00
Robin Ward
46efbac40e Add "Unread" filter to messages tab. Rename the other two views. 2013-08-30 12:32:05 -04:00
Sam
28466eb5b2 group the "suggested topics" by category correctly.
in the past new topics were not prioritizing current category and
new topics in a category were not being inserted before other unread topics
in other categories
2013-08-28 10:52:06 +10:00
Robin Ward
c0b051c9f6 Show Private Messages pill as a topic list rather than individual messages. 2013-08-24 16:58:16 -04:00
Navin
2e700dd26d Patch ActiveRecord::Relation#references and clean up 2013-08-16 14:53:40 +02:00
Navin
9ab2471a92 Fix rails4 deprecation warnings
That appear when running topic_spec.rb
2013-08-15 17:52:18 +02:00
Régis Hanol
4866f4d8f5 FIX: N+1 query for avatars 2013-08-14 15:25:05 +02:00
Robin Ward
1e730ccffe Suggested Topics tweak: Prefer topics in the same category 2013-08-08 13:18:52 -04:00
Robin Ward
0317cf9608 Show topics as a list of topics on the User Stream. 2013-07-25 15:56:20 -04:00
Robin Ward
147a239b28 Prefer unread topics in the same category 2013-07-18 14:47:59 -04:00
Stephan Kaag
6067736562 Refactor topic_query.rb for better readability
- Available options are defined and validated
- Internal methods are now protected
- Renamed some variables to be more consistent
2013-07-16 21:20:18 +02:00
Sam
ecf17cfebb work in progress, add fidelity to category group permissions (full, create posts, readonly) 2013-07-16 15:46:11 +10:00
Robin Ward
a43ed88699 Prefer your own topics in Suggested 2013-07-12 14:38:20 -04:00
Neil Lalonde
f39f44ddbf Categories page should not show invisible topics 2013-06-20 16:07:53 -04:00
Sam
e6e81efe85 correct information leak in page not found 2013-06-13 10:27:17 +10:00
Sam
73834370a5 work in progress, live unread and new counts 2013-05-30 16:49:57 +10:00
Robin Ward
9717a344c3 Refactor: Less reliance on views for logic for topic list 2013-05-29 13:28:07 -04:00
Sam
fcc7192fd2 User tracking state implementation progress for live unread / new counts 2013-05-23 15:22:06 +10:00
Sam
b5eff93a9d update message bus to support per client filtering
start work on user_tracking_state
fix can_ban? in guardian
expose protected scopes on topic_query we need
move guardian spec to use build as opposed to creating topics / posts / users
start work on user tracking spec
2013-05-21 16:39:51 +10:00
Sam
5cfcdc7ef0 backend for secure categories mostly done (todo pm groups) 2013-04-29 16:33:43 +10:00
Robin Ward
ee76f1926d Debugging Tool for Hot Topics 2013-04-02 18:00:53 -04:00
Robin Ward
b54054c1d3 No idea why these tests weren't failing for me in guard when I pushed last time. Fixed! 2013-04-01 14:54:53 -04:00
Robin Ward
1e6f8dc93a Include uncategorized pinned topics on 'Hot' 2013-04-01 13:49:35 -04:00
Robin Ward
473a64d39d Add score, percent_rank to topics. Adds HotTopic model and consolidated job to calculate
hotness. Note: People on Heroku will have to update their jobs to the new structure
in Heroku.md
2013-03-28 15:39:54 -04:00
Robin Ward
36269cfbaa Rename 'popular' to 'latest'. First stab at 'Hot' tab. 2013-03-27 16:21:23 -04:00
Karan Misra
5dfb04e4b3 Convert a lot of :a => b to a: b and bring peace to the world 2013-03-25 05:07:36 +05:30
Sam
3053831b97 don't show pinned stuff on the front page for anon 2013-03-11 18:30:25 -07:00
Sam
d6ca23a75b remove N+1 queries 2013-03-08 05:34:19 -08:00
Robin Ward
f8d8272406 Cleaned up TopicUserSpec, introduces clearing of pinned topics 2013-03-07 12:19:25 -05:00
Alexander
fd240c1967 RSS for topics in a category
Creates a new route for category RSS
2013-03-01 09:52:59 -08:00
Robin Ward
421f048458 Add a better from address to the email 2013-02-27 18:30:14 -05:00
Gosha Arinich
cafc75b238 remove trailing whitespaces ❤️ 2013-02-26 07:31:35 +03:00
Robin Ward
bb1156cee1 filter out closed/archived/invisible topics from suggested 2013-02-19 14:38:59 -05:00
Sam Saffron
87d83802b9 added option that allows users to decide when they consider topics new (default 2 days old or newer)
added site_setting to control the default new_topic_duration_minutes
added 10 minutes option for auto_track_topics_after_msecs, default bumped up to 5 mins
2013-02-14 17:36:14 +11:00
Robin Ward
21b5628528 Initial release of Discourse 2013-02-05 14:16:51 -05:00