Commit Graph

262 Commits

Author SHA1 Message Date
Guo Xiang Tan
996765562e DEV: Correct use of sanitize_sql_array in TopicQuery. 2020-09-08 10:44:35 +02:00
Guo Xiang Tan
05691b732d DEV: Address review comments for 5ed84d9885. 2020-09-08 10:44:27 +02:00
Guo Xiang Tan
1f6e8b642d SECURITY: Don't allow moderators to list PMs of all groups.
* Also return 404 when a user is trying to list PMs of a group that
cannot be accessed by the user.
2020-09-08 10:43:13 +02:00
Neil Lalonde
eb10109c99
Merge diffs from master 2020-06-24 13:48:37 -04:00
Neil Lalonde
607d00f780
Merge master 2020-06-24 13:47:36 -04:00
Vinoth Kannan
744bbf6904 FEATURE: exclude muted categories from the "top" topics list. 2020-05-08 00:34:53 +05:30
David Taylor
03818e642a
FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Sam Saffron
d0d5a138c3
DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
Dan Ungureanu
3d9c320aab
PERF: Cache Category.subcategory_ids (#9350)
Also reset category cache after backup restore.
2020-04-09 15:42:24 +03:00
Dan Ungureanu
b9d411a4eb
FIX: Topic.time_to_first_response should include sub-sub-categories (#9349) 2020-04-04 13:31:34 +03:00
Sam Saffron
980d2ed052
PERF: improve performance of category topic list
In some cases CTE caused pathologically bad query plans.
This optimises it so query runs by itself and caches for lifetime
of the topic query object.

This lightweight caching is done cause topic query will often
execute two queries (one for pinned and one for non pinned)
2020-03-05 11:46:20 -05:00
Sam Saffron
b4999acadd
PERF: improve performance of category topic list
In some cases CTE caused pathologically bad query plans.
This optimises it so query runs by itself and caches for lifetime
of the topic query object.

This lightweight caching is done cause topic query will often
execute two queries (one for pinned and one for non pinned)
2020-02-29 15:40:54 +11:00
Sam Saffron
18209e1daf
DEV: remove dead code
This code is not called anywhere, remove it
2020-02-29 15:05:09 +11:00
Dan Ungureanu
788ddcc407
FIX: Make topic query include topics from sub-sub-categories (#8709) 2020-01-20 17:06:58 +02:00
David Taylor
d1779346e8 FIX: topic_tracking_state when mute_all_categories_by_default is enabled 2020-01-06 18:22:42 +00:00
Neil Lalonde
875f0d8fd8
FEATURE: Tag synonyms
This feature adds the ability to define synonyms for tags, and the ability to merge one tag into another while keeping it as a synonym. For example, tags named "js" and "java-script" can be synonyms of "javascript". When searching and creating topics using synonyms, they will be mapped to the base tag.

Along with this change is a new UI found on each tag's page (for example, `/tags/javascript`) where more information about the tag can be shown. It will list the synonyms, which categories it's restricted to (if any), and which tag groups it belongs to (if tag group names are public on the `/tags` page by enabling the "tags listed by group" setting). Staff users will be able to manage tags in this UI, merge tags, and add/remove synonyms.
2019-12-04 13:33:51 -05:00
Vinoth Kannan
f83125f0c2 DEV: minor refactoring to reduce the code duplication. 2019-11-19 08:04:24 +05:30
Vinoth Kannan
57bbcf4c5d FIX: 'default_categories_muted' site setting not working for anonymous users. 2019-11-19 07:48:16 +05:30
Vinoth Kannan
3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Krzysztof Kotlarek
6e1fe22a9d
FEATURE: Dismiss new per category (#8330)
Ability to dismiss new topics per category.
2019-11-14 11:16:13 +11:00
Vinoth Kannan
ba5b78a348
FEATURE: support to mute all categories by default. (#8295)
Instead of enabling `suppress_from_latest` setting on many categories now we can enable `mute_all_categories_by_default` site setting. Then users should opt-in to categories for them to appear in the latest and categories pages.
2019-11-08 08:28:11 +05:30
Daniel Waterworth
200cef90ea FIX: TopicQuery doesn't react well to subcategories without definitions
Also:

Move includes call higher which makes it possible to run all of the
intermediate queries for easier debugging.

Add tests for TagsController with categories in the path.
2019-11-02 08:34:43 +00:00
Daniel Waterworth
790e1b7191 FIX: TopicQuery category lookup by slug
If we are searching for categories by their slugs, it doesn't make sense
to include subcategories since a slug, by itself, does not necessarily
uniquely identify a subcategory.

Similarly, the empty string as a slug is not a good category identifier.
2019-10-28 15:29:26 +00:00
Daniel Waterworth
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Gerhard Schlager
631315624d FIX: Topics with muted tag didn't show up when filtering by category and tag
It also removes the redundant `filter` parameter. Previously URLs looked like this:

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json?filter=tags/c/some-category/muted-tag/l/latest
```

But it looks like the `filter` parameter was only used to find out if topics with a muted tag should be removed or not. But the same thing can be accomplished by using the first tag ID. The following URL looks a lot cleaner.

```
http://example.com/tags/c/some-category/muted-tag/l/latest.json
```
2019-09-06 20:38:03 +02:00
Sam Saffron
0d5d478146 PERF: avoid filtering shared drafts when not used
In some very specific cases (large sites) shared drafts can introduce a
performance hit due to the mechanism used to filter out topics

This avoids the entire process when shared drafts are not enabled
2019-08-29 11:37:20 +10:00
Roman Rizzi
7c741fa0d6
FEATURE: Publish read state on group messages. (Originally introduced in #7989) (#8025)
* Revert "Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)""

This reverts commit 36425eb9f0.

* Fix: Show who read only if the attribute is enabled

* PERF: Precalculate the last post  readed by a group member

* Use book-reader icon instear of far-eye

* FIX: update topic groups correctly

* DEV: Tidy up read indicator update on write
2019-08-27 09:09:00 -03:00
romanrizzi
36425eb9f0 Revert "FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)"
This reverts commit 5dda5c2f7c.
2019-08-20 13:29:22 -03:00
Roman Rizzi
5dda5c2f7c
FEATURE: Publish read state on group messages. (#7989) [Undo revert] (#8024)
* Reenable: "FEATURE: Publish read state on group messages. (#7989)"

This reverts commit 67f5cc1ce8.

* FIX: Read indicator only appears when the group setting is enabled
2019-08-20 11:57:25 -03:00
romanrizzi
67f5cc1ce8 Revert "FEATURE: Publish read state on group messages. (#7989)"
This reverts commit 1630dae2db.
2019-08-20 10:24:34 -03:00
Roman Rizzi
1630dae2db
FEATURE: Publish read state on group messages. (#7989)
* Enable or disable read state based on group attribute

* When read state needs to be published, the minimum unread count is calculated in the topic query. This way, we can know if someone reads the last post

* The option can be enabled/disabled from the UI

* The read indicator will live-updated using message bus

* Show read indicator on every post

* The read indicator now shows read count and can be expanded to see user avatars

* Read count gets updated everytime someone reads a message

* Simplify topic-list read indicator logic

* Unsubscribe from message bus on willDestroyElement, removed unnecesarry values from post-menu, and added a comment to explain where does minimum_unread_count comes from
2019-08-20 09:46:57 -03:00
Sam Saffron
19e3b3b1bc PERF: speed up topic poster lookups
During profiling looking up topic users popped up as a hot path, this
change more than halved the amount of work it does

It reduces object allocations and method calls and avoids repeate translation
of common terms
2019-06-05 18:28:36 +10:00
Dan Ungureanu
c1e7a1b292 UX: Merge settings related to muted tags. (#7656) 2019-06-03 12:23:23 +10:00
Dan Ungureanu
8728850452 FEATURE: Mute topics tagged with both muted and unmuted tags. 2019-05-30 07:58:17 +08:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
Sam Saffron
0a5a6dfded DEV: stop mutating inputs as a side effect
We had quite a few cases in core where inputs are being mutated as a side
effect of calling a method.

This handles all the cases where specs caught this.

Mutating inputs makes code harder to reason about. Eg:

```
frog = "frog"
jump(frog)
puts frog
"fly" # ?????
```

This commit is part of a followup commit that adds # frozen_string_literal
to all our specs.
2019-04-30 10:25:53 +10:00
Maja Komel
422237391e FIX: fix notifications for flag PMs and show topics with moderator posts in inbox (#7331)
Some old moderator posts that were not correctly processed before this change and 993d8f34 are not listed under /u/username/messages
2019-04-25 11:15:13 +02:00
Sam Saffron
0c35b8b420 FEATURE: add suggested_topics_unread_max_days_old
This new site setting determines the maximum age of unread topics in
suggested. By default if you have any unread topics older than 90 days
they will be omitted from suggested.

This change was added for 2 reasons:

1. A performance safeguard, some users tend to collect a huge amount of
read state so it becomes super expensive to find unread

2. People who collect a large amount of unread are much more interested in
recent unread topics vs ancient unread topics, this makes suggested more
relevant

Also, this is a minor speed up for tests cause 3 expensive tests became 1.
2019-04-16 17:52:10 +10:00
Sam Saffron
0744d2ef73 PERF: speed up suggested unread
Previously we tried using a sub-query which has some terrible performance
implications, by adding 1 extra query we can eliminate the PG issue

A join to user_stats also is prone to the same slowdown
2019-04-05 15:55:29 +11:00
Sam Saffron
5f896ae8f7 PERF: Keep track of when a users first unread is
This optimisation avoids large scans joining the topics table with the
topic_users table.

Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.

The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
2019-04-05 12:44:45 +11:00
Sam
94b8ba4f8f FIX: remove slow platform detection from server side
Historically due to https://meta.discourse.org/t/why-is-discourse-so-slow-on-android/8823
we decreased page sizes of both home page and topic page on android by half.

This was done on the server side and as a side effect and caused page sizes on android
to mismatch between Android and non Android.

Unfortunately about a year ago googlebot started pretending it is Android,
this cause Google to start indexing pages as what android would see. So
it saw double the amount of pages in the index as what exists on desktop.
This in turn caused double the amount of indexing work and a large amount
of broken links on long topics.

This fix removes all special behavior which is no longer needed due to
other performance work in Discourse including raw handlebars on home page
and virtual dom on topic pages.

I tested we do not need this on Blu Advance 5.0 it has 1.3 GHZ mediatec mt6580
This phone retails for around $50 USD.

If we decide long term that we want any hacks like this we will shift them
to the client side. It can just hold data in memory without rendering.
2018-12-13 13:57:05 +11:00
David Taylor
6c71395bf6
FIX: Only hide shared draft topics from latest (#6737)
Previously we were hiding them from all topic lists, which can result in
topics being "stuck" in an unread state with no easy way to clear them.
2018-12-07 12:44:23 +00:00
David Taylor
0b1d660876
UX: Make shared drafts behaviour consistent for non-staff users (#6734)
This makes it easier to diagnose the problem when a public category
is set as the 'shared drafts category'. Doing this is not recommended.
2018-12-06 18:59:29 +00:00
Sam
e17a13ce19 FEATURE: additional "related messages" section
This splits out previous message correspondence from suggeted and instead
has a dedicated section called "related messages"
2018-11-12 13:04:42 +11:00
Sam
5c86e2d749 tweaks to related message list generation
- exclude users in groups I am in from related message search
- correctly limit number of related messages
2018-10-29 16:09:58 +11:00
Sam
9933059426 FEATURE: push related PMs to take first 3 slots
Previously the related PMs were last meaning you would have to work through
all unread to see them.

Also amends it so it either asks for related by group OR user not both.
2018-10-29 10:47:59 +11:00
Arpit Jalan
c0bb04d89d FIX: convert tag string to array when filtering topic list by tags 2018-10-08 08:56:25 +05:30
David Taylor
9bf522f227
FEATURE: Mixed case tagging (#6454)
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.

- If force_lowercase_tags is disabled, then mixed case tags are allowed.

- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.

- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.

- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.

- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
2018-10-05 10:23:52 +01:00