Commit Graph

2933 Commits

Author SHA1 Message Date
Martin Brennan
fbcc35b417
DEV: Remove PostAction/UserAction bookmark refs ()
We have not used anything related to bookmarks for PostAction
or UserAction records since 2020, bookmarks are their own thing
now. Deleting all this is just cleaning up old cruft.
2022-05-10 10:42:18 +10:00
Martin Brennan
222c8d9b6a
FEATURE: Polymorphic bookmarks pt. 3 (reminders, imports, exports, refactors) ()
A bit of a mixed bag, this addresses several edge areas of bookmarks and makes them compatible with polymorphic bookmarks (hidden behind the `use_polymorphic_bookmarks` site setting). The main ones are:

* ExportUserArchive compatibility
* SyncTopicUserBookmarked job compatibility
* Sending different notifications for the bookmark reminders based on the bookmarkable type
* Import scripts compatibility
* BookmarkReminderNotificationHandler compatibility

This PR also refactors the `register_bookmarkable` API so it accepts a class descended from a `BaseBookmarkable` class instead. This was done because we kept having to add more and more lambdas/properties inline and it was very messy, so a factory pattern is cleaner. The classes can be tested independently as well.

Some later PRs will address some other areas like the discourse narrative bot, advanced search, reports, and the .ics endpoint for bookmarks.
2022-05-09 09:37:23 +10:00
Jarek Radosz
3f0e767106
DEV: Use FakeLogger in RequestTracker specs ()
`TestLogger` was responsible for some flaky specs runs:

```
Error during failsafe response: undefined method `debug' for #<TestLogger:0x0000556c4b942cf0 @warnings=1>
Did you mean?  debugger
```

This commit also cleans up other uses of `FakeLogger`
2022-05-05 09:53:54 +08:00
David Taylor
c1db968740
DEV: Move hotlinked image information into a dedicated table ()
This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change.

For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 13:53:32 +01:00
David Taylor
0f772bdf5b
FEATURE: Optionally skip using full_name when suggesting usernames ()
This commit introduces a new site setting: `use_name_for_username_suggestions` (default true)

Admins can disable it if they want to stop using Name values when generating usernames for users. This can be useful if you want to keep real names private-by-default or, when used in conjunction with the `use_email_for_username_and_name_suggestions` setting, you would prefer to use email-based username suggestions.
2022-04-29 14:00:13 +01:00
Loïc Guitaut
008b700a3f DEV: Upgrade to Rails 7
This patch upgrades Rails to version 7.0.2.4.
2022-04-28 11:51:03 +02:00
Martin Brennan
3e4621c2cb
FEATURE: Polymorphic bookmarks pt. 2 (lists, search) ()
This pull request follows on from https://github.com/discourse/discourse/pull/16308. This one does the following:

* Changes `BookmarkQuery` to allow for querying more than just Post and Topic bookmarkables
* Introduces a `Bookmark.register_bookmarkable` method which requires a model, serializer, fields and preload includes for searching. These registered `Bookmarkable` types are then used when validating new bookmarks, and also when determining which serializer to use for the bookmark list. The `Post` and `Topic` bookmarkables are registered by default.
* Adds new specific types for Post and Topic bookmark serializers along with preloading of associations in `UserBookmarkList`
* Changes to the user bookmark list template to allow for more generic bookmarkable types alongside the Post and Topic ones which need to display in a particular way

All of these changes are gated behind the `use_polymorphic_bookmarks` site setting, apart from the .hbs changes where I have updated the original `UserBookmarkSerializer` with some stub methods.

Following this PR will be several plugin PRs (for assign, chat, encrypt) that will register their own bookmarkable types or otherwise alter the bookmark serializers in their own way, also gated behind `use_polymorphic_bookmarks`.

This commit also removes `BookmarkQuery.preloaded_custom_fields` and the functionality surrounding it. It was added in 0cd502a558 but only used by one plugin (discourse-assign) where it has since been removed, and is now used by no plugins. We don't need it anymore.
2022-04-22 08:23:42 +10:00
Martin Brennan
154afa60eb
FIX: Skip upload extension validation when changing security ()
When changing upload security using `Upload#update_secure_status`,
we may not have the context of how an upload is being created, because
this code path can be run through scheduled jobs. When calling
update_secure_status, the normal ActiveRecord validations are run,
and ours include validating extensions. In some cases the upload
is created in an automated way, such as user export zips, and the
security is applied later, with the extension prohibited from
use when normally uploading.

This caused the upload to fail validation on `update_secure_status`,
causing the security change to silently fail. This fixes the issue
by skipping the file extension validation when the upload security
is being changed.
2022-04-20 14:11:39 +10:00
Bianca Nenciu
056c7a3f30
DEV: Add test for web hooks and topic tags changes () 2022-04-19 20:48:32 +03:00
David Taylor
137e06a316
FIX: Ensure 'crop' always returns requested dimensions ()
Previously, 'crop' would resize the image to have the requested width, then crop the height to the requested value. This works when cropping images vertically, but not when cropping them horizontally.

For example, trying to crop a 500x500 image to 200x500 was actually resulting in a 200x200 image. Having an OptimizedImage with width/height columns mismatching the actual OptimizedImage width/height causes some unusual issues.

This commit ensures that a call to `OptimizedImage.crop(from, to, width, height)` will always return an image of the requested width/height. The `w x h^` syntax defines minimum width/height, while maintaining aspect ratio.
2022-04-19 11:21:24 +01:00
Martin Brennan
c6c633e041
FIX: Issues with incorrect unread and private message topic tracking state ()
This commit fixes two issues at play. The first was introduced
in f6c852b (or maybe not introduced
but rather revealed). When a user posted a new message in a topic,
they received the unread topic tracking state MessageBus message,
and the Unread (X) indicator was incremented by one, because with the
aforementioned perf commit we "guess" the correct last read post
for the user, because we no longer calculate individual users' read
status there. This meant that every time a user posted in a topic
they tracked, the unread indicator was incremented. To get around
this, we can just exclude the user who created the post from the
target users of the unread state message.

The second issue was related to the private message topic tracking
state, and was somewhat similar. Whenever a user created a new private
message, the New (X) indicator was incremented, and could not be
cleared until the page was refreshed. To solve this, we just don't
update the topic state for the user when the new_topic tracking state
message comes through if the user who created the topic is the
same as the current user.

cf. https://meta.discourse.org/t/bottom-of-topic-shows-there-is-1-unread-remaining-when-there-are-actually-0-unread-topics-remaining/220817
2022-04-19 11:37:01 +10:00
Bianca Nenciu
86c7e07428
FEATURE: Improve screened IPs roll up and extend for IPv6 ()
This commit improves the logic for rolling up IPv4 screened IP
addresses and extending it for IPv6. IPv4 addresses will roll up only
up to /24. IPv6 can rollup to /48 at most. The log message that is
generated contains the list of original IPs and new subnet.
2022-04-12 21:07:37 +03:00
Roman Rizzi
6f76a12e0a
FEATURE: Let sites add a sitemap.xml file. ()
* FEATURE: Let sites add a sitemap.xml file.

This PR adds the same features discourse-sitemap provides to core. Sitemaps are only added to the robots.txt file if the `enable_sitemap` setting is enabled and `login_required` disabled.

After merging , this change will take priority over the sitemap plugin because it will disable itself. We're also using the same sitemaps table, so our migration won't try to create it
again using `if_not_exists: true`.
2022-04-12 10:33:59 -03:00
Sam
cedcdb0057
FEATURE: allow for local theme js assets ()
Due to default CSP web workers instantiated from CDN based assets are still
treated as "same-origin" meaning that we had no way of safely instansiating
a web worker from a theme.

This limits the theme system and adds the arbitrary restriction that WASM
based components can not be safely used.

To resolve this limitation all js assets in about.json are also cached on
local domain.

{
  "name": "Header Icons",
  "assets" : {
    "worker" : "assets/worker.js"
  }
}

This can then be referenced in JS via:

settings.theme_uploads_local.worker

local_js_assets are unconditionally served from the site directly and
bypass the entire CDN, using the pre-existing JavascriptCache

Previous to this change this code was completely dormant on sites which
used s3 based uploads, this reuses the very well tested and cached asset
system on s3 based sites.

Note, when creating local_js_assets it is highly recommended to keep the
assets lean and keep all the heavy working in CDN based assets. For example
wasm files can still live on the CDN but the lean worker that loads it can
live on local.

This change unlocks wasm in theme components, so wasm is now also allowed
in `theme_authorized_extensions`

* more usages of upload.content

* add a specific test for upload.content

* Adjust logic to ensure that after upgrades we still get a cached local js
on save
2022-04-07 07:58:10 +10:00
Jarek Radosz
8e809149d2
DEV: Fix "overridden" typos ()
There are still some, but those are in actual code that's used outside core, so the change there would need to go through the deprecation cycle. That's a task for another day.
2022-04-06 23:17:20 +02:00
Sam
1598e6b489
FIX: users watching tags in open tag groups not notified ()
All users are members of the EVERYONE group, but this group is special and
is omitted from the group_users table. When checking permission we need to
make sure we also add a bypass.

This also fixes a very buggy test in post_alerter, it was confirming the
broken behavior due to fabricator flow.

When it defined the tag group the everyone group automatically had full access
then the additional permission fabricated just added one more group. After
fix was made to code the test started failing. Fabricators can be risky.
2022-04-06 11:43:57 +10:00
Blake Erickson
ec2930712d
FIX: 500 error when creating a user with an integer username ()
Via the API it is possible to create a user with an integer username. So
123 instead of "123". This causes the following 500 error:

```
NoMethodError (undefined method `unicode_normalize' for 1:Integer)
app/models/user.rb:276:in `normalize_username'
```

See: https://meta.discourse.org/t/222281
2022-04-04 15:15:32 -06:00
Mark VanLandingham
b974375239
DEV: Skip notifications without topic_id in ensure_consistency () 2022-03-30 09:56:35 -05:00
Martin Brennan
b8828d4a2d
FEATURE: Polymorphic bookmarks pt. 1 (CRUD) ()
This commit introduces a new use_polymorphic_bookmarks site setting
that is default false and hidden, that will be used to help continuous
development of polymorphic bookmarks. This setting **should not** be
enabled anywhere in production yet, it is purely for local development.

This commit uses the setting to enable create/update/delete actions
for polymorphic bookmarks on the server and client side. The bookmark
interactions on topics/posts are all usable. Listing, searching,
sending bookmark reminders, and other edge cases will be handled
in subsequent PRs.

Comprehensive UI tests will be added in the final PR -- we already
have them for regular bookmarks, so it will just be a matter of
changing them to be for polymorphic bookmarks.
2022-03-30 12:43:11 +10:00
Bianca Nenciu
6eb3d658ca
FIX: Do not wrap unaccent around tsqueries ()
tsqueries use quotes and having other characters that when unaccented
become quotes results in invalid tsqueries.
2022-03-25 19:10:05 +02:00
Bianca Nenciu
cbaf7c949b
FIX: Make sure max_oneboxes_per_post is enforced ()
PostAnalyzer and CookedPostProcessor both replace URLs with oneboxes.
PostAnalyzer did not use the max_oneboxes_per_post site and setting and
CookedPostProcessor replaced at most max_oneboxes_per_post URLs ignoring
the oneboxes that were replaced already by PostAnalyzer.
2022-03-23 17:36:08 +02:00
Rafael dos Santos Silva
100a84d345
DEV: Reserve assigned notification type ()
* DEV: Reserve assigned notification type

* update constants
2022-03-16 17:08:10 -03:00
Jarek Radosz
383f164672
FIX: Always include the first poster when converting to PM ()
Regressed in 
2022-03-15 10:51:10 +01:00
Sam
de9a031073
FEATURE: use canonical links in posts.rss feed ()
* FEATURE: use canonical links in posts.rss feed

Previously we used non canonical links in posts.rss

These links get crawled frequently by crawlers when discovering new
content forcing crawlers to hop to non canonical pages just to end up
visiting canonical pages

This uses up expensive crawl time and adds load on Discourse sites

Old links were of the form:

`https://DOMAIN/t/SLUG/43/21`

New links are of the form

`https://DOMAIN/t/SLUG/43?page=2#post_21`

This also adds a post_id identified element to crawler view that was
missing.

Note, to avoid very expensive N+1 queries required to figure out the
page a post is on during rss generation, we cache that information.

There is a smart "cache breaker" which ensures worst case scenario is
a "page drift" - meaning we would publicize a post is on page 11 when
it is actually on page 10 due to post deletions. Cache holds for up to
12 hours.

Change only impacts public post RSS feeds (`/posts.rss`)
2022-03-15 20:17:06 +11:00
Sam
3bf5692c72
FEATURE: prioritize group search order based on prefix match ()
Our @mention user search prioritized users based on prefix matches.

So if searching for `sa` we will display `sam`, `asam` in that order

Previously, we did not prioritize group matches based on prefix. This change ensures better parity.

Implementation notes:

1. User search only prioritizes based on username prefix, not name prefix. TBD if we want to change that.
2. @mention on client side will show 0 group matches if we fill up all the spots with user matches. TBD if we want to unconditionally show the first / second group match.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-03-03 16:57:52 +11:00
Krzysztof Kotlarek
ea3a58d051
FIX: indirectly muted categories for topic-tracking-state ()
Topics belonging to indirectly muted categories should be excluded from topic-tracking-state report.
2022-03-02 15:02:09 +11:00
David Taylor
c9dab6fd08
DEV: Automatically require 'rails_helper' in all specs ()
It's very easy to forget to add `require 'rails_helper'` at the top of every core/plugin spec file, and omissions can cause some very confusing/sporadic errors.

By setting this flag in `.rspec`, we can remove the need for `require 'rails_helper'` entirely.
2022-03-01 17:50:50 +00:00
Jarek Radosz
2fc70c5572
DEV: Correctly tag heredocs ()
This allows text editors to use correct syntax coloring for the heredoc sections.

Heredoc tag names we use:

languages: SQL, JS, RUBY, LUA, HTML, CSS, SCSS, SH, HBS, XML, YAML/YML, MF, ICS
other: MD, TEXT/TXT, RAW, EMAIL
2022-02-28 20:50:55 +01:00
Jarek Radosz
4020738eed
DEV: Fix specs deprecations () 2022-02-26 03:51:39 +01:00
Jeff Wong
d1bdb6c65d
FEATURE: upload an avatar option for uploading avatars with selectable avatars ()
* FEATURE: upload an avatar option for uploading avatars with selectable avatars

Allow staff or users at or above a trust level to upload avatars even when the site
has selectable avatars enabled.

Everyone can still pick from the list of avatars. The option to upload is shown
below the selectable avatar list.

refactored boolean site setting into an enum with the following values:

disabled: No selectable avatars enabled (default)
everyone: Show selectable avatars, and allow everyone to upload custom avatars
tl1: Show selectable avatars, but require tl1+ and staff to upload custom avatars
tl2: Show selectable avatars, but require tl2+ and staff to upload custom avatars
tl3: Show selectable avatars, but require tl3+ and staff to upload custom avatars
tl4: Show selectable avatars, but require tl4 and staff to upload custom avatars
staff: Show selectable avatars, but only allow staff to upload custom avatars
no_one: Show selectable avatars. No users can upload custom avatars

Co-authored-by: Régis Hanol <regis@hanol.fr>
2022-02-24 12:57:39 -08:00
Sam
d4d3580761
PERF: perform all cached counting in background ()
Previously cached counting made redis calls in main thread and performed
the flush in main thread.

This could lead to pathological states in extreme heavy load.

This refactor reduces load and cleans up the interface
2022-02-22 16:45:25 +00:00
David Taylor
f6c852bf8e
PERF: Use a shared message for replies to tracked topics ()
Previously we were publishing one messagebus message per user which was 'tracking' a topic. On large sites, this can easily be 1000+ messages. The important information in the message is common between all users, so we can manage with a single message on a shared channel, which will be much more efficient.

For user-specific values (notification_level and last_read_post_number), the JS app can infer values which are 'good enough'. Correct values will be loaded as soon as a topic-list containing the topic is visited.
2022-02-22 15:27:46 +00:00
Joffrey JAFFEUX
f5ec32bc8c
FEATURE: adds the user_promoted event to webhooks () 2022-02-22 10:57:18 +01:00
Martin Brennan
fa0c796baf
DEV: Fix SMTP bounce regexp ()
Never trust me with regexp. Follow up to
01ef1d08fc,
which did not take into account codes in
the format X.X.XX (with the 2 digits on the end)
2022-02-22 08:54:01 +10:00
Bianca Nenciu
90c3695ab0
FEATURE: Rename Reset Read bulk action to Defer ()
It is enabled only if defer is enabled in user options too and if the
button shows up in the topic's footer.
2022-02-21 22:45:01 +02:00
Bianca Nenciu
94883dd326
FIX: Show links with 0 clicks in top links section ()
This is useful to moderators to see if users post spam links.
2022-02-21 22:15:38 +02:00
David Taylor
5d6d3fb244
DEV: Remove child theme settings/variables from parent compilation ()
aa1442fdc3 split theme stylesheets so that every component gets its own stylesheet. Therefore, there is now no need for parent themes to collate the settings/variables of its children during scss compilation.

Technically this is a breaking change for any themes which depend on the settings/variables of their child components. That was never a supported/recommended arrangement, so we don't expect this to cause issues.
2022-02-21 11:15:35 +00:00
David Taylor
c8d956374d FIX: Use fresh theme setting values when compiling stylesheets
If a theme is updated to introduce a new setting AND immediately make use of it in a stylesheet, then an error was being shown. This is because the stylesheet compilation was using the theme's cached settings, and the cache is only cleared **after** the theme has finished compiling.

This commit updates the SCSS compilation to use uncached values for settings. A similar fix was applied to other parts of theme compilation back in 2020: (a51b8d9c66)
2022-02-21 09:45:14 +00:00
David Taylor
51afa579f7 DEV: Use the theme cache helper for settings
The previous Discourse.cache usage was different to how other theme-related caching is handled, and also requires reaching out to redis every time. The common theme cache is held in memory (as a DistributedCache)
2022-02-21 09:45:14 +00:00
Martin Brennan
01ef1d08fc
FIX: Conform EmailLog#bounce_error_code to RFC ()
This commit makes sure that the email log's bounce_error_code
conforms to the SMTP error code RFC on save, so that
it is always in the format X.X.X or XXX without any
additional string details. Also included is a migration
to fix this issue for past records.
2022-02-21 11:26:39 +10:00
Bianca Nenciu
38cbca3f67
FIX: Count clicks on links with query params ()
This did not work sometimes if a topic had the same URL with and without
query params because it did not try to select the best matching URL.
2022-02-18 14:47:56 +02:00
Bianca Nenciu
53f9a1a469
FEATURE: Add settings to scale daily flags limit ()
Similar site settings exist for likes and edits and the new ones work
in a similar way.

By default, users below TL2 have a limit of 20, the limit is increased
by 1.5 for TL2 users up to 30, by 2 for TL3 users up to 40 and by 3 for
TL4 users up to 60.
2022-02-18 14:44:32 +02:00
megothss
c71c107649
FIX: Don't accept accents in slug if generation_method == 'ascii' ()
* FIX: Don't accept accents in slug if generation_method == 'ascii'

Fixes bug reported in:
- https://meta.discourse.org/t/404-when-trying-to-edit-category-with-accent-in-slug/214762
- https://meta.discourse.org/t/formatting-and-accents-in-urls/215734/5

Assuming `SiteSetting.slug_generation_method == 'ascii'.

If the user provides a slug containing non-ascii characters while
creating the category, the user will receive a 404 error just
after saving the category since the slug will be escaped anyway but
Category.find_by_slug_path won't escape the category slug
causing the Edit Page of the category to be inaccessible.

This commit checks the provided slug and raises an error if the
provided slugcontains non-ascii characters ensuring that the
provided value is consistent with the site settings.

It also changes Category.find_by_slug_path to always escape the slug,
since if present, it is escaped anyway in Category.ensure_slug to
prevent the 404 in the Edit Category Page in case the user already
have some category with a non-ascii slug.

* Removed trailing whitespace
2022-02-17 13:46:06 +11:00
Krzysztof Kotlarek
a7d43cf1ec
FEATURE: mute subcategory when parent category is muted ()
When parent category or grandparent category is muted, then category should be muted as well.

Still, it can be overridden by setting individual subcategory notification level.

CategoryUser record is not created, mute for subcategories is purely virtual.
2022-02-17 00:42:02 +01:00
Dan Ungureanu
effbd6d3e4
FEATURE: Show error if invite to topic is invalid ()
This can happen if the topic to which a user is invited is in a private
category and the user was not invited to one of the groups that can see
that specific category.

This used to be a warning and this commit makes it an error.
2022-02-16 18:35:02 +02:00
Sam
33a0ad1b69
PERF: introduce site/global emoji cache ()
Previously calls such as `Emoji["smile"]` would force a full dehydration of
objects from Redis.

This introduces a version safe site and global emoji cache so lookups are
cheap. It eliminates iterating through the list of emojis and pulling from
redis.

Distributed cache uses a normalized name as the key and stores an Array tuple
with version and Emoji. Successful hits always confirm version matches.

Interface to Emoji object remains unchanged.

We opted for 2 caches to improve reuse on multisites. misses though will be
stored in both caches. If there is a hit on the global cache we can avoid
looking up in site local cache and storing a miss there.
2022-02-16 12:46:17 +11:00
David Taylor
94a47d037f
PERF: Reduce number of EXPIRE calls from CachedCounting ()
Previously we were calling `EXPIRE` every time we incremented a given key. Instead, we can call EXPIRE once when the key is first populated. A LUA script is used to make this as efficient as possible.

Consumers of this Concern use daily keys. Since we're now calling EXPIRE only at the beginning of the day, rather than throughout the day, the expire time has been increased from 3 to 4 days.
2022-02-15 16:55:21 +00:00
Rafael dos Santos Silva
4d3da70bc6
PERF: Use Redis SET EX GET instead of LUA script for counting ()
This will prevent Discourse from booting on Redis < 6.2.0
2022-02-15 10:36:07 -03:00
David Taylor
af24c10314 DEV: Improve theme error handling UX
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
2022-02-14 10:11:19 +00:00
Alan Guo Xiang Tan
8e5b945b0f
Revert "DEV: Improve theme error handling UX" ()
`PrettyText.cook` is breaking on some sites. Revert for now while we
investigate.

This reverts commit c81d369ab6.
2022-02-11 11:30:36 +08:00
Alan Guo Xiang Tan
b876ff6281
FIX: Update user stat counts when post/topic visibility changes. ()
Breakdown of fixes in this commit:

* `UserStat#topic_count` was not updated when visibility of
the topic changed.

* `UserStat#post_count` was not updated when post was hidden or
unhidden.

* `TopicConverter` was only incrementing or decrementing the counts by 1
even if a user has multiple posts in the topic.

* The commit turns off the verbose logging by default as it is just
noise to normal users who are not debugging this problem.
2022-02-11 09:00:58 +08:00
David Taylor
c81d369ab6 DEV: Improve theme error handling UX
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
2022-02-10 22:56:11 +00:00
Ayke Halder
5ff3a9c4bb
DEV: add native lazy loading for emojis () 2022-02-09 12:18:59 +01:00
Bianca Nenciu
f704deca17
FIX: Clear drafts only when post is created by real user ()
This commits adds a new advance_draft to PostCreator that controls if
the draft sequence will be advanced or not. If the draft sequence is
advanced then the old drafts will be cleared. This used to happen for
posts created by plugins or through the API and cleared user drafts
by mistake.
2022-02-09 10:37:38 +02:00
Blake Erickson
71f7f7ed49
FEATURE: Add external_id to topics ()
* FEATURE: Add external_id to topics

This commit allows for topics to be created and fetched by an
external_id. These changes are API only for now as there aren't any
front changes.

* add annotations

* add external_id to this spec

* Several PR feedback changes

- Add guardian to find topic
- 403 is returned for not found as well now
- add `include_external_id?`
- external_id is now case insensitive
- added test for posts_controller
- added test for topic creator
- created constant for max length
- check that it redirects to the correct path
- restrain external id in routes file

* remove puts

* fix tests

* only check for external_id in webhook if exists

* Update index to exclude external_id if null

* annotate

* Update app/controllers/topics_controller.rb

We need to check whether the topic is present first before passing it to the guardian.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2022-02-08 20:55:32 -07:00
Martin Brennan
0a738bd5bc
FEATURE: Allow sending group SMTP emails with from alias ()
This commit allows group SMTP emails to be sent with a
different from email address that has been set up as an
alias in the email provider. Emails from the alias will
be grouped correctly using Message-IDs in the mail client,
and replies to the alias go into the correct group inbox.
2022-02-07 13:52:01 +10:00
Alan Guo Xiang Tan
b7eacaed21
FIX: Handle addressable error when parsing an invalid URL. ()
Passing in an invalid URL would result in an `Addressable::URI::InvalidURIError` which we were not catching.
2022-02-07 11:25:42 +08:00
Alan Guo Xiang Tan
5bd55acf83
FIX: Add DB constraints for post & topic counter cache for UserStat ()
Ensures that `UserStat#post_count` and `UserStat#topic_count` does not
go below 0. When it does like it did now, we tend to have bugs in our
code since we're usually coding with the assumption that the count isn't
negative.

In order to support the constraints, our post and topic fabricators in
tests will now automatically increment the count for the respective
user's `UserStat` as well. We have to do this because our fabricators
bypasss `PostCreator` which holds the responsibility of updating `UserStat#post_count` and
`UserStat#topic_count`.
2022-02-07 11:23:34 +08:00
Alan Guo Xiang Tan
4aa9a813ec
FIX: Liking whispers should not contribute to Topic#like_count. ()
Non-staff users are not allowed to see whisper so this change prevents
non-staff user from seeing a like count that does not make sense to
them. In the future, we might consider adding another like count column
for staff user.

Follow-up to 4492718864
2022-02-03 16:24:33 +11:00
Bianca Nenciu
48e5d1af03
FIX: Improve top links section from user summary ()
* Do not extract links for hotlinked images
* Include only links that have been clicked at least once in user
summary
2022-01-24 11:33:23 +11:00
Alan Guo Xiang Tan
e4e37257cc FIX: Handle malformed URLs in TopicEmbed.absolutize_urls. 2022-01-21 11:18:54 +08:00
Blake Erickson
12f041de5d
FIX: Tag watching for everyone tag groups ()
* FIX: Tag watching for everyone tag groups

Tags in tag groups that have permissions set to everyone were not able
to be saved correctly. A user on their preferences page would mark the
tags that they wanted to save, but the watched_tags in the response
would be empty. This did not apply to admins, just regular users. Even
though the watched tags were being saved in the db, the user serializer
response was filtering them out. When a user refreshed their preferences
pages it would show zero watched tags.

This appears to be a regression introduced by:

0f598ca51e

The issue that needed to be fixed is that we don't track the "everyone"
group (which has an id of 0) in the group_users table. This is because
everyone has access to it, so why fill a row for every single user, that
would be a lot. The fix was to update the query to include tag groups
that had permissions set to the "everyone" group (group_id 0).

I also added another check to the existing spec for updating
watched tags for tags that aren't in a tag group so that it checks the
response body. I then added a new spec which updates watched tags for
tags in a tag group which has permissions set to everyone.

* Resolve failing tests

Improve SQL query syntax for including the "everyone" group with the id
of 0.

This commit also fixes a few failing tests that were introduced. It
turns out that the Fabrication of the Tag Group Permissions was faulty.
What happens when creating the tag groups without any permissions is
that it sets the permission to "everyone". If we then follow up with
fabricating a tag group permission on the tag group instead of having a
single permission it will have 2 (everyone + the group specified)! We
don't want this. To fix it I removed the fabrication of tag group
permissions and just set the permissions directly when creating the tag
group.

* Use response.parsed_body instead of JSON.parse
2022-01-18 15:02:29 -07:00
Roman Rizzi
5ee31cbf7d
FIX: Mark invites flash messages as HTML safe. ()
* FIX: Mark invites flash messages as HTML safe.
This change should be safe as all user inputs included in the errors are sanitized before sending it back to the client.

Context: https://meta.discourse.org/t/html-tags-are-explicit-after-latest-update/214220

* If somebody adds a new error message that includes user input and doesn't sanitize it, using html-safe suddenly becomes unsafe again. As an extra layer of protection, we make the client sanitize the error message received from the backend.

* Escape user input instead of sanitizing
2022-01-18 09:38:31 -03:00
Roman Rizzi
8b3d50713d
FIX: Pass category and tag IDs to the emit webhook event job. ()
* FIX: Pass category and tag IDs to the emit webhook event job.

Like webhooks won't fire when they're scoped to specific categories or tags because we're not passing the data to the job that emits it.

* Update config/initializers/012-web_hook_events.rb

Co-authored-by: Dan Ungureanu <dan@ungureanu.me>

Co-authored-by: Dan Ungureanu <dan@ungureanu.me>
2022-01-14 11:17:38 -03:00
Bianca Nenciu
5d35c38db2
FEATURE: Search screened IP address in blocks ()
An admin could search for all screened ip addresses in a block by
using wildcards. 192.168.* returned all IPs in range 192.168.0.0/16.
This feature allows admins to search for a single IP address in all
screened IP blocks. 192.168.0.1 returns all IP blocks that match it,
for example 192.168.0.0/16.

* FEATURE: Remove roll up button for screened IPs

* FIX: Match more specific screened IP address first
2022-01-11 09:16:51 +02:00
Jarek Radosz
5a50f18c0c
DEV: Avoid $ globals ()
Also:
* Remove an unused method (#fill_email)
* Replace a method that was used just once (#generate_username) with `SecureRandom.alphanumeric`
* Remove an obsolete dev puma `tmp/restart` file logic
2022-01-08 23:39:46 +01:00
janzenisaac
30388512ec
DEV: Retroactively add test for ignoring flagged responses () 2022-01-07 11:53:31 -06:00
Arpit Jalan
554ff07786
FIX: when composing a message do not suggest deleted posts users () 2022-01-07 17:06:11 +05:30
Andrei Prigorshnev
16d9b2755c
DEV: rename single_sign_on classes to discourse_connect () 2022-01-06 16:28:46 +04:00
Peter Zhu
c5fd8c42db
DEV: Fix methods removed in Ruby 3.2 ()
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
2022-01-05 18:45:08 +01:00
Arpit Jalan
702685b6a0
SECURITY: only show user suggestions with regular post () 2022-01-03 13:37:40 +05:30
Osama Sayegh
8e6988163f
FIX: Don't allow NULL values for notification_level in category_users () 2021-12-29 09:19:39 +11:00
Daniel Waterworth
5ff0b86b57
PERF: Make tests faster by prefabricating more things () 2021-12-22 11:09:43 -06:00
Andrei Prigorshnev
c202252190
FEATURE: when suggesting usernames skip input that consist entirely of disallowed characters () 2021-12-21 21:13:05 +04:00
Daniel Waterworth
7e0c1fb039
PERF: Make tests faster by prefabricating more things () 2021-12-20 12:59:10 -06:00
Martin Brennan
2d68e5d942
FEATURE: Scheduled problem checks for admin dashboard ()
This commit introduces scheduled problem checks for the admin dashboard, which are long running or otherwise cumbersome problem checks that will be run every 10 minutes rather than every time the dashboard is loaded. If these scheduled checks add a problem, the problem will remain until it is cleared or until the scheduled job runs again.

An example of a check that should be scheduled is validating credentials against an external provider.

This commit also introduces the concept of a `priority` to the problems generated by `AdminDashboardData` and the scheduled checks. This is `low` by default, and can be set to `high`, but this commit does not change any part of the UI with this information, only adds a CSS class.

I will be making a follow up PR to check group SMTP credentials.
2021-12-20 09:59:11 +10:00
Andrei Prigorshnev
9365c4b364
DEV: make sure we handle staged users correctly in DiscourseConnect ()
Some time ago, we made this fix to external authentication –  https://github.com/discourse/discourse/pull/13706. We didn't address Discourse Connect (https://meta.discourse.org/t/discourseconnect-official-single-sign-on-for-discourse-sso/13045) at that moment, so I wanted to fix it for Discourse Connect as well.

Turned out though that Discourse Connect doesn't contain this problem and already handles staged users correctly. This PR adds tests that confirm it. Also, I've extracted two functions in Discourse Connect implementation along the way and decided to merge this refactoring too (the refactoring is supported with tests).
2021-12-16 19:44:07 +04:00
Daniel Waterworth
102fa71ef3
PERF: Speed up the tests by pre-fabricating more things () 2021-12-15 11:41:14 -06:00
Alan Guo Xiang Tan
6fe4c8ae58
DEV: Fix intermittent failing spec. () 2021-12-15 09:51:46 +01:00
Dan Ungureanu
adb6202c94
FIX: Check if invite domain is valid ()
* FIX: Check if invite domain is valid

Previous regex checked for generic hostname, which is too generic for
this case.
2021-12-13 16:39:14 +11:00
Angus McLeod
df3886d6e5
FEATURE: Experimental support for group membership via google auth ()
This commit introduces a new site setting "google_oauth2_hd_groups". If enabled, group information will be fetched from Google during authentication, and stored in the Discourse database. These 'associated groups' can be connected to a Discourse group via the "Membership" tab of the group preferences UI. 

The majority of the implementation is generic, so we will be able to add support to more authentication methods in the near future.

https://meta.discourse.org/t/managing-group-membership-via-authentication/175950
2021-12-09 12:30:27 +00:00
Daniel Waterworth
3b0d46c659
FIX: make tests work with higher starting upload id () 2021-12-08 13:13:59 -06:00
Bianca Nenciu
b1c11d5787
FIX: Select correct topic draft for user ()
The old query could return multiple rows.
2021-12-08 15:23:44 +02:00
Bianca Nenciu
049bc33838
FIX: Update has_topic_draft when draft is updated ()
Current user state regarding the new topic draft was not updated when
the draft was created or destroyed.
2021-12-08 14:40:35 +02:00
Alan Guo Xiang Tan
4e67297a7c FIX: Missing allowed urls when displaying granualar API key scopes.
Follow-up to 3791fbd919
2021-12-07 10:17:17 +08:00
Andrei Prigorshnev
f3508065a3
FIX: auth incorrectly handles duplicate usernames () 2021-12-06 20:49:04 +04:00
Daniel Waterworth
63112f89a3
PERF: Shave off some test-suite time () 2021-12-03 14:54:07 -06:00
Michelle Bueno Saquetim Vendrame
9b5836aa1d
Add three reports ()
* Add report top_users_by_received_likes

* Add report top_users_by_received_likes_from_inferior_trust_level

* Add report top_users_by_likes_received_from_a_variety_of_people

* Add test to report_top_users_by_received_likes

* add top_users_by_likes_received_from_a_variety_of_people report test

* add top_users_by_likes_received_from_inferior_trust_level report tests
2021-12-02 22:41:55 +05:30
Andrei Prigorshnev
1c0022c195
FIX: extract and fix overriding of usernames by external auth () 2021-12-02 17:42:23 +04:00
Natalie Tay
0f598ca51e
SECURITY: Only show tags to users with permission () 2021-12-01 10:26:56 +08:00
Roman Rizzi
1fc06520bd
REFACTOR: Improve support for consolidating notifications. ()
* REFACTOR: Improve support for consolidating notifications.

Before this commit, we didn't have a single way of consolidating notifications. For notifications like group summaries, we manually removed old ones before creating a new one. On the other hand, we used an after_create callback for likes and group membership requests, which caused unnecessary work, as we need to delete the record we created to replace it with a consolidated one.

We now have all the consolidation rules centralized in a single place: the consolidation planner class. Other parts of the app looking to create a consolidable notification can do so by calling Notification#consolidate_or_save!, instead of the default Notification#create! method.

Finally, we added two more rules: one for re-using existing group summaries and another for deleting duplicated dashboard problems PMs notifications when the user is tracking the moderator's inbox. Setting the threshold to one forces the planner to apply this rule every time.

I plan to add plugin support for adding custom rules in another PR to keep this one relatively small.

* DEV: Introduces a plugin API for consolidating notifications.

This commit removes the `Notification#filter_by_consolidation_data` scope since plugins could have to define their criteria. The Plan class now receives two blocks, one to query for an already consolidated notification, which we'll try to update, and another to query for existing ones to consolidate.

It also receives a consolidation window, which accepts an ActiveSupport::Duration object, and filter notifications created since that value.
2021-11-30 13:36:14 -03:00
Loïc Guitaut
a5fbb90df4 FEATURE: Display pending posts on user’s page
Currently when a user creates posts that are moderated (for whatever
reason), a popup is displayed saying the post needs approval and the
total number of the user’s pending posts. But then this piece of
information is kind of lost and there is nowhere for the user to know
what are their pending posts or how many there are.

This patch solves this issue by adding a new “Pending” section to the
user’s activity page when there are some pending posts to display. When
there are none, then the “Pending” section isn’t displayed at all.
2021-11-29 10:26:33 +01:00
Dan Ungureanu
fa8cd629f1
DEV: Hash tokens stored from email_tokens ()
This commit adds token_hash and scopes columns to email_tokens table.
token_hash is a replacement for the token column to avoid storing email
tokens in plaintext as it can pose a security risk. The new scope column
ensures that email tokens cannot be used to perform a different action
than the one intended.

To sum up, this commit:

* Adds token_hash and scope to email_tokens

* Reuses code that schedules critical_user_email

* Refactors EmailToken.confirm and EmailToken.atomic_confirm methods

* Periodically cleans old, unconfirmed or expired email tokens
2021-11-25 09:34:39 +02:00
janzenisaac
8c7cc426b7
FEATURE: Notify responders of post removal ()
- Notify users whose posts were cascade deleted due to a flagged post
2021-11-24 09:28:20 -06:00
Bianca Nenciu
3ea8937157
FEATURE: Add email normalization rules setting ()
When this setting is turned on, it will check that normalized emails
are unique. Normalized emails are emails without any dots or plus
aliases.

This setting can be used to block use of aliases of the same email
address.
2021-11-24 11:30:06 +02:00
Alan Guo Xiang Tan
c0f278d358
DEV: Fix flaky specs due to 8226ab1099. ()
The users all shared the same `User#last_seen_at` column so depending on
how the database returned the records, the user that we're interested in
may be excluded from the update query.

Follow-up to 8226ab1099
2021-11-23 15:26:55 +08:00
Alan Guo Xiang Tan
8226ab1099
PERF: Updating first unread PM for user not respecting limits. ()
In b8c8909a9d, we introduced a regression
where users may have had their `UserStat.first_unread_pm_at` set
incorrectly. This commit introduces a migration to reset `UserStat.first_unread_pm_at` back to
`User#created_at`.

Follow-up to b8c8909a9d.
2021-11-23 12:51:54 +08:00
Jarek Radosz
d91d67a442
FEATURE: Allow theme settings to request refresh ()
Similar to site settings, adds support for `refresh` option to theme settings.

```yaml
super_feature_enabled:
  type: bool
  default: false
  refresh: true
```
2021-11-22 13:16:56 +01:00
Martin Brennan
8a3ab1cc43
DEV: Fix spec ()
Broken by 9f8ee8f137
2021-11-22 15:59:10 +11:00
Martin Brennan
9f8ee8f137
FIX: Don't publish notifications to MessageBus for inactive users ()
We are pushing /notification-alert/#{user_id} and /notification/#{user_id}
messages to MessageBus from both PostAlerter and User#publish_notification_state.
This can cause memory issues on large sites with many users. This commit
stems the bleeding by only sending these alert messages if the user
in question has been seen in the last 30 days, which eliminates a large
chunk of users on some sites.
2021-11-22 14:38:49 +11:00
Vinoth Kannan
fc1c76cfcc
FIX: exclude moderator_action post for reply count in user summary. ()
Previously, incorrect reply counts are displayed in the "top categories" section of the user summary page since we included the `moderator_action` and `small_action` post types.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-11-18 13:42:03 +05:30
Bianca Nenciu
cc1b45f58b
FIX: Convert URLs embedded topics to absolute form ()
Sometimes the expanded post contained broken relative URLs because they
were not converted to their absolute form.
2021-11-17 16:39:49 +11:00
Vinoth Kannan
c47a526371
FIX: exclude topics from muted tag in category featured list. ()
Topics from muted tags were visible in the categories page's featured topics section since we didn't filter it before.
2021-11-16 12:10:50 +05:30
Bianca Nenciu
3791fbd919
FEATURE: Add read-only scope to API keys ()
This commit adds a global read-only scope that can be used to create
new API keys.
2021-11-10 17:48:00 +02:00
David Taylor
5ac10e2e79
DEV: Update DiscourseConnect nonce errors to be more descriptive () 2021-11-09 17:39:05 +00:00
Vinoth Kannan
53b7220638
FIX: exclude suppressed category topics in digest even if unmuted. ()
Previously, suppressed category topics are included in the digest emails if the user visited that topic before and the `TopicUser` record is created with any notification level except 'muted'.
2021-11-03 12:47:09 +05:30
Jarek Radosz
ab374fff72
DEV: Move imap_helper to spec/support directory () 2021-10-29 20:46:25 +02:00
Bianca Nenciu
cb0958fcea
FIX: Hide links to muted topics and in categories list ()
* FIX: Hide links to muted topics

* FIX: Hide muted topics in categories list on mobile
2021-10-29 17:52:23 +03:00
Vinoth Kannan
deee715a2c
FIX: update flair group of all members if primary group setting changed. ()
Previously, if we enable the `primary_group` setting on a group then the `flair_group_id` of its' members are not affected.
2021-10-28 22:56:44 +05:30
Vinoth Kannan
c62242c6b3
FIX: should not receive topic invites from ignored users. ()
Previously, ignored users can send notifications by inviting the ignorer to topics or PMs.
2021-10-28 21:03:00 +05:30
Roman Rizzi
df3eb93973
DEV: Sanitize HTML admin inputs ()
* DEV: Sanitize HTML admin inputs

This PR adds on-save HTML sanitization for:

Client site settings
translation overrides
badges descriptions
user fields descriptions

I used Rails's SafeListSanitizer, which [accepts the following HTML tags and attributes](018cf54073/lib/rails/html/sanitizer.rb (L108))

* Make sure that the sanitization logic doesn't corrupt settings with special characters
2021-10-27 11:33:07 -03:00
Andrei Prigorshnev
19d95c64af
DEV: simplify username suggester ()
This PR doesn't change any behavior, but just removes code that wasn't in use. This is a pretty dangerous place to change, since it gets called during user's registration. At the same time the refactoring is very straightforward, it's clear that this code wasn't doing any work (it still needs to be double-checked during review though). Also, the test coverage of UserNameSuggester is good.
2021-10-27 14:41:24 +04:00
Dan Ungureanu
69f0f48dc0
DEV: Fix rubocop issues () 2021-10-27 11:39:28 +03:00
Bianca Nenciu
0c6f9d7c67
FIX: Show right message when permanently deleting topic () 2021-10-26 18:31:15 +03:00
Martin Brennan
2b40049abb
FIX: Remove superfluous topic allowed users on group invite ()
When inviting a group to a topic, there may be members of
the group already in the topic as topic allowed users. These
can be safely removed from the topic, because they are implicitly
allowed in the topic based on their group membership.

Also, this prevents issues with group SMTP emails, which rely
on the topic_allowed_users of the topic to send to and cc's
for emails, and if there are members of the group as topic_allowed_users
then that complicates things and causes odd behaviour.

We also ensure that the OP of the topic is not removed from
the topic_allowed_users when a group they belong to is added,
as it will make it harder to add them back later.
2021-10-22 08:57:51 +10:00
Arpit Jalan
d1fc759ac4
FIX: remove 'crawl_images' site setting () 2021-10-19 17:12:29 +05:30
Andrei Prigorshnev
b9164d22ee
FEATURE: add a hidden setting that enables using email as a source for username suggestions ()
This is a follow-up to https://github.com/discourse/discourse/pull/14541. This adds a hidden setting for restoring the old behavior for those users who rely on it. We'll likely deprecate this setting at some point in the future.
2021-10-19 14:37:39 +04:00
Jean
6275658e3d
FEATURE: Add setting to disable notifications for topic category edits () 2021-10-18 09:04:01 -04:00
Bianca Nenciu
c4843fc1c1
FEATURE: Allow admins to permanently delete posts and topics ()
Sometimes administrators want to permanently delete posts and topics
from the database. To make sure that this is done for a good reasons,
administrators can do this only after one minute has passed since the
post was deleted or immediately if another administrator does it.
2021-10-13 12:53:23 +03:00
Andrei Prigorshnev
88ecb83382
FEATURE: stop using email as source for username and name suggestions for Single Sign On ()
We don't want to be using emails as source for username and name suggestions in cases when it's possible that a user have no chance to intervene and correct a suggested username. It risks exposing email addresses.
2021-10-12 17:25:54 +04:00
Vinoth Kannan
eeecb2460d
FIX: nil the baked version after moving the posts. ()
Previosuly, quotes from original topics are rendered incorrectly since the moved posts are not rebaked.

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2021-10-12 17:31:18 +11:00
Krzysztof Kotlarek
0301b775a8
FIX: topic_tracking_state not erroring when missing user_stat ()
publish_unread should not error when UserStat is missing
2021-10-11 13:20:55 +11:00
Andrei Prigorshnev
e68dc0c598
DEV: add test cases that prove that sso can override username from "bob1" to "bob" ()
The problem was reported a long time ago (https://meta.discourse.org/t/cannot-change-username-with-sso-enabled/31307). It's hard to say if this bag didn't exist or was fixed later, but test cases in this PR prove that this works now.
2021-10-07 22:29:24 +04:00
Alan Guo Xiang Tan
4cade9d7f6
FIX: Topic.similar_to results in invalid query for certain locales. ()
For `zh_CN`, we use the `cppjieba_rb` gem to remove stop words so
calling `Search.prepare_data` may result in an empty string.
2021-10-04 11:40:22 +08:00
Roman Rizzi
4c2d5158c5
FIX: Follow the canonical URL when importing a remote topic. ()
FinalDestination now supports the `follow_canonical` option, which will perform an initial GET request, parse the canonical link if present, and perform a HEAD request to it.

We use this mode during embeds to avoid treating URLs with different query parameters as different topics.
2021-10-01 12:48:21 -03:00
Vinoth Kannan
c8d5c049eb
DEV: skip S3 CDN urls with different path in prefix. ()
Previously, while retrieving each upload urls in a post S3 CDN urls with different path in prefix (external urls technically) are considered as uploaded url. It created issue while checking missing uploads.
2021-10-01 12:25:17 +05:30
Dan Ungureanu
76a7b75d8a
DEV: Reuse can_invite_to_forum? in can_invite_to? ()
This commit resolves refactors can_invite_to? to use
can_invite_to_forum? for checking the site-wide permissions and then
perform topic specific checkups.

Similarly, can_invite_to? is always used with a topic object and this is
now enforced.

There was another problem before when `must_approve_users` site setting
was not checked when inviting users to forum, but was checked when
inviting to a topic.

Another minor security issue was that group owners could invite to
group topics even if they did not have the minimum trust level to do
it.
2021-09-29 17:40:16 +03:00
Alan Guo Xiang Tan
a1745e05ae
FIX: Do not publish post for PM topic tracking if not new for user. () 2021-09-29 13:54:24 +08:00
Martin Brennan
27699648ef
FEATURE: Go to last unread for topic-level bookmark links ()
Instead of going to the OP of the topic for topic-level bookmarks
(which are bookmarks where for_topic is true) when clicking on the
bookmark in the quick access menu or on the user bookmark list,
this commit takes the user to the last unread post in
the topic instead. This should be generally more useful than landing
on the unchanging OP.

To make this work nicely, I needed to add the last_read_post_number to
the BookmarkQuery based on the TopicUser association. It should not add
too much extra weight to the query, because it is limited to the user
that we are fetching bookmarks for.

Also fixed an issue where the bookmark serializer highest_post_number was
not taking into account whether the user was staff, which is when we
should use highest_staff_post_number instead.
2021-09-21 13:49:56 +10:00
Vinoth Kannan
28be284b27
FIX: use active record update_attribute instead of mini sql. ()
* DEV: use active record `save!` instead of mini sql.

The "save" method will trigger the before_save callback "match_primary_group_changes" for User model. Else `flair_group_id` won't be removed from the user.

* check whether the method `match_primary_group_changes` called or not.
2021-09-21 09:29:12 +08:00
Vinoth Kannan
a8b2e7e343
DEV: trash category definition topic instead of destroying. ()
After deleting a category, we should soft-delete the category definition topic instead of hard deleting it. Else it causes issues while doing the user merge action if the source user has an orphan post that belongs to the deleted topic.
2021-09-20 11:20:49 +05:30
Vinoth Kannan
a6de4a5ce9
DEV: use upload id to save in theme setting instead of URL. ()
When we use URL instead it creates the problem while changing the CDN hostname.
2021-09-16 07:58:53 +05:30
Martin Brennan
d1d2298a4c
DEV: Add for_topic column to bookmarks ()
This new column will be used to indicate that a bookmark
is at the topic level. The first post of a topic can be
bookmarked twice after this change -- with for_topic set
to true and with for_topic set to false.

A later PR will use this column for logic to bookmark the
topic, and then topic-level bookmark links will take you
to the last unread post in the topic.

See also 22208836c5
2021-09-15 11:29:22 +10:00
Martin Brennan
22208836c5
DEV: Ignore bookmarks.topic_id column and remove references to it in code ()
We don't need no stinkin' denormalization! This commit ignores
the topic_id column on bookmarks, to be deleted at a later date.
We don't really need this column and it's better to rely on the
post.topic_id as the canonical topic_id for bookmarks, then we
don't need to remember to update both columns if the bookmarked
post moves to another topic.
2021-09-15 10:16:54 +10:00
Mark VanLandingham
68bb7c5a66
DEV: Support translated title in desktop/notifications () 2021-09-14 09:57:38 -05:00
Rafael dos Santos Silva
2e0992c757
DEV: Allow TopicEmbed.import to optionally receive a list of tags ()
This will be used by the rss-polling plugin
2021-09-13 17:01:59 -03:00
Alan Guo Xiang Tan
bc23dcd30b
FIX: Don't publish PM archive events to acting user. ()
When a user archives a personal message, they are redirected back to the
inbox and will refresh the list of the topics for the given filter.
Publishing an event to the user results in an incorrect incoming message
because the list of topics has already been refreshed.

This does mean that if a user has two tabs opened, the non-active tab
will not receive the incoming message but at this point we do not think
the technical trade-offs are worth it to support this feature. We
basically have to somehow exclude a client from an incoming message
which is not easy to do.

Follow-up to fc1fd1b416
2021-09-10 09:20:50 +08:00
Alan Guo Xiang Tan
412587f70a
FEATURE: Publish read topic tracking events for private messages. ()
Follow-up to fc1fd1b416
2021-09-09 09:16:53 +08:00
Alan Guo Xiang Tan
1e05175364
PERF: Avoid running query unnecessarily when updating bookmark. ()
* Avoid loading an entire ActiveRecord object when saving and updating.
* Avoid running a DB query when `post_id` or `user_id` is not changed.
2021-09-09 10:50:26 +10:00
Alan Guo Xiang Tan
5de64b3630
FIX: Correct highest post number for read topic tracking state. () 2021-09-08 11:55:12 +08:00
Krzysztof Kotlarek
e3793e6d7c
FIX: better filter for groups search ()
Follow up of https://github.com/discourse/discourse/pull/14216

Allow plugins to register custom filter with block
2021-09-08 09:38:45 +10:00
Krzysztof Kotlarek
f859fd6bde
FEATURE: allow plugins to extend Groups ()
* add_permitted_group_param API for plugins
* add groups-interaction-custom-options outlet
* custom search can use custom group scope
2021-09-06 10:18:51 +10:00
Rafael dos Santos Silva
ef0471d7ae
DEV: Allow passing cook_method to TopicEmbed.import to override default ()
DEV: Allow passing cook_method to TopicEmbed.import to override default

This will be used in the rss-polling plugin when we want to have
oneboxes on feed content, like youtube for example.
2021-09-01 15:46:39 -03:00
Alan Guo Xiang Tan
f66007ec83
FEATURE: Display unread and new counts for messages. ()
There are certain design decisions that were made in this commit.

Private messages implements its own version of topic tracking state because there are significant differences between regular and private_message topics. Regular topics have to track categories and tags while private messages do not. It is much easier to design the new topic tracking state if we maintain two different classes, instead of trying to mash this two worlds together.

One MessageBus channel per user and one MessageBus channel per group. This allows each user and each group to have their own channel backlog instead of having one global channel which requires the client to filter away unrelated messages.
2021-08-25 11:17:56 +08:00
Joffrey JAFFEUX
2bbc97fda5
DEV: UserOption.user_tzinfo ()
Provides a safe way to retrieve the timezone of a user.

This is not used in core yet, but used in multiple plugins.
2021-08-19 21:56:14 +02:00
Arpit Jalan
7db3888f17
FIX: log proper error message when SSO nonce verification fails () 2021-08-18 18:44:12 +05:30
Rafael dos Santos Silva
560c13211a
DEV: Allow passing a category parameter when importing a topic ()
This will be used in the rss pooling plugin to address the feature
request at https://meta.discourse.org/t/-/200644?u=falco
2021-08-17 18:17:07 -03:00
Roman Rizzi
29bb79de37
FIX: Query the items in the queue to calculate a user's flagged post count. ()
When a staff member clicks on a user's number of flagged posts, we redirect them to the review queue, so it makes sense to count the number of items there to calculate the count.

We used to look at post action items to calculate this number, which doesn't match the number of items in the queue if old flags exist.
2021-08-12 14:20:46 -03:00
Alan Guo Xiang Tan
aed65ec16d SECURITY: User's read state for topic is leaked to unauthorized clients.
A user's read state for a topic such as the last read post number and the notification level is exposed.
2021-08-12 12:16:45 +08:00
Alan Guo Xiang Tan
9a60c83535 FIX: TopicTrackingState.report not including unread for staff posts.
Follow-up to e15c86e8c5
2021-08-12 11:10:49 +08:00
jbrw
20a6bad87e
FIX: Category group moderators can read flagged post meta_topics ()
When a post is flagged with the reason of 'Something Else' a brief message can be added by the user which subsequently creates a `meta_topic` private message. The group `moderators` is automatically added to this topic.

If category group moderation is enabled, and the post belongs to a category with a reviewable group, that group should also be added to the meta_topic.

Note: This extends the `notify_moderators` logic, and will add the reviewable group to the meta_topic, regardless of the settings of that group.
2021-08-11 18:11:22 -04:00
Arpit Jalan
3006de39d1
REVERT "FIX: do not show private group flair on user avatars" ()
This reverts commit fe3e18f981 and 0d8fd9ace6
2021-08-10 17:25:11 +05:30
Arpit Jalan
0d8fd9ace6
FIX: do not show flair bg color if flair is not visible ()
follow up to fe3e18f981
2021-08-06 20:53:23 +05:30
Bianca Nenciu
d9843d757a
FIX: Update draft count when sequence is increased ()
* FIX: Update draft count when sequence is increased

Sometimes users ended up having a draft count higher than the actual
number of drafts.

* FIX: Do not update draft count twice

The call to DraftSequence.next! above already does it.
2021-08-04 13:30:37 +03:00
Vinoth Kannan
1da0aa838f
FIX: use update_attribute method to trigger callbacks. ()
Group flair is not removed while removing a user from the group since the `before_save` callback methods are not triggered while using the `update_columns` method.
2021-08-04 11:54:46 +05:30
Jean
e7b8e75583
FEATURE: Add post edits count to user activity () 2021-08-02 10:15:53 -04:00
Alan Guo Xiang Tan
016efeadf6
FEATURE: New and Unread messages for user personal messages. ()
* FEATURE: New and Unread messages for user personal messages.

Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
2021-08-02 12:41:41 +08:00
Arpit Jalan
fe3e18f981
FIX: do not show private group flair on user avatars ()
Meta ref: https://meta.discourse.org/t/visible-flair-for-invisible-groups-is-that-on-purpose/167674
2021-08-02 06:21:00 +05:30
Bianca Nenciu
300db3d3fa
FIX: Update draft count after creating a post ()
When a post is created, the draft sequence is increased and then older
drafts are automatically executing a raw SQL query. This skipped the
Draft model callbacks and did not update user's draft count.

I fixed another problem related to a raw SQL query from Draft.cleanup!
method.
2021-07-29 17:06:11 +03:00
Andrew Schleifer
461cb96532 use more appropriate labels for chinese UI option 2021-07-27 22:47:59 +08:00
Bianca Nenciu
760c9a5698
FEATURE: Show draft count in user menu and activity ()
This commit adds the number of drafts a user has next to the "Draft"
label in the user preferences menu and activity tab. The count is
updated via MessageBus when a draft is created or destroyed.
2021-07-27 14:05:33 +03:00
Krzysztof Kotlarek
88aa0136e3
FIX: do not raise exception when svg path is nil ()
Bug was introduced here: f7ab852e12

If path is nil, it should not raise an exception and continue logging the error.
2021-07-26 12:35:27 +10:00
Joffrey JAFFEUX
5eb6e9281a
FIX: manually adds frowning_face_with_open_mouth for apple () 2021-07-21 23:27:20 +02:00
Penar Musaraj
2ce2c83bc9
FIX: Show user filter hints when typing @ in search ()
Will show the last 6 seen users as filtering suggestions when typing @ in quick search. (Previously the user suggestion required a character after the @.)

This also adds a default limit of 6 to the user search query, previously the backend was returning 20 results but a maximum of 6 results was being shown anyway.
2021-07-21 09:14:53 -04:00
Mark VanLandingham
80950d7b28
DEV: Add chat_mention notification type () 2021-07-19 14:52:12 -05:00
Penar Musaraj
361c8be547
PERF: Add scheduled job to delete old stylesheet cache rows () 2021-07-16 10:58:01 -04:00
Penar Musaraj
f7ab852e12
FIX: Issues with custom icons in themes ()
Fixes two issues:
- ignores invalid XML in custom icon sprite SVG file (and outputs an error if sprite was uploaded via admin UI)
- clears SVG sprite cache when deleting an `icons-sprite` upload in a theme
2021-07-14 15:18:29 -04:00
David Taylor
800c6e1a68 PERF: Improve topic_user.liked update performance when moving posts
Previously we would re-calculate topic_user.liked for all users who have ever viewed the source or destination topic. This can be very expensive on large sites. Instead, we can use the array of moved post ids to find which users are actually affected by the move, and restrict the update query to only check/update their records.

On an example site this reduced the `update_post_action_cache` time from ~27s to 300ms
2021-07-13 12:30:38 +01:00
Martin Brennan
9f275c12ab
FIX: Handle storage providers not implementing ACLs ()
When secure media is enabled or when upload secure status
is updated, we also try and update the upload ACL. However
if the object storage provider does not implement this we
get an Aws::S3::Errors::NotImplemented error. This PR handles
this error so the update_secure_status method does not error
out and still returns whether the secure status changed.
2021-07-09 11:31:44 +10:00
Bianca Nenciu
87c1e98571
FEATURE: Let users select flair ()
User flair was given by user's primary group. This PR separates the
two, adds a new field to the user model for flair group ID and users
can select their flair from user preferences now.
2021-07-08 10:46:21 +03:00
Mark VanLandingham
14a13dc192
FIX: Check type of existing reviewables when new reviewable is created () 2021-07-07 11:45:00 -05:00
dependabot[bot]
dda41cf253
Build(deps): Bump rubocop from 1.18.2 to 1.18.3 ()
* Build(deps): Bump rubocop from 1.18.2 to 1.18.3

Bumps [rubocop](https://github.com/rubocop/rubocop) from 1.18.2 to 1.18.3.
- [Release notes](https://github.com/rubocop/rubocop/releases)
- [Changelog](https://github.com/rubocop/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop/compare/v1.18.2...v1.18.3)

---
updated-dependencies:
- dependency-name: rubocop
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix offences

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2021-07-07 01:51:43 +02:00
Dan Ungureanu
34387c5a38
FEATURE: Warn if invited user cannot see topic ()
Users can invite people to topics from secured category, but they will
not be redirected to the topic after signing up unless they have the
permissions to view the topic. This commit shows a warning when invite
is saved if the topic is in a secured category and none of the invite
groups are allowed to see it.
2021-07-06 12:49:26 +03:00
Alan Guo Xiang Tan
37b8ce79c9
FEATURE: Add last visit indication to topic view page. ()
This PR also removes grey old unread bubble from the topic badges by
dropping `TopicUser#highest_seen_post_number`.
2021-07-05 14:17:31 +08:00
Dan Ungureanu
6ea4bbd2ec
DEV: Prefer .pluck_first over .pluck.first () 2021-07-02 10:03:54 +08:00
Martin Brennan
59582102d8
FIX: Rename ninja edit terminology to grace period ()
We renamed the site setting for this long ago, but there
were a few places left in the code base where "ninja edit"
needed to be turned into "grace period". Doing this here
to avoid combatative language.
2021-07-01 11:27:11 +10:00
jbrw
4728962f7d
FIX: Don’t translate TrustLevel name when generating links ()
We want to put the name of the trust level in to generated URLs, not the human-readable form.

i.e.:

`/admin/users/list/newuser`

rather than:

`/admin/users/list/new user`
2021-06-30 14:19:15 -04:00
Andrei Prigorshnev
11baf872ed
FIX: do not close the merged topic if the first post wasn't merged ()
When a topic is fully merged into another topic we close it and schedule it for deleting. But last time I changed this place I added a bug – when merging all posts in topic except the first one the topic was closing too.

If the OP is not merged into another topic, the original topic shouldn't be closed and marked for deletion. This PR fixes this.
2021-06-30 18:28:18 +04:00
Martin Brennan
03338f9086
FIX: Remove legacy topic timer code ()
The new topic timer backend code introduced six months ago
in 0034cbd is now used instead of this legacy code. It can be safely removed
now.
2021-06-29 09:16:25 +10:00
Alan Guo Xiang Tan
ff1c53dd6f FIX: Missing category edit icon.
Follow-up to 0e4b8c5318
2021-06-28 10:54:23 +08:00
Martin Brennan
87684f7c5e
FEATURE: Use group SMTP job and mailer instead of UserNotifications change ()
This PR backtracks a fair bit on this one https://github.com/discourse/discourse/pull/13220/files.

Instead of sending the group SMTP email for each user via `UserNotifications`, we are changing to send only one email with the existing `Jobs::GroupSmtpEmail` job and `GroupSmtpMailer`. We are changing this job and mailer along with `PostAlerter` to make the first topic allowed user the `to_address` for the email and any other `topic_allowed_users` to be the CC address on the email. This is to cut down on emails sent via SMTP, which is subject to daily limits from providers such as Gmail. We log these details in the `EmailLog` table now.

In addition to this, we have changed `PostAlerter` to no longer rely on incoming email email addresses for sending the `GroupSmtpEmail` job. This was unreliable as a user's email could have changed in the meantime. Also it was a little overcomplicated to use the incoming email records -- it is far simpler to reason about to just use topic allowed users.

This also adds a fix to include cc_addresses in the EmailLog.addressed_to_user scope.
2021-06-28 08:55:13 +10:00
Roman Rizzi
fa4e5e8dad
FEATURE: Render emojis on GitHub labels when oneboxing an issue. () 2021-06-25 14:48:36 -03:00
Alan Guo Xiang Tan
3b32b6bc13 DEV: Clean up state leak in Site tests. 2021-06-25 09:17:49 +08:00
Joffrey JAFFEUX
2654a6685c
DEV: adds support for bannered until ()
ATM it only implements server side of it, as my need is for automation purposes. However it should probably be added in the UI too as it's unexpected to have pinned_until and no bannered_until.
2021-06-24 11:35:36 +02:00
Alan Guo Xiang Tan
0e4b8c5318 PERF: Cache categories in Site model take 3.
Previous attempt resulted in custom fields going missing in the
serialized output.

This reverts commit 83a6ad32ff.
2021-06-24 13:30:51 +08:00
Jarek Radosz
046a875222
DEV: Improve script/downsize_uploads.rb ()
* Only shrink images that are used in Posts and no other models
* Don't save the upload if the size is the same
2021-06-24 00:09:40 +02:00
Penar Musaraj
c8f4f7c235
FIX: Ignore missing uploads in theme settings ()
In some rare cases, this could prevent the site from bootstrapping,
because theme settings are invoked early in the application.
2021-06-22 22:34:22 -04:00
Martin Brennan
5222247746
FEATURE: Add more columns to outbound EmailLog ()
This adds the following columns to EmailLog:

* cc_addresses
* cc_user_ids
* topic_id
* raw

This is to bring the EmailLog table closer in parity to
IncomingEmail so it can be better utilized for Group SMTP
and IMAP mailing.

The raw column contains the full content of the outbound email,
but _only_ if the new hidden site setting
enable_raw_outbound_email_logging is enabled. Most sites do not
need it, and it's mostly required for IMAP and SMTP sending.

In the next pull request, there will be a migration to backfill
topic_id on the EmailLog table, at which point we can remove the
topic fallback method on EmailLog.
2021-06-22 08:32:01 +10:00
Kane York
83a6ad32ff Revert "PERF: Cache categories in Site model take 2."
This reverts commit 06fa1efd3d.

Breakage in solved plugin
2021-06-21 12:25:04 -07:00
Bianca Nenciu
d1b2e9db3b
PERF: Update post uploads secure status in a job ()
When secure uploads are enabled, editing a post with many uploads can
cause a timeout because the store has to be contacted for each upload.
2021-06-21 19:15:24 +03:00
Alan Guo Xiang Tan
8e3691d537 PERF: Eager load Theme associations in Stylesheet Manager.
Before this change, calling `StyleSheet::Manager.stylesheet_details`
for the first time resulted in multiple queries to the database. This is
because the code was modelled in a way where each `Theme` was loaded
from the database one at a time.

This PR restructures the code such that it allows us to load all the
theme records in a single query. It also allows us to eager load the
required associations upfront. In order to achieve this, I removed the
support of loading multiple themes per request. It was initially added
to support user selectable theme components but the feature was never
completed and abandoned because it wasn't a feature that we thought was
worth building.
2021-06-21 11:06:58 +08:00
Alan Guo Xiang Tan
06fa1efd3d PERF: Cache categories in Site model take 2.
Follow-up to aa4f0aee67.

Fixed the security problem in the previous attempt.
2021-06-21 09:47:05 +08:00
Martin Brennan
6fe78cd542
FIX: Make sure reset-new for tracked is not limited by per_page count ()
When dismissing new topics for the Tracked filter, the dismiss was
limited to 30 topics which is the default per page count for TopicQuery.
This happened even if you specified which topic IDs you were
selectively dismissing. This PR fixes that bug, and also moves
the per_page_count into a DEFAULT_PER_PAGE_COUNT for the TopicQuery
so it can be stubbed in tests.

Also moves the unused stub_const method into the spec helpers
for cases like this; it is much better to handle this in one place
with an ensure. In a follow up PR I will clean up other specs that
do the same thing and make them use stub_const.
2021-06-17 08:20:09 +10:00
Martin Brennan
c659e3e95b
FIX: Make sure topic_user.bookmarked is synced in more places ()
When we call Bookmark.cleanup! we want to make sure that
topic_user.bookmarked is updated for topics linked to the
bookmarks that were deleted. Also when PostDestroyer calls
destroy and recover. We have a job for this already --
SyncTopicUserBookmarked -- so we just utilize that.
2021-06-16 08:30:40 +10:00
Roman Rizzi
4dc8c3c409
FEATURE: Blocking is optional when deleting a user from the review queue. ()
Subclasses must call #delete_user_actions inside build_actions to support user deletion. The method adds a delete user bundle, which has a delete and a delete + block option. Every subclass is responsible for implementing these actions.
2021-06-15 12:35:45 -03:00
Jarek Radosz
e36377d9ab
DEV: Don't user before(:all)/after(:all) ()
Leaking state and non-obvious order (before :all runs *before* RailsHelper.test_setup) are not worth it.
A replacement PR for . Fixes some flaky specs, e.g.
```
bin/rspec './spec/components/freedom_patches/translate_accelerator_spec.rb[1:3]' './spec/jobs/clean_up_user_export_topics_spec.rb[1:1]' --tag ~type:multisite --seed 35994
```

Also included:
* DEV: No need for locale reset (we do it anyway in rails_helper in `test_setup`)
2021-06-15 17:25:06 +02:00
Jarek Radosz
0815b4cc2e
DEV: Fix flaky test due to locale cache poisoning
A follow-up to 
2021-06-08 13:13:38 +10:00
Jarek Radosz
0fe9a36e89 DEV: Fix another flaky spec
The error was:

```
  1) ExtraLocalesController.client_overrides_exist? returns true if there are client-side translation overrides
     Failure/Error: expect(ExtraLocalesController.client_overrides_exist?).to eq(false)

       expected: false
            got: true

       (compared using ==)

       Diff:
       @@ -1 +1 @@
       -false
       +true

     # ./spec/requests/extra_locales_controller_spec.rb:162:in `block (3 levels) in <main>'
     # ./spec/rails_helper.rb:279:in `block (2 levels) in <top (required)>'
     # .gem/ruby/2.7.3/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```

Minimal repro:

```
bin/rspec './spec/models/trust_level_and_staff_setting_spec.rb[1:1:1]' './spec/requests/extra_locales_controller_spec.rb[1:3:2]' --tag ~type:multisite --seed 33616
```
2021-06-08 09:54:37 +08:00
Dan Ungureanu
36e0e6a322
FIX: Allow invites if must_approve_users is true () 2021-06-07 18:57:08 +03:00
Alan Guo Xiang Tan
3c1f4d5771 FIX: Clear post action types application serializer fragment cache.
The bug was introduced in dc10bdee3d
2021-06-04 09:14:49 +08:00