Commit Graph

878 Commits

Author SHA1 Message Date
Ted Johansson
3137e60653
DEV: Database backed admin notices (#26192)
This PR introduces a basic AdminNotice model to store these notices. Admin notices are categorized by their source/type (currently only notices from problem check.) They also have a priority.
2024-05-23 09:29:08 +08:00
Krzysztof Kotlarek
40d65dddf8
Revert "DEV: move post flags into database (#26951)" (#27102)
This reverts commit 7aff9806eb.
2024-05-21 16:21:07 +10:00
Krzysztof Kotlarek
7aff9806eb
DEV: move post flags into database (#26951)
This is preparation for a feature that will allow admins to define their custom flags. Current behaviour should stay untouched.
2024-05-21 13:15:32 +10:00
Régis Hanol
0e9451e93f
FIX: crawler view with unicode usernames (#27051)
When "unicode_usernames" is enabled, calling the "user_path" helper with a username containing some non ASCII character will break due to the route constraint we have on username.

This fixes the issue by always encoding the username before passing it to the "user_path" helper.

Internal ref - t/127547
2024-05-16 17:11:24 +02:00
Alan Guo Xiang Tan
7079698cdf
FIX: Use MaxMind supplied permalinks to download MaxMind databases (#26847)
This commit switches `DiscourseIpInfo.mmdb_download` to use the
permalinks supplied by MaxMind to download the MaxMind databases as
specified in
https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
which states:

```
To directly download databases, follow these steps:

1. In the "Download Links" column, click "Get Permalink(s)" for the desired database.
2. Copy the permalink(s) provided in the modal window.
3. Provide your account ID and your license key using Basic Authentication to authenticate.
```

Previously we are downloading from `https://download.maxmind.com/app/geoip_download` but this is not
documented anyway on MaxMind's docs so this URL can in theory break
in the future without warning. Therefore, we are taking a proactive
approach to download the databases from MaxMind the recommended way
instead of relying on a hidden URL. This old way of downloading the
databases with only a license key will be deprecated in 3.3 and be
removed in 3.4.
2024-05-09 15:11:56 +08:00
Alan Guo Xiang Tan
b6f6852bba
FIX: Make getCategoryIdByName theme migration helper case insensitive (#26878) 2024-05-06 12:42:58 +08:00
Alan Guo Xiang Tan
a6624af66e
DEV: Add isValidUrl helper function to theme migrations (#26817)
This commit adds a `isValidUrl` helper function to the context in
which theme migrations are ran in. This helper function is to make it
easier for theme developers to check if a string is a valid URL or path
when writing theme migrations. This can be helpful in cases when
migrating a string based setting to `type: objects` which contain `type:
string` properties with URL validations enabled.

This commit also introduces the `UrlHelper.is_valid_url?` method
which actually checks that the URL string is of the valid format instead of
only checking if the URL string is parseable which is what `UrlHelper.relaxed_parse` does
and is not sufficient for our needs.
2024-04-30 16:45:07 +08:00
Martin Brennan
edec941a87
FIX: Better tracking of topic visibility changes (#26709)
This commit introduces a few changes as a result of
customer issues with finding why a topic was relisted.
In one case, if a user edited the OP of a topic that was
unlisted and hidden because of too many flags, the topic
would get relisted by directly changing topic.visible,
instead of going via TopicStatusUpdater.

To improve tracking we:

* Introduce a visibility_reason_id to topic which functions
  in a similar way to hidden_reason_id on post, this column is
  set from the various places we change topic visibility
* Fix Post#unhide! which was directly modifying topic.visible,
  instead we use TopicStatusUpdater which sets visibility_reason_id
  and also makes a small action post
* Show the reason topic visibility changed when hovering the
  unlisted icon in topic status on topic titles
2024-04-29 10:34:46 +10:00
Osama Sayegh
2215fa0c8e
FIX: Pass values of objects typed settings to theme migrations (#26751)
This commit fixes a bug in theme settings migrations where values of `objects` typed theme settings aren't passed to migrations even when there are overriding values for those settings. What causes this bug is that, when creating the hash that contains all the overridden settings and will be passed to migrations, the values of `objects` typed settings are incorrectly retrieved from the `value` column (which is always nil for `objects` type) instead of `json_value`. `objects` settings are different from all other types in that they store their values in the `json_value` column and they need to be special-cased when retrieving their values.
2024-04-25 16:39:22 +03:00
Alan Guo Xiang Tan
b3f1192312
DEV: Add getCategoryIdByName helper function to theme migrations (#26601)
This commit adds a `getCategoryIdByName` helper function to the context in
which theme migrations are ran in. The purpose of this commit is to
allow themes which may have stored category names in theme settings to
migrate to objects typed theme settings which supports properties of
categories typed which stores the category ids in the value of the
property.
2024-04-22 09:01:53 +08:00
Natalie Tay
1fea2bf1c5
FEATURE: Merge user associated accounts, favouring the target user upon conflict (#26645) 2024-04-16 17:37:33 +08:00
Blake Erickson
72ac675e4e
FEATURE: Consolidate link notifications (#26567)
Just like we have for consolidating likes this adds similar
functionality for consolidating links.
2024-04-09 11:53:37 -06:00
Penar Musaraj
c4e8221d7e
UX: Improvements to user tips (#26480)
- Add a "Skip tips" button to first notification tip
- Add a "Skip tips" button to the admin guide tip
- Fixes the timeline tip showing when no timeline was present
- Fixes post menu tip showing when no "..." button is present
- Adds system tests
- Marks each tip as seen as soon as it is displayed so that refreshing,
clicking outside, etc. won't show it again
- Change just above means we no longer need a MessageBus track

Co-authored-by: Bianca Nenciu <nbianca@users.noreply.github.com>
2024-04-03 11:43:56 -04:00
Penar Musaraj
1eb70973a2
DEV: allow themes to render their own custom homepage (#26291)
This PR adds a theme modifier and route so that custom themes can opt to show their own homepage. See PR description for example usage.
2024-04-02 11:05:08 -04:00
Jarek Radosz
4c860995e0
DEV: Remove unnecessary rails_helper requiring (#26364) 2024-03-26 11:32:01 +01:00
Ted Johansson
5ee23fc394
DEV: Make all admins TL4 in tests (#25435)
Make admins TL4 by default in tests, foregoing the need to call refresh_auto_groups on them.
2024-03-26 11:41:12 +08:00
Ted Johansson
4ca41e0af2
DEV: Promote block problem checks to ProblemCheck (#26193)
In #26122 we promoted all problem checks defined as class methods on AdminDashboardData to their own first-class ProblemCheck instances.

This PR continues that by promoting problem checks that are implemented as blocks as well. This includes updating a couple plugins that have problem checks.
2024-03-20 08:52:25 +08:00
Ted Johansson
e2ee70c4e2
FIX: Amend broken Mailgun API key check (#26206) 2024-03-18 09:10:30 +10:00
Daniel Waterworth
8cade1e825
SECURITY: Prevent large staff actions causing DoS
This commit operates at three levels of abstraction:

 1. We want to prevent user history rows from being unbounded in size.
    This commit adds rails validations to limit the sizes of columns on
    user_histories,

 2. However, we don't want to prevent certain actions from being
    completed if these columns are too long. In those cases, we truncate
    the values that are given and store the truncated versions,

 3. For endpoints that perform staff actions, we can further control
    what is permitted by explicitly validating the params that are given
    before attempting the action,
2024-03-15 14:24:04 +08:00
Ted Johansson
ea5c3a3bdc
DEV: Move non scheduled problem checks to classes (#26122)
In AdminDashboardData we have a bunch of problem checks implemented as methods on that class. This PR absolves it of the responsibility by promoting each of those checks to a first class ProblemCheck. This way each of them can have their own priority and arbitrary functionality can be isolated in its own class.

Think "extract class" refactoring over and over. Since they were all moved we can also get rid of the @@problem_syms class variable which was basically the old version of the registry now replaced by ProblemCheck.realtime.

In addition AdminDashboardData::Problem value object has been entirely replaced with the new ProblemCheck::Problem (with compatible API).

Lastly, I added some RSpec matchers to simplify testing of problem checks and provide helpful error messages when assertions fail.
2024-03-14 10:55:01 +08:00
Natalie Tay
0b41b236d7
FIX: Avoid sending user emails if @ mentioning a staged user in a topic (#26102)
Avoid sending user emails if @ mentioning a staged user

Some cases, unknowingly mentioning a staged user would invite
them into topics, sending them an email about it.
2024-03-13 11:05:34 +08:00
Ted Johansson
2211ffa851
DEV: Move problem checks to app directory (#26120)
There are a couple of reasons for this.

The first one is practical, and related to eager loading. Since /lib is not eager loaded, when the application boots, ProblemCheck["identifier"] will be nil because the child classes aren't loaded.

The second one is more conceptual. There turns out to be a lot of inter-dependencies between the part of the problem check system that live in /app and the parts that live in /lib, which probably suggests it should all go in /app.
2024-03-11 13:36:22 +08:00
Martin Brennan
6bcbe56116
DEV: Use freeze_time_safe in more places (#25949)
Followup to 120a2f70a9,
uses new method to avoid time-based spec flakiness
2024-03-01 10:07:35 +10:00
Mark VanLandingham
6c2c690479
DEV: Add push notification filtering to MessageBus alerts (#25965) 2024-02-29 12:49:46 -06:00
Martin Brennan
df4197c8b8
FIX: Show deleted bookmark reminders in user bookmarks menu (#25905)
When we send a bookmark reminder, there is an option to delete
the underlying bookmark. The Notification record stays around.
However, if you want to filter your notifications user menu
to only bookmark-based notifications, we were not showing unread
bookmark notifications for deleted bookmarks.

This commit fixes the issue _going forward_ by adding the
bookmarkable_id and bookmarkable_type to the Notification data,
so we can look up the underlying Post/Topic/Chat::Message
for a deleted bookmark and check user access in this way. Then,
it doesn't matter if the bookmark was deleted.
2024-02-29 09:03:49 +10:00
Mark VanLandingham
b426f85a81
DEV: Add modifiers for plugins to customize push notification translation arguments (#25889) 2024-02-27 14:03:55 -06:00
Martin Brennan
a57280cb17
DEV: Change min_trust_level_to_allow_profile_background to trust level setting (#25721)
New setting name is profile_background_allowed_groups

c.f. https://meta.discourse.org/t/changes-coming-to-settings-for-giving-access-to-features-from-trust-levels-to-groups/283408
2024-02-19 10:47:47 +10:00
Bianca Nenciu
1403217ca4
FEATURE: Async load of category and chat hashtags (#25526)
This commit includes several changes to make hashtags work when "lazy
load categories" is enabled. The previous hashtag implementation use the
category colors CSS variables, but these are not defined when the site
setting is enabled because categories are no longer preloaded.

This commit implements two fundamental changes:

1. load colors together with the other hashtag information

2. load cooked hashtag data asynchronously

The first change is implemented by adding "colors" to the HashtagItem
model. It is a list because two colors are returned for subcategories:
the color of the parent category and subcategory.

The second change is implemented on the server-side in a new route
/hashtags/by-ids and on the client side by loading previously unseen
hashtags, generating the CSS on the fly and injecting it into the page.

There have been minimal changes outside of these two fundamental ones,
but a refactoring will be coming soon to reuse as much of the code
and maybe favor use of `style` rather than injecting CSS into the page,
which can lead to page rerenders and indefinite grow of the styles.
2024-02-12 12:07:14 +02:00
Ted Johansson
f0a46f8b6f
DEV: Automatically update groups for test users with explicit TL (#25415)
For performance reasons we don't automatically add fabricated users to trust level auto-groups. However, when explicitly passing a trust level to the fabricator, in 99% of cases it means that trust level is relevant for the test, and we need the groups.

This change makes it so that when a trust level is explicitly passed to the fabricator, the auto-groups are refreshed. There's no longer a need to also pass refresh_auto_groups: true, which means clearer tests, fewer mistakes, and less confusion.
2024-01-29 17:52:02 +08:00
Ted Johansson
7e5d2a95ee
DEV: Convert min_trust_level_to_tag_topics to groups (#25273)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_tag_topics site setting to tag_topic_allowed_groups.
2024-01-26 13:25:03 +08:00
Ted Johansson
57ea56ee05
DEV: Remove full group refreshes from tests (#25414)
We have all these calls to Group.refresh_automatic_groups! littered throughout the tests. Including tests that are seemingly unrelated to groups. This is because automatic group memberships aren't fabricated when making a vanilla user. There are two places where you'd want to use this:

You have fabricated a user that needs a certain trust level (which is now based on group membership.)
You need the system user to have a certain trust level.
In the first case, we can pass refresh_auto_groups: true to the fabricator instead. This is a more lightweight operation that only considers a single user, instead of all users in all groups.

The second case is no longer a thing after #25400.
2024-01-25 14:28:26 +08:00
Mark VanLandingham
7bad0b1d8a
DEV: Trigger DiscourseEvent on push notification send (#25375) 2024-01-22 15:30:35 -06:00
Ted Johansson
fb087b7ff6
DEV: Convert min_trust_to_post_links to groups (#25298)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_to_post_links  site setting to post_links_allowed_groups.

This isn't used by any of our plugins or themes, so very little fallout.
2024-01-18 14:08:40 +08:00
Penar Musaraj
f2cf5434f3
Revert "DEV: Convert min_trust_level_to_tag_topics to groups (#25258)" (#25262)
This reverts commit c7e3d27624 due to
test failures. This is temporary.
2024-01-15 11:33:47 -05:00
Ted Johansson
c7e3d27624
DEV: Convert min_trust_level_to_tag_topics to groups (#25258)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_tag_topics site setting to tag_topic_allowed_groups.
2024-01-15 20:59:08 +08:00
Roman Rizzi
47597219b1
FEATURE: Prefer topic_embed's cached content when summarizing (#25190) 2024-01-09 14:00:01 -03:00
Ted Johansson
53d40672a7
DEV: Convert min_trust_level_to_allow_user_card_background to groups (#24891)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_allow_user_card_background site setting to user_card_background_allowed_groups.

Nothing of note here. This is used in exactly one place, and there's no fallout.
2023-12-14 10:57:58 +08:00
Ted Johansson
294febf3c4
DEV: Convert min_trust_to_flag_posts setting to groups (#24864)
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_to_flag_posts site setting to flag_post_allowed_groups.

Note: In the original setting, "posts" is plural. I have changed this to "post" singular in the new setting to match others.
2023-12-13 17:18:42 +08:00
Krzysztof Kotlarek
702d0620d7
DEV: Convert min_trust_to_create_topic to groups (#24740)
This change converts the min_trust_to_create_topic site setting to
create_topic_allowed_groups.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change
- After a couple of months, we will remove the min_trust_to_create_topicsetting entirely.

Internal ref: /t/117248
2023-12-13 14:50:13 +11:00
Jarek Radosz
694b5f108b
DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
Jarek Radosz
6a66dc1cfb
DEV: Fix Lint/BooleanSymbol (#24747) 2023-12-06 13:19:09 +01:00
Jarek Radosz
7196613e2e
DEV: Fix various spec linting issues (#24672)
Duplicated specs, incorrect descriptions, incorrect assertions, incorrect filenames, old todo
2023-12-04 13:45:19 +01:00
Sam
b09422428d
DEV: update syntax tree to latest (#24623)
update format to latest syntax tree
2023-11-29 16:38:07 +11:00
Daniel Waterworth
6e161d3e75
DEV: Allow fab! without block (#24314)
The most common thing that we do with fab! is:

    fab!(:thing) { Fabricate(:thing) }

This commit adds a shorthand for this which is just simply:

    fab!(:thing)

i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-09 16:47:59 -06:00
Martin Brennan
3c5fb871c0 SECURITY: Filter unread bookmark reminders the user cannot see
There is an edge case where the following occurs:

1. The user sets a bookmark reminder on a post/topic
2. The post/topic is changed to a PM before or after the reminder
   fires, and the notification remains unread by the user
3. The user opens their bookmark reminder notification list
   and they can still see the notification even though they cannot
   access the topic anymore

There is a very low chance for information leaking here, since
the only thing that could be exposed is the topic title if it
changes to something sensitive.

This commit filters the bookmark unread notifications by using
the bookmarkable can_see? methods and also prevents sending
reminder notifications for bookmarks the user can no longer see.
2023-11-09 13:39:16 +11:00
Mark VanLandingham
e3f8e9c0fb
DEV: Email notification filter plugin API (#24271) 2023-11-08 10:29:00 -06:00
Mark VanLandingham
047cae4b3f
FEATURE: Improve push notification message for watching_category_or_tag notifications (#24228) 2023-11-06 10:13:23 -06:00
Osama Sayegh
3cadd6769e
FEATURE: Theme settings migrations (#24071)
This commit introduces a new feature that allows theme developers to manage the transformation of theme settings over time. Similar to Rails migrations, the theme settings migration system enables developers to write and execute migrations for theme settings, ensuring a smooth transition when changes are required in the format or structure of setting values.

Example use cases for the theme settings migration system:

1. Renaming a theme setting.

2. Changing the data type of a theme setting (e.g., transforming a string setting containing comma-separated values into a proper list setting).

3. Altering the format of data stored in a theme setting.

All of these use cases and more are now possible while preserving theme setting values for sites that have already modified their theme settings.

Usage:

1. Create a top-level directory called `migrations` in your theme/component, and then within the `migrations` directory create another directory called `settings`.

2. Inside the `migrations/settings` directory, create a JavaScript file using the format `XXXX-some-name.js`, where `XXXX` is a unique 4-digit number, and `some-name` is a descriptor of your choice that describes the migration.

3. Within the JavaScript file, define and export (as the default) a function called `migrate`. This function will receive a `Map` object and must also return a `Map` object (it's acceptable to return the same `Map` object that the function received).

4. The `Map` object received by the `migrate` function will include settings that have been overridden or changed by site administrators. Settings that have never been changed from the default will not be included.

5. The keys and values contained in the `Map` object that the `migrate` function returns will replace all the currently changed settings of the theme.

6. Migrations are executed in numerical order based on the XXXX segment in the migration filenames. For instance, `0001-some-migration.js` will be executed before `0002-another-migration.js`.

Here's a complete example migration script that renames a setting from `setting_with_old_name` to `setting_with_new_name`:

```js
// File name: 0001-rename-setting.js

export default function migrate(settings) {
  if (settings.has("setting_with_old_name")) {
    settings.set("setting_with_new_name", settings.get("setting_with_old_name"));
  }
  return settings;
}
```

Internal topic: t/109980
2023-11-02 08:10:15 +03:00
Mark VanLandingham
88874389d2
FIX: Send push notifications for category/tag watching notifications (#24196)
Problem and solution are outlined here on Meta - https://meta.discourse.org/t/watching-a-category-does-not-cause-push-notifications/282794
2023-11-01 10:06:33 -05:00
Bianca Nenciu
fd07c943ad
DEV: Refactor watched words (#24163)
- Ignore only invalid words, not all words if one of them is invalid

- The naming scheme for methods was inconsistent

- Optimize regular expressions
2023-11-01 16:41:10 +02:00
Martin Brennan
9db4eaa870
DEV: Change anonymous_posting_min_trust_level to a group-based setting (#24072)
No plugins or themes rely on anonymous_posting_min_trust_level so we
can just switch straight over to anonymous_posting_allowed_groups

This also adds an AUTO_GROUPS const which can be imported in JS
tests which is analogous to the one defined in group.rb. This can be used
to set the current user's groups where JS tests call for checking these groups
against site settings.

Finally a AtLeastOneGroupValidator validator is added for group_list site
settings which ensures that at least one group is always selected, since if
you want to allow all users to use a feature in this way you can just use
the everyone group.
2023-10-25 11:45:10 +10:00
David Taylor
48e3d5b409
DEV: Add failing test for pull-hotlinked codeblocks (#23682)
If a codeblock contains **exactly** the same markdown as an image which has been retrieved by the 'pull hotlinked' job, then it will be replaced with the new URL. This commit adds failing (skipped) tests for this issue.
2023-09-29 09:55:51 +01:00
Ted Johansson
928a6cd143
FIX: Delete fast typer reviewable when deleting user (#23162)
In most cases, deleting a user from outside the review UI will also delete any pending reviewables for that user. This was not working in some cases, e.g. for reviewables created due to "fast typer" violations.

This was happening because UserDestroyer only automatically resolves flagged posts.

After this change, in addition to existing checks, look for ReviewablePost where the post was created by the user and reject them if present.
2023-08-21 18:03:03 +08:00
Roman Rizzi
5683c90917
FIX: TopicSummarization workaround for Postgres' discrete range types (#23105)
Our code assumed the content_range interval was inclusive, but they are open-ended due to Postgres' [discrete range types](https://www.postgresql.org/docs/current/rangetypes.html#RANGETYPES-DISCRETE), meaning [1,2] will be represented as [1,3).

It also fixes some flaky tests due to test data not being correctly setup and the registry not being resetted after each test.
2023-08-15 14:16:06 -03:00
Roman Rizzi
7ca5ee6cd2
FEATURE: Stream topic summaries. (#23065)
When we receive the stream parameter, we'll queue a job that periodically publishes partial updates, and after the summarization finishes, a final one with the completed version, plus metadata.

`summary-box` listens to these updates via MessageBus, and updates state accordingly.
2023-08-11 15:08:49 -03:00
Martin Brennan
09223e5ae7
DEV: Remove enable_experimental_hashtag_autocomplete logic (#22820)
This commit removes any logic in the app and in specs around
enable_experimental_hashtag_autocomplete and deletes some
old category hashtag code that is no longer necessary.

It also adds a `slug_ref` category instance method, which
will generate a reference like `parent:child` for a category,
with an optional depth, which hashtags use. Also refactors
PostRevisor which was using CategoryHashtagDataSource directly
which is a no-no.

Deletes the old hashtag markdown rule as well.
2023-08-08 11:18:55 +10:00
Sérgio Saquetim
03690ccccf
DEV: Add :push_notification event and deprecate :post_notification_alert (#22917)
This commit introduces the :push_notification event and deprecates :post_notification_alert.

The old :post_notification_alert event was not triggered when pushing chat notifications and did not respect when the user was in "do not disturb" mode.

The new event fixes these issues.
2023-08-02 18:44:19 -03:00
Roman Rizzi
238d71bcad
FEATURE: Regenerate outdated summaries. (#22718)
Users unable to generate new summaries won't be able to regenerate them. They'll only see the warning saying it's outdated.
2023-07-20 15:25:46 -03:00
Martin Brennan
54001060ea
FIX: Termless hashtag search when a type is disabled (#22660)
When a type was disabled, the hashtag search _without_ a
term was erroring. This was because we weren't filtering
out the disabled types from types_in_priority_order first
like we were if there was a term provided.

This commit fixes that issue, and also makes it so
contexts_with_ordered_types and ordered_types_for_context
will only return hashtag types which are enabled.
2023-07-19 10:10:33 +10:00
Martin Brennan
b583872eed
DEV: Introduce enabled? API to hashtag data sources (#22632)
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
2023-07-18 09:39:01 +10:00
Roman Rizzi
61aeb2da90
FEATURE: Inline topic summary. Cached version accessible to everyone. (#22551)
* FEATURE:  Inline topic summary. Cached version accessible to everyone.

Anons and non-members of the `custom_summarization_allowed_groups_map` groups can see cached summaries for any accessible topic. After the first 12 hours and if the posts to summarize have changed, allowed users clicking on the button will automatically re-generate it.

* Ensure chat summaries work and prevent model hallucinations when there are no messages.
2023-07-12 11:21:51 -03:00
Krzysztof Kotlarek
0744d242c6
FIX: post alerter notification when topic directly watched (#22433)
In previous PR https://github.com/discourse/discourse/pull/22340 bug was introduced. Notifications were blocked when, even if topic was watched directly. New query is taking TopicUser into consideration.

In addition, in user interface, when `watched_precedence_over_muted` is not set, then value from SiteSetting should be displayed.
2023-07-06 11:27:23 +10:00
Krzysztof Kotlarek
134dcdd63a
FEATURE: allow user to override watched_precedence_over_muted setting (#22340)
Recently, site setting watched_precedence_over_muted was introduced - https://github.com/discourse/discourse/pull/22252

In this PR, we are allowing users to override it. The option is only displayed when the user has watched categories and muted tags, or vice versa.
2023-07-04 15:08:29 +10:00
Roman Rizzi
71ff38bab6
DEV: Fix lint and flaky summarization spec (#22303) 2023-06-27 13:18:10 -03:00
Roman Rizzi
f4e7a80600
DEV: Cache summarization strategy results. (#22230)
Updates the interface for implementing summarization strategies and adds a cache layer to summarize topics once.

The cache stores the final summary and each chunk used to build it, which will be useful when we have to extend or rebuild it.
2023-06-27 11:44:34 -03:00
Krzysztof Kotlarek
9cf981f1f1
FEATURE: new watched_precedence_over_muted setting (#22252)
New setting which allow admin to define behavior when topic is in watched category and muted topic and vice versa.

If watched_precedence_over_muted setting is true, that topic is still visible in list of topics and notification is created.

If watched_precedence_over_muted setting is false, that topic is not still visible in list of topics and notification is skipped as well.
2023-06-27 14:49:34 +10:00
Loïc Guitaut
0f4beab0fb DEV: Update the rubocop-discourse gem
This enables cops related to RSpec `subject`.

See https://github.com/discourse/rubocop-discourse/pull/32
2023-06-26 11:41:52 +02:00
Mark VanLandingham
114a9a10b7
DEV: Add spec for notification data modifier (#22223) 2023-06-21 08:57:25 -05:00
Krzysztof Kotlarek
2effcaa0f9
FIX: Update sidebar to be navigation menu (#22101)
Communities can use sidebar or header dropdown, therefore navigation menu is a better name settings in 2 places:

- Old user sidebar preferences;
- Site setting about default tags and categories.
2023-06-15 09:31:28 +10:00
Ted Johansson
a674c6c4c2
DEV: Update username in new quote format - Part 1 (#22032)
When we introduced the new quote format with full-name display name:

```
[quote="Ted Johansson, post:1, topic:2, username:ted"]
we overlooked the code responsible for rewriting quotes when a user's name is changed.
```

The functional part of this change adds support for the new quote format in the code that updates quotes when a user's username changes. See the test case in `spec/services/username_changer_spec.rb` for the details.

In addition, this change adds a regression test for PrettyText to cover the new quote format, and extracts the code responsible for rewriting raw and cooked quotes into its own `QuoteRewriter` class. The functionality of the latter is tested through the tests in `spec/services/username_changer_spec.rb`.
2023-06-14 16:14:11 +08:00
Régis Hanol
4cb3412a56
PERF: improve findAllMatches speed (#22083)
When we introduced unicode support in the regular expressions used in watched words (9a27803) we didn't realize the cost adding the `u` flag would be.

Turns out, it's pretty bad when you have lots of regular expressions to test. A customer had slightly less than 200 watched words, and it would freeze the browser for about 2s on the first check of those regular expressions (roughly 10ms per regular expression).

This commit introduces a new field (`word`) to the serialized watched words which is then converted to a very fast and cheap regular expression on the client-side. We use that regexp to quicly check whether a matcher is even worth trying so that we don't incure the cost of compiling the expensive unicode regexp.

This commit also busts the `WordWatcher` cache since we added a new field to be serialized.

One nice side effect of using `matchAll` instead of a `while / exec` loop is that the likeliness of having a bad regexp matching infinitely is vastly reduced 🙌
2023-06-13 18:34:28 +02:00
Selase Krakani
c45eb8a618
FIX: Create new PM notifications for watching_first_post groups (#21997)
At the moment, PMs to groups with default notification level set to
`watching_first_post` do not generate "emailable" notifications. This happens
because, topic user notification level which is indirectly derived
from the group's default notification level is set to `tracking` if the
group's notification level happens to be `watching_first_post`.

This leads to a `group_message_summary` notification being created
instead of a `private_message` notification which results in no email
alerts being sent when a  topic is created.

As this `watching_first_post` --> `tracking` switcheroo appears to be
intentional instead being a bug, this change extends `PostAlerter`'s
`notify_pm_users` method to create a `private_message` notification for
first posts created in a `watching_first_post` group even if the topic
user notification level is set to `tracking`
2023-06-08 17:41:44 +00:00
Sam
c2332d7505
FEATURE: reduce avatar sizes to 6 from 20 (#21319)
* FEATURE: reduce avatar sizes to 6 from 20

This PR introduces 3 changes:

1. SiteSetting.avatar_sizes, now does what is says on the tin.
previously it would introduce a large number of extra sizes, to allow for
various DPIs. Instead we now trust the admin with the size list.

2. When `avatar_sizes` changes, we ensure consistency and remove resized
avatars that are not longer allowed per site setting. This happens on the
12 hourly job and limited out of the box to 20k cleanups per cycle, given
this may reach out to AWS 20k times to remove things.

3.Our default avatar sizes are now "24|48|72|96|144|288" these sizes were
very specifically picked to limit amount of bluriness introduced by webkit.
Our avatars are already blurry due to 1px border, so this corrects old blur.

This change heavily reduces storage required by forums which simplifies
site moves and more.

Co-authored-by: David Taylor <david@taylorhq.com>
2023-06-01 10:00:01 +10:00
Osama Sayegh
bb3c05ba0e
DEV: Allow plugins to hook into user preferences update process on the server (#21737)
This commit introduces a new `within_user_updater_transaction` event that's triggered inside the transaction that saves user updates in `UserUpdater`. Plugins can hook into the transaction using the event to include custom changes in the transaction. Callbacks for this event receive 2 arguments:

1. the user being saved
2. the changed attributes that are passed to `UserUpdater`.

There's also new modifier in this commit called `users_controller_update_user_params` to allow plugins to allowlist custom params in the `UsersController` which eventually end up getting passed as attributes to the `UserUpdater` and the new `within_user_updater_transaction` event where they can be used to perform additional updates using the custom params.

-----

New API is used in https://github.com/discourse/discourse-mailinglist-integration/pull/1.
2023-05-26 03:26:38 +03:00
Martin Brennan
0b3cf83e3c
FIX: Do not cook icon with hashtags (#21676)
This commit makes some fundamental changes to how hashtag cooking and
icon generation works in the new experimental hashtag autocomplete mode.
Previously we cooked the appropriate SVG icon with the cooked hashtag,
though this has proved inflexible especially for theming purposes.

Instead, we now cook a data-ID attribute with the hashtag and add a new
span as an icon placeholder. This is replaced on the client side with an
icon (or a square span in the case of categories) on the client side via
the decorateCooked API for posts and chat messages.

This client side logic uses the generated hashtag, category, and channel
CSS classes added in a previous commit.

This is missing changes to the sidebar to use the new generated CSS
classes and also colors and the split square for categories in the
hashtag autocomplete menu -- I will tackle this in a separate PR so it
is clearer.
2023-05-23 09:33:55 +02:00
Natalie Tay
07061410d8
FIX: Anonymizing a user clears their user status too (#21673) 2023-05-22 13:18:09 +08:00
Bianca Nenciu
9a2780397f
FIX: Handle all UTF-8 characters (#21344)
Watched words were converted to regular expressions containing \W, which
handled only ASCII characters. Using [^[:word]] instead ensures that
UTF-8 characters are also handled correctly.
2023-05-15 12:45:04 +03:00
Martin Brennan
86204fa4f0
FIX: Hashtag subcategory ref incorrect when not highest-ranked type (#21163)
This commit fixes the following scenario:

1. The user is searching for hashtags in chat, where the subcategory
   type is not highest-ranked in priority order.
2. There can, but doesn't have to be, a higher-ranked matching chat
   channel that has the same slug as the subcategory.
3. Since it is not the highest-ranked type, the subcategory, which
   normally has a ref of parent:child, has its ref changed to
   child::category, which does not work

This was happening because whenever a hashtag type was not highest
ranked, if _any_ other hashtag results conflicted slugs, we would
append the ::type suffix. Now, we only append this suffix if a
higher-ranked type conflicts with the hashtag, and we use the current ref
to build the new typed ref to preserve this parent:child format as well,
it's more accurate.
2023-04-20 09:03:55 +10:00
Jarek Radosz
29e2e3ff3b
DEV: Fix random typos (#20937) 2023-04-03 19:27:32 +02:00
Jan Cernik
afe3e36363
DEV: Remove lazy-yt and replace with lazy-videos (#20722)
- Refactors the old plugin to remove jquery usage
- Adds support for Vimeo videos (default on) and Tiktok (experimental and default off)
2023-03-29 11:54:25 -04:00
Rafael dos Santos Silva
2a7bdb2d66
FIX: Push notification delay should not be longer than specified (#20864)
When user.last_seen was less than push_notification_time_window_mins we
where delaying the notification for the whole
push_notification_time_window_mins PLUS the time the user was away from.

Originally reported in https://meta.discourse.org/t/-/259688
2023-03-28 13:22:54 -03:00
Sam
4a3c13a37b
FIX: search index failing on certain posts (#20736)
During search indexing we "stuff" the index with additional keywords for
entities that look like domain names.

This allows searches for `cnn` to find URLs for `www.cnn.com`

The search stuffing attempted to keep indexes aligned at the correct positions
by remapping the indexed terms. However under certain edge cases a single
word can stem into 2 different lexemes. If this happened we had an off by
one which caused the entire indexing to fail.

We work around this edge case (and carry incorrect index positions) for cases
like this. It is unlikely to impact search quality at all given index position
makes almost no difference in the search algorithm.
2023-03-20 15:43:08 +11:00
Ted Johansson
39c2f63b35 SECURITY: Add FinalDestination::FastImage that's SSRF safe 2023-03-16 15:27:09 -06:00
Gerhard Schlager
12436d054d
DEV: Remove badge_granted_title column from user_profiles (#20476)
That column is obsolete since we added the `granted_title_badge_id` column in 2019 (56d3e29a69). Having both columns can lead to inconsistencies (mostly due to old data from before 2019).

For example, `BadgeGranter.revoke_ungranted_titles!` doesn't work correctly if `badge_granted_title` is `false` while `granted_title_badge_id` points to the badge that is used as title.
2023-03-08 13:37:20 +01:00
Natalie Tay
44b7706a2b
UX: Skip applying link-type watched words to user custom fields (#20465)
We currently apply type: :link watched words to custom user fields. This makes the user card pretty ugly because we don't allow html / links there. Additionally, the admin UI also does not say that we apply this to custom user fields, but only words in posts.

So this PR is to remove the replacement of link-type watch words for custom user fields.
2023-03-01 10:43:34 +08:00
Sam
e636abeb0d
FIX: do not notify admins on suppressed categories (#20238)
* FIX: do not notify admins on suppressed categories

Avoid notifying admins on categories where they are not explicitly members
in cases where SiteSetting.suppress_secured_categories_from_admin is
enabled.

This helps keep notification stream clean and avoids admins mistakenly
being invited to discussions that should be suppressed
2023-02-14 16:45:06 +11:00
Ted Johansson
25a226279a
DEV: Replace #pluck_first freedom patch with AR #pick in core (#19893)
The #pluck_first freedom patch, first introduced by @danielwaterworth has served us well, and is used widely throughout both core and plugins. It seems to have been a common enough use case that Rails 6 introduced it's own method #pick with the exact same implementation. This allows us to retire the freedom patch and switch over to the built-in ActiveRecord method.

There is no replacement for #pluck_first!, but a quick search shows we are using this in a very limited capacity, and in some cases incorrectly (by assuming a nil return rather than an exception), which can quite easily be replaced with #pick plus some extra handling.
2023-02-13 12:39:45 +08:00
Rafael dos Santos Silva
6e522e4aad
DEV: Move to Sass compilation to dart-sass (#19910)
This PR is a major change to Sass compilation in Discourse.

The new version of sass-ruby moves to dart-sass putting we back on the supported version of Sass. It does so while keeping compatibility with the existing method signatures, so minimal change is needed in Discourse for this change.

This moves us

From:
  - sassc 2.0.1 (Feb 2019)
  - libsass 3.5.2 (May 2018)

To:
  - dart-sass 1.58

This update applies the following breaking changes:

> 
> These breaking changes are coming soon or have recently been released:
> 
>  [Functions are stricter about which units they allow](https://sass-lang.com/documentation/breaking-changes/function-units) beginning in Dart Sass 1.32.0.
> 
>  [Selectors with invalid combinators are invalid](https://sass-lang.com/documentation/breaking-changes/bogus-combinators) beginning in Dart Sass 1.54.0.
> 
>  [/ is changing from a division operation to a list separator](https://sass-lang.com/documentation/breaking-changes/slash-div) beginning in Dart Sass 1.33.0.
> 
>  [Parsing the special syntax of @-moz-document will be invalid](https://sass-lang.com/documentation/breaking-changes/moz-document) beginning in Dart Sass 1.7.2.
> 
>  [Compound selectors could not be extended](https://sass-lang.com/documentation/breaking-changes/extend-compound) in Dart Sass 1.0.0 and Ruby Sass 4.0.0.


SCSS files have been migrated automatically using `sass-migrator division app/assets/stylesheets/**/*.scss`
2023-02-07 12:24:57 -03:00
Gerhard Schlager
e17c145e8d
FIX: Category hashtags weren't always found for sub-sub-categories (#20156)
The algorithm failed to find the correct category by slug when there are multiple sub-sub-categories with the same child-category name and the first child doesn't  have the correct grandchild.

So, searching for "child / grandchild" worked in the following case, it found (3):

- (1) parent 1
  - (2) child
    - (3) grandchild
- (4) parent 2
  - (5) child
    - (6) grandchild

But it failed to find the grandchild in the following case:

- (1) parent 1
  - (2) child
- (4) parent 2
  - (5) child
    - (6) grandchild

And this also fixes a flaky spec by forcing categories to always order by by `parent_category_id` and `id`.
This makes it possible to partly revert 60990aab55
2023-02-03 12:17:52 +01:00
Sam
651476e89e
FIX: domain searches not working properly for URLs (#20136)
If a post contains domain with a word that stems to a non prefix single
words will not match it.

For example: in happy.com, `happy` stems to `happi`. Thus searches for happy
will not find URLs with it included.

This bloats the index a tiny bit, but impact is limited.

Will require a full reindex of search to take effect. 

When we are done refining search we can consider a full version bump.
2023-02-03 09:55:28 +11:00
Selase Krakani
2e78045af1
FIX: Extend username updates to self-mentions (#20071)
Posts with self-mentions aren't updated with username updates. This happens
because mention `UserAction` entries aren't logged for self-mentions.

This change updates the lookup of `Post` and `PostRevision` with mentions to bypass
`UserAction` entries.
2023-02-02 12:33:42 +00:00
Sam
4570118a63
FIX: search index duplicate parser matching is too restrictive (#20129)
Previous regex did not allow for cases where a lexeme contains a : (colon)

This can happen when parsing URLs. New algorithm allows for this.
Test was amended to more clearly call out index problems
2023-02-02 12:17:19 +11:00
Jan Cernik
06817bd94f
FIX: Category permission change not creating a log (#20027)
It didn't create a log if the category was public { "everyone" => 1 }
2023-01-31 10:15:17 -03:00
Sam
07679888c8
FEATURE: allow restricting duplication in search index (#20062)
* FEATURE: allow restricting duplication in search index

This introduces the site setting `max_duplicate_search_index_terms`.
Using this number we limit the amount of duplication in our search index.

This allows us to more correctly weight title searches, so bloated posts
don't unfairly bump to the top of search results.

This feature is completely disabled by default and behind a site setting

We will experiment with it first. Note entire search index must be rebuilt
for it to take effect.


---------

Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2023-01-31 12:41:31 +11:00
David Taylor
79bea9464c
PERF: Move user-tips and narrative to per-user messagebus channels (#19773)
Using a shared channel with per-message permissions means that every client is updated with the channel's 'last_id', even if there are no messages available to them. Per-user channel names avoid this problem - the last_id will only be incremented when there is a message for the given user.
2023-01-30 11:48:09 +00:00
Martin Brennan
63fdb6dd65
FIX: Do not add empty use/svg tags in ExcerptParser (#19969)
There was an issue where if hashtag-cooked HTML was sent
to the ExcerptParser without the keep_svg option, we would
end up with empty </use> and </svg> tags on the parts of the
excerpt where the hashtag was, in this case when a post
push notification was sent.

Fixed this, and also added a way to only display a plaintext
version of the hashtag for cases like this via PrettyText#excerpt.
2023-01-24 14:40:24 +10:00
Alan Guo Xiang Tan
f122f24b35
SECURITY: Default tags to show count of topics in unrestricted categories (#19916)
Currently, `Tag#topic_count` is a count of all regular topics regardless of whether the topic is in a read restricted category or not. As a result, any users can technically poll a sensitive tag to determine if a new topic is created in a category which the user has not excess to. We classify this as a minor leak in sensitive information.

The following changes are introduced in this commit:

1. Introduce `Tag#public_topic_count` which only count topics which have been tagged with a given tag in public categories.
2. Rename `Tag#topic_count` to `Tag#staff_topic_count` which counts the same way as `Tag#topic_count`. In other words, it counts all topics tagged with a given tag regardless of the category the topic is in. The rename is also done so that we indicate that this column contains sensitive information. 
3. Change all previous spots which relied on `Topic#topic_count` to rely on `Tag.topic_column_count(guardian)` which will return the right "topic count" column to use based on the current scope. 
4. Introduce `SiteSetting.include_secure_categories_in_tag_counts` site setting to allow site administrators to always display the tag topics count using `Tag#staff_topic_count` instead.
2023-01-20 09:50:24 +08:00
Ted Johansson
421fbfd1c7
FIX: Fix flaky test resulting from PostAlerter keyword arguments (#19826)
We've been doing some work to support new keyword argument semantics in Ruby 3. As part of that we made some changes to `DiscourseEvent::TestHelper`. The backwards compatibility fix doesn't work if the method is called with an empty hash as the final argument. This fix adds a valid option to the final hash in the particular test.
2023-01-12 09:46:50 +08:00
David Taylor
cb932d6ee1
DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
Alan Guo Xiang Tan
673089a6b4
FIX: Error condition in SidebarSiteSettingsBackfiller (#19787) 2023-01-09 07:01:58 +08:00