When a screened IP address is matched because it is either blocked or
allowed it should update match_count. This did not work because it
tried to validate the IP address and it failed as it matched with
itself.
Previously we would trigger the event before the `Topic#deleted_at`
column has been updated making it hard for plugins to correctly work
with the model when its new state has not been persisted in the
database.
* FIX: Only modify secured sidebar links on user promotion/demotion
If a user is created populate their sidebar with the default
categories/tags that they have access to.
If a user is promoted to admin populate any new categories/tags that
they now have access to.
If an admin is demoted remove any categories/tags that they no longer
have access to.
This will only apply for "secured" categories. For example if these are
the default sitebar categories:
- general
- site feedback
- staff
and a user only has these sidebar categories:
- general
when they are promoted to admin they will only receive the "staff"
category. As this is a default category they didn't previously have
access to.
* Add spec, remove tag logic on update
Change it so that if a user becomes unstaged it used the "add" method
instead of the "update" method because it is essentially following the
on_create path.
On admin promotion/demotion remove the logic for updating sidebar tags because
we don't currently have the tag equivalent like we do for User.secure_categories.
Added the test case for when a user is promoted to admin it should
receive *only* the new sidebar categories they didn't previously have
access to. Same for admin demotion.
* Add spec for suppress_secured_categories_from_admin site setting
* Update tags as well on admin promotion/demotion
* only update tags when they are enabled
* Use new SidebarSectionLinkUpdater
We now have a SidebarSectionLinkUpdater
that was introduced in: fb2507c6ce
* remove empty line
* FEATURE: Show warning if group cannot be mentioned
A similar warning is displayed when the user cannot be mentioned because
they have not been invited to the topic.
* FEATURE: Resolve mentions for new topic
This commit improves several improvements and refactors
/u/is_local_username route to a better /composer/mentions route that
can handle new topics too.
* FEATURE: Show warning if only some are notified
Sometimes users are still notified even if the group that was mentioned
was not invited to the message. This happens because its members were
invited directly or are members of other groups that were invited.
* DEV: Refactor _warnCannotSeeMention
User options were serialized at the root level of CurrentUserSerializer,
but UserSerializer has a user_option field. This inconsistency caused
issues in the past because user_option fields had to be duplicated on
the frontend.
When uploads are stored on S3, by default Discourse will fetch the avatars and proxy them through to the requesting client. This is simple, but it can lead to significant inbound/outbound network load in the hosting environment.
This commit adds an optional redirect_avatar_requests GlobalSetting. When enabled, requests for user avatars will be redirected to the S3 asset instead of being proxied. This adds an extra round-trip for clients, but it should significantly reduce server load. To mitigate that extra round-trip for clients, a CDN with 'follow redirect' capability could be used.
Update failing spec which previously used non-staff user to create
hidden posts.
Also add new spec for non-staff use cases to prevent future
regressions.
In some cases (e.g. user notification emails) we
are passing an excerpted/stripped version of the
post HTML to Email::Styles, at which point the
<span> elements surrounding the hashtag text have
been stripped. This caused an error when trying to
remove that element to replace the text.
Instead we can just remove all elements inside
a.hashtag-cooked and replace with the raw #hashtag
text which will work in more cases.
The centralization helps in reducing code duplication in our code base
and more importantly, centralizing logic for guardian checks into a
single spot.
Users who can access the review queue can claim a pending reviewable(s) which means that the claimed reviewable(s) can only be handled by the user who claimed it. Currently, we show claimed reviewables in the user menu, but this can be annoying for other reviewers because they can't do anything about a reviewable claimed by someone. So this PR makes sure that we only show in the user menu reviewables that are claimed by nobody or claimed by the current user.
Internal topic: t/77235.
This PR adds separate notification indicators for PMs and reviewables that have arrived since the last time the user opened the notifications menu.
The PM indicator is the strongest one of all three indicators followed by the reviewable indicator and then finally the blue indicator. This means that if there's a new PM and a new reviewable, then the PM indicator will be shown.
Meta topic: https://meta.discourse.org/t/no-green-or-red-notification-bubbles/242783?u=osama.
Internal topic: t/82995.
Fixes broken behaviour of arrow buttons for certain users as the interval to scroll menu can be cancelled before the scrolling actually happens.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This is closer to git's redirect following behaviour. We prevented git
following redirects when we clone in order to prevent SSRF attacks.
Follow-up-to: 291bbc4fb9
While load testing our user creation code path in production, we
identified that executing the DB statement to update the `Group#user_count` column within a
transaction is creating a bottleneck for us. This is because the
creation of a user and addition of the user to the relevant groups are
done in a transaction. When we execute the DB statement to update
`Group#user_count` for the relevant group, a row level lock is held
until the transaction completes. This row level lock acts like a global
lock when the server is creating users that will be added to the same
group in quick succession.
Instead of updating the counter cache within a transaction which the
default ActiveRecord `counter_cache` option does, we simply update the
counter cache outside of the committing transaction.
Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
Co-authored-by: Rafael dos Santos Silva <xfalcox@gmail.com>
When doing local oneboxes we sometimes want to allow
SVGs in the final preview HTML. The main case currently
is for the new cooked hashtags, which include an SVG
icon.
SVGs will be included in local oneboxes via `ExcerptParser` _only_
if they have the d-icon class, and if the caller for `post.excerpt`
specifies the `keep_svg: true` option.
Adds stats for API and user API requests similar to regular page views.
This comes with a new report to visualize API requests per day like the
consolidated page views one.
This commit introduce a new API for registering callbacks, which we'll execute when a user gets destroyed, and the `delete_posts` opt is true. The chat plugin registers one callback and queues a job to destroy every message from that user in batches.
When searching for categories it is possible for
a child category to have a slug that matches the term
exactly, but will not be found by .lookup since we
don't return these categories unless the ref matches
parent:child.
Introduces a search_sort method to each hashtag data
source so they can provide their custom sort logic of
results, in category's case putting all matching slugs
to the top regardless of parent/child relationship
then sorting by text.
Follow up to 40e8912395
In this previous commit I introduced a bug that prevented
a legitimate case for an existing user to redeem an invite,
where the email/domain were both blank and the invite was
still redeemable by the user. Fixes the issue and adds more
specs for that case.
Previously we would unconditionally fetch all images via HTTP to grab
original sizing from cooked post processor in 2 different spots.
This was wasteful as we already calculate and cache this info in upload records.
This also simplifies some specs and reduces use of mocks.
I'm hesitant to call this a performance improvement since claiming a
reviewable is probably rare. However, this commit cuts out two DB
queries each time we have to publish a reviewable claimed message. More
importantly, publishing to groups scales much better than publishing to
users because we esstentially cap the number ids we have to load into
memory.
The `decodedMap` prop comes from https://github.com/terser/terser/pull/1190
> This also exposes a new `decodedMap` property on the result object. Decoded maps are free to create (it's a shallow clone of the `GenMapping` instance), and passing them to `@jridgewell/trace-mapping` is copy-free. With Babel [recently](https://github.com/babel/babel/pull/14497) adding a `decodedMap` field, a dev could pass from the Babel transpilation to Terser without any added memory use for sourcemaps.
This adds API scope for the user status. This also adds a get method to the user status controller. We didn't need a dedicated method that returns status before because the server returns status with user objects, but I think we need to provide this method for API clients.
Raw paths like `/test/path` are not supported natively in the CSP. This commit prepends the site's base URL to these paths. This allows plugins to add 'local' assets to the CSP without needing to hardcode the site's hostname.
This changes the hashtag search to first do a lookup to find
results where the slug exactly matches the
search term. Now when we search for hashtags, the
exact matches will be found first and put at the top of
the results.
`ChatChannelFetcher` has also been modified here to allow
for more options for performance -- we do not need to
query DM channels for secured IDs when looking up or searching
channels for hashtags, since they should never show in
results there (they have no slugs). Nor do we need to include
the channel archive records.
Also changes the limit of hashtag results to 20 by default
with a hidden site setting, and makes it so the scroll for the
results is overflowed.
In this PR, we're making sure when a theme upload which is used in the theme's CSS is missing it won't break the stylesheet precompilation process. See also: 6ebd2cecda
Adds the description as a title="" attribute on the hashtag
autocomplete search items for tags, categories, and channels.
These descriptions can be seen by the user since they are
able to see the results that are returned by the search via
Guardian checks.
This fix changes the hashtag-raw hashtags, which are
the ones that do not actually match anything, back
to the old style which does not look like mentions.
The user attributes are not updated between clients and that is a
problem with user tips because the same user tip will be displayed
multiple times, once for every client.
This will be used by plugins to handle the client side of their custom
post validations without having to overwrite the whole composer save
action as it was done in other plugins.
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
The tag ordering was inconsistent, because we were not
passing the correct order option to DiscourseTagging.filter_allowed_tags.
The order would change based on the limit provided. Now,
we can have a consistent order which is term exact match -> topic count ->
name.
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
* FEATURE: API to update user's discourse connect external id
This adds a special handling of updates to DiscourseConnect external_id
in the general user update API endpoint.
Admins can create, update or delete a user SingleSignOn record using
PUT /u/:username.json
{
"external_ids": {
"discourse_connect": "new-external-id"
}
}
The hidden site setting `suppress_secured_categories_from_admin` will
suppress visibility of categories without explicit access from admins
in a few key areas (category drop downs and topic lists)
It is not intended to be a security wall since admins can amend any site
setting. Instead it is feature that allows hiding the categories from the
UI.
Admins will still be able to see topics in categories without explicit
access using direct URLs or flags.
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
When opening the invite acceptance page when the user
was already logged in, we were still showing the Accept
Invitation prompt even if the user had already redeemed
the invitation and was present in the `InvitedUser` table.
This would lead to errors when the user clicked on the button.
This commit fixes the issue by hiding the Accept Invitation
button and showing an error message instead indicating that
the user had already redeemed the invitation. This only applies
to multi-use invite links.
The problem was reported as a problem with changing theme in user preferences, after saving a new theme the previously set user status was disappearing (https://meta.discourse.org/t/user-status/240335/42). Turned out though that the problem was more wide, changing pretty much any setting in user preferences apart from user status itself led to clearing the status.
* FEATURE: Default Composer Category Site Setting
- Create the default_composer_category site setting
- Replace general_category_id logic for auto selecting the composer
category
- Prevent Uncategorized from being selected if not allowed
- Add default_composer_category option to seeded categories
- Create a migration to populate the default_composer_category site
setting if there is a general_category_id populated
- Added some tests
* Add missing translation for the new site setting
* fix some js tests
* Just check that the header value is null
This commit adds some protections in InviteRedeemer to ensure that email
can never be nil, which could cause issues with inviting the invited
person to private topics since there was an incorrect inner join.
If the email is nil and the invite is scoped to an email, we just use
that invite.email unconditionally. If a redeeming_user (an existing
user) is passed in when redeeming an email, we use their email to
override the passed in email. Otherwise we just use the passed in
email. We now raise an error after all this if the email is still nil.
This commit also adds some tests to catch the private topic fix, and
some general improvements and comments around the invite code.
This commit also includes a migration to delete TopicAllowedUser records
for users who were mistakenly added to topics as part of the invite
redemption process.
Currently, moderators are able to set primary group for users
irrespective of the of the `moderators_manage_categories_and_groups` site
setting value.
This change updates Guardian implementation to honour it.
Currently, we have available three 2fa methods:
- Token-Based Authenticators
- Physical Security Keys
- Two-Factor Backup Codes
If the first two are deleted, user lose visibility of their backup codes, which suggests that 2fa is disabled.
However, when they try to authenticate, the account is locked, and they have to ask admin to fix that problem.
This PR is fixing the issue. User still sees backup codes in their panel and can use them to authenticate.
In next PR, I will improve UI to clearly notify the user when 2fa is fully disabled and when it is still active.
* Remove old bookmark column ignores to follow up b22450c7a8
* Change some group site setting checks to use the _map helper
* Remove old secure_media helper stub for chat
* Change attr_accessor to attr_reader for preloaded_custom_fields to follow up 70af45055a
It is likely that a new admin user was created as just a regular user
before being promoted to admin so this change will update the sidebar
link records for any users that are promoted to admin. This way if any
of the default side bar categories or tags are restricted to admins
these new admins will have those added to their sidebar as well.
You can easily replicate this issue locally (prior to this fix) by using
`rails admin:create` where it creates a user first, then it is promoted
to admin. This means it would receive the default categories of regular
user, but never receive the ones they should have access to as an admin.
As part of this change I did drop the `!` from
`SidebarSectionLink.insert_all` so that it would add any new records
that were missing, but not throw a unique constraint error trying to add
any existing records.
Follow up to: 1b56a55f50
And: e320bbe513
Previously the stylesheet cachebusting hash was based on the maximum mtime of files. This works well in development and during in-container updates (e.g. via docker_manager). However, when a fresh docker image is created for each deploy, the file mtimes will change even if the contents has not.
This commit changes the production logic to calculate the cachebuster from the filenames and contents of the relevant assets. This should be consistent across deploys, thereby improving cache hits and improving page load times.
Meta topic: https://meta.discourse.org/t/meta-theme-color-is-not-respecting-current-color-scheme/239815
Currently, the dark mode theme-color `<meta>` tag doesn't apply because the light mode tag has `media="all"`. This means that the dark mode `<meta>` tag with `media="(prefers-color-scheme: dark)"` won't override it. This PR updates the light mode tag to `media="(prefers-color-scheme: light)"` if `dark_scheme_id` is defined and leaves it as `media="all"` otherwise.
Since the system user is a regular user, it can have its
`allow_private_messages` user option turned off, which
with our current `can_send_private_message?(Discourse.system_user)`
check inside the CurrentUserSerializer, will prevent any
user from sending messages in the UI if the system user is not
accepting PMs.
This commit adds a new `can_send_private_messages?` method to
the Guardian, which can be used in serializers and not depend
on the system user. When the user actually sends a message
we still rely on the old `can_send_private_message?(target)`
call to see if they are allowed to send the message to the target.
The new method is just to say they can "generally" send
private messages.
Previously, we didn't have a site-wide setting to set the default behavior for user profile visibility and user presence features. But we already have a user preference for that.
Depends on: #18806
We have a banner that prompts to edit the welcome topic, so let's not
show it in the topic list until it has been edited. Previously this
banner covered the welcome topic, now the banner will be above the topic
list, so we need to hide the welcome topic.
* DEV: Add a dedicated Admin::StaffController base controller
The current parent(Admin:AdminController) for all admin-related controllers
uses a filter that allows only staff(admin, moderator) users.
This refactor makes Admin::AdminController filter for only admins as the name suggests and
introduces a base controller dedicated for staff-related endpoints.
* DEV: Set staff-only controllers parent to Admin::StaffController
Refactor staff-only controllers to inherit newly introduced
Admin::StaffController abstract controller. This conveys the
purpose of the parent controller better unlike the previously used parent
controller.
* FIX: Follow up fixes for password-reset error page
Pass in `base_url` to the template
Use `.html_safe` since the message now contains html
Follow up to: 9b1536fb83
* Update specs to pass in the base_url
This commit fixes a regression introduced in 8979adc where under certain conditions the groups syncing logic in Discourse Connect would try to add users to groups they're already members of and cause errors when users try to sign in using Discourse Connect.
Before this commit, we did not have guardian checks in place to determine if a
topic's title associated with a user badge should be displayed or not.
This means that the topic title of topics with restricted access
could be leaked to anon and users without access if certain conditions
are met. While we will not specify the conditions required, we have internally
assessed that the odds of meeting such conditions are low.
With this commit, we will now apply a guardian check to ensure that the
current user is able to see a topic before the topic's title is included
in the serialized object of a `UserBadge`.
The previous sidebar default tags and categories implementation did not
allow for a user to configure their sidebar to have no categories or
tags. This commit changes how the defaults are applied. When a user is being created,
we create the SidebarSectionLink records based on the `default_sidebar_categories` and
`default_sidebar_tags` site settings. SidebarSectionLink records are
only created for categories and tags which the user has visibility on at
the point of user creation.
With this change, we're also adding the ability for admins to apply
changes to the `default_sidebar_categories` and `default_sidebar_tags`
site settings historically when changing their site setting. When a new
category/tag has been added to the default, the new category/tag will be
added to the sidebar for all users if the admin elects to apply the changes historically.
Like wise when a tag/category is removed, the tag/category will be
removed from the sidebar for all users if the admin elects to apply the
changes historically.
Internal Ref: /t/73500
Before this commit, there was no way for us to efficiently check an
array of topics for which a user can see. Therefore, this commit
introduces the `TopicGuardian#can_see_topic_ids` method which accepts an
array of `Topic#id`s and filters out the ids which the user is not
allowed to see. The `TopicGuardian#can_see_topic_ids` method is meant to
maintain feature parity with `TopicGuardian#can_see_topic?` at all
times so a consistency check has been added in our tests to ensure that
`TopicGuardian#can_see_topic_ids` returns the same result as
`TopicGuardian#can_see_topic?`. In the near future, the plan is for us
to switch to `TopicGuardian#can_see_topic_ids` completely but I'm not
doing that in this commit as we have to be careful with the performance
impact of such a change.
This method is currently not being used in the current commit but will
be relied on in a subsequent commit.
Meta topic: https://meta.discourse.org/t/meta-theme-color-is-not-respecting-current-color-scheme/239815/7?u=osama.
This commit renders an additional `theme-color` `<meta>` tag for the dark scheme if the current user/request has a scheme selected for dark mode. We currently only render one `theme-color` tag which is always based on the user's selected scheme for light mode, but if the user also selects a scheme for dark mode and uses a device that's configured to use/prefer dark mode, the Discourse UI will be in dark mode, but any parts of the browser/OS UI that's colored based on the `theme-color` tag, would use a color from the user's selected light scheme and look inconsistent with the Discourse UI because the `theme-color` tag is based on the user's selected light scheme.
The additional `theme-color` tag has `media="(prefers-color-scheme: dark)"` and is based on the user's selected dark scheme which means any browser UI that's colored based on `theme-color` tags should be able to pick the right tag based on the user's preference for light/dark mode.
The `active` param on the create user endpoint requires that an api key
is used in the request header otherwise it is ignored, so adding this
distinction to the api docs.
Related to aeee7ed.
Before the change in aeee7ed, notifications for direct replies to your posts and notifications for replies in watched topics looked the same in the notifications menu -- they both used the arrow icon.
We decided in aeee7ed to distinguish them by changing "watched topics" notifications to use the bell icon because it was confusing for users who watch topics to see the same icon for direct replies and "watched topics". However, that change also means that non-power/new users who receive replies to topics _they create_ will get notifications with the bell icon because technically they're watching the topic, but the arrow icon is more appropriate for this case because we use it throughout the app to indicate "replies".
This commit adds a special-case so that if a user is watching a topic AND the topic is created by them, they receive notifications with the arrow icon (type `replied`) instead of the bell icon (type `posted`) for new posts in the topic.
Internal topic: t/79051.
Discourse Connect can be used to manage group memberships of users by including a `add_groups`, `remove_groups` or `groups` attribute in the Discourse Connect payload. However, additions/deletions of users from groups aren't logged to the groups logs (available at `/g/<group>/manage/logs`) which can cause confusions to admins they try to figure out when/how users were added or removed from a group. This commit makes Discourse Connect add entries to the groups logs when it makes changes to users' group memberships.
This test flakes occassionally, possibly because
of the arg ordering which we do not guarantee.
In future if this keeps occurring we may want to
try make expect_enqueued_with not care about argument
orders or the order of arrays etc within those arguments.
Adds sorting for the HashtagAutocompleteService to
sort the results by case-insensitive text _within_
the type sort order specified by the params. This
should fix some flaky specs as well.
Linking a commit from a GitHub pull request included the complete commit
message, instead of just the first line. The rest of the commit message
will be added to the body of the Onebox.
Building does not persist the object in the database which is
unrealistic since we're mostly dealing with persisted objects in
production.
In theory, this will result our test suite taking longer to run since we
now have to write to the database. However, I don't expect the increase
to be significant and it is actually no different than us adding new
tests which fabricates more objects.
Staged users are allowed to view topics they created in a read restricted category
when category has `Category#email_in` and
`Category#email_in_allow_strangers` configured.
* DEV: Use list controller and action
It used an empty action handler which just returned the app and it
required another request to get the topic list. By using the correct
controller and action we can preload the topic list.
Our theme system injects a magical `settings` object at the top of themes JS modules to allow theme authors to access the settings as configured by admins in the UI. Within this `settings` object, there are a couple of special objects `theme_uploads` and `theme_uploads_local` that contain URLs for all the assets/uploads that the theme has.
For test modules/files, the theme system also injects a `settings` object at the top of tests modules, but it's not the same object as the object that's injected in non-test files. The difference is that in tests we want the settings to have their default values as opposed to any custom values that may exist in the site's database. This ensures that test results are consistent no matter the site that runs them.
However, the `settings` object in tests files currently doesn't have the special objects `theme_uploads` and `theme_uploads_local` which means that if a theme includes an asset that's lazy-loaded, it's not possible to write tests for anything that depends on the lazy-loaded asset because the theme will not be able to load the asset during the tests since `theme_uploads_local` and `theme_uploads` don't exist. This PR adds these special objects inside the `settings` object for test files.
Internal topic: t/71825/52.
When PostRevisor is called with 'skip_validations: true' it can save
the post twice and one of the calls passes the correct 'validate: false'
argument, but the other one does not.
The filenames (minus the extensions) were being used as keys in a hash to pass to Terser, which meant that colocated connector files would overwrite each other. This commit moves the un-colocating earlier in the pipeline so that the fixed filenames are passed to Terser.
Followup to be3d6a56ce
This commit adds a new `/hashtag/search` endpoint and both
relevant JS and ruby plugin APIs to handle plugins adding their
own data sources and priority orders for types of things to search
when `#` is pressed.
A `context` param is added to `setupHashtagAutocomplete` which
a corresponding chat PR https://github.com/discourse/discourse-chat/pull/1302
will now use.
The UI calls `registerHashtagSearchParam` for each context that will
require a `#` search (e.g. the topic composer), for each type of record that
the context needs to search for, as well as a priority order for that type. Core
uses this call to add the `category` and `tag` data sources to the topic composer.
The `register_hashtag_data_source` ruby plugin API call is for plugins to
add a new data source for the hashtag searching endpoint, e.g. discourse-chat
may add a `channel` data source.
This functionality is hidden behind the `enable_experimental_hashtag_autocomplete`
flag, except for the change to `setupHashtagAutocomplete` since only core and
discourse-chat are using that function. Note this PR does **not** include required
changes for hashtag lookup or new styling.
Before, `sidebar_list_destination` was an attribute on UserOptionSerializer. The problem was that this attribute was added to user model only when the user entered the preferences panel. We want that attribute to be available all the time, therefore it was moved to CurrentUserSerializer.
This commit excludes posts from hidden topics from the latest posts and user activity RSS feeds. Additionally, it also excludes small actions from the first one.
Theme javascript is now minified using Terser, just like our core/plugin JS bundles. This reduces the amount of data sent over the network.
This commit also introduces sourcemaps for theme JS. Browser developer tools will now be able show each source file separately when browsing, and also in backtraces.
For theme test JS, the sourcemap is inlined for simplicity. Network load is not a concern for tests.
Previously, compiling theme 'extra_js' was done with a number of steps. Each theme_field would be compiled into its own value_baked column, and then the JavascriptCache content would be built by concatenating all of those compiled values.
This commit streamlines things by removing the value_baked step. The raw value of all extra_js theme_fields are passed directly to the ThemeJavascriptCompiler, and then the result is stored in the JavascriptCache.
In itself, this commit should not cause any behavior change. It is designed to open the door to more advanced compilation features which have interdependencies between different source files (e.g. template colocation, sourcemaps).
The previous implementation would attempt to fetch groups using the end-user's Google auth token. This only worked for admin accounts, or users with 'delegated' access to the `admin.directory.group.readonly` API.
This commit changes the approach to use a single 'service account' for fetching the groups. This removes the need to add permissions to all regular user accounts. I'll be updating the [meta docs](https://meta.discourse.org/t/226850) with instructions on setting up the service account.
This is technically a breaking change in behavior, but the existing implementation was marked experimental, and is currently unusable in production google workspace environments.
This commit fixes an issue where we had a typo in the
UserAction.stream query which meant that action_code_path
was not loaded correctly. Once that was fixed, we were also
not actually using the action_code_path in the user-stream-item,
so that has been fixed here too.
The bug this caused was that, when the link for the action was
clicked within the user-stream-item, the user would be redirected
to a URL ending with `[missing%20%%7Bpath%7D%20value]` because
the I18n call did not have the path present.
Previously, when the array had both nil and string values it returned the error "comparison of NilClass with String failed". Now I added the `.compact` method to prevent this issue as per @martin-brennan's suggestion https://github.com/discourse/discourse/pull/18431#discussion_r984204788
* FIX: Reset related site settings on general category delete
If the new seeded General category is deleted we also need to delete the
corresponding site setting for it so that we don't try and reference it.
This fixes a bug in the category dropdown composer.
This change creates the `clear_related_site_settings` after destroy
hook that could also be used by other features in the future, like maybe
when we have a `default_category_id` site_setting.
Looks like if `nil` out a site setting it is set to `0`?
```
[9] pry(main)> SiteSetting.general_category_id = nil
SiteSetting Load (0.4ms) SELECT "site_settings".* FROM "site_settings" WHERE "site_settings"."name" = 'general_category_id' LIMIT 1
=> nil
[10] pry(main)> SiteSetting.general_category_id
=> 0
```
That is why the tests check if the value is `< 1` and not `nil`.
* Use -1 instead of nil because it is the default
This commit introduces a new framework for building user tutorials as
popups using the Tippy JS library. Currently, the new framework is used
to replace the old notification spotlight and tips and show a new one
related to the topic timeline.
All popups follow the same structure and have a title, a description and
two buttons for either dismissing just the current tip or all of them
at once.
The state of all seen popups is stored in a user option. Updating
skip_new_user_tips will automatically update the list of seen popups
accordingly.
The previous fix in e83d35d6 was incorrect, and the stub in the test was never actually hit. This commit moves the error handling to the right place and updates the specs to ensure the stub is always used.
* Revert "Revert "FEATURE: Preload resources via link header (#18475)" (#18511)"
This reverts commit 95a57f7e0c.
* put behind feature flag
* env -> global setting
* declare global setting
* forgot one spot
Unless we have specified `override = true` in the DeprecatedSettings
class for an old -> new settings map, we should not allow people
to change the old setting in the UI and have it affect the new
setting.
* FEATURE: Hide Privacy Policy and TOS topics
As a way to simplify new sites this change will hide the privacy policy
and the TOS topics from the topic list. They can still be accessed and
edited though.
* add tests
This can no longer be used from the user interface and could be used to
generate useless topic invites notifications. This commit adds site
setting max_topic_invitations_per_minute to prevent invite spam.
Experiment moving from preload tags in the document head to preload information the the response headers.
While this is a minor improvement in most browsers (headers are parsed before the response body), this allows smart proxies like Cloudflare to "learn" from those headers and build HTTP 103 Early Hints for subsequent requests to the same URI, which will allow the user agent to download and parse our JS/CSS while we are waiting for the server to generate and stream the HTML response.
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
Adds a new upload field for a second dark mode category logo.
This alternative will be used when the browser is in dark mode (similar to the global site setting for a dark logo).
These errors tend to indicate that the upload is missing on the remote store. This is bad, but we don't want it to block the dominant-color calculation process. This commit catches errors when there is an HTTP error, and fixes the `base_store.rb` implementation when `FileHelper.download` returns nil.
When a user with an email matching those inside the
DISCOURSE_DEVELOPER_EMAILS env var log in, we make
them into admin users if they are not already. This
is used when setting up the first admin user for
self-hosters, since the discourse-setup script sets
the provided admin emails into DISCOURSE_DEVELOPER_EMAILS.
The issue being fixed here is that the new admins were
not being automatically added to the staff and admins
automatic groups, which was causing issues with the site
settings that are group_list based that don't have an explicit
staff override. All we need to do is refresh the automatic
staff, admin groups when admin is granted for the user.
This commit makes pending reviewables show up in the main tab (a.k.a. "all notifications" tab). Pending reviewables along with unread notifications are always shown first and they're sorted based on their creation date (most recent comes first).
The dismiss button currently only shows up if there are unread notifications and it doesn't dismiss pending reviewables. We may follow up with another change soon that allows makes the dismiss button work with reviewables and remove them from the list without taking any action on them.
Follow-up to 079450c9e4.
cf. e62e93f83a
This PR also makes it so `bot` (negative ID) and `system` users are always allowed
to send PMs, since the old conditional was just based on `enable_personal_messages`
Static topics are the seeded topics that are automatically created for every Discourse instance to hold the content for the FAQ, ToS and Privacy pages. These topics are allowed to bypass the minimum title length checks when they're edited by admins:
ba27ee1637/app/assets/javascripts/discourse/app/models/composer.js (L487-L496)
However, on the server-side, the "quality title" validations aren't skipped for static topics and that can cause confusion for admins when they change the title of a static topic to something that's short enough to fail the quality title validations. This commit ignores all quality title validations on static topics when they're edited by admins.
Internal topic: t/75745.
Skipped invites were not counted at all and some invites could generate
more than one error and resulted in a grand total that was not equal to
the count of bulk invites.
* FEATURE: Make General the default category
* Set general as the default category in the composer model instead
* use semicolon
* Enable allow_uncategorized_topics in create_post spec helper for now
* Check if general_category_id is set
* Enable allow_uncategorized_topics for test env
* Provide an option to the create_post helper to not set allow_uncategorized_topics
* Add tests to check that category… is not present and that General is selected automatically
This commit adds non-archived group messages and `group_message_summary` notifications in the messages tab in the user menu. With this change, the messages tab in the user menu now includes 3 types of items:
1. Unread `private_message` notifications (notifications when you receive a reply in a PM)
2. Unread and read `group_message_summary` notifications (notifications when there's a new message in a group inbox that you track)
3. Non-archived personal and group messages
Unread `private_message` notifications are always shown first, followed by unread `group_message_summary` notifications, and then everything else (messages and read `group_message_summary` notifications) sorted by recency (most recent first).
Internal topic: t/72976.
Currently, the reviewables tab in the user menu shows pending reviewables at the top of the menu and fills the remaining space in the menu with old/handled reviewables. This PR makes the revieables tab show only pending reviewables and hides the tab altogether from the menu if there are no pending reviewables. We're going to follow-up with another change soon that will show pending reviewables in the main tab of the user menu.
Internal topic: t/73220.
By default, only staff members have to confirm their old email when
changing it. This commit adds a site setting that when enabled will
always ask the user to confirm old email.
* SECURITY: moderator shouldn't be able to import a theme via API.
* DEV: apply `AdminConstraint` for all the "themes" routes.
Co-authored-by: Vinoth Kannan <svkn.87@gmail.com>
This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality.
This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site.
Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing.
This also keeps compatibility with the `secure-media-uploads` path, and changes new
secure URLs to be `secure-uploads`.
Deprecated settings:
* secure_media -> secure_uploads
* secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails
* secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
This commit introduces rails system tests run with chromedriver, selenium,
and headless chrome to our testing toolbox.
We use the `webdrivers` gem and `selenium-webdriver` which is what
the latest Rails uses so the tests run locally and in CI out of the box.
You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra
verbose logs of what selenium is doing to communicate with the system
tests.
By default JS logs are verbose so errors from JS are shown when
running system tests, you can disable this with
`SELENIUM_DISABLE_VERBOSE_JS_LOGS=1`
You can use `SELENIUM_HEADLESS=0` to run the system
tests inside a chrome browser instead of headless, which can be useful to debug things
and see what the spec sees. See note above about `bin/ember-cli` to avoid
surprises.
I have modified `bin/turbo_rspec` to exclude `spec/system` by default,
support for parallel system specs is a little shaky right now and we don't
want them slowing down the turbo by default either.
### PageObjects and System Tests
To make querying and inspecting parts of the page easier
and more reusable inbetween system tests, we are using the
concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in
our system tests. A "Page" here is generally corresponds to
an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`,
and this contains logic for querying components within the topic
such as "Posts".
I have also split "Modals" into their own entity. Further down the
line we may want to explore creating independent "Component"
contexts.
Capybara DSL should be included in each PageObject class,
reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl
For system tests, since they are so slow, we want to focus on
the "happy path" and not do every different possible context
and branch check using them. They are meant to be overarching
tests that check a number of things are correct using the full stack
from JS and ember to rails to ruby and then the database.
### CI Setup
Whenever a system spec fails, a screenshot
is taken and a build artifact is produced _after the entire CI run is complete_,
which can be downloaded from the Actions UI in the repo.
Most importantly, a step to build the Ember app using Ember CLI
is needed, otherwise the JS assets cannot be found by capybara:
```
- name: Build Ember CLI
run: bin/ember-cli --build
```
A new `--build` argument has been added to `bin/ember-cli` for this
case, which is not needed locally if you already have the discourse
rails server running via `bin/ember-cli -u` since the whole server is built and
set up by default.
Co-authored-by: David Taylor <david@taylorhq.com>
* FEATURE: add composer warning when user haven't been seen in a long time
When a user creates a PM and adds a recipient that hasn't been seen in a
long time then we'll now show a warning in composer indicating that the
user hasn't been seen in a long time.
* FIX: Recursively tag topics with missing ancestor tags
Given only a child tag, walk up the ancestry chain, get all of it's
ancestors for use in tagging a topic
* FIX: Ensure only one parent tag is returned for topic tagging
Current implementation selects and return first parent tag if child tag
has multiple parents.
This change updates recursive parent tag implementation to only return
parent tags via only one ancestry line.
* DEV: Add test case for tag cycles
Given we aren't performing a strict graph traversal to get a tag's
parent, cycles do not have any effect on the tags returned for topic
tagging.
By default, we won't include associated account ids in current user serializer. If the new hidden site setting `include_associated_account_ids` is enabled then we will add it in the serializer.
This will replace `enable_personal_messages` and
`min_trust_to_send_messages`, this commit introduces
the setting `personal_message_enabled_groups`
and uses it in all places that `enable_personal_messages`
and `min_trust_to_send_messages` currently apply.
A migration is included to set `personal_message_enabled_groups`
based on the following rules:
* If `enable_personal_messages` was false, then set
`personal_message_enabled_groups` to `3`, which is
the staff auto group
* If `min_trust_to_send_messages` is not default (1)
and the above condition is false, then set the
`personal_message_enabled_groups` setting to
the appropriate auto group based on the trust level
* Otherwise just set `personal_message_enabled_groups` to
11 which is the TL1 auto group
After follow-up PRs to plugins using these old settings, we will be
able to drop the old settings from core, in the meantime I've added
DEPRECATED notices to their descriptions and added them
to the deprecated site settings list.
This commit also introduces a `_map` shortcut method definition
for all `group_list` site settings, e.g. `SiteSetting.personal_message_enabled_groups`
also has `SiteSetting.personal_message_enabled_groups_map` available,
which automatically splits the setting by `|` and converts it into
an array of integers.
See https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499
for thorough reasoning.
This commit changes how we generate Message-IDs and do email
threading for emails sent from Discourse. The main changes are
as follows:
* Introduce an outbound_message_id column on Post that
is either a) filled with a Discourse-generated Message-ID
the first time that post is used for an outbound email
or b) filled with an original Message-ID from an external
mail client or service if the post was created from an
incoming email.
* Change Discourse-generated Message-IDs to be more consistent
and static, in the format `discourse/post/:post_id@:host`
* Do not send References or In-Reply-To headers for emails sent
for the OP of topics.
* Make sure that In-Reply-To is filled with either a) the OP's
Message-ID if the post is not a direct reply or b) the parent
post's Message-ID
* Make sure that In-Reply-To has all referenced post's Message-IDs
* Make sure that References is filled with a chain of Message-IDs
from the OP down to the parent post of the new post.
We also are keeping X-Discourse-Post-Id and X-Discourse-Topic-Id,
headers that we previously removed, for easier visual debugging
of outbound emails.
Finally, we backfill the `outbound_message_id` for posts that have
a linked `IncomingEmail` record, using the `message_id` of that record.
We do not need to do that for posts that don't have an incoming email
since they are backfilled at runtime if `outbound_message_id` is missing.
The `add_column` `limit` parameter has no effect on a postgres `text` column. Instead we can perform the check in ActiveRecord.
We never expect this condition to be hit - users cannot control this value. It's just a safety net.
This commits introduces a new SiteSetting.enable_new_user_profile_nav_groups
feature flag. When configured, users of the configured groups will see
the new user page navigation links.
As of this commit, only the user activity navigation link has been
converted to the newly proposed dropdown of navigation links.
Mobile support has not been considered.
Adds limits to location and website fields at model and DB level
to match the bio_raw field limits. A limit cannot be added at the
DB level for bio_raw because it is a postgres text field.
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images.
This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly.
Dominant color will be calculated:
1. When a new upload is created
2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored
3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job)
Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
Upgrading to Markdown.it v13 broke empty inline BBCodes. This works around the problem by adding an empty token before a closing token if the previous token was a BBCode token.
It also removes the unused `jump` attribute which was removed in Markdown.it v12.3
When tagging is enabled, we were correctly serializing tags by their name. However, when tagging was disabled we were attempting to serialize an entire Tag object which raises an error since ee07f6da7d.
https://meta.discourse.org/t/232885
`delete_previous!` deletes existing topics even when we cannot send a new one due to the `limit_once_per` option. The dashboard problems PM gets deleted the next time the job runs (30 minutes), so the inbox could be empty when
admins click on the summary notification.
Right now the experimental user menu sorts notifications the same way that the old menu does: unread high-priority notifications are shown first in reverse-chronological order followed by everything else also in reverse-chronological order. However, since the experimental user menu has dedicated tabs for some notification types and each tab displays a badge with the count of unread notifications in the tab, we feel like it makes sense to change how notifications are sorted in the experimental user menu to this:
1. unread high-priority notifications
2. unread regular notifications
3. all read notifications (both high-priority and regular)
4. within each group, notifications are sorted in reverse-chronological order (i.e. newest is shown first).
This new sorting logic applies to all tabs in the experimental user menu, however it doesn't change anything in the old menu. With this change, if a tab in the experimental user menu shows an unread notification badge for a really old notification, it will be surfaced to the top and prevents confusing scenarios where a user sees an unread notification badge on a tab, but the tab doesn't show the unread notification because it's too old to make it to the list.
Internal topic: t72199.
