### What is the problem?
We have recently added a new option to add user fields required for existing users. This is in contrast to requiring fields only on sign-up.
This revealed an existing problem. Consider the following:
1. User A signs up.
2. Admin adds a new user field required on sign-up. (Should not apply to User A since they already signed up.)
3. User A tries to update their profile.
**Expected behaviour:**
No problem.
**Actual behaviour:**
User A receives an error saying they didn't fill up all required fields.
### How does this fix it?
When updating profile, we only check that required fields that are "for all users" are filled. Additionally, we check that fields that were required on sign-up and have previously been filled are not blanked out.
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.
Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.
We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.
The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.
Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.
A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.
The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
Followup 3ff7ce78e7
Basing this setting on referrer was too brittle --
the referrer header can easily be ommitted or changed.
Instead, for the small amount of use cases that this
site setting serves, we can use a group-based setting
instead, changing it to `cross_origin_opener_unsafe_none_groups`
instead.
Background:
In order to redrive failed webhook events, an operator has to go through and click on each. This PR is adding a mechanism to retry all failed events to help resolve issues quickly once the underlying failure has been resolved.
What is the change?:
Previously, we had to redeliver each webhook event. This merge is adding a 'Redeliver Failed' button next to the webhook event filter to redeliver all failed events. If there is no failed webhook events to redeliver, 'Redeliver Failed' gets disabled. If you click it, a window pops up to confirm the operator. Failed webhook events will be added to the queue and webhook event list will show the redelivering progress. Every minute, a job will be ran to go through 20 events to redeliver. Every hour, a job will cleanup the redelivering events which have been stored more than 8 hours.
Followup to 527f02e99f,
I had to introduce defer_track_visit_v2 because discourse-docs
relied on defer_track_visit. Now that discourse-docs
is using the new method as of
discourse/discourse-docs@0d9365571b,
we can rename it in core. Then we will need one more PR
in both core and docs to remove usage of the "v2" method.
Add a new column - `user_agent` - to the `SearchLog` table.
This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
SiteSetting.hide_user_profiles_from_public raises a Forbidden, which disallows our after_action: add no index header from triggering.
This fix makes sure that the no index header gets added via before_action instead
Followup 2f2da72747
This commit moves topic view tracking from happening
every time a Topic is requested, which is susceptible
to inflating numbers of views from web crawlers, to
our request tracker middleware.
In this new location, topic views are only tracked when
the following headers are sent:
* HTTP_DISCOURSE_TRACK_VIEW - This is sent on every page navigation when
clicking around the ember app. We count these as browser page views
because we know it comes from the AJAX call in our app. The topic ID
is extracted from HTTP_DISCOURSE_TRACK_VIEW_TOPIC_ID
* HTTP_DISCOURSE_DEFERRED_TRACK_VIEW - Sent when MessageBus initializes
after first loading the page to count the initial page load view. The
topic ID is extracted from HTTP_DISCOURSE_DEFERRED_TRACK_VIEW.
This will bring topic views more in line with the change we
made to page views in the referenced commit and result in
more realistic topic view counts.
By default, secure sessions expire after 1 hour.
For OAuth authentication it should expire at the same time when the authentication cookie expires - `SiteSetting.maximum_session_age.hours`.
It is possible that the forum will not have persistent sessions, based on `persistent_sessions` site setting. In that case, with next username and password authentication we need to reset information about OAuth.
Bug introduced in this PR - https://github.com/discourse/discourse/pull/27547
I am changing many of these to notes or resolving them as is,
most of these I have not actively worked on in years so someone
else can work on them when we get to these areas again.
This commit continues work laid out by ffec8163b0 for the admin config page for the /about page. The last commit set up the user interface, and this one sets up all the wiring needed to make the input fields and save buttons actually work.
Internal topic: t/128544.
When bad data is provided in the URI for redirecting to a category,
Rails raises an `ActionController::Redirecting::UnsafeRedirectError`
error, leading to a 500 error.
This patch catches the exception to render a 404 instead.
Currently redirecting to an external URL through a permalink doesn’t
work because Rails raises a
`ActionController::Redirecting::UnsafeRedirectError` error.
This wasn’t the case before we upgraded to Rails 7.0.
This patch fixes the issue by using `allow_other_host: true` on the
redirect.
If, for whatever reasons, the user's locale is "blank" and an admin is accepting their group membership request, there will be an error because we're generating posts with the locale of recipient.
In order to fix this, we now use the `user.effective_locale` which takes care of multiple things, including returning the default locale when the user's locale is blank.
Internal ref - t/132347
If the `enforce_second_factor_on_external_auth` setting
is disabled and a user logs in with an OAuth method,
they don't automatically get redirected to /preferences/second-factor
on login. However, they can get there manually, and once there
they cannot leave.
This commit fixes the issue and allows them to leave
and also does some refactors to indicate to the client
what login method is used as a followup to
0e1102b332
We want to allow admins to make new required fields apply to existing users. In order for this to work we need to have a way to make those users fill up the fields on their next page load. This is very similar to how adding a 2FA requirement post-fact works. Users will be redirected to a page where they can fill up the remaining required fields, and until they do that they won't be able to do anything else.
Adds a checkbox to filter untranslated text strings in the admin UI, behind a hidden and default `false` site setting `admin_allow_filter_untranslated_text`.
Followup to 0e1102b332
Minor followup, makes the condition check against the
boolean val, see the difference here:
```ruby
!SiteSetting.enforce_second_factor_on_external_auth && "true"
=> "true"
```
vs:
```ruby
!SiteSetting.enforce_second_factor_on_external_auth && "true" == "true"
=> true
```
This ensures that the theme id is resolved as early as possible in the
request cycle. This is necessary for the custom homepage to skip
preloading the wrong data.
In this PR we introduced `enforce_second_factor_on_external_auth` setting https://github.com/discourse/discourse/pull/27506
When it is set to false and the user is authenticated via OAuth, then we should not enforce the 2fa configuration.
Some tooling may rely on an unsafe-none cross origin opener policy to work. This change adds a hidden site setting that can be used to list referrers where we add this header instead of the default one configured in cross_origin_opener_policy_header.
A new admin setting called `enforce_second_factor_on_external_auth`. It allows users to authenticate using external providers even when 2FA is forced with `enforce_second_factor` site setting.
* Load search results in displayed order so that when more categories are loaded on scroll, they appear at the end,
* Limit the number of subcategories that are shown per category and display 'show more' links,
This commit adds ability to fetch a subset of site settings from the `/admin/site_settings` endpoint so that it can be used in all places where the client app needs access to a subset of the site settings.
Additionally, this commit also introduces a new service class called `UpdateSiteSetting` that encapsulates all the logic that surrounds updating a site setting so that it can be used to update site setting(s) anywhere in the backend. This service comes in handy with, for example, the controller for the flags admin config area which may need to update some site settings related to flags.
Internal topic: t/130713.
After working on the Webhook events filter by Status, I noticed that the 'Delivered' and 'Failed' options do not take the status param when loading more than fifty Webhook events. It causes to load all Webhook events regardless of its status after the first load.
This PR is adding webhook events status for the filter to the param when loading more than fifty Webhook events.
* FEATURE: Add Filter for Webhook Events by Status
* Fixing multiple issues
* Lint
* Fixing multiple issues
* Change the range of the status for webhook events
In some admin user controller extensions, @user is used to derive certain values.
The grant_admin method requires @user as well, so we are adding it here. This is tested in the plugin that it is used in.
This commit fixes a problem where the user will not be able to reset
their password when they only have security keys and backup codes
configured.
This commit also makes the following changes/fixes:
1. Splits password reset system tests to
`spec/system/forgot_password_spec.rb` instead of missing the system
tests in `spec/system/login_spec.rb` which is mainly used to test
the login flow.
2. Fixes a UX issue where the `Use backup codes` or `Use authenticator
app` text is shown on the reset password form when the user does
not have either backup codes or an authenticator app configured.
This commit includes various UX improvements to the reset
password page:
* Introduce a `hide-application-header-buttons` helper to do the following:
* Hide Sign Up and Log In buttons, they are not necessary on this flow
* Hide the sidebar, it is a distraction on this flow
* Improve messaging when a 2FA confirmation is required first
* Improve display of server-side ActiveRecord model validation errors
in password form, e.g. instead of "is the same as your current password"
we do "The password is the same as your current password"
* Move password tip to next line below input and move caps lock hint
inline with Show/Hide password toggle
* Add system specs for 2FA flow on reset password page
* Fixes a computed property conflict issue on the password reset
page when toggling 2FA methods
Continued work on moderate flags UI.
In this PR admins are allowed to change the order of flags. The notify user flag is always on top but all other flags can be moved.
This makes it more obvious what's happening, and makes it much less likely that users will send repeated reset emails (and thereby hit the rate limit)
Followup to e97ef7e9af
This commit adds the ability for site administrators to mark users'
passwords as expired. Note that this commit does not add any client side
interface to mark a user's password as expired.
The following changes are introduced in this commit:
1. Adds a `user_passwords` table and `UserPassword` model. While the
`user_passwords` table is currently used to only store expired
passwords, it will be used in the future to store a user's current
password as well.
2. Adds a `UserPasswordExpirer.expire_user_password` method which can
be used from the Rails console to mark a user's password as expired.
3. Updates `SessionsController#create` to check that the user's current
password has not been marked as expired after confirming the
password. If the password is determined to be expired based on the
existence of a `UserPassword` record with the `password_expired_at`
column set, we will not log the user in and will display a password
expired notice. A forgot password email is automatically send out to
the user as well.
This commit removes the `/admin-revamp` routes which were introduced as a part of an experiment to revamp the admin pages. We still want to improve the admin/staff experience, but we're going to do them within the existing `/admin` routes instead of introducing a completely new route.
Our initial efforts to improve the Discourse admin experience is this commit which introduces the foundation for a new subroute `/admin/config` which will house various new pages for configuring Discourse. The first new page (or "config area") will be `/admin/config/about` that will house all the settings and controls for configuring the `/about` page of Discourse.
Internal topic: t/128544
The UsersController#modify_user_params method is deprecated and replaced with a plugin modifier (users_controller_update_user_params). It is marked for removal in 3.2. This PR removes it.
For some reason, despite iframe also indicating a
```
<meta name="robots" content="noindex">
```
.. Google is still indexing the embed/comment URLs. This causes links like http://\<site>/embed/comments\?topic_id\=6366 to be indexed instead of the topic.
This commit adds it explicitly in the header.
This commit moves the logic for crawler rate limits out of the application controller and into the request tracker middleware. The reason for this move is to apply rate limits to all crawler requests instead of just the requests that make it to the application controller. Some requests are served early from the middleware stack without reaching the Rails app for performance reasons (e.g. `AnonymousCache`) which results in crawlers getting 200 responses even though they've reached their limits and should be getting 429 responses.
Internal topic: t/128810.
This gives us daily fidelity of topic view stats
New table stores a row per topic viewed per day tracking
anonymous and logged on views
We also have a new endpoint `/t/ID/views-stats.json` to get the statistics for the topic.
We're planning to implement a feature that allows adding required fields for existing users. This PR does some preparatory refactoring to make that possible. There should be no changes to existing behaviour. Just a small update to the admin UI.
This PR introduces a basic AdminNotice model to store these notices. Admin notices are categorized by their source/type (currently only notices from problem check.) They also have a priority.
This PR aims to add bulk actions to the user's bookmarks.
After this feature, all users should be able to select multiple bookmarks and perform the actions of "deleting" or "clear reminders"
Instead of creating two separate Topics when a user (1) requests to join a group and (2) gets accepted in, this makes the acceptance message into a Post under the origin group request Topic.
Note this may have performance issues in some cases, will need to be monitored
Previous to this change we were bracketing on 50 id windows. They may end up
having zero posts we are searching for leading to posts.rss and .json returning
no results.
- avoids Post.last.id which is expensive
- order by id desc which is better cause we bracket on id
In 07ecbb5a3b we ensure the mentions in a group's activity page worked properly but we missed adding proper support for infinite loading.
The client is using the `before` parameter instead of the `before_post_id` to do the pagination.
This adds support for `before` as well as some tests to ensure it doesn't regress.
I also added tests to the group's activity posts as well since those were missing.
Finally I deleted some unused code (`group.messages_for`) which is not used anymore.
Context - https://meta.discourse.org/t/-/308044/9
When converting a PM to a public topic (and vice versa), if there was a validation error (like a topic already used, or a tag required or not allowed) the error message wasn't bubbled up nor shown to the user.
This fix ensures we properly stop the conversion whenever a validation error happens and bubble up the errors back to the user so they can be informed.
Internal ref - t/128795
Using the CategoryDrop on the categories page redirected the user to the
"latest topics" page with topics only from that category. With these
changes, selecting a category will take the user to a "subcategories
page" where only the subcategories of the selected property will be
displayed.
The watched word group's create, update and delete action logs were missing the translations. This PR will add those strings and will use the group key instead of watched word key where needed.
... wasn't working because it wasn't storing the proper "action" value.
Issue was that we were using the "action" parameter which is being used by Rails to determine which controller action to call.
We need to use the "action_key" parameter instead.
At the moment, there is no way to create a group of related watched words together. If a user needed a set of words to be created together, they'll have to create them individually one at a time.
This change attempts to allow related watched words to be created as a group. The idea here is to have a list of words be tied together via a common `WatchedWordGroup` record. Given a list of words, a `WatchedWordGroup` record is created and assigned to each `WatchedWord` record. The existing WatchedWord creation behaviour remains largely unchanged.
Co-authored-by: Selase Krakani <skrakani@gmail.com>
Co-authored-by: Martin Brennan <martin@discourse.org>
This commit introduces a few changes as a result of
customer issues with finding why a topic was relisted.
In one case, if a user edited the OP of a topic that was
unlisted and hidden because of too many flags, the topic
would get relisted by directly changing topic.visible,
instead of going via TopicStatusUpdater.
To improve tracking we:
* Introduce a visibility_reason_id to topic which functions
in a similar way to hidden_reason_id on post, this column is
set from the various places we change topic visibility
* Fix Post#unhide! which was directly modifying topic.visible,
instead we use TopicStatusUpdater which sets visibility_reason_id
and also makes a small action post
* Show the reason topic visibility changed when hovering the
unlisted icon in topic status on topic titles
Selecting the +subcategories option does not work sometimes when "lazy
load categories" is enabled because the subcategories may not be
fetched. This ensures that subcategories are loaded by requesting them
before being used.
Our 'page_view_crawler' / 'page_view_anon' metrics are based purely on the User Agent sent by clients. This means that 'badly behaved' bots which are imitating real user agents are counted towards 'anon' page views.
This commit introduces a new method of tracking visitors. When an initial HTML request is made, we assume it is a 'non-browser' request (i.e. a bot). Then, once the JS application has booted, we notify the server to count it as a 'browser' request. This reliance on a JavaScript-capable browser matches up more closely to dedicated analytics systems like Google Analytics.
Existing data collection and graphs are unchanged. Data collected via the new technique is available in a new 'experimental' report.
Fixes two issues:
- frontend was reloading the page when clicking-to-remove avatar
- backend wasn't allowing resetting the setting by deleting all avatars
For better performances when listing all the API keys.
Loading all the "api key scopes" is slow and not required when showing the list of all the api keys.
- Run the CSP-nonce-related middlewares on the generated response
- Fix the readonly mode checking to avoid empty strings being passed (the `check_readonly_mode` before_action will not execute in the case of these re-dispatched exceptions)
- Move the BlockRequestsMiddleware cookie-setting to the middleware, so that it is included even for unusual HTML responses like these exceptions
Full text search does not return ideal results for category dropdown.
Usually, in category dropdowns we want to search for categories as we
type. For example, while typing "theme", the dropdown should show
intermediary results for "t", "th", "the", "them" and finally "theme".
For some of these substrings (like "the"), full text search does not
return any results, which leads to an unpleasant user experience.
The breadcrumbs were updated everytime there were changes to the
categories which was not efficient and caused unnecessary rerendering
of the CategoryDrop elements when "lazy load categories" is enabled.
This commit also ensures that all category fields are serialized for
ancestors too for the categories#search endpoint.
Include categories when fetching admin/web_hooks and make
'extras' more useful. 'extras' is the mechanism we use to provide
context for rest objects.
However, previously:
* When you fetched many objects, extras was only set on the ResultSet,
not on each object,
* If you need derived data from extras, there wasn't a sensible place to
put this code. Now, you can create an 'ExtrasClass' static field on
your rest model and this class will be used for your extras data,
When lazy load categories is enabled, categories should be loaded with
user activity items and drafts because the categories may not be
preloaded on the client side.
Using around_action means `add_early_hint_header` is in the stack for every request, and gets included in the backtrace of any errors.
We can manage with an after_action instead, which avoids adding to the stack depth (and avoids people blaming me for unrelated application errors 😉)
We will be collecting the logo URL and the site's default locale values along with existing basic details to display the site on the Discourse Discover listing page. It will be included only if the site is opted-in by enabling the "`include_in_discourse_discover`" site setting.
Also, we no longer going to use `about.json` and `site/statistics.json` endpoints retrieve these data. We will be using only the `site/basic-info.json` endpoint.
Plugins that are hidden or disabled aren't shown in the plugins list at `/admin/plugins` because they cannot be changed. However, the `#show` route doesn't check for the plugin's state and responds with 200 and the plugin's info even if the plugin is hidden or disabled. This commit makes the `#show` route respond with 404 if the plugin is hidden or disabled.
Why this change?
Before this change, the validation error message shown to the user when
saving a theme objects setting is very cryptic. This commit changes the
validation error messages to be displayed on top of the editor instead.
Note that I don't think this way of displaying is the ideal state we
want to get to but given the time we have this will do for now.
This commit removes the 'experimental_preconnect_link_header' site setting, and the 'preload_link_header' site setting, and introduces two new global settings: early_hint_header_mode and early_hint_header_name.
We don't actually send 103 Early Hint responses from Discourse. However, upstream proxies can be configured to cache a response header from the app and use that to send an Early Hint response to future clients.
- `early_hint_header_mode` specifies the mode for the early hint header. Can be nil (disabled), "preconnect" (lists just CDN domains) or "preload" (lists all assets).
- `early_hint_header_name` specifies which header name to use for the early hint. Defaults to "Link", but can be changed to support different proxy mechanisms.
Why this change?
Previously, we were preloading the necessary metadata for
`adminCustomizeThemes.show.schema` route in the
`adminCustomizeThemes.show` route. This is wasteful because we're
loading data upfront when the objects setting editor may not be used.
This change also lays the ground work for a future commit where we need
to be shipping down additional metadata which may further add to the
payload.
Previously, when crawlers triggered a Discourse::InvalidAccess exception, they would be served the full Ember SPA. The SPA is not optimized for crawling, and so this is likely to cause problems for sites. This issue is particularly problematic when user profiles are hidden from the public via the `hide_user_profiles_from_public` setting, because the crawler would end up being 'soft-redirected' to the homepage in the SPA.
Why this change?
In https://web.dev/articles/preconnect-and-dns-prefetch, it describes
how hinting to the browser to preconnect to domains which we will
eventually use the connection for can help improve the time it takes to
load a page.
We are putting this behind an experimental flag so that we can test and
profile this in a production environment.
What does this change introduce?
Introduce a hidden experimental `experimental_preconnect_link_header`
site setting which when enabled will add the `preconnect` and
`dns-prefetch` resource hints to the response headers for full page load
requests.
This commit makes it so the site settings filter controls and
the list of settings input editors themselves can be used elsewhere
in the admin UI outside of /admin/site_settings
This allows us to provide more targeted groups of settings in different
UI areas where it makes sense to provide them, such as on plugin pages.
You could open a single page for a plugin where you can see information
about that plugin, change settings, and configure it with custom UIs
in the one place.
In future we will do this in "config areas" for other parts of the
admin UI.
This commit operates at three levels of abstraction:
1. We want to prevent user history rows from being unbounded in size.
This commit adds rails validations to limit the sizes of columns on
user_histories,
2. However, we don't want to prevent certain actions from being
completed if these columns are too long. In those cases, we truncate
the values that are given and store the truncated versions,
3. For endpoints that perform staff actions, we can further control
what is permitted by explicitly validating the params that are given
before attempting the action,
