his is a new feature that lets admins dismiss notices from the dashboard. This helps with self-service in cases where a notice is "stuck", while we work on provisions to prevent "sticking" in the first place.
In TopicController, in addition to ensure_can_move_posts!, we also
checked if the topic is private message in this line:
```ruby
raise Discourse::InvalidAccess if params[:archetype] == "private_message" && !guardian.is_staff?
```
However, this was not present in `guardian.can_move_posts?`. As a result,
the frontend topic view got an incorrect serialized result, thinking
that TL4 could move the private message post. In fact, once they tried
to move it, they got the `InvalidAccess` error message.
This commit fixes that TL4 will no longer sees the "move to" option in
the "select post" panel for a private message.
Anonymous users are "shadow" users created when an existing real user desires to post anonymously. This feature is off by default, but it can be enabled via the `allow_anonymous_posting` site setting. Those shadow users shouldn't be included in the users directory (`/u`).
In 14cf8eacf1, we added the
`user_search_similar_results` site setting which when enabled will use
trigram matching for similarity search in `UserSearch`. However, we
noted that adding the `index_users_on_username_lower_trgm` index is
causing the PG planner to not use the `index_users_on_username_lower`
index when the `=` operator is used against the `username_lower` column.
Based on the PG mailing list discussion where support for the `=`
operator in gist_trgm_ops was being considered, it stated that "I also have checked that btree_gist is preferred over pg_trgm gist
index for equality search." This is however quite different from reality
on our own PG clusters where the btree index is not preferred leading to
significantly slower queries when the `=` operator is used.
Since the pg_trgm gist index is only used for queries when the `user_search_similar_results` site setting
is enabled, we decided to drop the feature instead as it is hidden and
disabled by default. As such, we can consider it experiemental and drop
it without deprecation.
PG mailing list discussiong: https://www.postgresql.org/message-id/CAPpHfducQ0U8noyb2L3VChsyBMsc5V2Ej2whmEuxmAgHa2jVXg%40mail.gmail.com
The user directory (`/u`) excludes inactive and silenced users from the list, so for the sake parity, it makes sense to also exclude those users from the /about page stats.
Internal topic: t/70928.
- fetch models inside services
- validate `user_id` in contracts
- use policy objects
- extract more logic to actions
- write specs for services and action
Adds support for `input-date` field when calling
`fill_in` on a FormKit field. Capybara supports passing
a Date object to `fill_in(with: value)` for date inputs,
so there is nothing fancy that needs to be done to support this.
Followup 0323b366f3
This was happening because another spec was adding a
report using the plugin API, but there was nothing
resetting that, so later in the reports controller
when we did Report.singleton_methods, we ended up
with another report with no translation, causing another
error.
The user option 'hide_profile_and_presence' is necessary to figure out
if the user status has to be displayed or not. In order to avoid N+1s
generated by `include_status?` method, both `user_status` and
`user_option` relations have to be included.
`track_sql_queries` only returned queries that were executed by
ActiveRecord. All queries executed through DB.exec, DB.query and others
were not returned.
Currently, when the custom flag has the same name as the system flag (which is disabled) then it is not displayed. To fix the problem, `custom_` prefix as `name_key` is used to distinguish between the system and the custom flag.
I considered writing a migration to fix existing custom flags name key. However, at the end of migration I would need to run rails code to reset cache `Flag.reset_flag_settings!`. I decided to skip that step as it is a very edge case. If someone has the same flag name as the system flag, then all they have to do is edit the flag and click save.
In addition, I made 2 small fixes:
- edit flag title was missing translation;
- flag form UI was not showing that description is the required field.
The Report model spec was directly adding methods
to the Report class, which was causing errors in the
admin reports controller because it would look for
a translation of the report name (e.g. report_timeout_test)
like so `I18n.t("reports.#{type}.title")`, then get an
error because the translation did not exist.
This is fixed by using `Report.stubs` instead, which is
cleaned up after every test.
This commit adds a new `about_page_hidden_groups` setting to exclude members of specific groups from the admin and moderator lists on the /about page.
Internal topic: t/137717.
### UI changes
All of the UI changes described are gated behind the `use_legacy_pageviews`
site setting.
This commit changes the admin dashboard pageviews report to
use the "Consolidated Pageviews with Browser Detection" report
introduced in 2f2da72747 with
the following changes:
* The report name is changed to "Site traffic"
* The pageview count on the dashboard is counting only using the new method
* The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews"
* By default "known crawlers" and "other" sources of pageviews are hidden on the report
When `use_legacy_pageviews` is `true`, we do not show or allow running
the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`,
we do not show or allow running the following legacy reports:
* consolidated_page_views
* consolidated_page_views_browser_detection
* page_view_anon_reqs
* page_view_logged_in_reqs
### Historical data changes
Also part of this change is that, since we introduced our new "Consolidated
Pageviews with Browser Detection" report, some admins are confused at either:
* The lack of data before a certain date , which didn’t exist before
we started collecting it
* Comparing this and the current "Consolidated Pageviews" report data,
which rolls up "Other Pageviews" into "Anonymous Browser" and so it
appears inaccurate
All pageview data in the new report before the date where the _first_
anon or logged in browser pageview was recorded is now hidden.
This upgrade is designed to be fully backwards-compatible. Any icon names which have changed will be automatically remapped to the new name. For now, this will happen silently. In future, once core & official themes/plugins have been updated, we will start raising deprecation errors to help theme/plugin authors update their code.
Extracted from https://github.com/discourse/discourse/pull/28715
Announcement at https://meta.discourse.org/t/were-upgrading-our-icons-to-font-awesome-6/325349
Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
* FEATURE: Log tag group changes in staff action log
This commit records every change (add, change, delete) to a tag group in
the staff action log.
It uses a modal that was originally called ThemeChangeModal to display
changes, allowing staffs to see the specific changes clearly. The modal
is renamed to StaffActionLogChangeModal in this PR.
ref: https://meta.discourse.org/t/-/325011/14
Co-authored-by: Keegan George <kgeorge13@gmail.com>
This was added 10 years ago, but currently there's not a single use in our public and private plugins and no reference in third-party plugins on github
* FEATURE: Add user to topic_tags_changed event
Add user to topic_tags_changed event context
Update automation plugin with new arguments in event
Update tests for new arguments
relates to https://github.com/discourse/discourse-chat-integration/pull/214
* DEV: change variable name for better readability
changed `tags` to be payload and used `values_at` to get the values of the keys
This commit adds a link to the top of the new /about page, shown to admins only, to allow them to easily navigate to `/admin/config/about` where they can edit the /about page.
Internal topic: t/137546.
This patch removes the `with_service` helper from the code base.
Instead, we can pass a block with actions directly to the `.call` method
of a service.
This simplifies how to use services:
- use `.call` without a block to run the service and get its result
object.
- use `.call` with a block of actions to run the service and execute
arbitrary code depending on the service outcome.
It also means a service is now “self-contained” and can be used anywhere
without having to include a helper or whatever.
Currently, categories support designating only 1 group as a moderation group on the category. This commit removes the one group limitation and makes it possible to designate multiple groups as mods on a category.
Internal topic: t/124648.
This commit introduces a new hidden site setting: `group_pm_user_limit`, default to `1000` which will raise an error when attempting to create a PM target a large group.
This will bring significant improvements to install speed & storage requirements. For information on how it may affect you, see https://meta.discourse.org/t/324521
This commit:
- removes the `yarn.lock` and replaces with `pnpm-lock.yaml`
- updates workspaces to pnpm format
- adjusts package dependencies to work with pnpm's stricter resolution strategy
- updates Rails app to load modules from more specific node_modules directories
- adds a `.pnpmfile` which automatically cleans up old yarn-managed `node_modules` directories
- updates various scripts to call `pnpm` instead of `yarn`
- updates patches to use pnpm's native patch system instead of patch-package
- adds a patch for licensee to support pnpm
Currently, when the default locale is Japanese, the search for a topic
using its URL, path or ID doesn’t work as expected. It will either
return wrong results or no result at all.
The problem lies with how we process the provided terms in Japanese
mode. For example, if `http://localhost/t/-/55` is provided, currently
this will result in `http localhost t 5 5` to be searched for.
This patch addresses the issue by checking whether the provided term
needs segmenting. If the provided term is a number, or a path or a full
URL, then it doesn’t need segmenting. When that happens we skip the
processing we normally apply for Japanese, making the search return the
expected results.
TagUser.rb is used to set user notification levels for a tag, we don't have a unique index on the notification level itself. This means that there might be some weird case where a user may have multiple of the same notification level on a tag.
This PR adds a migration which de-duplicates this based on defaults, where we keep the earliest record in the event there is multiple notification level per-user-per-tag.
* add data migration to keep only unexpired or most recently expired user password
* refactor to 1:1 relationship between User and UserPassword
* add migration to remove redundant indexes on user passwords
A new setting attribute is used to define the areas (separated by `|`).
In addition, endpoint `/admin/config/site_settings.json` accepts new `filter_area` data.
A previous refactor has moved this function in the controller instead of the route making it inaccessible to the modal.
This commit is fixing this and also adding a spec.
This change introduces a new thread notification level allowing users to get notified when someone replies to the thread.
Users who watch a thread will get a green notification on the chat icon and a user notification (blue). User notifications are consolidated based on thread id to prevent cluttering the original users notification area.
---------
Co-authored-by: Régis Hanol <regis@hanol.fr>
This commit will allow plugin developers to enable/disable the custom homepage.
Usage:
```ruby
register_modifier(:custom_homepage_enabled) do |enabled, args|
true
end
```
Args might contain request and/or current_user.
This commit introduces a little bit of duplication
since the old plugin UIs not using the new plugin show
page look different from ones like AI and Gamification
which have been converted. We can use the new admin
header component on the plugins list, but for the other
pages we are manually rendering a breadcrumb trail and
the list of plugin tabs.
Over time as we convert more plugins to use the new UI
guidelines and show page we can get rid of this duplication.
* DEV: Split slow test in multiple smaller tests
This might be faster because the smaller chunks of the test may run in
parallel.
* DEV: Fabricate reviewables only once
* Do not delete created external upload stubs for 2 days
instead of 1 hour if enable_upload_debug_mode is true,
this aids with server-side debugging.
* If using an API call, return the detailed error message
if enable_upload_debug_mode is true. In this case the user
is not using the UI, so a more detailed message is appropriate.
* Add a prefix to log messages in ExternalUploadHelpers, to
make it easier to find these in logster.
Prior to this fix we were calling `fetch_stats` which is never checking if we have a cache entry. This call is making a lot of SQL calls, so it's better to use the cache.
We're seeing a lot of log noise coming from unhandled exceptions stemming from requests to TopicsController#show where id is passed in as an array.
In the implementation of the method, we assume that if id is present it will be a string. This is because one of the routes to this action uses :id as a URL fragment, and so must be a string. However, there are other routes that go to this endpoint as well. Some of them don't have this URL fragment, so you can pass an arbitrary id query parameter.
Instead of a downstream unhandled exception, we raise a Discourse::InvalidParameters upfront.
In this PR we introduced a new setting `enforce_second_factor_on_external_auth` which disables enforce 2FA when the user is authenticated with an external provider.
https://github.com/discourse/discourse/pull/27506
However, with the first registration with an external provider, we authenticate the user right after activation. In that case, we need to also keep information that the user was authenticated with an external OAuth provider.
We were not updating `searchTerm` when changing the input which was making us always send an empty q parameter.
This commit is also adding tests for:
- initial url with q param
- filtering the bookmarks through the input
Makes it easier to reach the group from the category security
tab, and moves the trash button to the right to avoid misclicks.
Also converts the category permission row to gjs
This is a follow-up of d749227e87.
This patch checks if the key `not_found` is present on the result object
instead of calling `#blank?` on the model, as it can trigger an
`ActiveRecord` relation.
To test the restricted routing when filling up required fields, we fill up the field and then navigate to the root path and checking that we're not redirected.
This is somewhat flaky, and the screenshot shows we are back at the profile page, without any prompt to fill up fields.
My hypothesis is in cases where the backend is "slow" to respond, we're navigating away from the page before the request finishes (which will redirect back to the profile page.)
This PR adds an expectation after saving, to wait until the unrestricted profile page is rendered, before navigating away.
This commit changes the cutoff number for the admins and mods lists on the new /about page from 12 to 6. If the admins or mods lists are bigger than 6, the about page will display the 6 most recently seen admins/mods, and tuck the rest away behind a "view more" button.
Given we are a single-page-app, we hijack the browser's default behavior
for keyboard shortcuts like End because on long topics (20+ posts) we
need to load the last post before really reaching the end of the page.
However, when the last post is already rendered, the End shortcut
currently does nothing, it takes the user to the start of the last post.
If that post is too long, the user will have to scroll down manually.
This change ensures that if the last post of a topic is already rendered
(whether it is in the viewport or not), pressing End will take the user
to the very bottom of the page.
Note for the reviewer: the test added here is for the general case, it
is too hard to test the case where the last post is already rendered,
that isn't covered here.
Follow-up to e3ae57ea7a
The previous commit added an `after_create` callback that triggers a refresh for the user directory whenever a `User` record is created. Theoretically, this approach should work, however, there's a gotcha in practice, because during a real user registration, when the `User` record is created in the database, it's not marked as active until the user verifies their email address and the user directory excludes inactive users, so the initial directory refresh triggered by the `after_create` callback becomes pointless.
To make a new user appear in the user directory immediately after sign up, we need to trigger a refresh via an `after_save` callback when they verify their email address and become active.
Currently, `FlagSerializer#short_description` properly provides
`base_path` to its translations, but `FlagSerializer#description` does
not. This breaks the link to guidelines when flagging a post, for
example.
This patch provides `base_path` for `FlagSerializer#description` too.
Prior this fix a topic created in the future could generate this exception:
```
Job exception: ERROR: a negative number raised to a non-integer power yields a complex result
```
This fix and spec should ensure we don't consider topics in the future as the rest of the `update_scores` function is doing at other places.
This check was checking the wrong scope, causing problems in certain edge conditions, for example:
1. Admin adds an "on signup" field that isn't editable after signup.
2. Admin adds a "for all users" field.
3. User goes and fills up the "for all users" field from 2.
4. User is now stuck on the required fields page without any fields showing.
With this change, we only consider "for all users" fields when asking if required custom fields are filled in.
Covers the use case of doing this in composer:
```javascript
page.execute_script("document.querySelector('.d-editor-input').setSelectionRange(6, 12);")
```
When searching for site texts for admin using the english
version of the text, previously we would show the english
version in the results _even if_ there was another locale
translated version available when a locale was selected
from the dropdown.
This commit adds a "Only show results in selected locale"
checkbox option which will instead make it so the results
shown are in the target locale, making it easier for translators
to tell when there is actually translations vs. missing tranlsations.
Followup 76c56c8284
The change introduced above made it so the expired
bookmark reminders were cleared when using the bulk
action menu for bookmarks. However this also affected
clearing reminders for bookmarks when sending notifications.
When clearing bookmark reminders after sending notifications,
we take into account the auto delete preference:
* never - The bookmark `reminder_at` date should not be cleared,
and the bookmark is kept.
* clear_reminder - The bookmark `reminder_at` date is cleared and
the bookmark is kept
The `never` option made it so "expired" bookmark reminder show
on the user's bookmark list.
This commit fixes the change from the other commit and only
forces clearing of `reminder_at` if using the bookmark bulk
action service.
If there is a codec issue or something trying to process a video file
for thumbnail generation, uploads could hang indefinitely. This fix
ensures that we continue the upload process even if we encounter an
error trying to generate a thumbnail for it.
If you’re viewing a tag and you switch to a different tag via the sidebar or the tags dropdown, after expanding the info section of the tag page via the wrench button, the info section keeps showing the previous tag's details instead of the new one.
This happens because the tag info section makes an ajax request to load the tag's details, and this request is made inside the `didInsertElement` hook which is only fired once when the component is rendered. To fix this, we need to set the result from the ajax request to null and add a `didUpdateAttrs` hook to trigger another request to load the info of the new tag.
Internal topic: t/134809.
This change is preventing the "is dirty check" from happening when clicking delete on this form. This was not good UX and was also causing bugs by leaving the form in a unexpected state.
This commit improves `TopicsController#show` to not load suggested and
related topics unless it is the last page of the topic's view.
Previously, we avoided loading suggested and related topics by the use
of conditionals in the `TopicViewSerializer` to avoid calling
`TopicView#suggested_topics` and `TopicView#related_topics`. However,
this pattern is not reliable as the methods can still be called from
other spots in the code base. Instead, we ensure that
`TopicView#include_suggested` and `TopicView#include_related` is set
correctly on the instance of `TopicView` which ensures that for the
given instance, `TopicView#suggested_topics` and
`TopicView#related_topics` will be a noop.
Previous to this change there is no clean way to apply keyboard shortcuts
to things such as "add poll" and other hidden options in the toolbar
This allows shortcuts to be specified similar to how they are on the toolbar
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
If we don't get a `videoWidth` back for a video don't try and generate a
thumbnail for it.
Also as part of this change I switched getImageData, the function
throwing the error, to use video.videoWidth instead of canvas.width
because it's very likely we were setting canvas.width too early before
the width could be read. Now that we are reading the value inside of the
setTimeout hopefully we will actually have a width. Just incase we don't
detect a width we will now exit early instead of throwing an error.
We only need to check for `0` and not null because the value is an
integer and will always return a 0 if it can't be read. https://developer.mozilla.org/en-US/docs/Web/API/HTMLVideoElement/videoWidth
See https://meta.discourse.org/t/322363
What did this fix?
===============
Previously, we only triggered this event in the `user.logged_out` method.
This resulted in the event being triggered only when the user was logged
out by the administrator or the site had strict logout mode enabled.
This bug affected customers who managed user status via webhooks.
meta topic: https://meta.discourse.org/t/user-log-out-event-not-triggered-in-webhooks/249464
This commit moves the business logic in the `Admin::UsersController#suspend` and `Admin::UsersController#silence` actions to dedicated service classes. There's no functional changes in this commit.
Internal topic: t/130014.
This commit implements 2 new metrics/stats in the /about page for the _estimated_ numbers of unique visitors from the EU and the rest of the world. This new feature is currently off by default, but it can be enabled by turning on the hidden `display_eu_visitor_stats` site settings via the rails console.
There are a number of assumptions that we're making here in order to estimate the number of unique visitors, specifically:
1. we're assuming that the average of page views per anonymous visitor is similar to the average number of page views that a logged-in visitor makes, and
2. we're assuming that the ratio of logged in visitors from the EU is similar to the ratio of anonymous visitors from the EU
Discourse keeps track of the number of both logged-in and anonymous page views, and also the number of unique logged-in visitors and where they're from. So with those numbers and the assumptions above, we can estimate the number of unique anonymous visitors from the EU and the rest of the world.
Internal topic: t/128480.
We need a way to disable certain checks programatically, e.g. on Discourse hosting. This PR adds a configuration option for this, and makes it so that disabled checks aren't run as part of #run_all.
This patch allows using an AR relation as a model in services without
fetching associated records. It will just check if the relation is empty
or not. In the former case, the execution will stop at that point, as
expected.
This commit introduces a new frontend API to add custom items to the "Site activity" section in the new /about page. The new API is called `addAboutPageActivity` and it works along side the `register_stat` serve-side API which serializes the data that the frontend API consumes. More details of how the two APIs work together is in the JSDoc comment above the API function definition.
Internal topic: t/128545/9.
This commit fixes a bug where the silence button is incorrectly displayed on the admin page of a staff user. It's not actually possible to silence a staff user because the backend correctly prevents it, but the frontend isn't checking if the button should be displayed.
Another small bug that this commit fixes is the similar users list not showing up inside the silence/suspend modals due to also a bug in the frontend.
I've also changed the way similar users are loaded so that they're not returned by the `admin/users#show` endpoint anymore and moved them into a new endpoint that the penalize modals (suspend and silence) can call directly to retrieve the list of users. This is done because the similar users list is never shown on the admin user page (`/admin/users/:user_id/:username`); they're only needed when the suspend or silence modals are opened.
Internal topic: t/130014.
Currently the push_notification_time_window_mins is set to 10
minutes by default. We want to change this to 1 minute, which is the
delay it has been set to on meta and our internal site already for over
a year now.
This conforms to the expectation users have that notifications are
delivered close to immediately.
This commit converts the Backups page in the admin interface
to follow our new admin interface guidelines.
As part of this work, I've also made `AdminPageHeader` and `AdminPageSubheader`
components that can be reused on any admin page for consistency, that handle
the title and action buttons and also breadcrumbs.
Also renamed `AdminPluginFilteredSiteSettings` to `AdminFilteredSiteSettings` since
it can be used generally to show a subset of filtered site settings, not only
settings for a plugin. Not sure if it's ideal to have to define a new route for this
for every config area, but not sure how else to do it right now.
This commit fixes two codepaths which where incorrectly working with capitalized usernames as we were doing a mix of username_lower and non lower username.
Also adds two specs for these cases.
This patch removes two freedom patches:
- `mail_disable_starttls.rb`: this has been fixed in the 2.8 release of
the mail gem, so we don’t need it anymore.
- `rails4.rb`: those methods have been deprecated for a while now and
should have been dropped with Discourse v3.2.
This commit adds a blue dot next to the "What's New"
link in the admin sidebar if the user has not seen the
new features yet, as a followup to 3e5976f843
which removed the tab on the dashboard that had this same
functionality.
When the admin visits the "What's New" page they count
as having seen all the features straight away. This could
be something we want to change, but for now this keeps the
same functionality.
* System user attachment size WIP
* spec check
* controller update
* add max to system_user_max_attachment_size_kb
* DEV: update to use static method for `max_attachment_size_for_user`
add test to use large image.
add check for failure.
* DEV: update `system_user_max_attachment_size_kb` default value to 0
remove unecessary test.
update tests to reflect the new default value of `system_user_max_attachment_size_kb`
* DEV: update maximum_file_size to check when is an attachment made by a system user
Add tests for when `system_user_max_attachment_size_kb` is over and under the limit
Add test for checking interaction with `max_attachment_size_kb`
* DEV: move `max_attachment_size_for_user` to private methods
* DEV: turn `max_attachment_size_for_user` into a static method
* DEV: typo in test case
* DEV: move max_attachment_size_for_user to private class method
* Revert "DEV: move max_attachment_size_for_user to private class method"
This reverts commit 5d5ae0b715.
---------
Co-authored-by: Gabriel Grubba <gabriel@discourse.org>
This has been split out from https://github.com/discourse/discourse/pull/28051
so we can use this same code in plugin specs before merging the core PR,
adds some helpers for creating local backup temp files
and cleaning them up.
We used to show New Features in a tab on the dashboard,
but this could get pushed down the page especially on
our hosting. In 043117ca13
we made a separate What's New page, so this commit removes
the dashboard tab and changes the admin notification to
send the admin to /admin/whats-new instead of the dashboard
tab.
* DEV: Removal of create_post_for_category_and_tag_changes setting
reverting commit: #65f35e1
and adding a migration to remove the setting
ref: t/132320
* DEV: change checks for zeros to check for nils
* DEV: remove create_post_for_category_and_tag_changes migration file
If anything goes wrong, we can always revert back to the previous state.
This commit fixes a bug in the redesigned about page where if there's no banner image configured for the page, the top of the page where the banner goes is occupied with large white space. Additionally, this commit also fixes a related bug in the admin config area for the /about page where it's not possible to remove the uploaded banner image.
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
When upgrading to Rails 7.1, we had some problems because we were using
several tagged loggers at the same time. They were all added to the main
broadcast logger shipped with Rails, but the Rails 7.1 codebase contains
a bug making a request being run as many times as there are tagged loggers.
The fix was to use the code from the Rails 7.2 codebase.
This patch adds a small spec to ensure the behavior will stay the proper
one in the future.
This change ensures native push notifications respect the site setting for push_notification_time_window_mins. Previously only web push notifications would account for the delay, now we can bring more consistency between Discourse in browser vs Hub, by applying the same delay strategy to both forms of push notifications.
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
Adds a new statistics (hidden from the UI, but available via the API) that tracks daily participating users.
A user is considered as "participating" if they have
- Reacted to a post
- Replied to a topic
- Created a new topic
- Created a new PM
- Sent a chat message
- Reacted to a chat message
Internal ref - t/131013
This commit continues on work laid out by 6039b513fe to redesign the /about page. In this commit, we add sections for showing the site admins and moderators.
The lists of admins and moderators display the 10 most recently seen admins/moderators, with a button to display the rest of admins or moderators. Admins or moderators that have not logged in to the site in the last year will not be shown. Clicking on an admin's or moderator's name/avatar will show their user card.
In development, Ember raises an error when previously-used values are updated during a render. This is to avoid 'backtracking', where parts of templates have to be re-rendered multiple times. In general, this kind of pattern should be avoided, and Ember's warning helps us do that.
However, for the deprecation warning banner, it is quite reasonable for some rendering to trigger a deprecation, and thereby require the global-notice to be re-rendered. We can use our `DeferredTrackedSet` to achieve that. Its `.add` method will delay adding an item to the Set until after the current render has completed.
Very similar to move up/down flag problem fixed here - https://github.com/discourse/discourse/pull/28272
Those are the steps to toggle the flag:
1. click toggle - `saving` CSS class is added;
2. request to backend;
3. `saving` CSS class is removed.
And check if the flag was toggle was:
```ruby
def has_saved_flag?(key)
has_css?(".admin-flag-item.#{key}.saving")
has_no_css?(".admin-flag-item.#{key}.saving")
end
```
If the save action is very fast, then the saving class is removed before the first check.
Therefore I decided to invert it, and once action is finished add `saved` CSS class.
Then we can have a quick positive check:
```ruby
def has_saved_flag?(key)
has_css?(".admin-flag-item.#{key}.saved")
end
```
Currently, in the list controller, when encountering an unsafe redirect
error, a 404 is rendered. The problem is that it’s done in a way that it
also logs a fatal error (because a `Discourse::NotFound` exception was
raised inside a `rescue_from` block).
This patch addresses that issue by simply rendering a 404 without
raising any error.
This commit adds a `MiniSchedulerLongRunningJobLogger` class which will
poll every 60 seconds for mini_scheduler jobs which are stuck. When it
detects that a job is stuck, it will log a warning message with the
current backtrace of the thread that is executing the job.
Note that for scheduled jobs which are executed at a frequency of less
than 30 minutes, we will log when the job has been executing for 30
minutes.
For scheduled jobs executed at a frequency of less than 2 hours, we will
log when the job has been executing for a duration greater than its
specified frequency.
For scheduled jobs executed at a frequency greater than 2 hours, we will
log as long as the job has been executing for more than 2 hours.
Those are the steps to move the flag:
1. open menu;
2. click move up - `saving` CSS class is added;
3. request to backend;
4. `saving` CSS class is removed.
To check if the action was finished we are using this method:
```
def move_up(key)
open_flag_menu(key)
find(".admin-flag-item__move-up").click
has_saved_flag?(key)
self
end
def has_saved_flag?(key)
has_css?(".admin-flag-item.#{key}.saving")
has_no_css?(".admin-flag-item.#{key}.saving")
end
```
However, sometimes specs were failing with `expected to find CSS ".admin-flag-item.spam.saving" but there were no matches`
I think that the problem is with those 2 lines:
```
find(".admin-flag-item__move-up").click
has_closed_flag_menu?
```
If the save action is very fast, then the `saving` class is removed before the first check.
Therefore, to determine that the move action is finished, I am checking if the menu is closed.
Currently, when a badly named category slug is provided, it can lead to
an infinite redirect.
This patch addresses the issue by properly unescaping `request.fullpath`
so the path is successfully rewritten and the redirect happens as
expected.
Currently to handle stub topics after merging, there are only options to (1) never delete a stub topic and (2) delete a stub topic after X amount of days. This adds the option to immediately delete a stub topic upon merge.
---------
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
Co-authored-by: Renato Atilio <renato@discourse.org>
This commit continues on work laid out by 6039b513fe to redesign the /about page. In this commit, we add the site age and a section on the right hand side to show site activities/statistics such as topics, posts, sign-ups, likes etc.
Following a recent refactor, some methods from `FlagSettings` have been
renamed (`custom_types` -> `additional_message_types`). The
`PostActionType` model was using `custom_types` but when the renaming
was done, it was renamed to `with_additional_message` instead of
`additional_message_types`, which under the right circumstances will
raise an error.
Admin can create up to 50 custom flags. It is limited for performance reasons.
When the limit is reached "Add button" is disabled and backend is protected by guardian.
This commit patches `Net::HTTP` to reduce the default timeouts of 60
seconds when we are processing a request. There are certain routes in
Discourse which makes external requests and if the proper timeouts are
not set, we risk having the Unicorn master process force restarting the
Unicorn workers once the `30` seconds timeout is reached. This can
potentially become a vector for DoS attacks and this commit is aimed at
reducing the risk here.
We support a low-level construct called "inline checks", which you can use to register a problem ad-hoc from within application code.
Problems registered by inline checks never show up in the admin dashboard, this is because when loading the dashboard, we run all realtime checks and look for problems. Because of an oversight, we considered inline checks to be "realtime", causing them to be run and clear their problem status.
To fix this, we don't consider inline checks to be realtime, to prevent them from running when loading the admin dashboard.
This change is mainly a refactor of the desktop notifications service to improve readability and have standardised values for tracking state for current user in regards to the Notification API and Push API.
Also improves readability when handling push notification jobs, especially in scenarios where the push_notification_time_window_mins site setting is set to 0, which will allow sending push notifications instantly.
We were writing theme-transpiler JS files to the filesystem on a per-process basis, and then immediately reading them back in. Plus, there was no cleanup mechanism, so the tmp directory would grow indefinitely.
This commit refactors things so that the `build.js` script outputs the theme-transpiler source to stdout. That way, we can read it directly into the process, and then into mini-racer, without needing to go via the filesystem. No cleanup required!
In production, the theme-transpiler is still cached in a file during `assets:precompile`
In the formkit conversion in 2ca06ba236
we missed setting a type for the UppyImageUploader for badges. Also,
we were not passing down the `image_url` as form data, so when we used
`data.image` for that field the badge was not updating in the UI after
page loads and the image URL was not loading for preview.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Before this commit, running `rspec --seed 22953 --format documentation spec/requests/admin/site_texts_controller_spec.rb:191 spec/lib/freedom_patches/translate_accelerator_spec.rb:109` will fail.
Setting `I18n.config.available_locales` is equivalent to hard coding the
locales for the entire process. It should not be set so that `I18n` will
fallback to `backend.locales`.
Sometimes the backtrace is quite big for failing specs, this env var
(RSPEC_EXCLUDE_NOISE_IN_BACKTRACE) can be set to
1 to remove backtrace from anything but spec or application code in
rspec. This makes it easier to see where the actual failure is
coming from, most of the time all the gem paths are noise.
When creating a shared draft, we're recording topic view stats on the draft and then pass those on when the draft is published, conflating the actual view count.
This fixes that by not registering topic views if the topic is a shared draft.
When `SiteSetting.review_every_post` is true and the category `require_topic_approval` system creates two reviewable items.
1. Firstly, because the category needs approval, the `ReviewableQueuePost` record` is created - at this stage, no topic is created.
2. Admin is approving the review. The topic and first post are created.
3. Because `review_every_post` is true `queue_for_review_if_possible` callback is evaluated and `ReviewablePost` is created.
4. Then `ReviewableQueuePost` is linked to the newly generated topic and post.
At the beginning, we were thinking about hooking to those guards:
```
def self.queue_for_review_if_possible(post, created_or_edited_by)
return unless SiteSetting.review_every_post
return if post.post_type != Post.types[:regular] || post.topic.private_message?
return if Reviewable.pending.where(target: post).exists?
...
```
And add something like
```
return if Reviewable.approved.where(target: post).exists?
```
However, because the callback happens in point 3. before the `ReviewableQueuePost` is linked to the `Topic`, it was not possible.
Therefore, when `ReviewableQueuePost` is creating a `Topic`, a new option called `:reviewed_queued_post` is passed to `PostCreator` to avoid creating a second `Reviewable`.
Currently, descriptions for flag types aren’t interpolated, returning
`%{base_path}` in their string, for example. This breaks the navigation
on the sites.
The behavior changed probably because of an upgrade of Ruby, as two
hashes were passed to `I18n.t` (`vars` and `default`) without using the
splat operator.
When using `Discourse.cache.fetch` with an expiry, there's a potential for a race condition due to how we read the data from redis.
The code used to be
```ruby
raw = redis.get(key) if !force
entry = read_entry(key) if raw
return entry if raw && !(entry == :__corrupt_cache__)
```
with `read_entry` defined as follow
```ruby
def read_entry(key)
if data = redis.get(key)
Marshal.load(data)
end
rescue => e
:__corrupt_cache__
end
```
If the value at "key" expired in redis between `raw = redis.get` and `entry = read_entry`, the `entry` variable would be `nil` despite `raw` having a value.
We would then proceed to return `entry` (which is `nil`) thinking it had a value, when it didn't.
The first `redis.get` can be skipped altogether and we can rely only on `read_entry` to read the data from redis. Thus avoiding the race condition and removing the double read operations.
Internal ref - t/132507
* SECURITY: Update default allowed iframes list
Change the default iframe url list to all include 3 slashes.
* SECURITY: limit group tag's name length
Limit the size of a group tag's name to 100 characters.
Internal ref - t/130059
* SECURITY: Improve sanitization of SVGs in Onebox
---------
Co-authored-by: Blake Erickson <o.blakeerickson@gmail.com>
Co-authored-by: Régis Hanol <regis@hanol.fr>
Co-authored-by: David Taylor <david@taylorhq.com>
Followup 4aea12fdcb
In certain config areas (like About) we want to be able
to fetch specific site settings by name. In this case,
sometimes we need to be able to fetch hidden settings,
in cases where a config area is still experimental.
Splitting out a different endpoint for this purpose
allows us to be stricter with what we return for config
areas without affecting the main site settings UI, revealing
hidden settings before they are ready.
Since switching to Maxmind permalinks to download the databases in
7079698cdf, we have received multiple
reports about rebuilds failing as `maxminddb:refresh` runs during
the rebuilds and failing to download the databases cases the rebuilds to
fail.
Downloading Maxmind databases should not sit in the critical rebuild
path but since we are close to the Discourse 3.3 release, we have opted
to just rescue all errors encountered when downloading the databases.
In the near future after the Discourse 3.3 release, we will be looking
at moving the downloading of maxmind databases out of the rebuild path.
We have a dedicated admin page (`/admin/customize/email_templates`) that lets admins customize all emails that Discourse sends to users. The way this page works is that it lists all translations strings that are used for emails, and the list of translation strings is currently hardcoded and hasn't been updated in years. We've had a number of new emails that Discourse sends, so we should add those templates to the list to let admins easily customize those templates.
Meta topic: https://meta.discourse.org/t/3-2-x-still-ignores-some-custom-email-templates/308203.
* FIX: Ensure JsLocaleHelper to obly outputs up-to-date translations
The old implementation forgot to filter out deprecated
translations, causing these translations to incorrectly override the new
locale in the frontend.
This commit fills in the forgotten where clause, filtering only the
up-to-date part.
Related meta topic: https://meta.discourse.org/t/outdated-translation-replacement-causing-missing-translation/314352
This patch fixes the `i18n:check` rake task which has been broken by
the `MessageFormat` upgrade.
It also adds a spec to ensure we generate valid MF code for all our
available locales.
Currently, when adding translation overrides, values aren’t validated
for MF strings. This results in being able to add invalid plural keys or
even strings containing invalid syntax.
This patch addresses this issue by compiling the string when saving an
override if the key is detected as an MF one.
If there’s an error from the compiler, it’s added to the model errors,
which in turn is displayed to the user in the admin UI, helping them to
understand what went wrong.
When we show user tips, we immediately send an AJAX request to mark the
tiup as seen. This is done in the background. However, when system tests
are run, sometimes that request is not completed before the test ends.
This causes the test to be flakey.
One way to fix this is to force the system test run to wait for the AJAX
request to complete. However, this is not ideal because it makes the
test suite slower on each run.
Instead, this commit removes the flakey assertion and adds an alternative
assertion in the frontend tests that ensures the background request is
sent when the user tip is shown.
Form Kit is our new form library/framework for unifying the way forms look across Discourse. The admin config area for the /about page is a new form that isn't currently used, so it makes sense for it to be one of the first forms to be migrated to Form Kit to test the library.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
We can get translations with invalid plural keys from Crowdin
or from custom overrides. Currently, this will raise an error and the
locales won’t be outputted at all.
This patch addresses this issue by using the new `strict: false` option
of our `messageformat-wrapper` gem, allowing to generate locales even if
there are invalid plural keys present.
* FIX: Add post id to the anchor to prevent two identical anchors
We generate anchors for headings in posts. This works fine if there is
only one post in a topic with anchors. The problem comes when you have
two or more posts with the same heading. PrettyText generates anchors
based on the heading text using the raw context of each post, so it is
entirely possible to generate the same anchor for two posts in the same
topic, especially for topics with template replies
Post1:
# heading
context
Post2:
# heading
context
When both posts are on the page at the same time, the anchor will only
work for the first post, according to the [HTML specification](https://html.spec.whatwg.org/multipage/browsing-the-web.html#scroll-to-the-fragment-identifier).
> If there is an a element in the document tree whose root is document
> that has a name attribute whose value is equal to fragment, then
> return the *first* such element in tree order.
This bug is particularly serious in forums with non-Latin languages,
such as Chinese. We do not generate slugs for Chinese, which results in
the heading anchors being completely dependent on their order.
```ruby
[2] pry(main)> PrettyText.cook("# 中文")
=> "<h1><a name=\"h-1\" class=\"anchor\" href=\"#h-1\"></a>中文</h1>"
```
Therefore, the anchors in the two posts must be in exactly the same by
order, causing almost all of the anchors in the second post to be
invalid.
This commit solves this problem by adding the `post_id` to the anchor.
The new anchor generation method will add `p-{post_id}` as a prefix when
post_id is available:
```ruby
[3] pry(main)> PrettyText.cook("# 中文", post_id: 1234)
=> "<h1><a name=\"p-1234-h-1\" class=\"anchor\" href=\"#p-1234-h-1\"></a>中文</h1>"
```
This way we can ensure that each anchor name only appears once on the
same topic. Using post id also prevents the potential possibility of the
same anchor name when splitting/merging topics.
We are investigating a memory leak in Sidekiq and saw the following line
when comparing heap dumps over time.
`Allocated IMEMO 14775 objects of size 591000/7389528 (in bytes) at:
/var/www/discourse/app/jobs/onceoff/onceoff.rb:36`
That line in question was doing a `.select { |klass| klass < self }` on
`ObjectSpace.each_object(Class)`. This for some reason is allocating a
whole bunch of `IMEMO` objects which are instruction sequence objects.
Instead of diving deeper into why this might be leaking, we can just
save our time by switching to an implementation that is more efficient
and does not require looping through a ton of objects.
Followup e954eb234e
Adds a test for the defer_track_view method on topic controller
to check that the early returns (nexts) work correctly
without errors.
Previously in these 2 PRs, we introduced a new site setting `SiteSetting.enforce_second_factor_on_external_auth`.
https://github.com/discourse/discourse/pull/27547https://github.com/discourse/discourse/pull/27674
When disabled, it should enforce 2FA for local login with username and password and skip the requirement when authenticating with oauth2.
We stored information about the login method in a secure session but it is not reliable. Therefore, information about the login method is moved to the database.
Previously, we couldn't change the user agent name dynamically for onebox requests. In this commit, a new hidden site setting `onebox_user_agent` is created to override the default user agent value specified in the [initializer](c333e9d6e6/config/initializers/100-onebox_options.rb (L15)).
Co-authored-by: Régis Hanol <regis@hanol.fr>
- Ensure main title is set as 'not visible' when removed from DOM
- `deactivate` -> `willTransition` to ensure proper behavior when navigating between multiple topics
Followup to bdec564d14
- Move topic-title on-screen detection to intersection-observer (via new modifier), and add a boolean to header service which indicates whether it's on-screen
- Move scroll-direction from Mixin to dedicated service. Teach it to pause scroll monitoring while transitions are in progress, to avoid reporting false changes in scroll direction. Also resets to a 'neutral' state after each navigation, which indicates the the user has not yet scrolled
- When entering a topic view, notify the header service which post is being targeted. It can then make an educated guess about whether the topic title is likely to be in-view
- Update header service `topicInfoVisible` to be a declarative getter, based on the three refactored sources of truth mentioned above
- Update legacy widget header to use the header service for topic info
All of these changes mean that the header no longer 'flickers' when navigating into topics on mobile. As well as the improved UX, this should also improve our Cumulative Layout Shift (CLS) web vital metrics.
* FEATURE: Clean up previously logged information after permanently deleting posts
When soft deleteing a topic or post, we will log some details in the
staff log, including the raw content of the post. Before this commit, we
will not clear the information in these records. Therefore, after
permanently deleting the post, `UserHistory` still retains copy of the
permanently deleted post. This is an unexpected behaviour and may raise
some potential legal issues.
This commit adds a behavior that when a post is permanently deleted, the
details column of the `UserHistory` associated with the post will be
overwritten to "(permanently deleted)". At the same time, for permanent
deletion, a new `action_id` is introduced to distinguish it from soft
deletion.
Related meta topic: https://meta.discourse.org/t/introduce-a-way-to-also-permanently-delete-the-sensitive-info-from-the-staff-logs/292546
There is a bug with chat type flags - "An error occurred: Applies to is not included in the list"
Flag.valid_applies_to_types is a set of core types and types registered by plugins `Set.new(DEFAULT_VALID_APPLIES_TO | DiscoursePluginRegistry.flag_applies_to_types)`
Using lamba should ensure that valid values are calculated dynamically.
This commit promotes the new topic bulk action
menu introduced in 89883b2f51
to the main method of bulk selecting and performing
actions on topics. The site setting flag gating this
feature is deleted, and the old bulk select code is
deleted as well.
The new modal shows a loading spinner while operations
are taking place, allows selecting the action from a dropdown
instead of having a 2-step modal flow,
and also supports additional options for some operations, e.g.
allowing Close silently.
Replaces the existing topic map with the experimental-topic-map made by @awesomerobot.
---------
Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
This commit introduces the foundation for a new design for the /about page that we're currently working on. The current version will remain available and still be the default until we finish the new version and are ready to roll out. To opt into the new version right now, add one or more group to the `experimental_redesigned_about_page_groups` site setting and members in those groups will get the new version.
Internal topic: t/128545.
This ensures that elasticsearch doesn't parse it as an object. There are
too many combination of job opts so we don't want elasticsearch to be
parsing and indexing this field as an object.
This improves the `TextSentinel` so that we don't consider CJK text as being uppercase and thus failing the validator.
It also optimizes the entropy computation by using native ruby `.bytes` to get all the bytes from the text.
It also tweaks the `seems_pronounceable?` and `seems_unpretentious?` check to use the `\p{Alnum}` unicode regexp group to account for non-latin languages.
Reference - https://meta.discourse.org/t/body-seems-unclear-error-when-users-are-typing-in-chinese/88715
Inspired by https://github.com/discourse/discourse/pull/27900
Co-authored-by: Paulo Magalhaes <mentalstring@gmail.com>
* FEATURE: Add logging for CustomEmoji
We didn't provide any logs for CustomEmoji before, nor did we record the
person who added any emoji in the database. As a result, the staff had
no way to trace back who added a certain emoji.
This commit adds a new column `user_id` to `custom_emojis` to record the
creator of an emoji. At the same time, a log is added for staff logs to
record who added or deleted a custom emoji.
If a user has a required action, e.g. adding a 2FA method or filling in new required fields, we disable client-side routing except to allowed pages.
This led to a situation where a user might navigate away from e.g. the profile page to look at the new ToS, and then being "stuck" due to not knowing how to get back to accept the new terms.
This PR makes it so that if you click any restricted link, instead of doing nothing we transition the user back to the page where they can take the required action.
User actions can trigger functions that render changes to the screen within the same cycle (e.g. pressing the reply button will cause the login modal to pop up), potentially impacting performance and causing some jank on slower devices.
This change inserts runAfterFramePaint where certain actions are triggered. Below are some screenshots indicating an improved INP for some of the buttons affected on controls with the highest INPs. The two places where this is added help with several actions, e.g. user + group cards, generic button action usage.
When tags contain an underscore we should allow filtering in the same way, previously due to the regex those with underscores were not being found when filtering.
This commit ensures that we reset the `missing_s3_uploads` status count
if there are no inventory files which are at least 2 days older than the
site's restored date.
Otherwise, a site with missing uploads but was subsequntly restored will
be continue to report missing uploads for 2 days.
Followup 560e8aff75
The linked commit allowed oneboxing private GitHub PRs,
issues, commits, and so on, but it didn't actually allow
oneboxing the root repo e.g https://github.com/discourse/discourse-reactions
We didn't have an engine for this, we were relying on OpenGraph
tags on the HTML rendering of the page like we do with other
oneboxes.
To fix this, we needed a new github engine for repos specifically.
Also, this commit adds a `data-github-private-repo` attribute to
PR, issue, and repo onebox HTML so we have an indicator of
whether the repo was private, which can be used for theme components
and so on.
Our old group SMTP SSL option was a checkbox,
but this was not ideal because there are actually
3 different ways SSL can be used when sending
SMTP:
* None
* SSL/TLS
* STARTTLS
We got around this before with specific overrides
for Gmail, but it's not flexible enough and now people
want to use other providers. It's best to be clear,
though it is a technical detail. We provide a way
to test the SMTP settings before saving them so there
should be little chance of messing this up.
This commit also converts GroupEmailSettings to a glimmer
component.
Allow admin to create custom flag which requires an additional message.
I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
