Why this change?
As part of our ongoing efforts to security harden the Discourse
application, we are adding the `cross_origin_opener_policy_header` site setting
which allows the `Cross-Origin-Opener-Policy` response header to be set on requests
that preloads the Discourse application. In more technical terms, only
GET requests that are not json or xhr will have the response header set.
The `cross_origin_opener_policy_header` site setting is hidden for now
for testing purposes and will either be released as a public site
setting or be remove if we decide to be opinionated and ship a default
for the `Cross-Origin-Opener-Policy` response header.
Followup to eea74e0e32. Site settings
which are a list without a list_type should also have the _map
extension added which returns an array based on split("|").
For example:
```
SiteSetting.post_menu_map
=> ["read", "like"]
```
Tries to fix the composer upload spec by making the upload
slow enough to allow clicking the Cancel button, and improves
generally the API for CDP network changes.
When navigating around, we make ajax requests with a parameter like `?filter=latest`. This results in the TopicQuery being set up with `filter: "latest"` as a string. The logic introduced in fd9a5bc0 checks for equality with `:latest` and `:unseen` symbols, which didn't work correctly in this situation
This commit makes the logic detect both strings and symbols, and adds a spec for the behaviour.
This adds support for oneboxing WEBP and AVIF images in posts and fixing
oneboxing fixes download remote images for those formats too.
Reported in https://meta.discourse.org/t/-/276433?u=falco
This could happen after you had already change the separation mode and would cause unexpected bugs.
This PR also adds more tests around using switch buttons with chat.
Reverts e2705df and re-lands #23187 and #23219.
The issue was incorrect order of execution of Rails' `assets:precompile` task in our own precompilation stack.
Co-authored-by: David Taylor <david@taylorhq.com>
Short answer -- the problem is the video thumbnail generator & uploader
code added a couple of months back in f144c64e13.
It was implemented as another Mixin which overrides `this._uppyInstance`
when uploading the video thumbnail after the initial upload is complete,
which means the composer's `this._uppyInstance` value is overridden,
and it loses all of its preprocessors & upload code.
This is generally a problem with the Mixin based architecture that I
used for the Uppy code, which we need to remove at some point and
refacotr.
The most ideal thing to do here would be to convert this video thumbnail
code into an Uppy
[postprocessor](https://uppy.io/docs/uppy/#addpostprocessorfn) plugin,
which runs on each upload after they are complete. I started looking
into this, and the main hurdle here is adding support to tracking the
progress of postprocessors to
[ExtendableUploader](cf42466dea/app/assets/javascripts/discourse/app/mixins/extendable-uploader.js)
so that is out of scope at this time.
The fix here makes it so the ComposerVideoThumbnailUppy code is no
longer a Mixin, but acts more like a normal class, a pattern which
we have used in chat. I also clean up a lot of the thumbnail uploader
code and remove some unnecessary things.
Attempted to add a system spec, but video streaming does not work
in Chrome for Testing at this time, and it is needed for the
onloadedmetadata event.
This commit adds some system specs to test uploads with
direct to S3 single and multipart uploads via uppy. This
is done with minio as a local S3 replacement. We are doing
this to catch regressions when uppy dependencies need to
be upgraded or we change uppy upload code, since before
this there was no way to know outside manual testing whether
these changes would cause regressions.
Minio's server lifecycle and the installed binaries are managed
by the https://github.com/discourse/minio_runner gem, though the
binaries are already installed on the discourse_test image we run
GitHub CI from.
These tests will only run in CI unless you specifically use the
CI=1 or RUN_S3_SYSTEM_SPECS=1 env vars.
For a history of experimentation here see https://github.com/discourse/discourse/pull/22381
Related PRs:
* https://github.com/discourse/minio_runner/pull/1
* https://github.com/discourse/minio_runner/pull/2
* https://github.com/discourse/minio_runner/pull/3
In most cases, deleting a user from outside the review UI will also delete any pending reviewables for that user. This was not working in some cases, e.g. for reviewables created due to "fast typer" violations.
This was happening because UserDestroyer only automatically resolves flagged posts.
After this change, in addition to existing checks, look for ReviewablePost where the post was created by the user and reject them if present.
* Minor style adjustments
* Removes "all" count because it's redundant to the count on New
* Updates generic class names with -- modifier to follow BEM and help avoid class name collisions
* Hides the toggle when bulk select is enabled (the UI ends up being too busy)
Manipulating theme module paths means that the paths you author are not the ones used at runtime. This can lead to some very unexpected behavior and potential module name clashes. It also meant that the refactor in 16c6ab8661 was unable to correctly match up theme connector js/templates.
While this could technically be a breaking change, I think it is reasonably safe because:
1. Themes are already forced to use relative paths when referencing their own modules (since they're namespaced based on the site-specific id). The only time this might be problematic is when theme tests reference modules in the theme's main `javascripts` directory
2. For things like components/services/controllers/etc. our custom Ember resolver works backwards from the end of the path, so adding `discourse/` in the middle will not affect resolution.
This is a bug that happens only when the current date is less than 90 days from a date on which the time zone transitions into or out of Daylight Savings Time.
In these conditions, bulk invites show the time of day of their expiration as being 1 hour later than the current time.
Whereas it should match the time of day the invite was generated.
This is because the server has not been using the user's timezone in calculating the expiration time of day. This PR fixes issue by considering the user's timezone when doing the date math.
https://meta.discourse.org/t/bulk-invite-logic-to-generate-expire-date-bug/274689
`ReviewableQueuedPost` got refactored a while back to use the more
appropriate `target_created_by` for the user of the post being queued
instead of `created_by`. The change was not extended to the `DELETE
/review/:id` endpoint leading to error responses for a user attempting
to deleting their own queued post.
This fix extends the `Reviewable` lookup implementation in
`ReviewablesController#destroy` and Guardian implementation to account
for this change.
This PR adds a new toggle to switch the (new) /new list between showing topics with new replies (a.k.a unread topics), new topics, or everything mixed together.
When hiding a post (essentially updating hidden, hidden_at, and hidden_reason_id) our callbacks are running the whole battery of post validations. This can cause the hiding to fail in a number of edge cases. The issue is similar to the one fixed in #11680, but applies to all post validations, none of which should apply when hiding a post.
After some code reading and discussion, none of the validations in PostValidator seem to be relevant when hiding posts, so instead of just skipping unique check, we skip all post validator checks.
After fbe0e4c we always pass a block into these methods.
So yield inside the export methods works and there is no need
anymore to wrap them into enumerators.
What is the problem here?
The `selenium-webdriver` gem is responsible for downloading the
right version of the `chromedriver` binary and it downloads it into the
`~/.cache/selenium` folder. THe problem here is that when a user runs `bin/turbo_rspec spec/system`
for the first time, all of the processes will try to download the
`chromedriver` binary to the same path at the same time and will lead
to concurrency errors.
What is the fix here?
Before running any RSpec suite, we first check if the `.cache/selenium`
folder is present. If it is not present, we use a file system lock to
download the `chromedriver` binary such that other processes that runs
after will not need to install the `chromedriver` binary.
The long term fix here is to get `selenium-manager` to download the `chromedriver` binary to a unique path for each
process but the `--cache-path` option for `selenium-manager` is currently not supported in `selenium-webdriver`.
So we have to order by calling `find_each(order: :desc)`.
Note that that will order rows by Id, not by `last_match_at`
as we tried before (though that didn't work).
What is the problem here?
When transiting between `/filter` routes with different `q` query
params, the input field is not updating to include the values in the `q`
query param. This was because we were setting the value of the input
field in the constructor of the controller but controllers are actually
singletons in Ember so setting the value of the input field is only done
once when the controller is initialised.
What is the fix here?
Instead of setting the value of the input field in the controller, we
set the value in the `setupController` hook in the route file.
We can no long user Webdriver - SeleniumHQ/selenium#11066. Bumping selenium-webdriver did the trick, as well as manually setting the user_agent for mobile system specs. Unsure what changed to make this necessary, but it is necessary to get the app to boot in mobile view.
* scrub non-a html tags from tag descriptions on create, strips all tags from tag description when displayed in tag hover
* test for tag description links
* UX: basic render-tag test
* UX: fix linting
* UX: fix linting
* fix broken tests
* Update spec/models/tag_spec.rb
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
* UX: use has_sanitizable_fields instead of has_scrubbable_fields to ensafen tag.description
---------
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
The category feature that automatically closes topics does it silently
This amends it so `rake topics:apply_autoclose` which does retroactive
closing will also do so silently.
Our code assumed the content_range interval was inclusive, but they are open-ended due to Postgres' [discrete range types](https://www.postgresql.org/docs/current/rangetypes.html#RANGETYPES-DISCRETE), meaning [1,2] will be represented as [1,3).
It also fixes some flaky tests due to test data not being correctly setup and the registry not being resetted after each test.
When we receive the stream parameter, we'll queue a job that periodically publishes partial updates, and after the summarization finishes, a final one with the completed version, plus metadata.
`summary-box` listens to these updates via MessageBus, and updates state accordingly.
The OpenComposer mixin comes from a time before we had a composer service. As well as being a general cleanup/refactor, this commit aims to removes interlinking between composer APIs and the discovery-related controllers which are being removed as part of #22622.
In summary, this commit:
- Removes OpenComposer mixin
- Adds and updates composer service APIs to support everything that `openComposer` did
- Updates consumers to call the composer service directly, instead of relying on the mixin (either directly, or via a route-action which bubbled up to some parent)
- Deprecates composer-related methods on `DiscourseRoute` and on the application route
This commit moves the calendar date and time picker shown in
the local dates modal into a core component that can be reused
in other places. Also add system specs to make sure there isn't
any breakages with this feature, and a section to the styleguide.
Why this change?
By default in the test environment, MessageBus used the memory backend
which means all messages are stored in an in-memory data structure. However,
the in-memory data structure is not cleared after each system test so we
have the potential to be leaking stuff between system tests.
Similarly for the defer queue which process work in another thread, we
want to ensure that the defer queue processes everything it has to do
before the transaction is rolled back.
Prior to this fix we would output an image with no width/height which would then bypass a large part of `CookedProcessorMixin` and have no aspect ratio. As a result, an image with no size would cause layout shift.
It also removes a fix for oneboxes in chat messages due to this case.
If a selenium finder takes the full wait duration to resolve, that means it has been written inefficiently. Most likely a matcher has been negated incorrectly.
This commit introduces a patch which will raise an error in this situation so that we can catch the issues while developing specs.
This commit also fixes chat's visit_thread helper. It was spinning on `has_css?(".chat-skeleton")` for the full selenium wait duration, and then returns false. That's because the thread is often already fully loaded before `has_css?` is even called. It's now updated to only look for the final expected state.
There is no decorateCooked equivalent for small action posts,
so we need to manually call decorateHashtags when there is a custom
message for small action posts in order for the hashtags to get
their coloured icon/square.
This commit removes any logic in the app and in specs around
enable_experimental_hashtag_autocomplete and deletes some
old category hashtag code that is no longer necessary.
It also adds a `slug_ref` category instance method, which
will generate a reference like `parent:child` for a category,
with an optional depth, which hashtags use. Also refactors
PostRevisor which was using CategoryHashtagDataSource directly
which is a no-no.
Deletes the old hashtag markdown rule as well.
This fixes:
- a regression from 30c152c, where navigating to a topic's last reply
via keyboard would lose track of the topic when returning to the topic
list
- an issue where if a topic's last post is a small post, navigating to it
via keyboard would not focus the post
Co-authored-by: David Taylor <david@taylorhq.com>
This brings the theme development experience (via the discourse_theme cli) closer to the experience of making javascript changes in Discourse core/plugins via Ember CLI. Whenever a change is made to a non-css theme field, all clients will be instructed to immediately refresh via message-bus.
In #20135 we prevented invalid inputs from being accepted in category setting form fields on the front-end. We didn't do anything on the back-end at that time, because we were still discussing which path we wanted to take. Eventually we decided we want to move this to a new CategorySetting model.
This PR moves the num_auto_bump_daily from custom fields to the new CategorySetting model.
In addition it sets the default value to 0, which exhibits the same behaviour as when the value is NULL.
Edit community section button is hidden in secondary/more section. However, when there are no secondary links, then more section is not shown. In that case, we should still display an edit button for admins, so they can edit the section.
This commit introduces the :push_notification event and deprecates :post_notification_alert.
The old :post_notification_alert event was not triggered when pushing chat notifications and did not respect when the user was in "do not disturb" mode.
The new event fixes these issues.
This plugin is no longer supported, and so we no longer need to run its tests in CI
(removing the comment and the 'Canned Replies' value from the array caused syntax_tree to change to the `%w` syntax)
Why this change?
This is a follow up to e8f7b62752.
Tracking of GC stats didn't really belong in the `MethodProfiler` class
so we want to extract that concern into its own class.
As part of this PR, the `track_gc_stat_per_request` site setting has
also been renamed to `instrument_gc_stat_per_request`.
What does this change do?
This change adds a hidden `track_gc_stat_per_request` site setting which
when enabled will track the time spent in GC, major GC count and minor
GC count during a request.
Why is this change needed?
We have plans to tune our GC in production but without any
instrumentation, we will not be able to know if our tuning is effective
or not. This commit takes the first step at instrumenting some basic GC
stats in core during a request which can then be consumed by the discourse-prometheus plugin.
Linking to the #feedback category can break if the category gets renamed or a different site locale is used. By using the correct hashtag (at the time of seeding) this issues can be avoided.
PresenceChannel configuration is cached using redis. That cache is used, and sometimes repopulated, during normal GET requests. When the primary redis server was readonly, that `redis.set` call would raise an error and cause the entire request to fail. Instead, we should ignore the failure and continue without populating the cache.
This commit introduces five rake tasks to help us with version bump procedures:
- `version_bump:beta` and `version_bump:minor_stable` are for our minor releases
- `version_bump:major_stable_prepare` and `version_bump:major_stable_merge` are for our major release process
- `version_bump:stage_security_fixes` is to collate multiple security fixes from private branches into a single branch for release
The scripts will stage the necessary commits in a branch and prompt you to create a PR for review. No changes to release branches or tags will be made without the PR being approved, and explicit confirmation of prompts in the scripts.
To avoid polluting the operator's primary working tree, the scripts create a temporary git worktree in a temporary directory and perform all checkouts/commits there.
FEATURE: Only approved flags for post counters
* Why was this change necessary?
The counters for flagged posts in the user's profile and user index from
the admin view include flags that were rejected, ignored or pending
review. This introduces unnecessary noise. Also the flagged posts
counter in the user's profile includes custom flags which add further
noise to this signal.
* How does it address the problem?
* Modifying User#flags_received_count to return posts with only approved
standard flags
* Refactoring User#number_of_flagged_posts to alias to
User#flags_received_count
* Updating the flagged post staff counter hyperlink to navigate to a
filtered view of that user's approved flagged posts to maintain
consistency with the counter
* Adding system tests for the profile page to cover the flagged posts
staff counter
This is a similar fix to 32d4810e2b
Why this change?
Prior to this change, there is a bug in `TopicsController#bulk`
where it does not dismiss new unred posts in sub-subcategories when the
`category_id` and `include_subcategories=true` params are present. This
is because the controller did not account for sub-subcategories when
fetching the category ids of the new topics that should be dismissed.
This commit fixes the problem by relying on the `Category.subcategory_ids` class
method which accounts for sub-subcategories.
We currently are accumulating orphaned upload references whenever drafts are deleted.
This change deals with future cases by adding a dependent strategy of delete_all on the Draft#upload_references association. (We don't really need destroy strategy here, since UploadReference is a simple data bag and there are no validations or callbacks on the model.)
It deals with existing cases through a migration that deletes all existing, orphaned draft upload references.
A previous change updated `ReviewableQueuedPost`'s `created_by`
to be consistent with other reviewable types. It assigns
the the creator of the post being queued to `target_created_by` and sets
the `created_by` to the creator of the reviewable itself.
This fix updates some of the `created_by` references missed during the
intial fix.
Internal oneboxes to posts that contained oneboxed github links to
commits or PRs with long enough commit messages to have the `show-more`
and the `excerpt hidden` classes in their html were being stripped of
their content resulting in empty internal oneboxes.
see: https://meta.discourse.org/t/269436
This fixes a regression introduced in:
0b3cf83e3c
By default, only 10 members are highlighted on group cards. However,
joining/leaving a big group via the buttons on the group card results in
up to 50 members being highlighted. For large groups, this causes the card
to move off-screen.
This happens because, while the initial render explicitly fetches only 10
members, we don't seem to apply the same limit as part of the member
reload performed when a user leaves/joins via the buttons on the card.
This PR fixes that by only making the first 10 users available for
highlight regardless of the number of members loaded in the store.
What is the problem here?
In multiple controllers, we are accepting a `limit` params but do not
impose any upper bound on the values being accepted. Without an upper
bound, we may be allowing arbituary users from generating DB queries
which may end up exhausing the resources on the server.
What is the fix here?
A new `fetch_limit_from_params` helper method is introduced in
`ApplicationController` that can be used by controller actions to safely
get the limit from the params as a default limit and maximum limit has
to be set. When an invalid limit params is encountered, the server will
respond with the 400 response code.
What is the context for this change?
Prior to this change, there is a bug in `TopicsController#reset_new`
where it does not dismiss new topics in sub-subcategories when the
`category_id` and `include_subcategories=true` params are present. This
is because the controller did not account for sub-subcategories when
fetching the category ids of the new topics that should be dismissed.
This commit fixes the problem by relying on the `Category.subcategory_ids` class
method which accounts for sub-subcategories.
Context of this change:
There are two site settings which an admin can configured to set the
default categories and tags that are shown for a new user. `default_navigation_menu_categories`
is used to determine the default categories while
`default_navigation_menu_tags` is used to determine the default tags.
Prior to this change when seeding the defaults, we will filter out the
categories/tags that the user do not have permission to see. However,
this means that when the user does eventually gain permission down the
line, the default categories and tags do not appear.
What does this change do?
With this commit, we have changed it such that all the categories and tags
configured in the `default_navigation_menu_categories` and
`default_navigation_menu_tags` site settings are seeded regardless of
whether the user's visibility of the categories or tags. During
serialization, we will then filter out the categories and tags which the
user does not have visibility of.
Embed Motoko service's primary URL is transiting from embed.smartcontracts.org to embed.motoko.org, this PR updates the Onebox logic to work for either domain.
This adds support for the `<=` and `<` version operators in `.discourse-compatibility` files. This allows for more flexibility (e.g. targeting the entire 3.1.x stable release via `< 3.2.0.beta1`), and should also make compatibility files to be more readable.
If an operator is not specified we default to `<=`, which matches the old behavior.
We recently added a "don't feed the trolls" feature which warns you about interacting with posts that have been flagged and are pending review. The problem is the warning persists even if an admin reviews the post and rejects the flag.
After this change we only consider active flags when deciding whether to show the warning or not.
Why this change?
We were verifying that a url for a section link in a custom sidebar
section is valid by passing the url string to `Router#recognize`.
If a `rootURL` has been set on the router, the url string that is passed
to `Router#recognize` has to start with the `rootURL`.
This commit fixes the problem by ensuring that `RouteInfoHelper` adds
the application subfolder path before calling `Router#recognize` on the
url string.
Why this change?
When setting up the `IntersectionObserver`, we did not account for the
top margin and padding causing no intersection event to fire when the
last tag is load into view. This commits fixes the problem by setting a
bottom margin using the `rootMargin` option when setting up the
`IntersectionObserver`.
This commit also improves the test coverage surrounding the loading of
more tags.
Why this change?
We're already displaying a category's description as the title attribute
on the category section link. We should do the same for tags as well.
Allow anonymous users (logged-in, but set to anonymous posting) to like posts
---------
Co-authored-by: Emmett Ling <eling@zendesk.com>
Co-authored-by: Nat <natalie.tay@discourse.org>
* Why was this change necessary?
The current logic in the user.hbs template file does not render the
trust level element for the user's info panel when the user is TL0,
because 0 is treated as falsey in the `if` conditional block.
Ref: https://meta.discourse.org/t/tl0-not-displayed-on-users-profile-pages/271779/10
* How does it address the problem?
This PR adds a predicate helper method local to the user controller that
includes an additional check which returns true if the trust_level of
the user is 0 on top of the existing logic. This allows TL0 users to
have their trust level rendered correctly in their profile's info panel.
* DEV: Skip srcset for onebox thumbnails
In an effort to preserve bandwidth especially for mobile devices this
change will prevent upscaled srcset attributes from being added to
onebox thumbnail images.
Besides checking the html for onebox classes, our database structure for
uploads does not distinguish between regular images and onebox thumbnail
images, but all upload images in discourse do have a thumbnail. By
default this thumbnail is what is used for the non-upscaled image for
onebox images, so we should only use that thumbnail. Because the
rendered onebox image size is likely smaller than the upload thumbnail
size there really shouldn't be a need to upscale.
Recently we started giving admins a notice in the advice panel when their translations have become outdated due to changes in core. However, we didn't include any additional information.
This PR adds more information about the outdated translation inside the site text edit page, together with an option to dismiss the warning.
Followup to b583872eed
and 54001060ea
Another place where we need to filter hashtag types to
only enabled ones is PrettyText, though the latter PR
above should also already make it so the correct priority
types are passed.
This is causing errors in the email processing workflow
for some customers (presumably ones with tagging disabled).
When a type was disabled, the hashtag search _without_ a
term was erroring. This was because we weren't filtering
out the disabled types from types_in_priority_order first
like we were if there was a term provided.
This commit fixes that issue, and also makes it so
contexts_with_ordered_types and ordered_types_for_context
will only return hashtag types which are enabled.
* CHROME_LOAD_EXTENSIONS_MANIFEST - An env var with a path to a file
that contains one path per line. These are paths to extensions installed
in chrome that the user wants to load while running system specs.
Useful to run things like Ember Inspector.
* CHROME_DISABLE_FORCE_DEVICE_SCALE_FACTOR - On some systems the
--force-device-scale-factor=1 argument makes the UI for chrome
super small, add a way to disable this.
Performing a `Delete User`/`Delete and Block User` reviewable actions for a
queued post reviewable from the `review.show` route results in an error
popup even if the action completes successfully.
This happens because unlike other reviewable types, a user delete action
on a queued post reviewable results in the deletion of the reviewable
itself. A subsequent attempt to reload the reviewable record results in
404. The deletion happens as part of the call to `UserDestroyer` which
includes a step for destroying reviewables created by the user being
destroyed. At the root of this is the creator of the queued post
being set as the creator of the reviewable as instead of the system
user.
This change assigns the creator of the reviewable to the system user and
uses the more approapriate `target_created_by` column for the creator of the
post being queued.
Why this change?
The `legacy` navigation menu option for the `navigation_menu` site
setting will be removed shortly after the release of Discourse 3.1 in
the first beta release of Discourse 3.2. Therefore, we're adding an
admin dashboard warning to give sites on the `legacy` navigation menu a
heads up.
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
Why was the test flaky?
The test relied on the fact that visiting a topic would marked its
post as unread. However, we did not actually stay on the topic long
enough in some cases for it to be considered read based on the logic in
our client side code.
This commit fixes the flakiness by ensuring that the post has actually
been read before navigating away.
Why this change?
The user id in a fixture file was hardcoded to 666. Once we've
fabricated enough user objects until the sequence for `User#id` reaches
666, the specs in vanilla_body_parser_spec.rb will fail.
What is the fix here?
This commit increases the user id to a large integer which we will
likely never hit in the next 10-20 years.
1) Bookmarking posts and topics topic level bookmarks clears all topic bookmarks from the topic bookmark button if more than one post is bookmarked
Failure/Error: expect(Bookmark.where(user: current_user).count).to eq(0)
expected: 0
got: 2
Why this change?
Prior to this change, we would only return tags that are used in at
least one public topic. However, this is confusing for users because the
tag could be used in a restricted category and that is not considered a
"public" topic. Instead, we will just display all the tags in the edit
tags navigation modal as long as it is visible to the user.
Wikimedia provides a thumbnail url for its images, so we should use that
for oneboxes instead of the full-size image. Because the size of the
onebox image we display is quite small anyways the thumbnail wikimedia
provides should suffice and will save bandwidth.
See: https://meta.discourse.org/t/264039
We recently introduced this advice to admins when some translation overrides are outdated or using unknown interpolation keys:
However we missed the case where the original translation key has been renamed or altogether removed. When this happens they are no longer visible in the admin interface, leading to the confusing situation where we say there are outdated translations, but none are shown.
Because we don't explicitly handle this case, some deleted translations were incorrectly marked as having unknown interpolation keys. (This is because I18n.t will return a string like "Translation missing: foo", which obviously has no interpolation keys inside.)
This change adds an additional status, deprecated for TranslationOverride, and the job that checks them will check for this status first, taking precedence over invalid_interpolation_keys. Since the advice only checks for the outdated and invalid_interpolation_keys statuses, this fixes the problem.
Follow up to: 56e792d
Adds a test to check that there is an api scope for the t/external_id
route. Plus checks many other topic routes that should have scopes.
Using the lastViewedTopicId indiscriminately can cause strange scrolling behavior when navigating to a **different** topic list after viewing a topic. We only want to refocus the topic when going 'back' to the same topic list which originally triggered the navigation.
Previously we were implementing scroll reset/memorization on a per-page basis. Many of these approaches relied on the `didInsertElement` hook, which is no longer appropriate since Discourse changed to use the 'loading slider' strategy for page transitions.
This commit rips out all of our custom scroll resetting/memorizing, and implements those things in a generic service. There are two features:
1. After every route transition, scroll to the top of the page
2. When using browser back/forward buttons, restore the last known scroll position for those routes
To opt-out of the behaviour, individual routes can add a scrollOnTransition boolean to their RouteInfo metadata using Ember's `buildRouteInfoMetadata` hook.
Why this change?
Prior to this change, dismissing unreads posts did not publish the
changes across clients for the same user. As a result, users can end up
seeing an unread count being present but saw no topics being loaded when
visiting the `/unread` route.
Why this change?
Prior to this change, the ordering of the tags shown in the email subject
was non-deterministic as there was no specific order specified. This
problem was exposed by a flaky test which we had.
What is the fix?
This commit orders the tags used in the email subject first by the
`Tag#public_topic_count` column in descending order and then the `Tag#name`
column in ascending order.
Why this change?
Group mention notifications are currently placed in the "Others" tab
of the user menu which is odd considering that mentioned notifications
are in the reply tab. This commit changes it such that group mention
notifications are displayed in the reply tab as well.
This commit makes sure we don't load all data into memory when doing CSV exports.
The most important change here made to the recently introduced export of chat
messages (3ea31f4). We were loading all data into memory in the first version, with
this commit it's not the case anymore.
Speaking of old exports. Some of them already use find_each, and it worked as
expected, without loading all data into memory. And it will proceed working as
expected after this commit.
In general, I made sure this change didn't break other CSV exports, first manually, and
then by writing system specs for them. Sadly, I haven't managed yet to make those
specs stable, they work fine locally, but flaky in GitHub actions, so I've disabled them
for now.
I'll be making more changes to the CSV exports code soon, those system specs will be
very helpful. I'll be running them locally, and I hope I'll manage to make them stable
while doing that work.
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.
The only exception to this is caches that aren't caching things from
postgres.
This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.
Reapplying this - it now doesn't use after_commit if skip_db is set
* FEATURE: Inline topic summary. Cached version accessible to everyone.
Anons and non-members of the `custom_summarization_allowed_groups_map` groups can see cached summaries for any accessible topic. After the first 12 hours and if the posts to summarize have changed, allowed users clicking on the button will automatically re-generate it.
* Ensure chat summaries work and prevent model hallucinations when there are no messages.
This adds an option to allow non-image s3 files to be downloaded through CDN URL.
Addresses the issues in:
* meta.discourse.org/t/s3-cdn-url-not-being-used-on-non-image-uploads/175332
* meta.discourse.org/t/s3-uploads-using-cdn-for-pdfs/213218
Why is this change being made?
We've decided that the previous "community" section should look more
like a primary section that holds the most important navigation links
for the site and the word "community" doesn't quite fit that
description. Therefore, we've made the decision to drop the
section heading for the community section.
As part of removing the section heading, the following changes are made
as well:
1. Button to customize the section has been moved to the "footer" of the
"More..." section when `navigation_menu` site setting is set to `sidebar`.
When `navigation_menu` is set to `header dropdown`, a button to customize
the section is shown inline.
2. The section will no longer be collapsable.
3. The title of the section is no longer customisable as it is no longer
displayed. As a technical note, we have not dropped any previous
customisations of the section's title previously in case we have to
bring back the header in the future.
4. The new topic button that was previously present in the header has
been removed alongside the header. Admins can add a custom section
link to the `/new-topic` route if there would like to make it easier for
users to create a new topic in the sidebar.
This PR adds a feature to help admins stay up-to-date with their translations. We already have protections preventing admins from problems when they update their overrides. This change adds some protection in the other direction (where translations change in core due to an upgrade) by creating a notice for admins when defaults have changed.
Terms:
- In the case where Discourse core changes the default translation, the translation override is considered "outdated".
- In the case above where interpolation keys were changed from the ones the override is using, it is considered "invalid".
- If none of the above applies, the override is considered "up to date".
How does it work?
There are a few pieces that makes this work:
- When an admin creates or updates a translation override, we store the original translation at the time of write. (This is used to detect changes later on.)
- There is a background job that runs once every day and checks for outdated and invalid overrides, and marks them as such.
- When there are any outdated or invalid overrides, a notice is shown in admin dashboard with a link to the text customization page.
Known limitations
The link from the dashboard links to the default locale text customization page. Given there might be invalid overrides in multiple languages, I'm not sure what we could do here. Consideration for future improvement.
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.
The only exception to this is caches that aren't caching things from
postgres.
This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.
Follow-up to b27e12445d
This commit adds 2 new site settings `default_sidebar_link_to_filtered_list` and `default_sidebar_show_count_of_new_items` to control the default values for the navigation menu preferences that were added in the linked commit (`sidebar_link_to_filtered_list` and `sidebar_show_count_of_new_items` respectively).
1) Edit Category when editing a category with form templates set should have form templates enabled and showing the selected templates
Failure/Error: expect(category_page).to have_selected_template(selected_templates)
expected `#<PageObjects::Pages::Category:0x00007fdb278fbd30>.has_selected_template?("template_0,template_1")` to be truthy, got false
Wait for CSS rather than trying to compare attr directly
and also make sure the ids are always in order.
This introduces a PLATFORM_KEY_MODIFIER const that
can be used both client and server side, to determine
whether we should be using the Meta or Ctrl key based
on whether the user is on Windows/Linux or Mac.
In previous PR https://github.com/discourse/discourse/pull/22340 bug was introduced. Notifications were blocked when, even if topic was watched directly. New query is taking TopicUser into consideration.
In addition, in user interface, when `watched_precedence_over_muted` is not set, then value from SiteSetting should be displayed.
Previously , the test was flaky and failing with a selenium stale
element error because we were retrieving the tag nodes with `all` and
then calling `.map(&:text)` on it. However, there is a chance that a
re-render happens and those nodes will end up being stale resulting in
the selenium error.
Why this change?
By ensuring the reset happens in an `ensure` code block, we ensure that
the code will always be run even if code fails or an error is raised.
This helps to prevent leaking custom network condition states and
improves the stability of our system tests.
Recently, site setting watched_precedence_over_muted was introduced - https://github.com/discourse/discourse/pull/22252
In this PR, we are allowing users to override it. The option is only displayed when the user has watched categories and muted tags, or vice versa.
What is the problem?
Before this change, we were relying on the `/tags` endpoint which
returned all the tags that are visible to a give user on the site leading to potential performance problems.
The attribute keys of the response also changes based on the `tags_listed_by_group` site setting.
What is the fix?
This commit fixes the problems listed above by creating a dedicate `#list` action in the
`TagsController` to handle the listing of the tags in the edit
navigation menu tags modal. This is because the `TagsController#index`
action was created specifically for the `/tags` route and the response
body does not really map well to what we need. The `TagsController#list`
action added here is also much safer since the response is paginated and
we avoid loading a whole bunch of tags upfront.
What is the problem?
This regressed in fe294ab1a7 and we did
not have any tests on mobile to catch the regression. The problem was
that we were conditionally rendering the edit nav menu modals component
in the sidebar. However, the sidebar is collapsed on mobile when a
button is clicked. When the sidebar collapses, the edit nav menu modals
ended up being destroyed with it.
Why this change?
A new component based API for modals was introduced in
b3a23bd9d6. This commit moves the edit
navigation menu tags and categories modal to the new API.
What is the problem?
Before this change, the edit navigation menu tags modal was not
displaying tags that belonged to a tag_group when the tags_listed_by_group
site setting was set to true. This is because we are relying on the
/tags endpoint which returned tags in various keys depending on the
tags_listed_by_group site setting. When the site setting is set to
true, tags under belonging to tag groups were returned in the
extra.tag_groups attribute.
What is the fix?
This commit fixes it by pushing all tags in returned under the
`tag_groups` attribute into the list of tags to displayed. In a
following commit, we will move away from the `/tags` endpoint to a
dedicated route to handle the listing of tags in the modal.
]When changing fonts in the `/wizard/steps/styling` step of
the wizard, users would not see the font loaded straight away,
having to switch to another one then back to the original to
see the result. This is because we are using canvas to render
the style preview and this fails with a Chrome-based intervention
when font loading is taking too long:
> [Intervention] Slow network is detected. See
https://www.chromestatus.com/feature/5636954674692096 for more details.
Fallback font will be used while loading:
https://sea2.discourse-cdn.com/business7/fonts/Roboto-Bold.ttf?v=0.0.9
We can get around this by manually loading the fonts selected using
the FontFace JS API when the user selects them and before rerendering
the canvas. This just requires preloading more information about the
fonts if the user is admin so the wizard can query this data.
Before this change, links which required full reload because they are not in ember routes like `/my/preferences` or links to docs like `/pub/*` were treated as real external links. Therefore, they were opening in self window or new tab based on user `external_links_in_new_tab` setting.
To be consistent with behavior when full reload links are in the post, they are treated as internal and always open in the same window.
Recently, we added the option for watched tag/categories to take precedence over muted tag/categories. Therefore, `remove_muted_tags` is using `category_users` to check if categories are not watched. There was missing join in CategoryList which was causing an error.
This is the first of a number of PRs aimed at helping admins manage their translation overrides. It simply adds a list of available interpolation keys below the input field when editing an override.
It also includes custom interpolation key.
Introduced in cec68b3e2c,
this is flaky because if you click the back button before
the route is fully transitioned to the loaded thread,
we end up going to the history _before_ the thread list,
which ends up being the channel.
We need to make sure that everything is loaded for the
thread first, meaning the skeleton is not there.
Also exclude some noise from the capybara logs (image load failures)
What is the problem?
When an admin changes the default_sidebar_categories or default_sidebar_tags site settings and opts to backfill the setting,
we currently enqueue a sidekiq job to run the backfilling operation. When an admin changes those settings multiple times
within a short time frame, multiple sidekiq jobs with different backfilling parameters will be enqueued.
This is problematic if multiple jobs are executed concurrently as it may lead to situations where a job
with “outdated” site setting values is completed after a job with the “latest” site setting values.
What is the fix?
By setting `cluster_concurrency` to `1`, we ensure that only one of such
backfilling job will execute across all the sidekiq processes that are
deployed at any point in time. Since Sidekiq pops off job in the order
in which they are pushed, limiting the cluster concurrency here will
allow us to execute the enqueued `Jobs::BackfillSidebarSiteSettings`
jobs serially.
Why this change?
We are currently not fully satisfied with the current way to edit the
categories and tags that appears in the sidebar where the user is
redirected to the tracking preferences tab in the user's profile causing
the user to lose context of the current page. In addition, the dropdown
to select categories or tags limits the amount of information we can
display.
Since editing or adding a custom categories section is already using a
modal, we have decided to switch editing the categories and tags that
appear in the sidebar to use a modal as well.
This commit removes the `new_edit_sidebar_categories_tags_interface_groups` site setting and
make the modals the default for all users.
Updates the interface for implementing summarization strategies and adds a cache layer to summarize topics once.
The cache stores the final summary and each chunk used to build it, which will be useful when we have to extend or rebuild it.
New setting which allow admin to define behavior when topic is in watched category and muted topic and vice versa.
If watched_precedence_over_muted setting is true, that topic is still visible in list of topics and notification is created.
If watched_precedence_over_muted setting is false, that topic is not still visible in list of topics and notification is skipped as well.
Why does this change do?
If the `fixed_category_positions` is `false`, we want to order the
categories in the edit navigation menu categories modal by name. This
makes it easier to filter through a large list of categories.
This commit also fixes a bug where we were unintentionally mutating the
`this.site.categories` array.
While we are unable to support OAUTH2 with pop3 (due to upstream dependency ruby/net-pop#16), we are adding the support for mail pollers plugin. Doing so, it would be possible to write a plugin which then uses other ways (microsoft graph sdk for example) to poll emails from a mailbox.
The idea is that a plugin would define a class which inherits from Email::Poller and defines a poll_mailbox static method which returns an array of strings. Then the plugin could call register_mail_poller(<class_name>) to have it registered. All the configuration (oauth2 tokens, email, etc) could be managed by sitesettings defined in the plugin.
This change adds support retroactively updating display names in the new quote format when the user's name is changed. It happens through a background job that is triggered by a callback when a user is saved with a new name.
Why is this change required?
Previously, the tests in `viewing_sidebar_as_anonymous_user_spec.rb` was
flaky because the ordering of the tags changes depending on what the
auto generated tag names are. If a tag name is generated with the name
`tag10`, it would then be sorted before `tag9` which messes up the
ordering specified in our tests. This commit fixes the problem by
specifying the tag names instead of relying on the auto generated ones
by fabricator.
What does this change do?
This change adds a dropdown filter that allows a user to filter by
selected or unselected categories/tags in the edit navigation menu
modal.
For the categories modal, parent categories that do not match the
dropdown filter will be displayed as disabled since those parent
categories need to be displayed to maintain the hieracy of the child
child categories.
Why this change?
Predicate matchers are poor at providing good error messages when it
fails if all the predicate matcher does is to return a boolean. Prior to
this change, we were using `has_css? && all?` to assert for the tag
section links. There are two problems here. Firstly, when one of the matchers
fail, the error message does not provide any indication of which matcher
failed making it hard to debug failures. Secondly, the matchers were not
able to assert for the ordering of the tag section links which is an
important behaviour to assert for.
This commit changes `PageObjects::Components::Sidebar#has_tag_section_links?`
such that we make use of assertions to ensure ordering. The usage of
`all` will also provide a clear error message when things go wrong.
Why this change?
There was alot of duplication between the edit navigation menu tags/categories modal which
was making it hard to introduce new changes as the work had to be
duplicated into multiple places.
This commit mainly extracts the duplicated code into common components
such that it is easier to make styling changes across both modals.
This PR splits up the preference that controls the count vs dot and destination of sidebar links, which is really hard to understand, into 2 simpler checkboxes:
The new preferences/checkboxes are off by default, but there are database migrations to switch the old preference to the new ones so that existing users don't have to update their preferences to keep their preferred behavior of sidebar links when this changed is rolled out.
Internal topic: t/103529.
What this change?
When a user opens the modal to edit tags or categories for the
navigation menu, we want to input filter to have focus. This commit
fixes that by doing the following:
1. Changes <DModal> component such that it prioritises elements with the
autofocus attribute first.
2. Adds `autofocus` to the input elements on the edit tags/categories
modal form.
Don't cache user_fields on users separately from custom_fields, since they can get out of sync.
---------
Co-authored-by: Daniel Waterworth <me@danielwaterworth.com>
When a site does not have `default_navigation_menu_tags`
site setting set, anonymous users should be shown the site's top tags as
a default in the tags section. However, this regressed in 9fad71809c
and we ended up showing anonymous users a tags section with only the
`All Tags` section link.
As part of this commit, I have also refactored the QUnit acceptance
tests to system tests which are much easier to work with.
What does this change do?
This change adds the deselect all and reset to defaults buttons to the
edit navigation menu tags modal. The deselect all button when
clicked deselects all the selected tags in the modal. If the user
saves with no tags selected, the user's tags section in the
navigation menu will be set to the site's top tags.
The reset to defaults button is only shown when the
`default_navigation_menu_tags` site setting has been configured.
When clicked, the user's tags section in the navigation menu is
automatically set to the tags defined by the
`default_navigation_menu_tags` site setting.
There is a problem that unread and new count is not updated to reflecting topicTrackingState.
It is because discourseComputed on Category is not working properly with topicTrackingState. Moving it to component level is making counter reliable.
What does this change do?
This commit adds an input filter to filter through the tag checkboxes in the
modal to edit tags that are shown in the user's navigation menu. The
filtering is a simple matching of the given filter term against the
names of the tags.
What does this change do?
This change is a first pass for adding a modal used to edit tags that appears in
the navigation menu. As the feature is being worked on in phases, it is
currently hidden behind the `new_edit_sidebar_categories_tags_interface_groups` site setting.
The following features will be worked on in future commits:
1. Input filter to filter through the tgas
2. Button to reset tag selection to default navigation menu tags site
settings
3. Button to deselect all current selection
Why this change?
The comment consists of an output that was copied from RSpec's default
output. This has the potential to mess with systems that are parsing
RSpec's output to fetch the spec failures as those systems are usually
looking for the first occurence of `Failures:`
We use it like this:
expect(message.created_at).to eq_time(created_at)
The problem is that if one of the values or both of them are `nil` the matcher fails
with this error:
NoMethodError: undefined method `-' for nil:NilClass
This commit adds support for `nil` values. If both time values are `nil` they are equal,
if only one value is `nil` they aren't.
This commit adds an aria-label attribute to cooked hashtags using
the post/chat message decorateCooked functionality. I have just used
the inner content of the hashtag (the tag/category/channel name) for
the label -- we can reexamine at some point if we want something
different like "Link to dev category" or something, but from what I
can tell things like Twitter don't even have aria-labels for hashtags
so the text would be read out directly.
This commit also refactors any ruby specs checking the HTML of hashtags
to use rspec-html-matchers which is far clearer than having to maintain
the HTML structure in a HEREDOC for comparison, and gives better spec
failures.
c.f. https://meta.discourse.org/t/hashtags-are-getting-a-makeover/248866/23?u=martin
https://meta.discourse.org/t/markdown-preview-and-result-differ/263878
The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
What does this change do?
This change adds the deselect all and reset to defaults buttons to the
edit navigation menu categories modal. The deselect all button when
click deselects all the selected categories in the modal. If the user
saves with no categories selected, the user's categories section in the
navigation menu will be set to the site's top categories.
The reset to defaults button is only shown when the
`default_navigation_menu_categories` site setting has been configured.
When clicked, the user's categories section in the navigation menu is
automatically set to the categories defined by the
`default_navigation_menu_categories` site setting.
Previously, it will add a "Re:" prefix to all the emails passing through the group SMTP mailer. It's creating confusion while receiving the mail for the first time on a PM.
1. `everything` was changed to `topics`
2. Path for my posts translation is `sidebar.sections.community.links.my_posts.content` not `sidebar.sections.community.links.my/posts.content`
Different environments run on different hardware so response times vary
based on hardware. Instead of hardcoding the timeout for
`SystemHelpers#try_until_success` to 2 seconds, we change it such that
it follows `Capybara.default_wait_timeout` which we have configured to
be higher in environments which runs on lousier hardware.
This change should reduce the amount of flakiness we're seeing on CI
with tests that rely on `SystemHlpers#try_until_success`.
Upstream added a capital 'T' to the 'Translation missing' message in https://github.com/ruby-i18n/i18n/commit/c5c6e753f3. This caused our translate accelerator patch to diverge, and the change in case affected a number of our specs. This commit updates the translate accelerator to match the upstream casing, and introduces a spec to detect future divergence.
Today I learnt that `has_link?("text of link")` by default does an
includes instead of looking for a link with the exact text. This is not
the behaviour I want so I'm changing
`PageObjects::Components::Sidebar.has_section_link?` to use the
`exact_text` option instead.
This method is a huge footgun in production, since it calls
the Redis KEYS command. From the Redis documentation at
https://redis.io/commands/keys/:
> Warning: consider KEYS as a command that should only be used in
production environments with extreme care. It may ruin performance when
it is executed against large databases. This command is intended for
debugging and special operations, such as changing your keyspace layout.
Don't use KEYS in your regular application code.
Since we were only using `delete_prefixed` in specs (now that we
removed the usage in production in 24ec06ff85)
we can remove this and instead rely on `use_redis_snapshotting` on the
particular tests that need this kind of clearing functionality.
Added a new modifier hook to allow plugins to modify the @mentions
extracted from a cooked text.
Use case: Some plugins may change how the mentions are cooked to prevent
them from being confused with user or group mentions and display the user
card.
This modifier hook allows the plugin to filter the mentions detected or add new ways
to add mentions into cooked text.
Communities can use sidebar or header dropdown, therefore navigation menu is a better name settings in 2 places:
- Old user sidebar preferences;
- Site setting about default tags and categories.
* FEATURE: Implement max_tags_per_email_subject
* made it so only max_tags_per_email_subject is responsible for tags in emails when the feature is enabled
* added locales for implemented siteSettings
* reworded locale for enable_max_tags_per_email_subject
* added min value for max_tags_per_email_subject
* Implemented suggested changes to spec description
Currently, groups owned by moderators are not visible to them on the
groups page. This happens because, the group visibility queries don't
account for non-admin staff user group ownership.
This change updates the group visibility scope queries to account for a
moderator(non-admin staff user) group ownership.
When we introduced the new quote format with full-name display name:
```
[quote="Ted Johansson, post:1, topic:2, username:ted"]
we overlooked the code responsible for rewriting quotes when a user's name is changed.
```
The functional part of this change adds support for the new quote format in the code that updates quotes when a user's username changes. See the test case in `spec/services/username_changer_spec.rb` for the details.
In addition, this change adds a regression test for PrettyText to cover the new quote format, and extracts the code responsible for rewriting raw and cooked quotes into its own `QuoteRewriter` class. The functionality of the latter is tested through the tests in `spec/services/username_changer_spec.rb`.
Anonymization is among the most expensive operations we can perform with
extreme potential to impact the database. To mitigate risk we only allow a
single anonymization across the entire cluster concurrently.
This commit introduces support for `cluster_concurrency 1`. When you set that on a Job it will only allow 1 concurrent execution per cluster.
* FEATURE: Content custom summarization strategies.
This PR establishes a pattern for plugins to register alternative ways of summarizing content by extending a class that defines an interface.
Core controls which strategy we'll use and who has access to it through the `summarization_strategy` and `custom_summarization_allowed_groups`. It also defines the UI for summarizing topics.
Other plugins can access this summarization mechanism and implement their features, removing cross-plugin customizations, as it currently happens between chat and the discourse-ai plugin.
* Group membership validation and rate limiting
* Work with objects instead of classes
* Port summarization feature from discourse-ai to chat
* Rename available summaries to 'Top Replies' and 'Summary'
When we introduced unicode support in the regular expressions used in watched words (9a27803) we didn't realize the cost adding the `u` flag would be.
Turns out, it's pretty bad when you have lots of regular expressions to test. A customer had slightly less than 200 watched words, and it would freeze the browser for about 2s on the first check of those regular expressions (roughly 10ms per regular expression).
This commit introduces a new field (`word`) to the serialized watched words which is then converted to a very fast and cheap regular expression on the client-side. We use that regexp to quicly check whether a matcher is even worth trying so that we don't incure the cost of compiling the expensive unicode regexp.
This commit also busts the `WordWatcher` cache since we added a new field to be serialized.
One nice side effect of using `matchAll` instead of a `while / exec` loop is that the likeliness of having a bad regexp matching infinitely is vastly reduced 🙌
This patch sets some limits on custom fields:
- an entity can’t have more than 100 custom fields defined on it
- a custom field can’t hold a value greater than 10,000,000 characters
The current implementation of custom fields is relatively complex and
does an upsert in SQL at some point, thus preventing to simply add an
`ActiveRecord` validation on the custom field model without having to
rewrite a part of the existing logic.
That’s one of the reasons this patch is implementing validations in the
`HasCustomField` module adding them to the model including the module.
Clicking on TOC heading anchors in a subfolder setup was breaking the current URL for users.
Other than the fix this change introduces the ability to test the subfolder setup in system specs.
