Add a new column - `user_agent` - to the `SearchLog` table.
This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
The user serializer groups method previously relied on the members_visible_groups to determine groups that the user should be able to see, however this setting was intended for visibility of group members (which is entirely different).
The result of this could be seen when choosing a primary group from user preferences -> account, due to the serializer the group name was not visible when members_visible_groups was set to owners.
In our official Docker image, running git commands results in the
following error:
```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:
git config --global --add safe.directory /var/www/discourse
```
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.
Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
Instead of the deprecated 'min trust level to allow ignore' in order to reduce the number of deprecation notices in the logs.
This tweaks a few serializers so that the 'can_ignore_users?` property is always coming from the server and properly used on the client-side.
SiteSetting.hide_user_profiles_from_public raises a Forbidden, which disallows our after_action: add no index header from triggering.
This fix makes sure that the no index header gets added via before_action instead
When tag preference in group and site settings are both used with same default notification level it will break new users signups because it tries to create duplicate records in the tag_users table which can’t happen because we have a unique index set.
If an existing user (John) accepts an invite created by Kenny to a group, John may be seen as invited by Kenny, despite already having an account on the site.
This fix removes the bug by excluding invites that determine the invited_by after the user's creation date. The delay buffer in the query accounts for invites that also create the user at the same time.
Badges can have their associated image uploads deleted. When this happens, any user who has that badge will have their profile page error out.
After this fix, when deleting an upload that's associated with a badge, we nullify the foreign key ID on the badge. This makes the existing safeguard work correctly.
There's currently a race condition in the following spec:
65be7a7880/spec/system/admin_about_config_area_spec.rb (L70-L95)
where the form can be saved before the image uploader field has finished uploading the selected image and causing the assertion at line 94 to fail with the following error:
```
Failure/Error: expect(SiteSetting.about_banner_image.sha1).to eq(Upload.generate_digest(image_file))
NoMethodError:
undefined method `sha1' for nil
[Screenshot Image]: /__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_admin_about_config_area_page_the_general_settings_card_can_saves_its_fields_to_their_corresponding_site_settings_312.png
~~~~~~~ JS LOGS ~~~~~~~
http://localhost:31338/assets/vendor.js 15902:14 "WARNING: uppy needs a unique id, pass one in to the component implementing this mixin"
~~~~~ END JS LOGS ~~~~~
./spec/system/admin_about_config_area_spec.rb:94:in `block (3 levels) in <main>'
./spec/rails_helper.rb:552:in `block (3 levels) in <top (required)>'
./spec/rails_helper.rb:552:in `block (2 levels) in <top (required)>'
./spec/rails_helper.rb:513:in `block (3 levels) in <top (required)>'
./spec/rails_helper.rb:503:in `block (2 levels) in <top (required)>'
./spec/rails_helper.rb:460:in `block (2 levels) in <top (required)>'
./vendor/bundle/ruby/3.3.0/gems/webmock-3.23.1/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'
```
This PR fixes the problem by making the system test wait for the image to finish uploading (with 10 seconds timeout) before carrying out the rest of the system test.
Followup 2f2da72747
When the "Consolidated Pageviews with Browser Detection (Experimental)"
report was introduced, we started counting the original
"page_view_logged_in" and "page_view_anon" ApplicationRequest
data as "Other Pageviews", subtracting
"page_view_anon_browser" and "page_view_logged_in_browser" from
this number.
However we unknowingly automatically started counting these
browser-based page views, which are a subset of the total
"page_view_logged_in" and "page_view_anon" counts, in the
original "Pageviews" report, leading to double counting
which meant that when you looked at the data for each
report side-by-side the data didn't add up.
This commit fixes the issue by not counting the "browser"
pageviews in the Pageviews report, and making the code where
we were only counting certain types of requests for this
report more plain, explicitly stating which types of requests
we want.
When a topic embed is run with either no tags argument or a nil tag argument
this should not affect any existing tags.
Only update topic tags when tags argument is explicitly empty.
Before checking if flags were reordered on the topic page, we need to ensure that the reorder action was finished. To achieve it "saving" CSS is added and removed when AJAX call is completed.
Followup 2f2da72747
This commit moves topic view tracking from happening
every time a Topic is requested, which is susceptible
to inflating numbers of views from web crawlers, to
our request tracker middleware.
In this new location, topic views are only tracked when
the following headers are sent:
* HTTP_DISCOURSE_TRACK_VIEW - This is sent on every page navigation when
clicking around the ember app. We count these as browser page views
because we know it comes from the AJAX call in our app. The topic ID
is extracted from HTTP_DISCOURSE_TRACK_VIEW_TOPIC_ID
* HTTP_DISCOURSE_DEFERRED_TRACK_VIEW - Sent when MessageBus initializes
after first loading the page to count the initial page load view. The
topic ID is extracted from HTTP_DISCOURSE_DEFERRED_TRACK_VIEW.
This will bring topic views more in line with the change we
made to page views in the referenced commit and result in
more realistic topic view counts.
By default, secure sessions expire after 1 hour.
For OAuth authentication it should expire at the same time when the authentication cookie expires - `SiteSetting.maximum_session_age.hours`.
It is possible that the forum will not have persistent sessions, based on `persistent_sessions` site setting. In that case, with next username and password authentication we need to reset information about OAuth.
Bug introduced in this PR - https://github.com/discourse/discourse/pull/27547
* FIX: Division by zero error on WebHookEventsDailyAggregate
* DEV: Update implementation of WebHookEventsDailyAggregate to handle division by zero error
I am changing many of these to notes or resolving them as is,
most of these I have not actively worked on in years so someone
else can work on them when we get to these areas again.
This commit continues work laid out by ffec8163b0 for the admin config page for the /about page. The last commit set up the user interface, and this one sets up all the wiring needed to make the input fields and save buttons actually work.
Internal topic: t/128544.
While creating a new category if the user didn't specify a value for `minimum_required_tags` input but clicked it then it returned the "PG::NotNullViolation: null value in column 'minimum_required_tags'" error.
When bad data is provided in the URI for redirecting to a category,
Rails raises an `ActionController::Redirecting::UnsafeRedirectError`
error, leading to a 500 error.
This patch catches the exception to render a 404 instead.
Currently redirecting to an external URL through a permalink doesn’t
work because Rails raises a
`ActionController::Redirecting::UnsafeRedirectError` error.
This wasn’t the case before we upgraded to Rails 7.0.
This patch fixes the issue by using `allow_other_host: true` on the
redirect.
If, for whatever reasons, the user's locale is "blank" and an admin is accepting their group membership request, there will be an error because we're generating posts with the locale of recipient.
In order to fix this, we now use the `user.effective_locale` which takes care of multiple things, including returning the default locale when the user's locale is blank.
Internal ref - t/132347
When using the full page search and filtering down to a specific topic, the sort order was overwritten to by by "post_number".
This was confusing because we allow different type of sort order in the full search page.
This fixes it by only sorting by post_number when there's no "global" sort order defined.
Since the "new topic map" uses the search endpoint behind the scene, this also fixes the "most likes" popup.
Context - https://meta.discourse.org/t/searching-order-seems-to-be-broken-when-searching-in-topic/312303
* DEV: add db migration to filter out invalid csp script source values
* DEV: insert UserHistory row during data migration to track old value for content_security_policy_script_src site setting
When visiting a user profile, and then opening the search, there's an option to filter down by posts made by that user.
When clicking that option, it used to pre-fill the "search bar" with "@<username>" to filter down the search.
This restore this behaviour and add a system spec to ensure it doesn't regress.
Context - https://meta.discourse.org/t/in-posts-by-search-option-does-not-work-when-clicked/312916
We want to allow admins to make new required fields apply to existing users. In order for this to work we need to have a way to make those users fill up the fields on their next page load. This is very similar to how adding a 2FA requirement post-fact works. Users will be redirected to a page where they can fill up the remaining required fields, and until they do that they won't be able to do anything else.
Followup 96a0781bc1
When sending emails where secure uploads is enabled
and secure_uploads_allow_embed_images_in_emails is
true, we attach the images to the email, and we
do some munging with the final email so the structure
of the MIME parts looks like this:
```
multipart/mixed
multipart/alternative
text/plain
text/html
image/jpeg
image/jpeg
image/png
```
However, we were not specifying the `boundary` of the
`multipart/mixed` main content-type of the email, so
sometimes the email would come through appearing to
have an empty body with the entire thing attached as
one attachment, and some mail parsers considered the
entire email as the "epilogue" and/or "preamble".
This commit fixes the issue by specifying the boundary
in the content-type header per https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
Adds a checkbox to filter untranslated text strings in the admin UI, behind a hidden and default `false` site setting `admin_allow_filter_untranslated_text`.
Followup to 0e1102b332
Minor followup, makes the condition check against the
boolean val, see the difference here:
```ruby
!SiteSetting.enforce_second_factor_on_external_auth && "true"
=> "true"
```
vs:
```ruby
!SiteSetting.enforce_second_factor_on_external_auth && "true" == "true"
=> true
```
* Removed the link from the title, so the settings can only be accessed via the settings button on the right
* Added an icon to the "Learn more" link to indicate that it opens a new window
* Made various styling adjustments
* DEV: Upgrade Rails to 7.1
* FIX: Remove references to `Rails.logger.chained`
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
* DEV: Make parameters optional to all FakeLogger methods
* FIX: Set `override_level` on Logster loggers (#27519)
A followup to f595d599dd
* FIX: Don’t duplicate Rack response
---------
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
In some instances, the `modifications` of `tags` hasn't been properly serialized as a Ruby array but rather as a string (I've seen `""`, `"[]"`, and `"[\"\"]"`).
This generates an error when we try to `filter_tags` and remove `hidden_tags` (which is an array) from `tags` which might be a string.
Internal ref - t/131126
I wasn't able to figure out the root cause of this so I reverted the behavior that was introduced ~6 years ago in f2c060bdf2
This ensures that the theme id is resolved as early as possible in the
request cycle. This is necessary for the custom homepage to skip
preloading the wrong data.
Previously filter route was not setting topic list, this meant that
keyboard navigation using "G" "J" was not functioning.
This amends it by ensuring the list is set after looking up the model.
* DEV: Upgrade Rails to 7.1
* FIX: Remove references to `Rails.logger.chained`
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
* DEV: Make parameters optional to all FakeLogger methods
* FIX: Set `override_level` on Logster loggers (#27519)
A followup to f595d599dd
* FIX: Don’t duplicate Rack response
---------
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
In this PR we introduced `enforce_second_factor_on_external_auth` setting https://github.com/discourse/discourse/pull/27506
When it is set to false and the user is authenticated via OAuth, then we should not enforce the 2fa configuration.
We recently fixed a problem where secure upload images weren't re-attached when sending the activity summary e-mail.
This fix contained a bug that would lead to n copies of the e-mail body being included, n being the number of duplicates. This is because #fix_parts_after_attachments! was called once per attachment, and adding more parts to the multipart e-mail.
This PR fixes that by:
Adding a failing test case for the above.
Moving the looping over multiple posts into #fix_parts_after_attachments! itself.
When a post is cooked the links are extracted and `TopicLink` instances
are created for each of them. These links are used in various places,
including the topic view, user summary page, etc.
In previous commit 48e5d1a, hotlinked images from Oneboxes have been
ignored from the texts, but hotlinked images turned into Lightboxes
were still extracted.
The test that checks that securely uploaded images are re-attached to the digest e-mail wasn't rendering the actual digest e-mail template. This change fixes that.
Many site settings can be distructive or have huge side-effects
for a site that the admin may not be aware of when changing it.
This commit introduces a `requires_confirmation` attribute that
can be added to any site setting. When it is true, a confirmation
dialog will open if that setting is changed in the admin UI,
optionally with a custom message that is defined in client.en.yml.
If the admin does not confirm, we reset the setting to its previous
clean value and do not save the new value.
Some tooling may rely on an unsafe-none cross origin opener policy to work. This change adds a hidden site setting that can be used to list referrers where we add this header instead of the default one configured in cross_origin_opener_policy_header.
For Topic Embeds, we would prefer <article> to be the main article in a topic, rather than a table cell <td> with potentially a lot of data. However, in an example URL like here, the table cell (the very large code snippet) is seen as the Topic Embed's article due to the determined content weight by the Readability library we use.
In the newly released 0.7.1 cantino/ruby-readability#94, the library has a new option to exclude the library's default <td> element into content weighting. This is more in line with the original library where they only weighted <p>. So this PR excludes the td, as seen in the tests, to allow the actual article to be seen as the article. This PR also adds the details tag into the allow-list.
Followup 6b872c4c53
Even though we were showing a validation error for a reject
reason that was too long, we were still sending an email and
doing other operations on the user which we are rejecting.
This commit fixes this by validating the reviewable model
before attempting to do anything else after the reason is set.
A new admin setting called `enforce_second_factor_on_external_auth`. It allows users to authenticate using external providers even when 2FA is forced with `enforce_second_factor` site setting.
* Revert "FIX: Set `override_level` on Logster loggers (#27519)"
This reverts commit c1b0488c54.
* Revert "DEV: Make parameters optional to all FakeLogger methods"
This reverts commit 3318dad7b4.
* Revert "FIX: Remove references to `Rails.logger.chained`"
This reverts commit f595d599dd.
* Revert "DEV: Upgrade Rails to 7.1"
This reverts commit 081b00391e.
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
Currently when a cache entry is corrupt, we log the event without doing
anything else. It means the cache is still corrupt, and the proper value
isn’t computed again.
Normally, it’s very rare the cache becomes corrupt, but it can happen
when upgrading Rails for example and the cache format changes. This is
normally handled automatically by Rails but since we’re using a custom
cache class, we have to do it ourselves.
This patch takes the same approach the Rails team did, when a cache
entry is corrupt, we treat it as a miss, recomputing the proper value
and caching it in the new format.
Wasn't quite handling the cases where a closing bracket `]` was used in the value of one of the attributes.
```markdown
[chat quote=user channel="[broken]"]
```
Would not be correctly parsed because we would _greedily_ use the first `]` as the end of the tag even though it might be a valid character when inside proper quotes.
c39a4de139/app/assets/javascripts/discourse-markdown-it/src/features/bbcode-block.js (L62)
Re-wrote the `parseBBCodeTag` to properly handle the following cases
- A closing tag (aka `[/name]`) which are easy since they don't have any attributes
- An old `[quote=...]` format we used that doesn't uses quotes but still has various attributes of the form `key:value`
- All three valid BBCode opening tag formats we support
- `[name]` without any attributes
- `[name=foo]` with a default value
- `[name foo=bar]` with some attributes
Ended up having to fix/rewrite the few bbcode rules that were using the `parseBBCodeTag` function, namely `d-wrap` and `discourse-local-dates`.
While working on this, I think I also found a way to get rid the of shims we had in place so that plugins could use the `parseBBCodeTag` function.
Reference - https://meta.discourse.org/t/having-a-right-bracket-in-a-channel-name-breaks-all-quotes-from-that-channel/308439
* Load search results in displayed order so that when more categories are loaded on scroll, they appear at the end,
* Limit the number of subcategories that are shown per category and display 'show more' links,
This commit adds ability to fetch a subset of site settings from the `/admin/site_settings` endpoint so that it can be used in all places where the client app needs access to a subset of the site settings.
Additionally, this commit also introduces a new service class called `UpdateSiteSetting` that encapsulates all the logic that surrounds updating a site setting so that it can be used to update site setting(s) anywhere in the backend. This service comes in handy with, for example, the controller for the flags admin config area which may need to update some site settings related to flags.
Internal topic: t/130713.
This commits introduces the `sidekiq_report_long_running_jobs_minutes`
global setting which allows a site administrator to log a warning in the
Rails log when a Sidekiq job has been running for too long.
The warning is logged with the backtrace of the thread that is
processing the Sidekiq job to make it easier to figure out what a
sidekiq job is stuck on.
When we turn on settings automatically for customers,
we sometimes use `.set_and_log` which will make a staff
action log for the site setting change. This is fine, but
there is no context for customers.
This change allows setting a message with `.set_and_log`, which
will be stored in the `details` column of the staff action log
created, which will show up on `/admin/logs/staff_action_logs`
---------
Co-authored-by: Kelv <kelv@discourse.org>
In #26642 we introduced a change that re-attaches securely uploaded images in the digest e-mail. However, this change assumed that the type argument to the Email::Sender constructor would be a symbol, but when it is coming from the UserEmail job it is a string. This PR fixes that.
This introduces the syntax of
`category:a,b,c` which will search across multiple categories.
Previously there was no way to allow search across a wide selection of
categories.
After working on the Webhook events filter by Status, I noticed that the 'Delivered' and 'Failed' options do not take the status param when loading more than fifty Webhook events. It causes to load all Webhook events regardless of its status after the first load.
This PR is adding webhook events status for the filter to the param when loading more than fifty Webhook events.
This commit tries another work around for the `Socket::ResolutionError: getaddrinfo: Temporary failure in name resolution`
error we are seeing on CI.
The problem with the previous workaround is that `Capybara.using_session` will attempt to resolve `localhost`
before yielding the block which means our retry code is not hit.
This problem may be related to https://bugs.ruby-lang.org/issues/20172
which hints at us potentially not being able to spin up threads on CI so
I'm adding a debugging statement when stuff fails.
decorator-transforms (https://github.com/ef4/decorator-transforms) is a modern replacement for babel's plugin-proposal-decorators. It provides a decorator implementation using modern browser features, without needing to enable babel's full suite of class feature transformations. This improves the developer experience and performance.
In local testing with Google's 'tachometer' tool, this reduces Discourse's 'init-to-render' time by around 3-4% (230ms -> 222ms).
It reduces our initial gzip'd JS payloads by 3.2% (2.43MB -> 2.35MB), or 7.5% (14.5MB -> 13.4MB) uncompressed.
This was previously reverted in 97847f6. This version includes a babel transformation which works around the bug in Safari <= 15.
For Cloudflare compatibility issues, check https://meta.discourse.org/t/311390
Not all HTML elements are converted into Markdown. Some are kept as HTML.
Without this fix XML/HTML entities that are formatted as text instead of code are swallowed by Discourse.
This also fixes quotes in the `title` attribute of the `<abbr>` tag.
Previously `HtmlToMarkdown` always converted HTML tables into Markdown tables. That lead to some badly formatted Markdown tables, e.g. when the table contained `rowspan` or `colspan`. This solves the issue by using very basic HTML tables in those cases.
When running ` rspec spec/services/external_upload_manager_spec.rb`
in the development environment, tests were failing with the following
error:
```
NameError:
uninitialized constant FakeS3::Aws
```
This commit introduces a hidden `s3_inventory_bucket` site setting which
replaces the `enable_s3_inventory` and `s3_configure_inventory_policy`
site setting.
The reason `enable_s3_inventory` and `s3_configure_inventory_policy`
site settings are removed is because this feature has technically been
broken since it was introduced. When the `enable_s3_inventory` feature
is turned on, the app will because configure a daily inventory policy for the
`s3_upload_bucket` bucket and store the inventories under a prefix in
the bucket. The problem here is that once the inventories are created,
there is nothing cleaning up all these inventories so whoever that has
enabled this feature would have been paying the cost of storing a whole
bunch of inventory files which are never used. Given that we have not
received any complains about inventory files inflating S3 storage costs,
we think that it is very likely that this feature is no longer being
used and we are looking to drop support for this feature in the not too
distance future.
For now, we will still support a hidden `s3_inventory_bucket` site
setting which site administrators can configure via the
`DISCOURSE_S3_INVENTORY_BUCKET` env.
This is a follow up to 9ff0805a1d. We
noticed that `localhost` can fail to resolve in other spots of the app
and not just in selenium-webdriver.
From the failing tests we have seen, the `getaddrinfo: Temporary failure in name resolution` error is only
seen from within the `Capybara.using_session` block. This commit aims to
ensure that `localhost` can be resolve after the new session is started.
* FEATURE: Add Filter for Webhook Events by Status
* Fixing multiple issues
* Lint
* Fixing multiple issues
* Change the range of the status for webhook events
We want to get rid of the old topic bulk actions modal
and use the new dropdown (currently gated behind
experimental_topic_bulk_actions_enabled_groups). To do
this we need to use the new dropdown in all places in the
UI.
This commit changes the full page search UI to use the new
topic bulk actions dropdown if experimental_topic_bulk_actions_enabled_groups
is enabled, and makes some minor refactors to make this work.
Also add a spec for both the old and new functionality.
This commit fixes a problem where the user will not be able to reset
their password when they only have security keys and backup codes
configured.
This commit also makes the following changes/fixes:
1. Splits password reset system tests to
`spec/system/forgot_password_spec.rb` instead of missing the system
tests in `spec/system/login_spec.rb` which is mainly used to test
the login flow.
2. Fixes a UX issue where the `Use backup codes` or `Use authenticator
app` text is shown on the reset password form when the user does
not have either backup codes or an authenticator app configured.
The mistake was made when flags were moved to the database. The `notify_moderators` (something else) flag should be the last position on the list.
This commit contains 3 changes:
- update fixtures order;
- remove position and enable from fixtures (they can be overridden by admin and we don't want seed to restore them);
- migration to fix data if the order was not changed by admin.
This commit updates the `UserPasswordExpirer.expire_user_password`
method to update `UserPassword#password_expired_at` when an existing
`UserPassword` record exists with the same `password_salt`,
`password_hash` and `password_algorithm`. This is to prevent the unique
validation error on `UserPassword#user_id` and
`UserPassword#password_hash` from being raised when the method is called
twice for a user that has not changed its password.
This commit includes various UX improvements to the reset
password page:
* Introduce a `hide-application-header-buttons` helper to do the following:
* Hide Sign Up and Log In buttons, they are not necessary on this flow
* Hide the sidebar, it is a distraction on this flow
* Improve messaging when a 2FA confirmation is required first
* Improve display of server-side ActiveRecord model validation errors
in password form, e.g. instead of "is the same as your current password"
we do "The password is the same as your current password"
* Move password tip to next line below input and move caps lock hint
inline with Show/Hide password toggle
* Add system specs for 2FA flow on reset password page
* Fixes a computed property conflict issue on the password reset
page when toggling 2FA methods
Continued work on moderate flags UI.
In this PR admins are allowed to change the order of flags. The notify user flag is always on top but all other flags can be moved.
On Github Actions, system tests which uses `Capybara#using_session` are
failing intermittently with the error "Socket::ResolutionError: getaddrinfo: Temporary failure in name resolution"
when `Selenium::WebDriver::Platform.localhost` tries to resolve
`localhost`.
Too much time has been spent trying to figure out why so we are giving
up here and just retrying the resolution of `localhost` on Github
Actions.
This makes it more obvious what's happening, and makes it much less likely that users will send repeated reset emails (and thereby hit the rate limit)
Followup to e97ef7e9af
This commit adds the ability for site administrators to mark users'
passwords as expired. Note that this commit does not add any client side
interface to mark a user's password as expired.
The following changes are introduced in this commit:
1. Adds a `user_passwords` table and `UserPassword` model. While the
`user_passwords` table is currently used to only store expired
passwords, it will be used in the future to store a user's current
password as well.
2. Adds a `UserPasswordExpirer.expire_user_password` method which can
be used from the Rails console to mark a user's password as expired.
3. Updates `SessionsController#create` to check that the user's current
password has not been marked as expired after confirming the
password. If the password is determined to be expired based on the
existence of a `UserPassword` record with the `password_expired_at`
column set, we will not log the user in and will display a password
expired notice. A forgot password email is automatically send out to
the user as well.
Even when the admin sidebar sections are collapsed, they should expand while filtering. When the filter is removed, sections should go back to the previous state.
In addition, trim whitespace from the filter section.
This commit re-introduces the "Move to Inbox" and "Move to Archive"
bulk topic actions, which we had in the old modal but had not yet added
to the new "experimental" dropdown, which isn't really experimental at
this point.
Once this is merged we can remove the old modal and only
rely on the new dropdown.
Prior to this fix we were opening a modal before closing the `DMenu` modal, given `DModal` expects only one modal at a time it was closing the latest modal and instantly closing the one we just opened.
This commit introduces the following changes which allows a site
administrator to mark `Upload` records with the `s3_file_missing`
verification status which will result in the `Upload` record being ignored when
`Discourse.store.list_missing_uploads` is ran on a site where S3 uploads
are enabled and `SiteSetting.enable_s3_inventory` is set to `true`.
1. Introduce `s3_file_missing` to `Upload.verification_statuses`
2. Introduce `Upload.mark_invalid_s3_uploads_as_missing` which updates
`Upload#verification_status` of all `Upload` records from `invalid_etag` to `s3_file_missing`.
3. Introduce `rake uploads:mark_invalid_s3_uploads_as_missing` Rake task
which allows a site administrator to change `Upload` records with
`invalid_etag` verification status to the `s3_file_missing`
verificaton_status.
4. Update `S3Inventory` to ignore `Upload` records with the
`s3_file_missing` verification status.
When uploading a video, the composer will now show a thumbnail image in
the composer preview instead of just the video placeholder image.
If `enable_diffhtml_preview` is enabled the video will be rendered in
the composer preview and is playable.
Introduced back in 2022 in
e3d495850d,
our new more specific message-id format for inbound and
outbound emails has now been in use for a very long time,
we can remove the support for the old formats:
`topic/:topic_id/:post_id.:random@:host`
`topic/:topic_id@:host`
`topic/:topic_id.:random@:host`
We are seeing a weird resolution error on Github actions with the
following backtrace:
```
Failure/Error:
visit File.join(
GlobalSetting.relative_url_root || "",
"/session/#{user.encoded_username}/become.json?redirect=false",
)
Socket::ResolutionError:
getaddrinfo: Temporary failure in name resolution
```
Switch to use `127.0.0.1` instead of forcing a name resolution.
In 958437e7dd we ensured that the email summaries are properly sent based on 'digest_attempted_at' for people who barely/never visit the forum.
This fixed the "frequency" of the email summaries but introduced a bug where the digest would be sent even though there wasn't anything new since for some users.
The logic we use to compute the threshold date for the content to be included in the digest was
```ruby
@since = opts[:since] || user.last_seen_at || user.user_stat&.digest_attempted_at || 1.month.ago
```
It was working as expected for users who haven never been seen but for users who have connected at least once, we would use their "last_seen_at" date as the "threshold date" for the content to be sent in a summary 😬
This fix changes the logic to be the most recent date amongst the `last_seen_at`, `digest_attempted_at` and `1.month.ago` so it's correctly handling cases where
- user has never been seen nor emailed a summary
- user has been seen in a while but has recently been sent a summary
- user has been sent a summary recently but hasn't been seen in a while.
This commit moves the logic for crawler rate limits out of the application controller and into the request tracker middleware. The reason for this move is to apply rate limits to all crawler requests instead of just the requests that make it to the application controller. Some requests are served early from the middleware stack without reaching the Rails app for performance reasons (e.g. `AnonymousCache`) which results in crawlers getting 200 responses even though they've reached their limits and should be getting 429 responses.
Internal topic: t/128810.
This gives us daily fidelity of topic view stats
New table stores a row per topic viewed per day tracking
anonymous and logged on views
We also have a new endpoint `/t/ID/views-stats.json` to get the statistics for the topic.
When selected some text inside a post, we offer the ability to "fast edit" the selected text without opening the composer.
However, there are certain cases where this isn't working quite a expected, due to the fact that we have some text in the "cooked" version of the post that isn't literally in the "raw" version of the post.
This ensures that whenever someone selects the within
- a quote
- a onebox
- an encrypted message
- a "cooked" date
we directly show the composer instead of showing the fast edit modal and then leaving the user with an invisible error.
Internal ref. t/128400
This commit updates `S3Inventory#files` to ignore S3 inventory files
which have a `last_modified` timestamp which are not at least 2 days
older than `BackupMetadata.last_restore_date` timestamp.
This check was previously only in `Jobs::EnsureS3UploadsExistence` but
`S3Inventory` can also be used via Rake tasks so this protection needs
to be in `S3Inventory` and not in the scheduled job.
After flags were moved to the database, with each save they are changing available PostActionTypes. Therefore, flag specs should clear the state before and after each example not just before.
In addition, we need to clear `nil` counts for dynamically created flags from serializer.
* FEATURE: add agree and edit
adds agree and edit - an alias for agree and keep -- but with a client action to
edit the post in the composer before the flag is agreed with
---------
Co-authored-by: Juan David Martinez <juan@discourse.org>
We're planning to implement a feature that allows adding required fields for existing users. This PR does some preparatory refactoring to make that possible. There should be no changes to existing behaviour. Just a small update to the admin UI.
This commit updates `Post#each_upload_url` to reject URLs that do not
have a host which matches `Discourse.current_hostname` but follows the
`/uploads/short-url` uploads URL format. This situation most commonly
happen when users copy upload URL link between different Discourse
sites.
For plugins with only an "enabled" site setting, it doesn't
make sense to take them to the site settings page, since the
toggle switch in the list can be used to change enabled/disabled.
This will not be the case for plugins that have their own custom
config page (like Automation), but we will deal with this when
we actually overhaul this plugin to use the new show page.
Also adds another rspec fixture of a test plugin.
This PR introduces a basic AdminNotice model to store these notices. Admin notices are categorized by their source/type (currently only notices from problem check.) They also have a priority.
* DEV: allow reply_by_email, visit_link_to_respond strings to be modified by plugins
* DEV: separate visit_link_to_respond and reply_by_email modifiers out
This PR aims to add bulk actions to the user's bookmarks.
After this feature, all users should be able to select multiple bookmarks and perform the actions of "deleting" or "clear reminders"
Instead of creating two separate Topics when a user (1) requests to join a group and (2) gets accepted in, this makes the acceptance message into a Post under the origin group request Topic.
This fixes the `PrettyText.make_all_links_absolute` to better handle subfolder.
In subfolder, when given the cooked version of a post, links to mentions includes the `Discourse.base_path` prefix. Adding the `Discourse.base_url` was doubling the `Discourse.base_path`.
The issue was hidden behind the specs which was stubbing `Discourse.base_url` instead of relying on `Discourse.base_path`.
This fixes both the "algorithm" used in `PrettyText.make_all_links_absolute` to better handle this case and correct the specs to properly handle subfolder cases.
There are lots of changes in the specs due to a refactoring to use squiggly heredoc strings for easier reading and less escaping.
- FIX: properly scope category changes to what the current user can see
- UX: previous category is now highlighted in "red", new category is highlighted in "green"
- PERF: no need to serialize the categories
- FIX: properly track wiki
- FIX: properly track post_type (aka. Staff Color)
- FIX: properly track making a topic a PM
- FIX: never show the category changes when a topic is made a PM
- PERF: post_revision serializer is now more leaner (never includes title changes when post_number > 1, never includes user changes if there aren't any)
- UX: always sort the tags by name
Note this may have performance issues in some cases, will need to be monitored
Previous to this change we were bracketing on 50 id windows. They may end up
having zero posts we are searching for leading to posts.rss and .json returning
no results.
- avoids Post.last.id which is expensive
- order by id desc which is better cause we bracket on id
(experimental)
The initial implementation of glimmer topic-list and related components. Does not include new APIs and isn't compatible with existing customization. That's gonna come in future PRs.
Enabled by adding groups to `experimental_glimmer_topic_list_groups` setting.
AuthProvider#enabled_setting=, used primarily by plugins, has been deprecated since version 2.9, in favour of Authenticator#enabled?. This PR confirms we are seeing no more usage and removes the method.
In 07ecbb5a3b we ensure the mentions in a group's activity page worked properly but we missed adding proper support for infinite loading.
The client is using the `before` parameter instead of the `before_post_id` to do the pagination.
This adds support for `before` as well as some tests to ensure it doesn't regress.
I also added tests to the group's activity posts as well since those were missing.
Finally I deleted some unused code (`group.messages_for`) which is not used anymore.
Context - https://meta.discourse.org/t/-/308044/9
Whenever one creates, updates, or deletes a post, we should keep the `topic.word_count` counter in sync.
Context - https://meta.discourse.org/t/-/308062
- login with username/password
- login with username/password and 2FA
- login with username/password back up code
- login with magic link
- login with magic link and 2FA
- login with magic link and back up code
- login when 2FA is required
- reset password
---
- signup and activate account
- signup with invite code
- signup with invite link
- signup and approve account
- signup and auto approve account
- signup with blocked domain
---
- basic login with Facebook
- basic login with Google
- basic login with Github
- basic login with Twitter
- basic login with Discord
- basic login with Linkedin
When converting a PM to a public topic (and vice versa), if there was a validation error (like a topic already used, or a tag required or not allowed) the error message wasn't bubbled up nor shown to the user.
This fix ensures we properly stop the conversion whenever a validation error happens and bubble up the errors back to the user so they can be informed.
Internal ref - t/128795
When "unicode_usernames" is enabled, calling the "user_path" helper with a username containing some non ASCII character will break due to the route constraint we have on username.
This fixes the issue by always encoding the username before passing it to the "user_path" helper.
Internal ref - t/127547
This commits updates `FinalDestination#get` to not forward
`Authorization` header on redirects since most HTTP clients I tested like
curl and wget does not it.
This also fixes a recent problem in `DiscourseIpInfo.mmdb_download`
where we will fail to download the databases when both `GlobalSetting.maxmind_account_id` and
`GlobalSetting.maxmind_license_key` has been set. The failure is due to
the bug above where the redirected URL given by MaxMind does not accept
an `Authorization` header.
The users directory is updated on a daily cadence. However, when a site is new and doesn't have many users, it can be confusing that a user who has just joined doesn't show up in the users until a day after they join. To eliminate this confusion, this commit triggers a refresh for the users directory as soon as as a user joins, if the site is in bootstrap mode. The reason for the conditional trigger is that refreshing the users directory is an expensive operation and doing it often on a large site with many users could lead to performance problems.
Internal topic: t/126076.
This commit changes request method for "categories/search" from GET to
POST to make sure that long filters can be passed to the server. For
example, category selectors with many categories are setting the full
list of selected category IDs to ensure these are filtered out from the
list of choices. This can result in a long URL that exceeds the maximum
length.
Some sites are still on the legacy "hamburger dropdown"
navigation_menu setting. In this case to avoid confusion,
we want to show both the sidebar icon and the header dropdown
hamburger when visiting the admin portal. Otherwise, the
hamburger switches sides from right to left for admins
and takes on different behaviour.
The hamburger in this case _only_ shows the main panel, not
other sidebar panels like the admin one.
The watched word group's create, update and delete action logs were missing the translations. This PR will add those strings and will use the group key instead of watched word key where needed.
This reverts commit 0f4520867b.
This has led to two problems:
1. An incompatibility with Cloudflare's "auto minify" feature. They've deprecated this feature because of incompatibility with modern JS syntax. But unfortunately it will remain enabled on existing properties until 2024-08-05.
2. Discourse fails to boot in Safari 15. This is strange, because Safari does support all the required features in our production JS bundles. Even more strangely, things start working as soon as you open the developer tools. That suggests the cause could be a Safari bug rather than a simple incompatibility.
Reverting while we work out a path forward on both those issues.
This commit switches `DiscourseIpInfo.mmdb_download` to use the
permalinks supplied by MaxMind to download the MaxMind databases as
specified in
https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
which states:
```
To directly download databases, follow these steps:
1. In the "Download Links" column, click "Get Permalink(s)" for the desired database.
2. Copy the permalink(s) provided in the modal window.
3. Provide your account ID and your license key using Basic Authentication to authenticate.
```
Previously we are downloading from `https://download.maxmind.com/app/geoip_download` but this is not
documented anyway on MaxMind's docs so this URL can in theory break
in the future without warning. Therefore, we are taking a proactive
approach to download the databases from MaxMind the recommended way
instead of relying on a hidden URL. This old way of downloading the
databases with only a license key will be deprecated in 3.3 and be
removed in 3.4.
In 95a82d608d, we lowered the default for
`Onebox.options.max_download_kb` from 10mb to 2mb for security hardening
purposes. However, this resulted in multiple bug reports where seemingly
nomral URLs stopped being oneboxed. It turns out that lowering
`Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised
more often for more URLs in `Onebox::Helpers.fetch_response` which
`Onebox::Helpers.fetch_html_doc` relies on. When
`Onebox::Helpers::DownloadTooLarge` is raised in
`Onebox::Helpers.fetch_response`, we throw away whatever response body
which we have already downloaded at that point. This is not ideal
because Nokogiri can parse incomplete HTML documents and there is a
really high chance that the incomplete HTML document still contains the
information which we need for oneboxing.
Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not
throw away the response body when the size of the response body exceeds
`Onebox.options.max_download_size`. Instead, we just take whatever
response which we have and get Nokogiri to parse it.
This can happen for various reasons including rate limiting and middleware bugs. This should resolve the warning we're seeing in the logs
```
RequestTracker.get_data failed : NoMethodError : undefined method `[]' for nil:NilClass
```
decorator-transforms (https://github.com/ef4/decorator-transforms) is a modern replacement for babel's plugin-proposal-decorators. It provides a decorator implementation using modern browser features, without needing to enable babel's full suite of class feature transformations. This improves the developer experience and performance.
In local testing with Google's 'tachometer' tool, this reduces Discourse's 'init-to-render' time by around 3-4% (230ms -> 222ms).
It reduces our initial gzip'd JS payloads by 3.2% (2.43MB -> 2.35MB), or 7.5% (14.5MB -> 13.4MB) uncompressed.
DropdownMenu component is meant as a way to describe the content of menus.
Syntax:
```
<DropdownMenu as |dm|>
<dm.item class="test">
First
</dm.item>
<dm.divider class="foo" />
<dm.item class="bar">
Second
</dm.item>
</DropdownMenu>
```
This commits updates
`PageObjects::Components::NavigationMenu::Base#click_edit_tags_button`
to wait for `.sidebar-tags-form` to be present before returning. This is
essential because the client side app has to load the tags from the
server when the modal is open. If we don't wait for all the tags to be
loaded, it makes it hard to reason about the state of the UI when
interacting with the modal. In the case of "allows a user to deselect all tags in the modal which will display the site's top tags" which
was flaky, the system test was interacting with the UI when the tags are
still loading.
This keeps coming up in user testing as something
we want to get rid of. The `navigation_menu` setting
has been set to sidebar by default for some time now,
and we are rolling out admin sidebar widely. It just
doesn't make sense to let people turn this off in
the first step of the wizard -- we _want_ people to
use the sidebar.
menus and tooltips are now appended to their own portals. The service are the only responsible for managing the instances, prior to this commit, services could manage one instance, but the DMenu and DTooltip components could also take over which could cause unexpected states.
This change also allows nested menus/tooltips.
Other notable changes:
- few months ago core copied the CloseOnClickOutside modifier of float-kit without removing the float-kit one, this commit now only use the core one.
- the close function is now trully async
- the close function accepts an instance or an identifier as parameter
Whenever a post already failed "lightweight" validations, we skip all the expensive validations (that cooks the post or run SQL queries) so that we reply as soon as possible.
Also skip validating polls when there's no "[/poll]" in the raw.
Internal ref - t/115890
This spec has been failing forever on my machine. I guess I have a "better" version of pngquant?
---------
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
If there's ever a circular reference in categories, don't go into an infinite loop when generating the category slug.
Instead, keep track of parent ids, and bail out as soon as we're encountering one more than once.
In #22851 we added a dependent strategy for deleting upload references when a draft is destroyed. This, however, didn't catch all cases, because we still have some code that issues DELETE drafts queries directly to the database. Specifically in the weekly cleanup job handled by Draft#cleanup!.
This PR fixes that by turning the raw query into an ActiveRecord #destroy_all, which will invoke the dependent strategy that ultimately deletes the upload references. It also includes a post migration to clear orphaned upload references that are already in the database.
In this PR we started redirecting to the guide page after the wizard - https://github.com/discourse/discourse/pull/26696
The guide will require rebrand and until it is ready, we should redirect to `/latest`
This commit introduces the `run_theme_migration` spec helper to allow
theme developers to write RSpec tests for theme migrations. For example,
this allows the following RSpec test to be written in themes:
```
RSpec.describe "0003-migrate-small-links-setting migration" do
let!(:theme) { upload_theme_component }
it "should set target property to `_blank` if previous target component is not valid or empty" do
theme.theme_settings.create!(
name: "small_links",
theme: theme,
data_type: ThemeSetting.types[:string],
value: "some text, #|some text 2, #, invalid target",
)
run_theme_migration(theme, "0003-migrate-small-links-setting")
expect(theme.settings[:small_links].value).to eq(
[
{ "text" => "some text", "url" => "#", "target" => "_blank" },
{ "text" => "some text 2", "url" => "#", "target" => "_blank" },
],
)
end
end
```
This change is being introduced because we realised that writting just
javascript tests for the migrations is insufficient since javascript
tests do not ensure that the migrated theme settings can actually be
successfully saved into the database. Hence, we are introduce this
helper as a way for theme developers to write "end-to-end" migrations
tests.
* Simplify config nav link generation to always inject the Settings
tab
* Auto-redirect to the first non-settings config link (if there is one)
when the user lands on /admin/plugins/:plugin_id
* Add `extras` to admin plugin serializer so plugins can add more
data on first load
* Add PikadayCalendar page object for system specs, extracted from the
CalendarDateTimePicker to make it more generic.
