Note this commit also fixes an issue where the edit post actions was trying to focus the edit textarea, but was using jquery functions on a DOM node.
scrollTo is not available on IE11 but that shouldn't cause much trouble.
When secure media is enabled and an attachment is marked as secure we want to use the full url instead of the short-url so we get the same access control post protections as secure media uploads.
Meta report: https://meta.discourse.org/t/excessive-requests-to-uploads-lookup-urls-leading-to-429-response/143119
* The data-orig-src attribute was not being removed from cooked
video and audio so the composer was infinitely trying to get the
URLs for them, which would never resolve to anything
* Also the code that retrieved the short URL was unscoped, and was
getting everything on the page. if running from the composer we
now scope to the preview window
* Also fixed a minor issue where the element href for the video
and audio tags was not being set when the short URL was found
* Add uploads:sync_s3_acls rake task to ensure the ACLs in S3 are the correct (public-read or private) setting based on upload security
* Improved uploads:disable_secure_media to be more efficient and provide better messages to the user.
* Rename uploads:ensure_correct_acl task to uploads:secure_upload_analyse_and_update as it does more than check the ACL
* Many improvements to uploads:secure_upload_analyse_and_update
* Make sure that upload.access_control_post is unscoped so deleted posts are still fetched, because they still affect the security of the upload.
* Add escape hatch for capture_stdout in the form of RAILS_ENABLE_TEST_STDOUT. If provided the capture_stdout code will be ignored, so you can see the output if you need.
* FIX: Add aria-labels to topic list items
Before this fix you could navigate the topic list using a screen reader
and a keyboard but some of the items were not as descriptive as they
could be. The newly added labels make it easier to understand what you
are tabbing over.
context:
https://meta.discourse.org/t/accessibility-aria-attributes-are-not-defined-for-links-under-replies-category/142539
* Update app/assets/javascripts/discourse/lib/utilities.js.es6
Co-Authored-By: Régis Hanol <regis@hanol.fr>
* Multiline fix
* Fix more tests
Co-authored-by: Régis Hanol <regis@hanol.fr>
* PERF: Allow passing an existing list of user field ids when loading
This avoids the need for running `UserField.pluck(:id)` for each user that is serialized
* Memoize user_fields to avoid rebuilding hash ever time
The server and client used two different formats for preload keys. The
server was using 'topic_list_c/SLUG/l/latest', but the client was using
'topic_list_c/SLUG/ID/l/latest'.
This commit is an addition to 374534f00e.
PMs will now display an envelope icon next to the topic title in search results. This is especially useful when searching using `in:all`.
Co-authored-by: adam j hartz <hz@mit.edu>
In IE11, the browser returns the cached HTML response, rather than the JSON formatted response. Adding the `.json` suffix ensures that the cache is not shared. Same root cause as b0211772
Introduces a new site setting `max_notifications_per_user`.
Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.
To disable this safeguard set the setting to 0.
Enforcement happens weekly.
This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
When looking for the first paragraph with content in a post,
it was matching the lightboxed image paragraph as "<p></p>".
Fix that and other potential empty paragraphs with the
p:not(:empty) selector.
Add a new selector to find the image links in lightboxed
images as valid content for emails.
This commit also fixes a deprecation warning as the previous component was overriding a computed property from the group model.
Finally a test has been added as this is the only place where we use list-setting outside of the settings, this was highly subject to regressions.
* Also fixes an issue where if webp was a downloaded hotlinked
image and then secure + sent in an email, it was not being
redacted because webp was not a supported media format in
FileHelper
* Webp originally removed as an image format in
https://github.com/discourse/discourse/pull/6377
and there was a spec to make sure a .bin webp
file did not get renamed from its type to webp.
However we want to support webp images now to make
sure they are properly redacted if secure media is
on, so change the example in the spec to use tiff,
another banned format, instead
This is because the TOTP gem identifies as a colon as an addressable
protocol. The solution for now is to remove the colon in the issuer
name.
Changing the issuer changes the token values, but now it was completely
broken for colons so this should not be breaking anyone new.
d7d4612b2d removed the duplicate call to initState(). However, we are relying on a side effect of the duplicate call for subfolder sites to function correctly when accessed without a trailing slash. To avoid a large refactor before the stable release, this commit restores the old behavior.
Long term we should look at migrating to Ember's built-in location library, rather than maintaining our own (very similar) version
https://github.com/emberjs/ember.js/blob/master/packages/%40ember/-internals/routing/lib/location/history_location.ts
When admin changes a user's email from the preferences page of that user:
* The user will not be sent an email to confirm that their
email is changing. They will be sent a reset password email
so they can set the password for their account at the new
email address.
* The user will still be sent an email to their old email to inform
them that it was changed.
* Admin and staff users still need to follow the same old + new
confirm process, as do users changing their own email.
If a group mention could be notified on preview it was given an `<a>`
tag with the `.notify` class. When cooked it would display differently.
This patch makes the server side cooking match the client preview.
This normalizes it so we only carry one place for grabbing disk space size
It also normalizes the command made so it uses Discourse.execute_command
which splits off params in a far cleaner way.
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways
1. We cache the result in `Discourse.os_hostname` so it is only ever called once
2. We prefer to use Socket.gethostname which avoids making a shell command
This improves performance as we are not spawning hostname processes throughout
the app lifetime
Now if a group is visible but unmentionable, users can search for it
when composing by typing with `@`, but it will be rendered without the
grey background color.
It will also no longer pop up a JIT warning saying "You are about to
mention X people" because the group will not be mentioned.
* FIX: when unread reply notification exists don't create new
From time to time, the user is creating a reply post and then they want to add additional details. They edit an existing post and for example, add a quote from a previous one.
In that situation, if the user to whom reply was directed to already have the unread notification, we should not create the new one.
That behaviour was mentioned here: https://meta.discourse.org/t/reply-then-edit-to-add-quote-notification-redundancy/138358
* FIX: dont create new notification if already exists
* Because custom emoji count as post "uploads" we were
marking them as secure when updating the secure status for post uploads.
* We were also giving them an access control post id, which meant
broken image previews from 403 errors in the admin custom emoji list.
* We now check if an upload is used as a custom emoji and do not
assign the access control post + never mark as secure.
Styling based on element-ids, it is impossible for themes/plugins to display multiple cards on a single page. Using classes is a more flexible approach. The element-ids are maintained for backwards compatibility with existing plugins/themes.
### UI Changes
If `SiteSetting.enable_bookmarks_with_reminders` is enabled:
* Clicking "Bookmark" on a topic will create a new Bookmark record instead of a post + user action
* Clicking "Clear Bookmarks" on a topic will delete all the new Bookmark records on a topic
* The topic bookmark buttons control the post bookmark flags correctly and vice-versa
Disabled selecting the "reminder type" for bookmarks in the UI because the backend functionality is not done yet (of sending users notifications etc.)
### Other Changes
* Added delete bookmark route (but no UI yet)
* Added a rake task to sync the old PostAction bookmarks to the new Bookmark table, which can be run as many times as we want for a site (it will not create duplicates).
* FEATURE: export defaultRenderTag function
This can be helpful for plugins e.g. when combined with replaceTagRenderer: a plugin may want to customize rendering for some tags and let others be rendered with default code
* refactor: don't change version
..as per review comment, https://github.com/discourse/discourse/pull/8941#discussion_r378406809
The `DiscourseLocation.initState` function was accidently renamed in 0431942f (select-kit-2) to `initOptions`. This means that the ember router does not automatically call the function after the router is initialized.
For a long time, we have been calling the `initState` function in the `init` function of discourse-location, which caused an imperceptible URL change to the the root domain, before switching back to the correct subfolder URL when ember called `initState`. This commit removes that call from the initializer, so `initState` is only called once (by ember).
Relevant ember code: https://github.com/emberjs/ember.js/blob/v3.12.2/packages/@ember/-internals/routing/lib/system/router.ts#L695-L699
Meta topic: https://meta.discourse.org/t/secure-media-uploads-expire/140894
This fixes the issue where if secure media was enabled, audio
and video files would do an initial load using the presigned
URL for the media to get metadata information e.g. duration of
track/video. However this started the expiry countdown for the
URL, so when a user pressed play on the media after 15 seconds
the media would be expired and AWS would return a 403 error.
We do not preload media if secure media is enabled. Otherwise
we just set the preload type to "metadata" which is the browser
default anyway.
The commit: 75069ff179
allows users to remove their primary group, but this introduced a bug
where if you were to edit any other profile info like location or
website which is a form on a separate page then the flair dropdown,
would cause the selected flair to be removed.
This fix ensures that if the `primary_group_id` parameter is missing
from the update payload it does not remove the existing
`primary_group_id`. It will only remove the `primary_group_id` if it is
present in the payload and empty.
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.
Context at https://meta.discourse.org/t/-/113249
* FIX: Properly convert quotes to Markdown
When quoting a quote it used to convert the quote header, including the
user avatar and username, into a image and some text and then the
contents. This also caused issues when quoting full paragraphs (or when
selecting paragraphs by triple-clicking) because the user avatar and
name from the following quote would also be included.
This commit implements the support necessary to convert
<aside class="quote"> elements to proper Discourse quotes.
Commit aa24be1 made it possible to build data attributes from image's
Markdown and this changes ensure that the resize controls still work
when data attributes are present.
Previously, `notify_first_post_users` was loading all users into memory simultaneously, which can cause Sidekiq to run out of memory for large sites. `notify_post_users` was loading every user one-by-one in a loop.
This commit makes both these functions load users in batches of 100. This should make the memory usage of `notify_first_post_users` lower, and reduce the number of queries required in `notify_post_users`.
Extracted from #8772
This will allow developers (in rails development mode only) to log pre-loaded JSON app data to the browser console for inspection.
When a tag is restricted to a secured category that the user can't see,
the message was saying that it wasn't restricted to any categories.
Now it will say it's restricted to categories you can't access.
* DEV: Use Ember 3.12.2
* Add Ember version to ThemeField's DEPENDENT_CONSTANTS
* DEV: Use `id` instead of `elementId` (See: https://github.com/emberjs/ember.js/issues/18147)
* FIX: Don't leak event listeners (bug introduced in 999e2ff)
Our current algorithm for picking the number of notifications to display
when expanding the notifications relies on magic numbers.
Previously we only allowed for header and an estimate of maximum height of
notification container, this is not ideal as there is padding at the bottom
and top of the notification container
This adds a special number for padding.
The longer term fix though is to render the notification panel off screen
then grab the correct count, finally adding it back into view with.
This would allow for large fonts, small fonts, custom themes and much more.
This reverts commit 7133fd8c89.
Unfortunately performCheckSize may have certain situations in topic
navigation where it is unsafe to call.
We need to add som more safety here prior to merging.
This fix ensures that the site setting `post_edit_time_limit` does not
bypass the limit of the site setting `min_trust_to_edit_post`. This
prevents a bug where users that did not meet the minimum trust level to
edit could edit the title of topics.
Previously it would go to the "html" page when refreshing on the "css" page, and would open an invalid empty-state page when trying to go to the "email style" tab when already on it.
This also enables`@action` use in plugin connectors.
Setting `actions` earlier allows `setupComponents` to use them, for example, when setting up event listeners.
Those are the same arguments that are passed into `after-d-editor` outlet. This will enable plugins that attach to editor preview to be conditionally enabled, usually only for the composer.
Plugins that will use this: discourse-canned-responses, discourse-zoom.
Previously you'd get a server side generic error due to a password check
failing. Now the input element has a maxlength attribute and the server
side will respond with a nicer error message if the value is too long.
If our reply tree somehow ends up with cycles or other odd
structures, we only want to consider a reply once, at the first
level in the tree that it appears.
* DEV: Add data-notification-level attribute to category UI
* Show muted categories on the category page by default
This reverts commit ed9c21e42c.
* Remove redundant spec - muted categories are now visible by default
It seems in some situations replies have been moved to other topics but
the `PostReply` table has not been updated. I will try and fix this in a
follow up PR, but for now this fix ensures that every time we ask a post
for its replies that we restrict it to the same topic.
This commit adds support for an optional "logout" parameter in the
payload of the /session/sso_provider endpoint. If an SSO Consumer
adds a "logout=true" parameter to the encoded/signed "sso" payload,
then Discourse will treat the request as a logout request instead
of an authentication request. The logout flow works something like
this:
* User requests logout at SSO-Consumer site (e.g., clicks "Log me out!"
on web browser).
* SSO-Consumer site does whatever it does to destroy User's session on
the SSO-Consumer site.
* SSO-Consumer then redirects browser to the Discourse sso_provider
endpoint, with a signed request bearing "logout=true" in addition
to the usual nonce and the "return_sso_url".
* Discourse destroys User's discourse session and redirects browser back
to the "return_sso_url".
* SSO-Consumer site does whatever it does --- notably, it cannot request
SSO credentials from Discourse without the User being prompted to login
again.
This new iteration of select-kit focuses on following best principales and disallowing mutations inside select-kit components. A best effort has been made to avoid breaking changes, however if you content was a flat array, eg: ["foo", "bar"] You will need to set valueProperty=null and nameProperty=null on the component.
Also almost every component should have an `onChange` handler now to decide what to do with the updated data. **select-kit will not mutate your data by itself anymore**
If someone only had security keys enabled, the icon to say they had 2FA enabled would not show in the admin staff user list. It would only show if they had TOTP enabled.
This fixes a bug which caused '{{#unless var}}' to act the same as
'{{#if true}}' because 'unless' was transforming the conditional value
to 'undefined'.
For example /t/ URLs were being replaced if they contained secure-media-uploads so if you made a topic called "Secure Media Uploads Are Cool" the View Topic link in the user notifications would be stripped out.
Refactored code so this secure URL detection happens in one place.
When 'categories topics' setting is set to 0, the system will
automatically try to find a value to keep the two columns (categories
and topics) symmetrical.
The value is computed as 1.5x the number of top level categories and at
least 5 topics will always be returned.
If using {{#if showFooter}} in a template, showFooter is never set to true on a group's g/groupname/activity/topics route (it's correctly set on other group routes like group-activity-posts)
Basically, say you had already downloaded a certain image from a certain URL
using pull_hotlinked_images and the onebox. The upload would be stored
by its sha as an upload record. Whenever you linked to the same URL again
in a post (e.g. in our case an og:image on review.discourse) we would
would reuse the original upload record because of the sha1.
However when you turned on secure media this could cause problems as
the first post that uses that upload after secure media is enabled
will set the access control post for the upload to the new post.
Then if the post is deleted every single onebox/link to that same image
URL will fail forever with 403 as the secure-media-uploads URL fails
if the access control post has been deleted.
To fix this when cooking posts and pulling hotlinked images, we only
allow using an original upload by URL if its access control post
matches the current post, and if the original_sha1 is filled in,
meaning it was uploaded AFTER secure media was enabled. otherwise
we just redownload the media again to be safe, as the URL will always
be new then.
Regression was created here:
https://github.com/discourse/discourse/pull/8750
When tag or category is added and the user is watching that category/tag
we changed notification type to `edited` instead of `new post`.
However, the logic here should be a little bit more sophisticated.
If the user has already seen the post, notification should be `edited`.
However, when user hasn't yet seen post, notification should be "new
reply". The case for that is when for example topic is under private
category and set for publishing later. In that case, we modify an
existing topic, however, for a user, it is like a new post.
Discussion on meta:
https://meta.discourse.org/t/publication-of-timed-topics-dont-trigger-new-topic-notifications/139335/13
Previously the badge was granted one month after the last time the badge was granted. The exact date shifted by one day each month. The new logic tries to grant the badge always at the beginning of a new month by looking at new users of the previous month. The "granted at" date is set to the end of the previous month.
Adds a new route `/u/{username}/card.json`, which has a reduced number of fields. This change is behind a hidden site setting, so we can test compatibility before rolling out.
With this change the script:
* Actually removes original large-sized images
* Doesn't save processed files if their size has increased
* Prevents inconsistent state
* DEV: Remove `.large-image` selector
This selector is no longer used in core and there's no reference to it in any of `all-the-plugins`.
* FIX: Adjust the broken image placeholder border
When pull_hotlinked_images tried to run on posts with secure media (which had already been downloaded from external sources) we were getting a 404 when trying to download the image because the secure endpoint doesn't allow anon downloads.
Also, we were getting into an infinite loop of pull_hotlinked_images because the job didn't consider the secure media URLs as "downloaded" already so it kept trying to download them over and over.
In this PR I have also refactored secure-media-upload URL checks and mutations into single source of truth in Upload, adding a SECURE_MEDIA_ROUTE constant to check URLs against too.
* FEATURE: Replace existing badge owners when using the bulk award feature
* Use ActiveRecord to sanitize title update query, Change replace checkbox text
Co-Authored-By: Robin Ward <robin.ward@gmail.com>
Co-authored-by: Robin Ward <robin.ward@gmail.com>
* UX: Invalid CSV error message now includes information about the malformed line
* Update config/locales/server.en.ym and use line_number instead of lineno
Co-Authored-By: Robin Ward <robin.ward@gmail.com>
Co-authored-by: Robin Ward <robin.ward@gmail.com>
Add TopicUploadSecurityManager to handle post moves. When a post moves around or a topic changes between categories and public/private message status the uploads connected to posts in the topic need to have their secure status updated, depending on the security context the topic now lives in.
For consistency this PR introduces using custom markdown and short upload:// URLs for video and audio uploads, rather than just treating them as links and relying on the oneboxer. The markdown syntax for videos is  and for audio it is .
This is achieved in discourse-markdown-it by modifying the rules for images in mardown-it via md.renderer.rules.image. We return HTML instead of the token when we encounter audio or video after | and the preview renders that HTML. Also when uploading an audio or video file we insert the relevant markdown into the composer.
Let's say post #2 quotes post number #1. If a user decides to quote the
quote in post #2, it should keep the information of post #1
("user_1, post: 1, topic: X"), instead of replacing with current post
info ("user_2, post: 2, topic: X").
When editing site texts from
/admin/customize/site_texts/
you can edit badge titles (aka name) and this will update any users that
have that badge currently set as their title. This fix prevents a badge
description text from being set as their title if an admin updates the
badge description text or any other badge fields that aren't the title.
The data-vocabulary.org schema is being deprecated.
We're now using the BreadcrumList data from the latest and greatest schema.org.
FIX: categories_breadcrumb helper to support more than 2 levels of categories.
- Increase size of textarea when displaying generated codes
- Adjust maxlength of input field in JS UI
- Adjust maxlength of input field in no_ember UI
Follow-up to bff9880d63
On some customer forums we are randomly getting a "You must select a valid user" error when sending a PM even when all parameters seem to be OK. This is an attempt to track it down with more data.
There is a feature, that when tag or category is added to the topic,
customers who are watching that category or tag are notified.
The problem is that it is using default notification type "new post"
It would be better to use "new post" only when there really is a new
post and "edited" when categories or tags were modified.
Previously if local login via email was disabled because of the site setting or because SSO was enabled, we were raising a 500 error. We now raise a 403 error instead; we shouldn't raise 500 errors on purpose, instead keeping that code for unhandled errors. It doesn't make sense in the context of what we are validating either to raise a 500.
This fix allows a user to remove their currently assigned primary group
if the Site Setting `user selected primary groups` is enabled.
Before this fix, if a user selected "none" for their primary group it
would silently fail and never be updated.
The timezone should only be initialized when there is no timezone stored in the DB yet. This also fixes an error that happened in dev mode whenever you switch between user preference tabs.
ReviewableScore#types extend the PostActionTypes with their own, storing the result inside a class variable. To avoid overwriting an existing flag, we need to calculate the next flag ID using these types instead of the PostAction ones. Since we first call the score types to calculate the id, this list gets memoized, leaving us with an outdated list.
To fix this, we now reload ReviewableScore#types after replacing flags.
* When we refactored away the admin-login route we introduced a bug where admins could not log into an SSO enabled site, because of a check in the email_login route that disallowed this.
* Allow admin to get around this check.
* DEV: Fix the function prototype observers deprecation
DEPRECATION: Function prototype extensions have been deprecated, please migrate from function(){}.observes('foo') to observer('foo', function() {}). [deprecation id: function-prototype-extensions.observes] See https://deprecations.emberjs.com/v3.x/#toc_function-prototype-extensions-observes for more details.
* DEV: Fix the function prototype event listeners deprecation
DEPRECATION: Function prototype extensions have been deprecated, please migrate from function(){}.on('foo') to on('foo', function() {}). [deprecation id: function-prototype-extensions.on] See https://deprecations.emberjs.com/v3.x/#toc_function-prototype-extensions-on for more details.
* DEV: Simplify `default as` imports
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This breaking change was originally a deprecation fix for overriding a computed property `none`.
There are 4 uses of `rootNone` in core and "all-the-plugins":
1. in discourse-chat-integration, admin-plugins-chat-edit-rule.hbs - changed behavior, that I'd consider a fix - `rootNoneLabel` is now used regardless of `siteSettings.allow_uncategorized_topics` value, which I believe was an originally intended behavior (i.e. it most likely hasn't been tested with disabled uncategorized topics)
2. in discourse-slack-official, plugins-slack.hbs - the same as 1.
3. in core, edit-category-general.hbs (in this PR) - no change in behavior
4. in discourse-googlebooks, edit-category-general.hbs - no change in behavior (since `allowUncategorized="true"` is also passed as an argument)
* when we dragged the topic-timeline handle past the last post
in a topic we were not closing the timeline as we would if
scrolling to a specific post
* this also fixes the issue where when scrolling past the end of
the topic with a massive last post, none of the post could be
seen
### General Changes and Duplication
* We now consider a post `with_secure_media?` if it is in a read-restricted category.
* When uploading we now set an upload's secure status straight away.
* When uploading if `SiteSetting.secure_media` is enabled, we do not check to see if the upload already exists using the `sha1` digest of the upload. The `sha1` column of the upload is filled with a `SecureRandom.hex(20)` value which is the same length as `Upload::SHA1_LENGTH`. The `original_sha1` column is filled with the _real_ sha1 digest of the file.
* Whether an upload `should_be_secure?` is now determined by whether the `access_control_post` is `with_secure_media?` (if there is no access control post then we leave the secure status as is).
* When serializing the upload, we now cook the URL if the upload is secure. This is so it shows up correctly in the composer preview, because we set secure status on upload.
### Viewing Secure Media
* The secure-media-upload URL will take the post that the upload is attached to into account via `Guardian.can_see?` for access permissions
* If there is no `access_control_post` then we just deliver the media. This should be a rare occurrance and shouldn't cause issues as the `access_control_post` is set when `link_post_uploads` is called via `CookedPostProcessor`
### Removed
We no longer do any of these because we do not reuse uploads by sha1 if secure media is enabled.
* We no longer have a way to prevent cross-posting of a secure upload from a private context to a public context.
* We no longer have to set `secure: false` for uploads when uploading for a theme component.
FIX: raised a proper NotFound exception when filtering groups by username with invalid username.
FIX: properly filter the groups based on current user visibility when viewing another user's groups.
DEV: Guardian.can_see_group?(group) is now using Guardian.can_see_groups(groups) instead of duplicating the same code.
FIX: spec for groups_controller#index when group directory is disabled for logged in user.
FIX: groups_controller.sortable specs to actually test all sorting combinations.
DEV: s/response_body/body/g for slightly shorter spec code.
FIX: rewrote the "view another user's groups" specs to test all group_visibility and members_group_visibility combinations.
DEV: Various refactoring for cleaner and more consistent code.
* UI: Mass grant a badge from the admin ui
* Send the uploaded CSV and badge ID to the backend
* Read the CSV and grant badge in batches
* UX: Communicate the result to the user
* Don't award if badge is disabled
* Create a 'send_notification' method to remove duplicated code, slightly shrink badge image. Replace router transition with href.
* Dynamically discover current route
Now that the spec is finished use the unprefixed API, which was also moved
from window to navigator.
Still uses feature detection so it fail gracefully when not available in
the user agent.
The previous concurrency-safe implementation relied on catching an
index conflict and following through appropriately. Unfortunately
those conflicts were logged to Postgres and there is no easy way
to turn them off.
This solution approaches the problem differently. It should still
be safe under concurrency and not log errors.
