* FEATURE: use canonical links in posts.rss feed
Previously we used non canonical links in posts.rss
These links get crawled frequently by crawlers when discovering new
content forcing crawlers to hop to non canonical pages just to end up
visiting canonical pages
This uses up expensive crawl time and adds load on Discourse sites
Old links were of the form:
`https://DOMAIN/t/SLUG/43/21`
New links are of the form
`https://DOMAIN/t/SLUG/43?page=2#post_21`
This also adds a post_id identified element to crawler view that was
missing.
Note, to avoid very expensive N+1 queries required to figure out the
page a post is on during rss generation, we cache that information.
There is a smart "cache breaker" which ensures worst case scenario is
a "page drift" - meaning we would publicize a post is on page 11 when
it is actually on page 10 due to post deletions. Cache holds for up to
12 hours.
Change only impacts public post RSS feeds (`/posts.rss`)
In the API keys page where admins can create API keys with restricted scopes, each scope shows a list of URLs that it allows. But currently, this list of allowed URLs shows incomplete URLs for scopes that are added by plugins. For example, the allowed URL for the "run queries" scope of the data-explorer plugin is shown as `/queries/:id/run` when the correct URL for this scope is `/admin/plugins/explorer/queries/:id/run`. The first 3 segments of the path are the mount path of the plugin's engine and it's missing because the routes set of the engine doesn't include the mount path. To fix this, this commit gets the mount path and prepends it to the URL so the complete URL is shown to the user.
It's not possible to write tests for this change because plugins are not loaded in the test environment by default when core's tests suite is running.
The user can select what happens with a bookamrk after it expires. New
option allow bookmark's reminder to be kept even after it has expired.
After a bookmark's reminder notification is created, the reminder date
will be highlighted in red until the user resets the reminder date.
User can do that using the new Clear Reminder button from the dropdown.
This change adds support for the categories endpoint to have an api
scope. Only adds GET scope for listing categories and for fetching a
single category.
See: https://meta.discourse.org/t/218080/4
Our @mention user search prioritized users based on prefix matches.
So if searching for `sa` we will display `sam`, `asam` in that order
Previously, we did not prioritize group matches based on prefix. This change ensures better parity.
Implementation notes:
1. User search only prioritizes based on username prefix, not name prefix. TBD if we want to change that.
2. @mention on client side will show 0 group matches if we fill up all the spots with user matches. TBD if we want to unconditionally show the first / second group match.
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
* FEATURE: upload an avatar option for uploading avatars with selectable avatars
Allow staff or users at or above a trust level to upload avatars even when the site
has selectable avatars enabled.
Everyone can still pick from the list of avatars. The option to upload is shown
below the selectable avatar list.
refactored boolean site setting into an enum with the following values:
disabled: No selectable avatars enabled (default)
everyone: Show selectable avatars, and allow everyone to upload custom avatars
tl1: Show selectable avatars, but require tl1+ and staff to upload custom avatars
tl2: Show selectable avatars, but require tl2+ and staff to upload custom avatars
tl3: Show selectable avatars, but require tl3+ and staff to upload custom avatars
tl4: Show selectable avatars, but require tl4 and staff to upload custom avatars
staff: Show selectable avatars, but only allow staff to upload custom avatars
no_one: Show selectable avatars. No users can upload custom avatars
Co-authored-by: Régis Hanol <regis@hanol.fr>
Previously cached counting made redis calls in main thread and performed
the flush in main thread.
This could lead to pathological states in extreme heavy load.
This refactor reduces load and cleans up the interface
Previously we were publishing one messagebus message per user which was 'tracking' a topic. On large sites, this can easily be 1000+ messages. The important information in the message is common between all users, so we can manage with a single message on a shared channel, which will be much more efficient.
For user-specific values (notification_level and last_read_post_number), the JS app can infer values which are 'good enough'. Correct values will be loaded as soon as a topic-list containing the topic is visited.
aa1442fdc3 split theme stylesheets so that every component gets its own stylesheet. Therefore, there is now no need for parent themes to collate the settings/variables of its children during scss compilation.
Technically this is a breaking change for any themes which depend on the settings/variables of their child components. That was never a supported/recommended arrangement, so we don't expect this to cause issues.
If a theme is updated to introduce a new setting AND immediately make use of it in a stylesheet, then an error was being shown. This is because the stylesheet compilation was using the theme's cached settings, and the cache is only cleared **after** the theme has finished compiling.
This commit updates the SCSS compilation to use uncached values for settings. A similar fix was applied to other parts of theme compilation back in 2020: (a51b8d9c66)
The previous Discourse.cache usage was different to how other theme-related caching is handled, and also requires reaching out to redis every time. The common theme cache is held in memory (as a DistributedCache)
This commit makes sure that the email log's bounce_error_code
conforms to the SMTP error code RFC on save, so that
it is always in the format X.X.X or XXX without any
additional string details. Also included is a migration
to fix this issue for past records.
Similar site settings exist for likes and edits and the new ones work
in a similar way.
By default, users below TL2 have a limit of 20, the limit is increased
by 1.5 for TL2 users up to 30, by 2 for TL3 users up to 40 and by 3 for
TL4 users up to 60.
We validate the *format* of email addresses in many places with a match against
a regex, often with very slightly different syntax.
Adding a separate EmailAddressValidator simplifies the code in a few spots and
feels cleaner.
Deprecated the old location in case someone is using it in a plugin.
No functionality change is in this commit.
Note: the regex used at the moment does not support using address literals, e.g.:
* localpart@[192.168.0.1]
* localpart@[2001:db8::1]
* FIX: Don't accept accents in slug if generation_method == 'ascii'
Fixes bug reported in:
- https://meta.discourse.org/t/404-when-trying-to-edit-category-with-accent-in-slug/214762
- https://meta.discourse.org/t/formatting-and-accents-in-urls/215734/5
Assuming `SiteSetting.slug_generation_method == 'ascii'.
If the user provides a slug containing non-ascii characters while
creating the category, the user will receive a 404 error just
after saving the category since the slug will be escaped anyway but
Category.find_by_slug_path won't escape the category slug
causing the Edit Page of the category to be inaccessible.
This commit checks the provided slug and raises an error if the
provided slugcontains non-ascii characters ensuring that the
provided value is consistent with the site settings.
It also changes Category.find_by_slug_path to always escape the slug,
since if present, it is escaped anyway in Category.ensure_slug to
prevent the 404 in the Edit Category Page in case the user already
have some category with a non-ascii slug.
* Removed trailing whitespace
When parent category or grandparent category is muted, then category should be muted as well.
Still, it can be overridden by setting individual subcategory notification level.
CategoryUser record is not created, mute for subcategories is purely virtual.
This can happen if the topic to which a user is invited is in a private
category and the user was not invited to one of the groups that can see
that specific category.
This used to be a warning and this commit makes it an error.
This commit introduces two new APIs for handling unused uploads, one
can be used to exclude uploads in bulk when the data model allow and
the other one excludes uploads one by one.
Previously calls such as `Emoji["smile"]` would force a full dehydration of
objects from Redis.
This introduces a version safe site and global emoji cache so lookups are
cheap. It eliminates iterating through the list of emojis and pulling from
redis.
Distributed cache uses a normalized name as the key and stores an Array tuple
with version and Emoji. Successful hits always confirm version matches.
Interface to Emoji object remains unchanged.
We opted for 2 caches to improve reuse on multisites. misses though will be
stored in both caches. If there is a hit on the global cache we can avoid
looking up in site local cache and storing a miss there.
Previously we were calling `EXPIRE` every time we incremented a given key. Instead, we can call EXPIRE once when the key is first populated. A LUA script is used to make this as efficient as possible.
Consumers of this Concern use daily keys. Since we're now calling EXPIRE only at the beginning of the day, rather than throughout the day, the expire time has been increased from 3 to 4 days.
Whenever we got a bounced email in the Email::Receiver we
previously would just set bounced: true on the EmailLog and
discard the status/diagnostic code. This commit changes this
flow to store the bounce error code (defined in the RFC at
https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml)
not just in the Email::Receiver, but also via webhook events
from other mail services and from SNS.
This commit does not surface the bounce error in the UI,
we can do that later if necessary.
There is a couple of layers of caching for theme JavaScript in Discourse:
The first layer is the `javascript_caches` table in the database. When a theme
with JavaScript files is installed, Discourse stores each one of the JavaScript
files in the `theme_fields` table, and then concatenates the files, compiles
them, computes a SHA1 digest of the compiled JavaScript and store the results
along with the SHA1 digest in the `javascript_caches` table.
Now when a request comes in, we need to render `<script>` tags for the
activated theme(s) of the site. To do this, we retrieve the `javascript_caches`
records of the activated themes and generate a `<script>` tag for each record.
The `src` attribute of these tags is a path to the `/theme-javascripts/:digest`
route which simply responds with the compiled JavaScript that has the requested
digest.
The second layer is a distributed cache whose purpose is to make rendering
`<script>` a lot more efficient. Without this cache, we'd have to query the
`javascript_caches` table to retrieve the SHA1 digests for every single
request. So we use this cache to store the `<script>` tags themselves so that
we only have to retrieve the `javascript_caches` records of the activated
themes for the first request and future requests simply get the cached
`<script>` tags.
What this commit does it ensures that the SHA1 digest in the
`javascript_caches` table stay the same across compilations by adding an order
by id clause to the query that loads the `theme_fields` records. Currently, we
specify no order when retrieving the `theme_fields` records so the order in
which they're retrieved can change across compilations and therefore cause the
SHA1 to change even though the individual records have not changed at all.
An inconsistent SHA1 digest across compilations can cause the database cache
and the distributed cache to have different digests and that causes the
JavaScript to fail to load (and if the theme heavily customizes the site, it
gives the impression that the site is broken) until the cache is cleared.
This can happen in busy sites when 2 concurrent requests recompile the
JavaScript files of a theme at the same time (this can happen when deploying a
new Discourse version) and request A updates the database cache after request B
did, and request B updates the distributed cache after request A did.
Internal ticket: t60783.
Co-authored-by: David Taylor <david@taylorhq.com>
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
In 8e5b945b0f, we reverted the commit but
at the same time resulted in Theme::BASE_COMPILER_VERSION going
backwards which caused problems with themes caching.
This commit bumps the version to clear all the caches.
Follow-up to 8e5b945b0f
Breakdown of fixes in this commit:
* `UserStat#topic_count` was not updated when visibility of
the topic changed.
* `UserStat#post_count` was not updated when post was hidden or
unhidden.
* `TopicConverter` was only incrementing or decrementing the counts by 1
even if a user has multiple posts in the topic.
* The commit turns off the verbose logging by default as it is just
noise to normal users who are not debugging this problem.
- Update UI to improve contrast
- Make it clear that the message is only shown to administrators
- Add theme name and id to the console output
- Parse the error backtrace to identify the theme-id for post-decoration errors
- Improve console output to include the theme name / URL
- Add `?safe_mode=no_custom` to the admin panel link, so that it will work even if the theme is causing the site to break
This commits adds a new advance_draft to PostCreator that controls if
the draft sequence will be advanced or not. If the draft sequence is
advanced then the old drafts will be cleared. This used to happen for
posts created by plugins or through the API and cleared user drafts
by mistake.
* FEATURE: Add external_id to topics
This commit allows for topics to be created and fetched by an
external_id. These changes are API only for now as there aren't any
front changes.
* add annotations
* add external_id to this spec
* Several PR feedback changes
- Add guardian to find topic
- 403 is returned for not found as well now
- add `include_external_id?`
- external_id is now case insensitive
- added test for posts_controller
- added test for topic creator
- created constant for max length
- check that it redirects to the correct path
- restrain external id in routes file
* remove puts
* fix tests
* only check for external_id in webhook if exists
* Update index to exclude external_id if null
* annotate
* Update app/controllers/topics_controller.rb
We need to check whether the topic is present first before passing it to the guardian.
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
* Apply suggestions from code review
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
This commit allows group SMTP emails to be sent with a
different from email address that has been set up as an
alias in the email provider. Emails from the alias will
be grouped correctly using Message-IDs in the mail client,
and replies to the alias go into the correct group inbox.
Ensures that `UserStat#post_count` and `UserStat#topic_count` does not
go below 0. When it does like it did now, we tend to have bugs in our
code since we're usually coding with the assumption that the count isn't
negative.
In order to support the constraints, our post and topic fabricators in
tests will now automatically increment the count for the respective
user's `UserStat` as well. We have to do this because our fabricators
bypasss `PostCreator` which holds the responsibility of updating `UserStat#post_count` and
`UserStat#topic_count`.
