* REFACTOR: Rename SiteSetting.disable_edit_notifications to disable_system_edit_notifications
- The older name could cause some confusion because the setting does not disable all edit notifications, only system ones.
* FIX: Add frozen_string_literal: true in the migration
* DEV: Deprecate 'disable_edit_notifications'
* DEV: group_list site settings should store IDs instead of group names
* Ship site setting to know when we should migrate group_list settings
* Migrate existing group_list site settings
* Bump migration timestamp and don't set null when migrating is not possible.
DEV: deprecate `invite.via_email` in favor of `invite.emailed_status`
This commit adds a new column `emailed_status` in `invites` table for
tracking email sending status.
0 - not required
1 - pending
2 - bulk pending
3 - sending
4 - sent
For normal email invites, invite record is created with emailed_status
set to 'pending'.
When bulk invites are sent invite record is created with emailed_status
set to 'bulk pending'.
For invites that generates link, invite record is created with
emailed_status set to 'not required'.
When invite email is in queue emailed_status is updated to 'sending'
Once the email is sent via `InviteEmail` job the invite emailed_status
is updated to 'sent'.
- adds a migration renaming FA4 icon names in badges
- allows all icons to be used in badges (previously was limited to icons prefixed with fa-)
- renames remaining FA 4.7 icons equivalents
In order for this to work the Backuper stores a couple of site settings
in the new backup_metadata table, because the old setting values might
not be available on restore anymore.
Groups can now be marked as visible to "logged on users". All automatic groups (except `everyone`) are now visible to "logged on users", previously they were marked as public but suppressed in the group page for non-staff.
This allows you to temporarily disable components without having to remove them from a theme.
This feature is very handy when doing quick fix engineering.
Adds a second factor landing page that centralizes a user's second factor configuration.
This contains both TOTP and Backup, and also allows multiple TOTP tokens to be registered and organized by a name. Access to this page is authenticated via password, and cached for 30 minutes via a secure session.
* Remove unused method
* Prefabricate user in category_user_spec.rb
* FIX: Remove notification_level from category_users unique indexes
* FIX: CategoryUser#batch_set wasn't updating pre-existing records
* Improve tests for CategoryUser#batch_set
* FIX: changed was being reported incorrectly
* DEV: Rewrote query to do a bulk insert
* DEV: remove unnecessary parentheses
You can now add javascript files under `/javascripts/*` in a theme, and they will be loaded as if they were included in core, or a plugin. If you give something the same name as a core/plugin file, it will be overridden. Support file extensions are `.js.es6`, `.hbs` and `.raw.hbs`.
This feature allows end users to "defer" topics by marking them unread
The functionality is default disabled. This also introduces the new site
setting default_other_enable_defer: to enable this by default on new user
accounts.
Previously we used custom fields to denote a user was anonymous, this was
risky in that custom fields are prone to race conditions and are not
properly dedicated, missing constraints and so on.
The new table `anonymous_users` is properly protected. There is only one
possible shadow account per user, which is enforced using a constraint.
Every anonymous user will have a unique row in the new table.
The migration for the ToS signup field happend in 2014. Everyone who hasn't updated yet needs to live with the English text "Terms of Service". There's no need to keep these unused translations forever.
User.publish_notifications_state is called every time a notification is
created, this can become a very critical code path.
On some heavy notification related sites this can be a major CPU user on PG
This index makes it much cheaper to publish notification state, cause a
simple index lookup does the trick.
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.
Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
This is a feature that used to be present in discourse-assign but is
much easier to implement in core. It also allows a topic to be assigned
without it claiming for review and vice versa and allows it to work with
category group reviewers.
This change shows a notification number besides the flag icon in the
post menu if there is reviewable content associated with the post.
Additionally, if there is pending stuff to review, the icon has a red
background.
We have also removed the list of links below a post with the flag
status. A reviewer is meant to click the number beside the flag icon to
view the flags. As a consequence of losing those links, we've removed
the ability to undo or ignore flags below a post.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
This is a regression as a result of 7896c74c2b. Most instances would have ran the migrations and some might have run this migration with the incorrect query. Impact of this is small for now but I'm fixing this for correctness purposes.
All these columns must never include duplicates for a single post
otherwise code breaks. All are defined in post.rb in the top in constants
but we usually prefer not to ref constants in case they change and migration
becomes inconsistent.
This commit adds some improvements to native app banners for iOS and Android
- iOS and Android now have separate settings for native app banners
- app banners will now only show for users on TL1 and up
- app ids are now in a hidden site setting to allow sites to switch to their own app, if desired
- iOS only: the site URL is passed to the app arguments
A scheduled job runs `Post.rebake_old` with a limit of 80 which does a
look up for the latest posts that have not been baked to the latest
version. Before this commit, the query would run using the primary key's
index and does a reverse scan. However the query performance quicky
becomes bad as more and more of the latest posts have been baked to the
latest version.
This optimisation avoids large scans joining the topics table with the
topic_users table.
Previously when a user carried a lot of read state we would have to join
the entire read state with the topics table. This operation would slow down
home page and every topic page. The more read state you accumulated the
larger the impact.
The optimisation helps people who clean up unread, however if you carry
unread from years ago it will only have minimal impact.
A new checkbox has been added to the Tags tab of the category settings modal
which is used when some tags and/or tag groups are restricted to the category,
and all other unrestricted tags should also be allowed.
Default is the same as the previous behaviour: only allow the specified set of
tags and tag groups in the category.
"Rejecting" a user in the queue is equivalent to deleting them, which
would then making it impossible to review rejected users. Now we store
information about the user in the payload so if they are deleted things
still display in the Rejected view.
Secondly, if a user is destroyed outside of the review queue, it will
now automatically "Reject" that queue item.
Conversely, if a user is deactivated the reviewable should automatically
be rejected.
Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
This functionality was never supported but before the new review queue
it didn't have any errors. Now the combination of settings is prevented
and existing sites with sso enabled will be migrated to remove invite
only.
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.
Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
- s3_force_path_style was added as a Minio specific url scheme but it has never been well supported in our code base.
- Our new migrate_to_s3 rake task does not work reliably with path style urls too
- Minio has also added support for virtual style requests i.e the same scheme as AWS S3/DO Spaces so we can rely on that instead of using path style requests.
- Add migration to drop s3_force_path_style from the site_settings table
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by
* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
Previous instagram records didn’t have unique index, as a result it's seems like in some cases we have duplicate ids, which is causing issues with the associated_accounts which does have unique index.
It's sorted by `updated_at DESC` on the thought that the most recent one is more likely to be the one we want to keep.
For some reason, setting a text site setting to empty in the past can
result in either an empty string or null as the value. This migration
is a follow up to 1981add261.
If the depercated SiteSetting.logo_url was set to "" previously, it will
not have been migrated by the onceoff job which we added. As such, we
need a migration to migrate the blank override so that the default for
certain upload site settings will not be used.
* FEATURE: Add `IgnoredUsersSummary` daily job
## Why?
This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).
We want to:
1. Send an automatic group PM that goes out to moderators
2. When {x} users have Ignored the same user, threshold defined by a site setting, default of 5
3. Only send this message every X days which is defined by another site setting
* Adds warnings to the "Edit Category" dialog
* Doesn't hide the "Security" tab on the "Edit Category" dialog anymore. Instead, it shows an explanation why permissions can't be changed.
* Makes the category name translatable
* Hides the category name from the edit dialog (it can be customized by overriding the translation)
* Creates a translation override if the category has been renamed in the past
Generally we should never be touching AR objects in migrations, this is
super risky as we may end up with invalid schema cache.
This code from 2013 did it unconditionally. This change amends it so:
1. We only load up schema if we have no choice
2. We flush the cache before and after
This makes this migration far less risky.
This commit copies `login_required.welcome_message` translation override to `login_required.welcome_message_invite_only` if the `login_required.welcome_message_invite_only` override is blank. This is done so as to ensure that login page customization remains same after [this commit](93eb0a0690).
New `about.json` fields (all optional):
- `authors`: An arbitrary string describing the theme authors
- `theme_version`: An arbitrary string describing the theme version
- `minimum_discourse_version`: Theme will be auto-disabled for lower versions. Must be a valid version descriptor.
- `maximum_discourse_version`: Theme will be auto-disabled for lower versions. Must be a valid version descriptor.
A localized description for a theme can be provided in the language files under the `theme_metadata.description` key
The admin UI has been re-arranged to display this new information, and give more prominence to the remote theme options.
- Themes can supply translation files in a format like `/locales/{locale}.yml`. These files should be valid YAML, with a single top level key equal to the locale being defined. For now these can only be defined using the `discourse_theme` CLI, importing a `.tar.gz`, or from a GIT repository.
- Fallback is handled on a global level (if the locale is not defined in the theme), as well as on individual keys (if some keys are missing from the selected interface language).
- Administrators can override individual keys on a per-theme basis in the /admin/customize/themes user interface.
- Theme developers should access defined translations using the new theme prefix variables:
JavaScript: `I18n.t(themePrefix("my_translation_key"))`
Handlebars: `{{theme-i18n "my_translation_key"}}` or `{{i18n (theme-prefix "my_translation_key")}}`
- To design for backwards compatibility, theme developers can check for the presence of the `themePrefix` variable in JavaScript
- As part of this, the old `{{themeSetting.setting_name}}` syntax is deprecated in favour of `{{theme-setting "setting_name"}}`
- adds migration to enable CSP for new sites
- removes "EXPERIMENTAL" labels from setting names
- sets CSP violation report to default off
- adds CSP-related note to GTM setting
We query this table when getting composer messages with the queries
such as:
```
SELECT 1 AS one FROM "user_histories"
WHERE "user_histories"."target_user_id" = 1 AND
"user_histories"."action" = 9 AND
"user_histories"."topic_id" = 105794
LIMIT 1
```
This index ensures this query remains very quick, regardless of user
history size.
This was done to pick up 3 changes
1. New pngquant which will result in much smaller images
2. Placeholder images which are missing from old posts
3. Retina images missing from old posts
Also picks up on Image Magick upgrade which slightly alters resize algorithm.
Rebake trickles per: `rebake_old_posts_count` site setting. (100 per 15 minutes)
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.
We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
Do not send an activation email to users invited via email. They
already confirmed their email address by clicking the invite link.
Users invited via link will need to confirm their email address before
they can login.
Changes to functionality
- Removed syncing of user metadata including gender, location etc.
These are no longer available to standard Facebook applications.
- Removed the remote 'revoke' functionality. No other providers have
it, and it does not appear to be standard practice in other apps.
- The 'facebook_no_email' event is no longer logged. The system can
cope fine with a missing email address.
Data is migrated to the new user_associated_accounts table.
facebook_user_infos can be dropped once we are confident the data has
been migrated successfully.
A generic implementation of Auth::Authenticator which stores data in the
new UserAssociatedAccount model. This should help significantly reduce the duplicated
logic across different auth providers.
This splits off the logic between SSO keys used incoming vs outgoing, it allows to far better restrict who is allowed to log in using a site.
This allows for better auditing of the SSO provider feature
* FEATURE: add branch option to remote theme import
* FIX: Add missing variable in params
* FIX: Add missing param for import_theme method
* SPEC: Add test methods for branch support in git import
* FIX: Add missing space to scss style
* Do not assume default branch as master
* Change branch field placeholder
* FIX: add missing div start tag
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.
The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.
```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```
To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.
- If force_lowercase_tags is disabled, then mixed case tags are allowed.
- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.
- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.
- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.
- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
Introduces a hidden setting (default is 0.1) that erodes bounce score
every time we send an email. This means that erratic failures are less
painful cause system auto corrects
Previously we used width and height for thumbnails, new code ensures
1. We auto correct width and height
2. We added extra columns for thumbnail_width and height, this is determined
by actual upload and no longer passed in as a side effect
3. Optimized Image now stores filesize which can be used for analysis, decisions
Also
- fixes Android image manifest as a side effect
- fixes issue where a thumbnail generated that is smaller than the upload is no longer used
```
change_column :topic_views, :ip_address, :inet, null: true
```
translates to
```
"ALTER TABLE \"topic_views\" ALTER COLUMN \"ip_address\" TYPE inet"
```
which locks the whole table while the migration is taking place.
### navigate_to_first_post_after_read setting for categories
When enabled on categories logged on users will return to OP after
reading the entire category. (useful for documentation categories)
### num_auto_bump_daily
Set a number of topics that will automatically bump daily on a category.
- Every 15 minutes we will check if any category has this setting
- Categories with the setting are shuffled
- We exclude pinned, closed, category description and archived topics
- Maximum of 1 topic for the list of categories is bumped till limit reached per category
- We always try to bump oldest first
- Limit is elastic using a RateLimiter that ensures that we only bump N per day
Also some minor organisation on category settings
Froze strings on category.rb
* Phase 0 for user-selectable theme components
- Drops `key` column from the `themes` table
- Drops `theme_key` column from the `user_options` table
- Adds `theme_ids` (array of ints default []) column to the `user_options` table and migrates data from `theme_key` to the new column.
- Removes the `default_theme_key` site setting and adds `default_theme_id` instead.
- Replaces `theme_key` cookie with a new one called `theme_ids`
- no longer need Theme.settings_for_client
Introduce new patterns for direct sql that are safe and fast.
MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API
- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder
See more at: https://github.com/discourse/mini_sql
* Feature: Push notifications for Android
Notification config for desktop and mobile are merged.
Desktop notifications stay as they are for desktop views.
If mobile mode, push notifications are enabled.
Added push notification subscriptions in their own table, rather than through
custom fields.
Notification banner prompts appear for both mobile and desktop when enabled.
Note, current practice if for plugins to submit PRs to core for any
migrations required for plugins, so we can better control schema.
Especially if core tables are being touched.
In this case index has close to zero cost unless akismet is installed
This reduces the akismet admin query from 20ms on every new page load to 0.5ms
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.
The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
This feature can be enabled by choosing a destination for the
`shared drafts category` site setting.
* Staff members can create shared drafts, choosing a destination
category for the topic when it is published.
* Shared Drafts can be viewed in their category, or above the
topic list for the destination category where it will end up.
* When the shared draft is ready, it can be published to the
appropriate category by clicking a button on the topic view.
* When published, Drafts change their timestamps to the current
time, and any edits to the original post are removed.
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.
New setting explicitly only removes from latest list but leaves the
category list alond
implemented review items.
Blocking previous codes - valid 2-factor auth tokens can only be authenticated once/30 seconds.
I played with updating the “last used” any time the token was attempted but that seemed to be overkill, and frustrating as to why a token would fail.
Translatable texts.
Move second factor logic to a helper class.
Move second factor specific controller endpoints to its own controller.
Move serialization logic for 2-factor details in admin user views.
Add a login ember component for de-duplication
Fix up code formatting
Change verbiage of google authenticator
add controller tests:
second factor controller tests
change email tests
change password tests
admin login tests
add qunit tests - password reset, preferences
fix: check for 2factor on change email controller
fix: email controller - only show second factor errors on attempt
fix: check against 'true' to enable second factor.
Add modal for explaining what 2fa with links to Google Authenticator/FreeOTP
add two factor to email signin link
rate limit if second factor token present
add rate limiter test for second factor attempts
Locking a post prevents it from being edited. This is useful if the user
has posted something which has been edited out, and the staff members don't
want them to be able to edit it back in again.
https://meta.discourse.org/t/search-logs-page/73281/11?u=techapj
This commit adds following features:
- support for tracking click through to user, tag and category
- new filter for search type (header, full page)
This commit also removes "most viewed topic" field from search logs page because we are now tracking multiple click through entities, so topic is not a special entity anymore. This also improves query perf. The query now takes `20.5ms` to runs, as opposed to `655.9ms` previously.
* Add user_home configuration option
* Use the new user_home preference to actually show the right home page
* Fix trailing whitespace
* Update user_option_serializer.rb
* Fix JavaScript default homepage tests
* Use an object instead of a giant switch
* Remove trailing whitespace
* Make the default `user_home` set to `null` instead of `0`
* Rename user_home to homepage_id
* Split alias levels in mentionable and messageable levels.
* Fixed some tests.
* Set messageable level to everyone by default.
* By defaults, groups are not mentionable or messageable.
* Made staff groups messageable by the system.
This change-set allows setting different defaults for different locales.
It also:
- Adds extensive testing around site setting validation
- raises deprecation error if site setting has the default property based on env
- relocated site settings for dev and tests in the initializer
- deprecated client_setting in the site setting's loading process
- ensure it raises when a enum site setting being set
- default_locale is promoted to `required` category.
- fixes incorrect default setting and validation
- fixes ensure type check for site settings
- creates a benchmark for site setting
- sets reasonable defaults for Chinese
There are 4 visibility levels
- public (default)
- members only
- staff
- owners
Note, admins and group owners ALWAYS have visibility to groups
Migration treated old "non public" as "members only"
Figuring out what unread topics a user has is a very expensive
operation over time.
Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)
When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.
This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.
We use pg functions, instead of nested queries here to assist the
planner.
This feature introduces the concept of themes. Themes are an evolution
of site customizations.
Themes introduce two very big conceptual changes:
- A theme may include other "child themes", children can include grand
children and so on.
- A theme may specify a color scheme
The change does away with the idea of "enabled" color schemes.
It also adds a bunch of big niceties like
- You can source a theme from a git repo
- History for themes is much improved
- You can only have a single enabled theme. Themes can be selected by
users, if you opt for it.
On a technical level this change comes with a whole bunch of goodies
- All CSS is now compiled using a custom pipeline that uses libsass
see /lib/stylesheet
- There is a single pipeline for css compilation (in the past we used
one for customizations and another one for the rest of the app
- The stylesheet pipeline is now divorced of sprockets, there is no
reliance on sprockets for CSS bundling
- CSS is generated with source maps everywhere (including themes) this
makes debugging much easier
- Our "live reloader" is smarter and avoid a flash of unstyled content
we run a file watcher in "puma" in dev so you no longer need to run
rake autospec to watch for CSS changes
- avoid access denied on bad cookie, instead just nuke it
- avoid marking a token unseen for first minute post rotation
- log path in user auth token logs
Revamped system for managing authentication tokens.
- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes
New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.
Also introduces weekly job to expire old auth tokens.
- Regular users are not notified of whispers
- Regular users no longer have "stuck" topics in unread
- Additional tracking for staff highest post number
- Remove a bunch of unused columns in topics table
previously we supported blanket read and write for user API, this
change amends it so we can define more limited scopes. A scope only
covers a few routes. You can not grant access to part of the site and
leave a large amount of the information hidden to API consumer.
FIX: warning about popup dimensions when using facebook login
Rules are:
- On account creation we always import
- If you already have an avatar uploaded, nothing is changed
- If you have no avatar uploaded, we upload from facebook on login
- If you have no avatar uploaded, we select facebook unless gravatar already selected
This also fixes SSO issues where on account creation accounts had missing avatar uploads
This feature ensures session cookie lifespan is extended
when user is online.
Also decreases session timeout from 90 to 60 days.
Ensures all users (including logged on ones) get expiring sessions.
Users can no longer opt-in for "public" edit history
if site owner disables it.
This feature adds cost and complexity to post rendering since
user options need to be premeptively loaded for every user in the
stream. It is also confusing to explain to communities with private edit
history.
