Commit Graph

3931 Commits

Author SHA1 Message Date
David Taylor
b1b8c37e62 FIX: Correctly import theme assets with spaces in filename
This issue is caused by the filename restrictions in `OptimizedImage#ensure_safe_paths!`. Difficult to add a test for this because image optimization is bypassed in test mode.
2019-01-18 15:20:11 +00:00
David Taylor
f40c97c367 Remove support for theme_var and theme_color_var fields
These are remnants of the old 'customization' system, and have been superseded by theme settings
2019-01-18 12:04:23 +00:00
Guo Xiang Tan
95e3841974 FIX: Remove old reference and use MiniScheduler::Stat. 2019-01-18 16:36:11 +08:00
Vinoth Kannan
2818afb267 FIX: Do not hide posts automatically when flagged by staff user 2019-01-18 01:33:55 +05:30
Kris
675bf94133 UX: Bump up base font size 1px, add smaller text size option 2019-01-17 10:30:34 -05:00
David Taylor
24b59afe2a FIX: Wrap theme javascript inside IIFE to prevent using global namespace 2019-01-17 14:44:10 +00:00
David Taylor
5bfa661a87 FIX: Theme settings are referenced in JS by settings, not themeSetting 2019-01-17 12:24:21 +00:00
David Taylor
880311dd4d
FEATURE: Support for localized themes (#6848)
- Themes can supply translation files in a format like `/locales/{locale}.yml`. These files should be valid YAML, with a single top level key equal to the locale being defined. For now these can only be defined using the `discourse_theme` CLI, importing a `.tar.gz`, or from a GIT repository.

- Fallback is handled on a global level (if the locale is not defined in the theme), as well as on individual keys (if some keys are missing from the selected interface language).

- Administrators can override individual keys on a per-theme basis in the /admin/customize/themes user interface.

- Theme developers should access defined translations using the new theme prefix variables:
  JavaScript: `I18n.t(themePrefix("my_translation_key"))`
  Handlebars: `{{theme-i18n "my_translation_key"}}` or `{{i18n (theme-prefix "my_translation_key")}}`

- To design for backwards compatibility, theme developers can check for the presence of the `themePrefix` variable in JavaScript

- As part of this, the old `{{themeSetting.setting_name}}` syntax is deprecated in favour of `{{theme-setting "setting_name"}}`
2019-01-17 11:46:11 +00:00
Guo Xiang Tan
aa2cc4ab31 FIX: Liked notification consolidation has to account for user like frequency setting. 2019-01-17 14:33:23 +08:00
Sam
384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Bianca Nenciu
7d84648d11 FEATURE: Remove full quotes only from new posts. (#6862) 2019-01-17 13:24:32 +11:00
Sam
d5ecf8e8c2 FIX: properly clean out post and user actions on destroy user
This corrects 2 issues:

First is a regression with d7c08e21 for some reason dependent :delete_all
respects default scopes where-as dependent :destroy bypasses it.

Secondly, we were keeping orphan user actions around on user destroy, this
ensures we remove all the user actions not only ones that originated by
the user.

So for example: if I like a post of user A we create a user action saying I
did that, but once user A is deleted we were not removing the action leading
to an orphan action in the database.
2019-01-17 12:40:30 +11:00
Sam
d7c08e217a PERF: delete user and post actions vs destroy on user destroy
Users can have 100s of thousands of post and user actions, we do not want
to destroy each individually cause the tracking is enormous and the amount
of queries we would need is enormous.

This gives up on the `after_commit` hook on `post_actions` which ships a message
to clients to synchronize a post, so some phantom post_actions may remain
in the UX in the rare occasion we delete a user. The phantoms will be gone
on reload.
2019-01-17 11:19:37 +11:00
Régis Hanol
58008dd2fe FIX: sort topics/posts/links in descending order in user summaries
This should only change the order on freshly imported instances with no likes.
This makes the user summary show the latest topics/posts/links instead of the firsts until the users get some likes.
2019-01-16 23:58:46 +01:00
David Taylor
b46b6e72d1 FIX: Only add the trust level name for numeric trust levels
Previously the TrustLevelAndStaffSetting would display 'new user' for staff/admin
2019-01-16 12:04:12 +00:00
Guo Xiang Tan
f06b773d25 FIX: Don't show liked consolidated notification when frequency is never. 2019-01-16 17:08:59 +08:00
Guo Xiang Tan
ebe65577ed
FEATURE: Consolidate likes notifications. (#6879) 2019-01-16 10:40:16 +08:00
Arpit Jalan
c4422fdc13 UX: sort admins and moderators by last seen date on /about page
UX: only show active admins and moderators on /about page
2019-01-15 17:26:11 +05:30
Vinoth Kannan
59e5af466b FIX: call method only if it exists
discourse-objects creating empty Discourse object in window. It fails the _registerPluginCode method call since it is not exist.
2019-01-15 15:55:32 +05:30
David Taylor
1ebd3dbbd0
FEATURE: Allow the base font size to be changed on a per-user basis (#6859) 2019-01-14 13:21:46 +00:00
Robin Ward
95f263995d FIX: Previous annotations were broken 2019-01-11 14:30:19 -05:00
Robin Ward
a3839495e0 Update annotations 2019-01-11 12:19:43 -05:00
Guo Xiang Tan
2956c52e57 FIX: Show title as home logo if title has been set and logo is blank.
https://meta.discourse.org/t/default-text-logo-not-working/103936
2019-01-11 15:46:58 +08:00
Joffrey JAFFEUX
1fdbf0fc9b
FIX: uses compiler version to force rebaking of themes (#6870)
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 12:36:13 +01:00
Joffrey JAFFEUX
f9648de897
DEV: upgrades from Ember 2.13 to Ember 3.5.1 (#6808)
Co-Authored-By: Bianca Nenciu <nbianca@users.noreply.github.com>
Co-Authored-By: David Taylor <david@taylorhq.com>
2019-01-10 11:06:01 +01:00
Guo Xiang Tan
d10694150e Revert "FIX: Partial reply key search in email sent logs."
This reverts commit e9b2018bc8.
2019-01-10 10:05:56 +08:00
Guo Xiang Tan
cb0f6d653b DEV: Minor fixes to b63b399799. 2019-01-10 09:38:22 +08:00
Saurabh Patel
b63b399799 DEV: remove uploaded_meta_id column from category (#6725)
* DEV: remove uploaded_meta_id column from category

* remove uploaded_meta part
2019-01-10 09:37:21 +08:00
Guo Xiang Tan
e9b2018bc8 FIX: Partial reply key search in email sent logs.
Follow up to c85b9c6ed3
2019-01-10 09:25:14 +08:00
Rafael dos Santos Silva
f73fe36772 FEATURE: PWA compatibility checks in the Dashboard (#6850) 2019-01-09 08:46:11 +08:00
Sam
7e4a43afc6 PERF: run ImageMagick conversions with nice 10
This lowers the priority of all image resizes so they do not clog CPU on
machines running low on resources
2019-01-09 09:21:58 +11:00
Sam
e08a3f719c FEATURE: push post rebake regular task to low priority queue
This allows us to run regular rebakes without starving the normal queue.

It additionally adds the ability to specify queue with `Jobs.enqueue` so
we can specifically queue a job with lower priority using the `queue` arg.
2019-01-09 08:57:20 +11:00
Vinoth Kannan
8f602be2fe FEATURE: keep the topic in closed status until the community flags are handled 2019-01-08 16:13:10 +05:30
Maja Komel
6c8069c65a FIX: properly escape embed url 2019-01-07 18:11:43 +01:00
Robin Ward
5eaf3cb104 Adjusts the minimum_flag_threshold for TL3/TL4 actions
Before this patch, a high trust level user could flag something
and have an action be taken, as well as skipping the flag queue.

Now, if a TL3/TL4 cause an action, the flag will skip the minimum
visibility check and allow staff to review it.
2019-01-04 13:16:44 -05:00
Régis Hanol
95e5f8380d FEATURE: Allow plugins to add custom emoji translations
FIX: buildTranslationTree was erroring when translations overlapped (ie. ":-)" and ":-))")
FIX: emoji translations wasn't working properly when translations overlapped
2019-01-04 15:27:46 +01:00
David Taylor
5bf16d7d10 FEATURE: Topic timer for bumping a topic in the future 2019-01-04 13:08:04 +00:00
Sam
9c91e68351 PERF: remove image optimization throttling from Sidekiq
Previously we only allowed one image optimization per machine, this meant there
was cross talk between avatar resizing and Sidekiq. This could lead to large
amounts of starvation when optimized image version changed which in turn could
block the Sidekiq queue.

This increases amount of allowed load on machines but this is preferable to
having crosstalk between avatar resizing and Sidekiq.
2019-01-04 18:44:44 +11:00
Vinoth Kannan
75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Sam
8b7a2d1cb7 FEATURE: add setting to bypass sending redis CLIENT commands
Some cloud providers (Google Memorystore) do not support any CLIENT commands

By setting :id to nil in the redis config hash we can avoid these commands.

This adds a special global setting GCE users can enable:
`DISCOURSE_REDIS_SKIP_CLIENT_COMMANDS = true`
2019-01-04 15:08:33 +11:00
Sam
70269c7c97 FEATURE: tighter limits on per cluster post rebakes
We have the periodical job that regularly will rebake old posts. This is
used to trickle in update to cooked markdown. The problem is that each rebake
can issue multiple background jobs (post process and pull hotlinked images)

Previously we had no per-cluster limit so cluster running 100s of sites could
flood the sidekiq queue with rebake related jobs.

New system introduces a hard limit of 300 rebakes per 15 minutes across a
cluster to ensure the sidekiq job is not dominated by this.

We also reduced `rebake_old_posts_count` to 80, which is a safer default.
2019-01-04 09:24:46 +11:00
Arpit Jalan
d1597683f3 Revert "FIX: trim trailing slash from topic links"
This reverts commit 993f847a2c.

There is an edge case where the link click redirect fails when the URL has trailing slash. Need to figure out a better fix for this.
2019-01-03 17:29:22 +05:30
Arpit Jalan
993f847a2c FIX: trim trailing slash from topic links 2019-01-03 12:38:36 +05:30
Sam
570877da3c FEATURE: store thumbnail algorithm version in optimized image table
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.

We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
2019-01-03 17:07:30 +11:00
Gerhard Schlager
e8053d6e7d FIX: Polls didn't work in imported posts
Imports skip validation of posts, but polls are only created during the validation phase.
2019-01-02 15:26:57 +01:00
Sam
766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Arpit Jalan
70fdc10365
FEATURE: move posts to new/existing PM (#6802) 2018-12-31 17:17:22 +05:30
Joffrey JAFFEUX
f1269fa807
FEATURE: Add Top Uploads report (#6825)
Co-Authored-By: I am very Pro-Grammer. <khalilovcmded@users.noreply.github.com>
2018-12-28 20:48:54 +01:00
Vinoth Kannan
2b006c0429 FEATURE: Invalidate broken images cache on Rebuild HTML action 2018-12-26 23:22:07 +05:30
Régis Hanol
5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Sam Saffron
303a535dba PERF: automatic upload size calculation not persisted
Previously if upload had missing width and height we would calculate
on first use BUT we (me) forgot to save this to the database

This was particularly bad on home page cause category images (when old)
miss dimensions.
2018-12-26 17:19:29 +02:00
Joffrey JAFFEUX
f637286db5
UX: improves flags-status report (#6773)
- link to post
- cover more post action types
2018-12-26 10:29:33 +01:00
Joffrey JAFFEUX
0402f0f357
UX: new site setting to define activity metrics displayed on dashboard 2018-12-26 10:29:07 +01:00
Kyle Zhao
61dcd7c52c FIX: validate YAML before save to avoid race condition 2018-12-25 15:08:17 +02:00
Vinoth Kannan
a16bb32865 FIX: Do not raise error if most liked user is deleted 2018-12-25 01:04:55 +05:30
Gerhard Schlager
126be6c478 FIX: Resetting bump date shouldn't fail when first post is hidden 2018-12-21 17:43:52 +01:00
David Taylor
7feabd9e49 PERF: Eradicate N+1 queries from the theme admin page 2018-12-21 11:03:58 +02:00
Vinoth Kannan
2909e7fbdf FIX: Subfolder path is missing in category permalinks after slug update 2018-12-21 09:29:54 +05:30
Arpit Jalan
bb67ca9d21 FIX: use correct post object when logging exception 2018-12-19 22:17:37 +05:30
Bianca Nenciu
2eefe6d5d6 FIX: Use CDN for logos and icons. (#6698) 2018-12-19 10:20:48 +01:00
Vinoth Kannan
9c6f77f9da DEV: Remove the unique_searches column from select query 2018-12-19 02:45:48 +05:30
Robin Ward
6080e3a2c0 FIX: Broken spec 2018-12-18 14:55:09 -05:00
Robin Ward
662cfc416b FEATURE: Show a blurry preview when lazy loading images
This generates a 10x10 PNG thumbnail for each lightboxed image.
If Image Lazy Loading is enabled (IntersectionObserver API) then
we'll load the low res version when offscreen. As the image scrolls
in we'll swap it for the high res version.

We use a WeakMap to track the old image attributes. It's much less
memory than storing them as `data-*` attributes and swapping them
back and forth all the time.
2018-12-19 01:57:30 +08:00
Robin Ward
e593d68beb Use an options hash instead of boolean parameters 2018-12-19 01:57:30 +08:00
Vinoth Kannan
341a6bd78a
REFACTOR: Calculate CTR in SearchLog model and hide unique column (#6791) 2018-12-18 19:13:46 +05:30
Maja Komel
98d09c90ac Current user serializer groups (ef7f84b follow-up) 2018-12-18 09:05:45 +01:00
Bianca Nenciu
1023003eba FIX: Strip remote url before import. (#6762) 2018-12-17 15:27:49 +01:00
Gerhard Schlager
1a8ca68ea3 FEATURE: Improve backup stats on admin dashboard
* Dashboard doesn't timeout anymore when Amazon S3 is used for backups
* Storage stats are now a proper report with the same caching rules
* Changing the backup_location, s3_backup_bucket or creating and deleting backups removes the report from the cache
* It shows the number of backups and the backup location
* It shows the used space for the correct backup location instead of always showing used space on local storage
* It shows the date of the last backup as relative date
2018-12-17 11:35:11 +01:00
Guo Xiang Tan
c0aae16f6b FIX: Clear anon cache when disabling readonly mode.
`SiteSerializer#is_readonly` is cached for an anonymous user so we have
to clear the cache when disabling readonly mode. Otherwise, the site may
appear to be in readonly mode for an extended period of time.
2018-12-17 17:27:44 +08:00
Neil Lalonde
ef0e84e3d9 FIX: clear the site_contact_username setting if the user's staff privileges are revoked 2018-12-14 16:52:44 -05:00
David Taylor
1960236822
FIX: Suspicious login detection (#6772) 2018-12-14 16:30:34 +00:00
Joffrey JAFFEUX
03014b0d05
FEATURE: adds security tab to dashboard (#6768)
This commit also includes the new staff_logins report
2018-12-14 13:47:59 +01:00
Maja Komel
9f89aadd33 FIX: delete all posts in batches without hijack (#6747) 2018-12-14 11:04:18 +01:00
Sam
7ee9a6a7ec SECURITY: do not delete avatars uploads when deleting accounts
We rely on the clean up uploads job to do this safely
2018-12-13 16:26:07 +11:00
Gerhard Schlager
688755baf2 DEV: Improve specs and handle invalid email token
Follow-up to 7977b09025
2018-12-11 18:04:10 +01:00
David Taylor
3fedb2ad20 DEV: Style and performance improvements
Follow-up from 9db8291
2018-12-11 09:58:20 +00:00
Sam
671469bcc7 FIX: URLs containing two # would fail to work
Some URLs in browsers are non compliant and contain twos `#` this commit adds
special handling for this edge case by auto encoding any fragments containing `#`
2018-12-11 18:03:13 +11:00
Gerhard Schlager
7977b09025 FEATURE: Activate users invited via email when invite is redeemed
Do not send an activation email to users invited via email. They
already confirmed their email address by clicking the invite link.
Users invited via link will need to confirm their email address before
they can login.
2018-12-11 00:09:53 +01:00
David Taylor
9db829134c
FIX: Use database to persist metadata during social registration (#6750)
Previously was using the cookie_store, which is limited to 4kb. This caused issues for providers sending large volumes of metadata about a user.
2018-12-10 15:10:06 +00:00
David Taylor
160d29b18a
REFACTOR: Migrate TwitterAuthenticator to use ManagedAuthenticator (#6739)
No changes to functionality. TwitterAuthenticator goes from 136 lines to 24, and all twitter-specific logic elsewhere has been deleted 🎉
2018-12-07 15:39:06 +00:00
Saurabh Patel
9e3143445b DEV:add uploaded_meta option in category for category meta image (#6724) 2018-12-07 16:24:07 +01:00
Guo Xiang Tan
dcf9c6da59 DEV: Don't publish post messages to non-human users. 2018-12-06 08:24:13 +08:00
Régis Hanol
aea2d8bbeb FIX: properly secure poll message bus
Co-authored-by: Sam <sam.saffron@gmail.com>
2018-12-05 21:27:49 +01:00
David Taylor
a1d9aeda8b REFACTOR: split TopicLink#extract_from into multiple methods
Also rename some confusing variables
2018-12-05 18:54:30 +01:00
David Taylor
37249c9a32 FIX: Do not reset link counts when post is rebaked
This was an indentation mistake introduced in 44eba0b. Pretty understandable, considering we are indented 8 levels deep in this method. Will follow-up with a refactor to improve this.
2018-12-05 18:54:30 +01:00
Guo Xiang Tan
978f0db109 SECURITY: Require groups to be given when inviting to a restricted category. (#6715) 2018-12-05 16:43:07 +01:00
Vinoth Kannan
57ba4b7cb2
DEV: remove unnecessary safe nav operators (#6730) 2018-12-05 20:07:18 +05:30
Vinoth Kannan
fb78414229 Add missing safe navigation operator 2018-12-05 16:23:43 +05:30
Vinoth Kannan
d33d031742
FEATURE: Filter topic and post web hook events by tags (#6726)
* FEATURE: Filter topic and post web hook events by tags

* Add a spec test with unmatched tags
2018-12-05 14:44:06 +05:30
Régis Hanol
3c9c95ac83 Update Rubocop to 0.60 2018-12-04 10:48:16 +01:00
Sam
aa97f6fdba FEATURE: disable notifications for small actions that are whispers
Previously we would notify on small actions if they were whispers
this inconsistently lead to all sorts of problems including

- collapsed "N replies" after assign
- empty push notifications

New behavior adds an api to explicitly send push notifications as well
if needed: create_notification_alert
2018-12-04 17:54:42 +11:00
David Taylor
9248ad1905 DEV: Enable Style/SingleLineMethods and Style/Semicolon in Rubocop (#6717) 2018-12-04 11:48:13 +08:00
Penar Musaraj
f8e6a37858 FIX: raise exception when getting dimensions of missing image
- follow-up on 0eacd45ab1
2018-12-03 10:19:49 -05:00
Guo Xiang Tan
125bb46052 Remove unused constant in the codebase. 2018-12-03 11:04:47 +08:00
Sam
236c755d62 FIX: do not store key tracking last seen time indefinitely
UserStat has some special logic to keep adding time read if repeat calls
are made in intervals less than 100 seconds. This is called regularly
when we update read timings on a topic.

We only need to cache this key in redis for 100 seconds, however previously
we would keep it forever, 1 key per user. This has potential of bloating
a very large amount of keys for no longer active users in redis.
2018-12-03 08:35:26 +11:00
Kyle Zhao
488fba3c5f
FEATURE: allow plugins and themes to extend the default CSP (#6704)
* FEATURE: allow plugins and themes to extend the default CSP

For plugins:

```
extend_content_security_policy(
  script_src: ['https://domain.com/script.js', 'https://your-cdn.com/'],
  style_src: ['https://domain.com/style.css']
)
```

For themes and components:

```
extend_content_security_policy:
  type: list
  default: "script_src:https://domain.com/|style_src:https://domain.com"
```

* clear CSP base url before each test

we have a test that stubs `Rails.env.development?` to true

* Only allow extending directives that core includes, for now
2018-11-30 09:51:45 -05:00
David Taylor
208005f9c9 REFACTOR: Migrate FacebookAuthenticator to use ManagedAuthenticator
Changes to functionality
  - Removed syncing of user metadata including gender, location etc.
    These are no longer available to standard Facebook applications.
  - Removed the remote 'revoke' functionality. No other providers have
    it, and it does not appear to be standard practice in other apps.
  - The 'facebook_no_email' event is no longer logged. The system can
    cope fine with a missing email address.

Data is migrated to the new user_associated_accounts table.
facebook_user_infos can be dropped once we are confident the data has
been migrated successfully.
2018-11-30 11:18:11 +00:00
David Taylor
534e1b1b18 DEV: Introduce Auth::ManagedAuthenticator
A generic implementation of Auth::Authenticator which stores data in the
new UserAssociatedAccount model. This should help significantly reduce the duplicated
logic across different auth providers.
2018-11-30 11:18:11 +00:00
Penar Musaraj
0eacd45ab1 FIX: refactor ImageSizer.resize
reverts 140d9c2
2018-11-29 15:28:45 -05:00
Penar Musaraj
140d9c2910 FIX: call ImageSizer only if width/height are available 2018-11-29 15:03:02 -05:00