Commit Graph

3892 Commits

Author SHA1 Message Date
Sam
8b7a2d1cb7 FEATURE: add setting to bypass sending redis CLIENT commands
Some cloud providers (Google Memorystore) do not support any CLIENT commands

By setting :id to nil in the redis config hash we can avoid these commands.

This adds a special global setting GCE users can enable:
`DISCOURSE_REDIS_SKIP_CLIENT_COMMANDS = true`
2019-01-04 15:08:33 +11:00
Sam
70269c7c97 FEATURE: tighter limits on per cluster post rebakes
We have the periodical job that regularly will rebake old posts. This is
used to trickle in update to cooked markdown. The problem is that each rebake
can issue multiple background jobs (post process and pull hotlinked images)

Previously we had no per-cluster limit so cluster running 100s of sites could
flood the sidekiq queue with rebake related jobs.

New system introduces a hard limit of 300 rebakes per 15 minutes across a
cluster to ensure the sidekiq job is not dominated by this.

We also reduced `rebake_old_posts_count` to 80, which is a safer default.
2019-01-04 09:24:46 +11:00
Arpit Jalan
d1597683f3 Revert "FIX: trim trailing slash from topic links"
This reverts commit 993f847a2c.

There is an edge case where the link click redirect fails when the URL has trailing slash. Need to figure out a better fix for this.
2019-01-03 17:29:22 +05:30
Arpit Jalan
993f847a2c FIX: trim trailing slash from topic links 2019-01-03 12:38:36 +05:30
Sam
570877da3c FEATURE: store thumbnail algorithm version in optimized image table
Previously we had no idea what algorithm generated thumbnails, this starts tracking the version.

We also bumped up the version to force all optimized images to be generated. This is important cause we recently introduced pngquant which results in much smaller images.
2019-01-03 17:07:30 +11:00
Gerhard Schlager
e8053d6e7d FIX: Polls didn't work in imported posts
Imports skip validation of posts, but polls are only created during the validation phase.
2019-01-02 15:26:57 +01:00
Sam
766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Arpit Jalan
70fdc10365
FEATURE: move posts to new/existing PM (#6802) 2018-12-31 17:17:22 +05:30
Joffrey JAFFEUX
f1269fa807
FEATURE: Add Top Uploads report (#6825)
Co-Authored-By: I am very Pro-Grammer. <khalilovcmded@users.noreply.github.com>
2018-12-28 20:48:54 +01:00
Vinoth Kannan
2b006c0429 FEATURE: Invalidate broken images cache on Rebuild HTML action 2018-12-26 23:22:07 +05:30
Régis Hanol
5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Sam Saffron
303a535dba PERF: automatic upload size calculation not persisted
Previously if upload had missing width and height we would calculate
on first use BUT we (me) forgot to save this to the database

This was particularly bad on home page cause category images (when old)
miss dimensions.
2018-12-26 17:19:29 +02:00
Joffrey JAFFEUX
f637286db5
UX: improves flags-status report (#6773)
- link to post
- cover more post action types
2018-12-26 10:29:33 +01:00
Joffrey JAFFEUX
0402f0f357
UX: new site setting to define activity metrics displayed on dashboard 2018-12-26 10:29:07 +01:00
Kyle Zhao
61dcd7c52c FIX: validate YAML before save to avoid race condition 2018-12-25 15:08:17 +02:00
Vinoth Kannan
a16bb32865 FIX: Do not raise error if most liked user is deleted 2018-12-25 01:04:55 +05:30
Gerhard Schlager
126be6c478 FIX: Resetting bump date shouldn't fail when first post is hidden 2018-12-21 17:43:52 +01:00
David Taylor
7feabd9e49 PERF: Eradicate N+1 queries from the theme admin page 2018-12-21 11:03:58 +02:00
Vinoth Kannan
2909e7fbdf FIX: Subfolder path is missing in category permalinks after slug update 2018-12-21 09:29:54 +05:30
Arpit Jalan
bb67ca9d21 FIX: use correct post object when logging exception 2018-12-19 22:17:37 +05:30
Bianca Nenciu
2eefe6d5d6 FIX: Use CDN for logos and icons. (#6698) 2018-12-19 10:20:48 +01:00
Vinoth Kannan
9c6f77f9da DEV: Remove the unique_searches column from select query 2018-12-19 02:45:48 +05:30
Robin Ward
6080e3a2c0 FIX: Broken spec 2018-12-18 14:55:09 -05:00
Robin Ward
662cfc416b FEATURE: Show a blurry preview when lazy loading images
This generates a 10x10 PNG thumbnail for each lightboxed image.
If Image Lazy Loading is enabled (IntersectionObserver API) then
we'll load the low res version when offscreen. As the image scrolls
in we'll swap it for the high res version.

We use a WeakMap to track the old image attributes. It's much less
memory than storing them as `data-*` attributes and swapping them
back and forth all the time.
2018-12-19 01:57:30 +08:00
Robin Ward
e593d68beb Use an options hash instead of boolean parameters 2018-12-19 01:57:30 +08:00
Vinoth Kannan
341a6bd78a
REFACTOR: Calculate CTR in SearchLog model and hide unique column (#6791) 2018-12-18 19:13:46 +05:30
Maja Komel
98d09c90ac Current user serializer groups (ef7f84b follow-up) 2018-12-18 09:05:45 +01:00
Bianca Nenciu
1023003eba FIX: Strip remote url before import. (#6762) 2018-12-17 15:27:49 +01:00
Gerhard Schlager
1a8ca68ea3 FEATURE: Improve backup stats on admin dashboard
* Dashboard doesn't timeout anymore when Amazon S3 is used for backups
* Storage stats are now a proper report with the same caching rules
* Changing the backup_location, s3_backup_bucket or creating and deleting backups removes the report from the cache
* It shows the number of backups and the backup location
* It shows the used space for the correct backup location instead of always showing used space on local storage
* It shows the date of the last backup as relative date
2018-12-17 11:35:11 +01:00
Guo Xiang Tan
c0aae16f6b FIX: Clear anon cache when disabling readonly mode.
`SiteSerializer#is_readonly` is cached for an anonymous user so we have
to clear the cache when disabling readonly mode. Otherwise, the site may
appear to be in readonly mode for an extended period of time.
2018-12-17 17:27:44 +08:00
Neil Lalonde
ef0e84e3d9 FIX: clear the site_contact_username setting if the user's staff privileges are revoked 2018-12-14 16:52:44 -05:00
David Taylor
1960236822
FIX: Suspicious login detection (#6772) 2018-12-14 16:30:34 +00:00
Joffrey JAFFEUX
03014b0d05
FEATURE: adds security tab to dashboard (#6768)
This commit also includes the new staff_logins report
2018-12-14 13:47:59 +01:00
Maja Komel
9f89aadd33 FIX: delete all posts in batches without hijack (#6747) 2018-12-14 11:04:18 +01:00
Sam
7ee9a6a7ec SECURITY: do not delete avatars uploads when deleting accounts
We rely on the clean up uploads job to do this safely
2018-12-13 16:26:07 +11:00
Gerhard Schlager
688755baf2 DEV: Improve specs and handle invalid email token
Follow-up to 7977b09025
2018-12-11 18:04:10 +01:00
David Taylor
3fedb2ad20 DEV: Style and performance improvements
Follow-up from 9db8291
2018-12-11 09:58:20 +00:00
Sam
671469bcc7 FIX: URLs containing two # would fail to work
Some URLs in browsers are non compliant and contain twos `#` this commit adds
special handling for this edge case by auto encoding any fragments containing `#`
2018-12-11 18:03:13 +11:00
Gerhard Schlager
7977b09025 FEATURE: Activate users invited via email when invite is redeemed
Do not send an activation email to users invited via email. They
already confirmed their email address by clicking the invite link.
Users invited via link will need to confirm their email address before
they can login.
2018-12-11 00:09:53 +01:00
David Taylor
9db829134c
FIX: Use database to persist metadata during social registration (#6750)
Previously was using the cookie_store, which is limited to 4kb. This caused issues for providers sending large volumes of metadata about a user.
2018-12-10 15:10:06 +00:00
David Taylor
160d29b18a
REFACTOR: Migrate TwitterAuthenticator to use ManagedAuthenticator (#6739)
No changes to functionality. TwitterAuthenticator goes from 136 lines to 24, and all twitter-specific logic elsewhere has been deleted 🎉
2018-12-07 15:39:06 +00:00
Saurabh Patel
9e3143445b DEV:add uploaded_meta option in category for category meta image (#6724) 2018-12-07 16:24:07 +01:00
Guo Xiang Tan
dcf9c6da59 DEV: Don't publish post messages to non-human users. 2018-12-06 08:24:13 +08:00
Régis Hanol
aea2d8bbeb FIX: properly secure poll message bus
Co-authored-by: Sam <sam.saffron@gmail.com>
2018-12-05 21:27:49 +01:00
David Taylor
a1d9aeda8b REFACTOR: split TopicLink#extract_from into multiple methods
Also rename some confusing variables
2018-12-05 18:54:30 +01:00
David Taylor
37249c9a32 FIX: Do not reset link counts when post is rebaked
This was an indentation mistake introduced in 44eba0b. Pretty understandable, considering we are indented 8 levels deep in this method. Will follow-up with a refactor to improve this.
2018-12-05 18:54:30 +01:00
Guo Xiang Tan
978f0db109 SECURITY: Require groups to be given when inviting to a restricted category. (#6715) 2018-12-05 16:43:07 +01:00
Vinoth Kannan
57ba4b7cb2
DEV: remove unnecessary safe nav operators (#6730) 2018-12-05 20:07:18 +05:30
Vinoth Kannan
fb78414229 Add missing safe navigation operator 2018-12-05 16:23:43 +05:30
Vinoth Kannan
d33d031742
FEATURE: Filter topic and post web hook events by tags (#6726)
* FEATURE: Filter topic and post web hook events by tags

* Add a spec test with unmatched tags
2018-12-05 14:44:06 +05:30
Régis Hanol
3c9c95ac83 Update Rubocop to 0.60 2018-12-04 10:48:16 +01:00
Sam
aa97f6fdba FEATURE: disable notifications for small actions that are whispers
Previously we would notify on small actions if they were whispers
this inconsistently lead to all sorts of problems including

- collapsed "N replies" after assign
- empty push notifications

New behavior adds an api to explicitly send push notifications as well
if needed: create_notification_alert
2018-12-04 17:54:42 +11:00
David Taylor
9248ad1905 DEV: Enable Style/SingleLineMethods and Style/Semicolon in Rubocop (#6717) 2018-12-04 11:48:13 +08:00
Penar Musaraj
f8e6a37858 FIX: raise exception when getting dimensions of missing image
- follow-up on 0eacd45ab1
2018-12-03 10:19:49 -05:00
Guo Xiang Tan
125bb46052 Remove unused constant in the codebase. 2018-12-03 11:04:47 +08:00
Sam
236c755d62 FIX: do not store key tracking last seen time indefinitely
UserStat has some special logic to keep adding time read if repeat calls
are made in intervals less than 100 seconds. This is called regularly
when we update read timings on a topic.

We only need to cache this key in redis for 100 seconds, however previously
we would keep it forever, 1 key per user. This has potential of bloating
a very large amount of keys for no longer active users in redis.
2018-12-03 08:35:26 +11:00
Kyle Zhao
488fba3c5f
FEATURE: allow plugins and themes to extend the default CSP (#6704)
* FEATURE: allow plugins and themes to extend the default CSP

For plugins:

```
extend_content_security_policy(
  script_src: ['https://domain.com/script.js', 'https://your-cdn.com/'],
  style_src: ['https://domain.com/style.css']
)
```

For themes and components:

```
extend_content_security_policy:
  type: list
  default: "script_src:https://domain.com/|style_src:https://domain.com"
```

* clear CSP base url before each test

we have a test that stubs `Rails.env.development?` to true

* Only allow extending directives that core includes, for now
2018-11-30 09:51:45 -05:00
David Taylor
208005f9c9 REFACTOR: Migrate FacebookAuthenticator to use ManagedAuthenticator
Changes to functionality
  - Removed syncing of user metadata including gender, location etc.
    These are no longer available to standard Facebook applications.
  - Removed the remote 'revoke' functionality. No other providers have
    it, and it does not appear to be standard practice in other apps.
  - The 'facebook_no_email' event is no longer logged. The system can
    cope fine with a missing email address.

Data is migrated to the new user_associated_accounts table.
facebook_user_infos can be dropped once we are confident the data has
been migrated successfully.
2018-11-30 11:18:11 +00:00
David Taylor
534e1b1b18 DEV: Introduce Auth::ManagedAuthenticator
A generic implementation of Auth::Authenticator which stores data in the
new UserAssociatedAccount model. This should help significantly reduce the duplicated
logic across different auth providers.
2018-11-30 11:18:11 +00:00
Penar Musaraj
0eacd45ab1 FIX: refactor ImageSizer.resize
reverts 140d9c2
2018-11-29 15:28:45 -05:00
Penar Musaraj
140d9c2910 FIX: call ImageSizer only if width/height are available 2018-11-29 15:03:02 -05:00
Penar Musaraj
03deda2147
Upgrade to FontAwesome 5 (take two) (#6673)
* Add missing icons to set

* Revert FA5 revert

 This reverts commit 42572ff

* use new SVG syntax in locales

* Noscript page changes (remove login button, center "powered by" footer text)

* Cast wider net for SVG icons in settings

- include any _icon setting for SVG registry (offers better support for plugin settings)

- let themes store multiple pipe-delimited icons in a setting

- also replaces broken onebox image icon with SVG reference in cooked post processor

* interpolate icons in locales

* Fix composer whisper icon alignment

* Add support for stacked icons

* SECURITY: enforce hostname to match discourse hostname

This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname

* load SVG sprite with pre-initializers

* FIX: enable caching on SVG sprites

* PERF: use JSONP for SVG sprites so they are served from CDN

This avoids needing to deal with CORS for loading of the SVG

Note, added the svg- prefix to the filename so we can quickly tell in
dev tools what the file is

* Add missing SVG sprite JSONP script to CSP

* Upgrade to FA 5.5.0

* Add support for all FA4.7 icons

- adds complete frontend and backend for renamed FA4.7 icons

- improves performance of SvgSprite.bundle and SvgSprite.all_icons

* Fix group avatar flair preview

- adds an endpoint at /svg-sprites/search/:keyword

- adds frontend ajax call that pulls icon in avatar flair preview even when it is not in subset

* Remove FA 4.7 font files
2018-11-26 16:49:57 -05:00
Bianca Nenciu
172b3bf4d3 FIX: Fix broken theme field URLs. (#6622) 2018-11-23 16:11:05 +01:00
Guo Xiang Tan
3f636b2d19 FIX: Check whether group is mentionable by user when cooking post. 2018-11-22 16:16:33 +08:00
Saurabh Patel
d984323e23 FEATURE: Show change name of user in staff logs (#6647)
https://meta.discourse.org/t/admins-changing-users-name-not-username-should-be-logged/99511
2018-11-22 10:13:02 +08:00
Joffrey JAFFEUX
e860c8b844
FIX: adds support for missing reports from old dashboard (#6624) 2018-11-19 12:20:05 +01:00
Joffrey JAFFEUX
078705bcc0
minor refactoring (#6615) 2018-11-15 21:41:05 +01:00
Joffrey JAFFEUX
761003c8bd
FIX: makes sure an hex is always at 6 chars (#6608)
eg: #aaa should be converted to #aaaaaa
2018-11-15 00:52:47 +01:00
Maja Komel
c701036034 FIX: reset bump date resets bumped_at to the last regular post in topic (#6605) 2018-11-14 18:56:22 +01:00
Régis Hanol
c78dcde973 FIX: only send originalText when we need to 2018-11-14 17:47:59 +01:00
Bianca Nenciu
fce0a0ccc8 FEATURE: Compute distance between logins to generate login alerts. (#6562) 2018-11-14 13:26:47 +01:00
Guo Xiang Tan
2220345a8b Fix the build. 2018-11-14 17:32:32 +08:00
Guo Xiang Tan
72370b9c36 Add deprecation warnings for url based site settings. 2018-11-14 16:09:26 +08:00
Guo Xiang Tan
44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
David Taylor
17bc82765b FEATURE: Log password changes in UserHistory (#6600) 2018-11-14 08:32:42 +08:00
Kyle Zhao
38a9bc740d FIX: change title when primary group changes (#6602) 2018-11-14 08:28:41 +08:00
Sam
80ceb57c76 DEV: add API endpoint to destroy_timings only of last post
Previously API only allowed you to nuke all timings from a topic,
new API is less punishing and allows you just to remove 1 post.
2018-11-13 16:07:48 +11:00
Robin Ward
0cb33d2b52 UX: Rename Most Disagreed Flaggers report to "User Flagging Ratio" 2018-11-12 16:23:37 -05:00
David Taylor
d89ffbeffd
FEATURE: Add button to delete unused tags (#6587)
This is particularly useful if you have uploaded a CSV file, and wish
to bulk-delete all of the tags that you uploaded.
2018-11-12 16:24:34 +00:00
Bianca Nenciu
5af9a69a3b FIX: Do not check for suspicious login when impersonating. (#6534)
* FIX: Do not check for suspicious login when impersonating.

* DEV: Add 'impersonate' parameter to log_on_user.
2018-11-12 15:34:12 +01:00
Maja Komel
012da86a07 FIX user directory time period count (#6586) 2018-11-12 15:30:05 +01:00
Sam
d1684a1652 PERF: avoid left joins and distinct on posts_for query
Previously this was extremely expensive due to the `distict` and
`left join` clauses, this eliminates it and makes query much faster
2018-11-12 13:26:41 +11:00
David Taylor
c3c9299c44 FIX: Improve performance for Group.posts_for (#6588) 2018-11-12 08:44:20 +08:00
Joffrey JAFFEUX
f8838ac9cd
FIX: fix/removes broken elements of old dashboard 2018-11-09 17:18:19 +01:00
Gerhard Schlager
24e5be3f0c FIX: Relative links in translations should work with subfolder 2018-11-08 23:31:05 +00:00
Sam
42572ff138 Revert font awesome 5 changes
We are still pushing ahead on this 100% just need a bit longer to prepare
all plugins
2018-11-08 16:12:18 +11:00
Penar Musaraj
005e1ecb9b
FEATURE: Update Font Awesome to v5.4.1 and SVGs (#6557)
* First take on subsetting svg icons

* FontAwesome 5 svg subset WIP

* Include icons from plugins/badges into svg sprite subset

* add svg icon support to themes

* Add spec for SvgSprite

* Misc. SVG icon fixes

* Use FA5 svgs in local-dates plugin

* CSS adjustments, fix SVG icons in group flair

* Use SVG icons in poll plugin

* Add SVG icons to /wizard
2018-11-07 13:05:43 -05:00
Sam
0a442e319c FIX: correct svg handling for images
We regressed and optimized images no longer worked with svg

The following adds the correct logic to simply copy file for svgs
and bypasses resizing for svg avatars
2018-11-07 15:29:26 +11:00
Guo Xiang Tan
7534042427 DEV: Update annotations. 2018-11-07 11:11:19 +08:00
Bianca Nenciu
2070edf889 FIX: Clarify User.group_locked_trust_level.
* Rename User.group_locked_trust_level to User.group_granted_trust_level.

* Remove the column from users table.
2018-11-07 10:27:44 +08:00
Joffrey JAFFEUX
8067f8a32c
FIX: disables dates filtering on most_disagreed_flags report (#6556) 2018-11-02 14:42:52 +01:00
Joffrey JAFFEUX
4e0f033fae
FEATURE: adds ignored flags to most_disagreed_flags report (#6554) 2018-11-02 11:08:00 +01:00
Kyle Zhao
f9b36820ef
FIX: only extract script tags with certain types (#6553)
`script` tags with custom types (e.g. `text/template`) are not executed
by the browser, and should not be extracted into an external theme
JavaScript
2018-11-01 16:01:46 -04:00
Robin Ward
ec91450aae FEATURE: Track how many user flags are agreed/disagreed/ignored
Display the percentage when reviewing flags.
2018-11-01 09:59:50 -04:00
Sam
23423ba112 correct spec and error reporting
previous commit misused warn_exception which caused a spec to fail
2018-10-31 13:38:05 +11:00
Bianca Nenciu
e0ccd36dbe FEATURE: Suspicious logins report. (#6544) 2018-10-30 22:51:58 +00:00
Joffrey JAFFEUX
b2585524a9
FEATURE: adds a most disagreed flaggers report 2018-10-26 15:59:04 +02:00
Penar Musaraj
ed9c21e42c FEATURE: hide muted categories from /categories list (#6531) 2018-10-26 11:34:39 +11:00
Régis Hanol
d17c8df926 Only check for suspicious login for staff members 2018-10-26 00:29:28 +02:00
Bianca Nenciu
6a3767cde7 FEATURE: Warn users via email about suspicious logins. (#6520)
* FEATURE: Warn users via email about suspicious logins.

* DEV: Move suspicious login check to a job.
2018-10-25 09:45:31 +00:00