Commit Graph

194 Commits

Author SHA1 Message Date
Penar Musaraj
102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Gerhard Schlager
cc7b24b88b FIX: Regularly reset unknown extension of uploads
Discourse will try to detect the extension the next time the upload is needed. Maybe there was just a transient error the last time.
2019-08-21 10:23:20 +02:00
Guo Xiang Tan
7bd93eba3e FIX: Gravatar uploads being dependent on authorized_extensions. 2019-08-01 16:24:09 +08:00
Guo Xiang Tan
73a45048a0 FIX: Upload#short_url generates incorrect URL when extension is nil. 2019-06-19 09:10:50 +08:00
Penar Musaraj
f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677)
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Guo Xiang Tan
bb98785569 DEV: Fix broken Upload#base62_sha1. 2019-06-04 14:10:46 +08:00
Guo Xiang Tan
f0620e7118 FEATURE: Support [description|attachment](upload://<short-sha>) in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Penar Musaraj
7c9fb95c15 Temporarily revert "FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan
b1d3c678ca FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603) 2019-05-28 11:18:21 -04:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan
761e1e25f7 Remove tables incorrectly excluded in Upload.migrate_to_new_scheme.
Follow up to 4a03b21f80.
2019-04-25 20:27:52 +08:00
Vinoth Kannan
84cfe30c1f Use update_all instead of save to select and update particular columns
9ec6ef85b6
2019-04-25 11:56:57 +05:30
Guo Xiang Tan
4a03b21f80 Exclude large tables when remapping in Upload.migrate_to_new_scheme.
Those tables do not carry any information about uploads.
2019-04-25 11:55:48 +08:00
Guo Xiang Tan
be88ac8b7e Additional old Upload#url format for Upload.migrate_to_new_scheme. 2019-04-24 18:19:25 +08:00
Guo Xiang Tan
9ec6ef85b6 Don't validate post when saving in Upload.migrate_to_new_scheme. 2019-04-24 17:17:31 +08:00
Guo Xiang Tan
3c9495b989 DEV: Put a mutex around Upload.migrate_to_new_scheme.
This ensures that only one migration is running at any given point in
time across the instances.
2019-04-24 17:07:10 +08:00
Guo Xiang Tan
55f406bb79 Follow up to 845b68f9ab. 2019-04-24 16:57:45 +08:00
Guo Xiang Tan
845b68f9ab Update possible upload url format for Upload.migrate_to_new_scheme. 2019-04-24 16:26:54 +08:00
Guo Xiang Tan
c811e59898 Avoid hardcoding db name in Upload.migrate_to_new_scheme. 2019-04-24 15:25:24 +08:00
Guo Xiang Tan
573ddcbbd0 Disable migrate_to_new_scheme once there is nothing to migrate. 2019-04-24 14:00:00 +08:00
Guo Xiang Tan
3094a603b7 Follow up to 149411ec90. 2019-04-24 12:20:53 +08:00
Guo Xiang Tan
149411ec90 PERF: Speed up Upload.migrate_to_new_scheme by limiting remap scope.
Doing a `LIKE` on `Post#raw` and `Post#cooked` takes forever on large
sites.
2019-04-24 11:56:48 +08:00
Vinoth Kannan
914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Guo Xiang Tan
52c4711e80 DEV: Fix method that was incorrectly made private. 2019-04-03 12:37:50 +08:00
Guo Xiang Tan
4037a2fd0b FIX: Restrict scope of old scheme upload migration.
Some sites have external URLs that don't even match `%/uploads/%' and
some sites surprise me with URLs that contains the default path when it
is a site in a multisite cluster. We can't do anything about those.
2019-04-03 11:56:41 +08:00
Guo Xiang Tan
feb731bffd FIX: Regenerate optimized images instead of migrating from old scheme.
`OptimizedImage.migrate_to_new_scheme` was optimizing optimized images
which we don't need to do. Regnerating the optimized image is way easier.
2019-04-03 09:45:02 +08:00
Guo Xiang Tan
d0a1922ad9 FIX: Upload.migrate_to_new_scheme undefined error when external image fails to download. 2019-03-28 16:00:13 +08:00
Guo Xiang Tan
161a3c3870 DEV: Don't optimize image when migrating to new scheme.
The image has already been uploaded, the migrate to new scheme job's is
just to correct the scheme and not the content of the upload.
2019-03-26 15:12:10 +08:00
Guo Xiang Tan
0ac6a58b09 FIX: Skip validation when saving in Upload#migrate_to_new_scheme. 2019-03-26 14:28:39 +08:00
Guo Xiang Tan
40b03e717b FIX: Upload#migrate_to_new_scheme should not migrate system uploads. 2019-03-14 12:45:03 +08:00
Guo Xiang Tan
b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Guo Xiang Tan
58b0e945bd
UX: Lightbox support for image uploader. (#7034) 2019-02-21 10:13:37 +08:00
Robin Ward
95f263995d FIX: Previous annotations were broken 2019-01-11 14:30:19 -05:00
Robin Ward
a3839495e0 Update annotations 2019-01-11 12:19:43 -05:00
Vinoth Kannan
75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Sam Saffron
303a535dba PERF: automatic upload size calculation not persisted
Previously if upload had missing width and height we would calculate
on first use BUT we (me) forgot to save this to the database

This was particularly bad on home page cause category images (when old)
miss dimensions.
2018-12-26 17:19:29 +02:00
Robin Ward
662cfc416b FEATURE: Show a blurry preview when lazy loading images
This generates a 10x10 PNG thumbnail for each lightboxed image.
If Image Lazy Loading is enabled (IntersectionObserver API) then
we'll load the low res version when offscreen. As the image scrolls
in we'll swap it for the high res version.

We use a WeakMap to track the old image attributes. It's much less
memory than storing them as `data-*` attributes and swapping them
back and forth all the time.
2018-12-19 01:57:30 +08:00
Robin Ward
e593d68beb Use an options hash instead of boolean parameters 2018-12-19 01:57:30 +08:00
Penar Musaraj
f8e6a37858 FIX: raise exception when getting dimensions of missing image
- follow-up on 0eacd45ab1
2018-12-03 10:19:49 -05:00
Penar Musaraj
0eacd45ab1 FIX: refactor ImageSizer.resize
reverts 140d9c2
2018-11-29 15:28:45 -05:00
Penar Musaraj
140d9c2910 FIX: call ImageSizer only if width/height are available 2018-11-29 15:03:02 -05:00
Guo Xiang Tan
44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Arpit Jalan
3e17ef0507
Merge pull request #6414 from techAPJ/user-field-unique-key
FEATURE: add external details to user fields
2018-09-20 22:29:39 +05:30
Sam
df45e82377 SECURITY: only allow picking of avatars created by self (#6417)
* SECURITY: only allow picking of avatars created by self

Also adds origin tracking to all uploads including de-duplicated uploads
2018-09-19 22:33:10 -07:00
Arpit Jalan
72be638728 FEATURE: add external details to user fields 2018-09-20 08:10:51 +05:30
Guo Xiang Tan
4a92c5b2d6 UploadRecovery should recover attachments too. 2018-09-19 10:44:36 +08:00
Guo Xiang Tan
3884e99e88 Add extra protection in Upload#get_from_url.
In case the extension goes missing from the URL.
2018-09-12 00:12:14 -07:00
Guo Xiang Tan
b3469bea2d FIX: Uploads not being linked correctly to posts.
Regression due to 1f636c445b.
2018-09-11 23:50:23 -07:00
Guo Xiang Tan
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00