Commit Graph

67 Commits

Author SHA1 Message Date
Penar Musaraj
f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677)
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Guo Xiang Tan
a3938f98f8 Revert changes to FileStore::S3Store#path_for in f0620e7118.
There are some places in the code base that assumes the method should
return nil.
2019-05-29 18:39:07 +08:00
Guo Xiang Tan
f0620e7118 FEATURE: Support [description|attachment](upload://<short-sha>) in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Penar Musaraj
7c9fb95c15 Temporarily revert "FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan
b1d3c678ca FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603) 2019-05-28 11:18:21 -04:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan
243fb8d9ad Fix the build. 2019-03-13 17:39:07 +08:00
Vinoth Kannan
563b953224 DEV: Add 'backfill_etags_' to the method name since it also backfilling the etags 2019-02-19 21:54:35 +05:30
Vinoth Kannan
0472bd4adc FIX: Remove 'backfill_etags' keyword argument from 'uploads:missing' rake task
And etags backfilling code is optimized
2019-02-15 00:34:35 +05:30
Vinoth Kannan
7b5931013a Update rake task to backfill etags from s3 inventory 2019-02-14 05:18:06 +05:30
Vinoth Kannan
b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Vinoth Kannan
75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Rishabh
cae5ba7356 FIX: Ensure that multisite s3 uploads are tombstoned correctly (#6769)
* FIX: Ensure that multisite uploads are tombstoned into the correct paths

* Move multisite specs to spec/multisite/s3_store_spec.rb
2018-12-19 13:32:32 +08:00
Rishabh
503ae1829f FIX: All multisite upload paths should start with /uploads/default/.. (#6707) 2018-12-03 12:04:14 +08:00
Rishabh
05a4f3fb51 FEATURE: Multisite support for S3 image stores (#6689)
* FEATURE: Multisite support for S3 image stores

* Use File.join to concatenate all paths & fix linting on multisite/s3_store_spec.rb
2018-11-29 12:11:48 +08:00
Vinoth Kannan
bcdf5b2f47 DEV: improve missing uploads query and skip checking file size 2018-11-27 02:21:33 +05:30
Vinoth Kannan
4ccf9d28eb Remove trailing whitespaces 2018-11-27 01:15:29 +05:30
Vinoth Kannan
fd272eee44 FEATURE: Make uploads:missing task compatible with s3 uploads 2018-11-27 00:54:51 +05:30
Guo Xiang Tan
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00
Guo Xiang Tan
8496537590 Add RECOVER_FROM_S3 to uploads:list_posts_with_broken_images rake task. 2018-09-10 15:14:30 +08:00
Sam
5d96809abd FIX: improve support for subfolder S3 CDN 2018-08-22 12:31:13 +10:00
Sam
f5142861e5 Revert "Revert "FIX: upload URLs from S3 on subfolder installs""
This reverts commit 26c96e97e5.

We have no choice but to run this code
2018-08-22 11:31:33 +10:00
Sam
26c96e97e5 Revert "FIX: upload URLs from S3 on subfolder installs"
This reverts commit 357df2ff4f.
2018-08-22 10:51:40 +10:00
Neil Lalonde
357df2ff4f FIX: upload URLs from S3 on subfolder installs 2018-08-21 14:58:55 -04:00
Guo Xiang Tan
aafff740d2 Add FileStore::S3Store#copy_file. 2018-08-08 11:30:34 +08:00
Andrew Schleifer
dba22bbde2 rollback changes
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
Andrew Schleifer
52e9f49ec1 fix s3 subfolders harder
specifically, include the folder in absolute_base_url
2018-07-06 16:28:40 -05:00
Andrew Schleifer
4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Régis Hanol
5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Régis Hanol
93dfc87b99 FIX: always set the 'content_type' when storing a file on S3 2016-10-17 19:16:29 +02:00
Guo Xiang Tan
3141c179f7 REFACTOR: Get bucket name from S3Helper. 2016-08-19 14:08:37 +08:00
Guo Xiang Tan
7ff1f6cb9d Allow custom bucket name for FileStore::S3Store. 2016-08-16 15:25:42 +08:00
Guo Xiang Tan
205be0d044 Remove unused require. 2016-08-15 21:58:55 +08:00
Guo Xiang Tan
0433163866 FEATURE: Support subfolders in SiteSetting.s3_backup_bucket. 2016-08-15 16:14:51 +08:00
Guo Xiang Tan
aa5de3c40a FEATURE: Support subfolders in S3 bucket name.
This commit also fixes a bug where s3 uploads are not
moved to a tombstone folder when removed.
2016-08-15 13:07:41 +08:00
Guo Xiang Tan
3378ee223f FIX: Incorrect path being passed to S3Store#remove_file. 2016-08-15 11:35:30 +08:00
Hu Ming
f8a12d4940 Add support for AWS cn (#4327) 2016-07-14 16:56:09 +02:00
Régis Hanol
5169bcdb6e FIX: httpshttps ultra secure URLs 2016-06-30 16:55:01 +02:00
Régis Hanol
81a699e2b0 better support for mixed content 2015-06-01 17:49:58 +02:00
Régis Hanol
56f077db69 FIX: optimized images fail if source is remote and S3 is disabled 2015-06-01 11:13:56 +02:00
Régis Hanol
5a143c0c6e storage engines refactor 2015-05-29 18:39:47 +02:00
Régis Hanol
8e7bfd0f29 FIX: automatically growing uploads tree 2015-05-28 01:03:24 +02:00
Régis Hanol
83d2b59fc3 FIX: s3 endpoint when using 'us-east-1' region 2015-05-27 17:50:49 +02:00
Sam
a988cd5abe FIX: redirect to CDN avatar for s3 avatars 2015-05-27 12:02:57 +10:00
Régis Hanol
a5d93c6705 FIX: undefined method 'max_file_size_kb' 2015-05-26 16:39:41 +02:00
Régis Hanol
a797f7c664 FIX: properly handle images when using 's3_cdn_url' 2015-05-26 11:47:33 +02:00
Sam
e17f614771 FIX: fallback to local store when uploads are not on S3 2015-05-26 13:08:31 +10:00
Régis Hanol
bb0c2813ac FEATURE: generate (avatar) thumbnails in a background task
FIX: keep the "uploading..." indicator until the server replies via the MessageBus
FIX: text was disapearing when uploading an avatar

PERF: always use a region for S3 (defaults to 'us-east-1')
FEATURE: ApplyCDN middleware when using S3
FIX: use the same pattern to store files on S3 and locally
PERF: keep a local cache of uploads when generating thumbnails
FEATURE: migrate_to_s3 rake task
2015-05-25 17:59:00 +02:00