Gerhard Schlager
f2dc59d61f
FEATURE: Add hidden setting to include S3 uploads in backups
2019-07-09 14:04:16 +02:00
Penar Musaraj
03805e5a76
FIX: Ensure lightbox image download has correct content disposition in S3 ( #7845 )
2019-07-04 11:32:51 -04:00
Vinoth Kannan
b7830680b6
DEV: use cdn url to download the external uploads to local.
2019-06-06 19:17:19 +05:30
Penar Musaraj
f00275ded3
FEATURE: Support private attachments when using S3 storage ( #7677 )
...
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Guo Xiang Tan
a3938f98f8
Revert changes to FileStore::S3Store#path_for
in f0620e7118
.
...
There are some places in the code base that assumes the method should
return nil.
2019-05-29 18:39:07 +08:00
Guo Xiang Tan
f0620e7118
FEATURE: Support [description|attachment](upload://<short-sha>)
in MD take 2.
...
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Penar Musaraj
7c9fb95c15
Temporarily revert "FEATURE: Support [description|attachment](upload://<short-sha>)
in MD. ( #7603 )"
...
This reverts commit b1d3c678ca
.
We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan
b1d3c678ca
FEATURE: Support [description|attachment](upload://<short-sha>)
in MD. ( #7603 )
2019-05-28 11:18:21 -04:00
David Taylor
ef660d5a3e
FIX: Return consistent character encodings when downloading S3 uploads
...
Net::HTTP always returns ASCII-8BIT encoding. File.read auto-detects the encoding. This leads to an encoding inconsistency between a fresh download, and a cached download. This commit ensures all downloaded files are treated equally, by always returning the cached version from the filesystem, even during initial download.
One symptom of this problem is during theme exports: https://meta.discourse.org/t/116907
Related ruby ticket: https://bugs.ruby-lang.org/issues/2567
2019-05-17 11:27:00 +01:00
Sam Saffron
30990006a9
DEV: enable frozen string literal on all files
...
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.
Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Guo Xiang Tan
5b934cb33d
FIX: Error when trying to move the same file to tombstone.
...
If an optimized image is destroyed when a previous similar optimized
image is already placed in the tombstone, `FileUtils.move` will blow up.
2019-04-24 16:47:36 +08:00
Guo Xiang Tan
243fb8d9ad
Fix the build.
2019-03-13 17:39:07 +08:00
Guo Xiang Tan
b0c8fdd7da
FIX: Properly support defaults for upload site settings.
2019-03-13 16:36:57 +08:00
Régis Hanol
664e90bd17
FIX: ensure local images use local CDN when uploads are stored on S3
...
When the S3 store was enabled, we were only applying the S3 CDN.
So all images stored locally, like the emojis, were never put on the local CDN.
Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls'
in order to get final URLs.
I also removed the unnecessary PrettyText.add_s3_cdn method since this is already
handled in the CookedPostProcessor.
2019-02-20 19:24:38 +01:00
Vinoth Kannan
563b953224
DEV: Add 'backfill_etags_' to the method name since it also backfilling the etags
2019-02-19 21:54:35 +05:30
Vinoth Kannan
0472bd4adc
FIX: Remove 'backfill_etags' keyword argument from 'uploads:missing' rake task
...
And etags backfilling code is optimized
2019-02-15 00:34:35 +05:30
Vinoth Kannan
7b5931013a
Update rake task to backfill etags from s3 inventory
2019-02-14 05:18:06 +05:30
Vinoth Kannan
b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats ( #6867 )
2019-02-01 10:10:48 +05:30
Vinoth Kannan
f94c0283b2
FIX: Use correct version when generating file path for optimized image ( #6871 )
2019-01-11 18:35:38 +05:30
Vinoth Kannan
75dbb98cca
FEATURE: Add S3 etag value to uploads table ( #6795 )
2019-01-04 14:16:22 +08:00
Régis Hanol
5381096bfd
PERF: new 'migrate_to_s3' rake task
2018-12-26 17:34:49 +01:00
Rishabh
cae5ba7356
FIX: Ensure that multisite s3 uploads are tombstoned correctly ( #6769 )
...
* FIX: Ensure that multisite uploads are tombstoned into the correct paths
* Move multisite specs to spec/multisite/s3_store_spec.rb
2018-12-19 13:32:32 +08:00
Rishabh
503ae1829f
FIX: All multisite upload paths should start with /uploads/default/.. ( #6707 )
2018-12-03 12:04:14 +08:00
Rishabh
871d4543cc
FIX: Use File.join for relative_base_url, fix spec
2018-11-29 09:49:56 +05:30
Rishabh
05a4f3fb51
FEATURE: Multisite support for S3 image stores ( #6689 )
...
* FEATURE: Multisite support for S3 image stores
* Use File.join to concatenate all paths & fix linting on multisite/s3_store_spec.rb
2018-11-29 12:11:48 +08:00
Vinoth Kannan
bcdf5b2f47
DEV: improve missing uploads query and skip checking file size
2018-11-27 02:21:33 +05:30
Vinoth Kannan
4ccf9d28eb
Remove trailing whitespaces
2018-11-27 01:15:29 +05:30
Vinoth Kannan
fd272eee44
FEATURE: Make uploads:missing task compatible with s3 uploads
2018-11-27 00:54:51 +05:30
Guo Xiang Tan
ce6a0a5e9e
FIX: Moving upload to tombstone should update modification time.
...
A upload created a long time ago will be nuked from the tombstone
immediately if it gets deleted.
2018-09-18 10:48:29 +08:00
Guo Xiang Tan
e1b16e445e
Rename FileHelper.is_image?
-> FileHelper.is_supported_image?
.
2018-09-12 09:22:28 +08:00
Guo Xiang Tan
8496537590
Add RECOVER_FROM_S3
to uploads:list_posts_with_broken_images
rake task.
2018-09-10 15:14:30 +08:00
Sam
5d96809abd
FIX: improve support for subfolder S3 CDN
2018-08-22 12:31:13 +10:00
Sam
f5142861e5
Revert "Revert "FIX: upload URLs from S3 on subfolder installs""
...
This reverts commit 26c96e97e5
.
We have no choice but to run this code
2018-08-22 11:31:33 +10:00
Sam
26c96e97e5
Revert "FIX: upload URLs from S3 on subfolder installs"
...
This reverts commit 357df2ff4f
.
2018-08-22 10:51:40 +10:00
Neil Lalonde
357df2ff4f
FIX: upload URLs from S3 on subfolder installs
2018-08-21 14:58:55 -04:00
Guo Xiang Tan
aafff740d2
Add FileStore::S3Store#copy_file
.
2018-08-08 11:30:34 +08:00
Andrew Schleifer
dba22bbde2
rollback changes
...
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
Andrew Schleifer
52e9f49ec1
fix s3 subfolders harder
...
specifically, include the folder in absolute_base_url
2018-07-06 16:28:40 -05:00
Régis Hanol
448e2fe1a2
FIX: properly delete files in the download cache
2018-07-04 18:18:39 +02:00
Andrew Schleifer
4be0e31459
fix s3_cdn_url when the s3 bucket contains a folder
2018-05-23 15:51:02 -05:00
Régis Hanol
5f4f617689
FIX: cache_file storage cleanup logic was wrong
...
https://meta.discourse.org/t/68296
2018-01-18 17:00:04 +01:00
Sam
70bb2aa426
FEATURE: allow specifying s3 config via globals
...
This refactors handling of s3 so it can be specified via GlobalSetting
This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3
It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan
8cc8010564
Maintain backwards compatibility before Jobs::MigrateUploadExtensions
runs.
2017-08-03 11:56:55 +09:00
Neil Lalonde
83011045c8
fix rubocop offenses
2017-07-31 11:59:16 -04:00
Neil Lalonde
5d528f0d15
Merge pull request #4958 from dmacjam/search_posts_by_filetype
...
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
Guo Xiang Tan
5012d46cbd
Add rubocop to our build. ( #5004 )
2017-07-28 10:20:09 +09:00
Neil Lalonde
d8c27e3871
Merge branch 'master' into search_posts_by_filetype
2017-07-25 14:41:20 -04:00
Jakub Macina
677267ae78
Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests.
2017-07-12 17:19:27 +02:00
Jakub Macina
8c445e9f17
Fix backend code for searching by a filetype as a combination of uploads and topic links. Add rspec test for extracting file extension in upload.
2017-07-06 19:19:31 +02:00
Régis Hanol
5d63a7f4a6
FIX: pull hotlinked images even when they have no extension
2017-06-13 13:27:05 +02:00