github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-23 22:26:26 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
42,063 Commits 190 Branches 500 Tags 938 MiB
87684f7c5e
Commit Graph

145 Commits

Author SHA1 Message Date
Guo Xiang Tan
b1d3c678ca FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603) 2019-05-28 11:18:21 -04:00
David Taylor
ef660d5a3e FIX: Return consistent character encodings when downloading S3 uploads 
Net::HTTP always returns ASCII-8BIT encoding. File.read auto-detects the encoding. This leads to an encoding inconsistency between a fresh download, and a cached download. This commit ensures all downloaded files are treated equally, by always returning the cached version from the filesystem, even during initial download.

One symptom of this problem is during theme exports: https://meta.discourse.org/t/116907

Related ruby ticket: https://bugs.ruby-lang.org/issues/2567
 2019-05-17 11:27:00 +01:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files 
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
 2019-05-13 09:31:32 +08:00
Guo Xiang Tan
5b934cb33d FIX: Error when trying to move the same file to tombstone. 
If an optimized image is destroyed when a previous similar optimized
image is already placed in the tombstone, `FileUtils.move` will blow up.
 2019-04-24 16:47:36 +08:00
Guo Xiang Tan
243fb8d9ad Fix the build. 2019-03-13 17:39:07 +08:00
Guo Xiang Tan
b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Régis Hanol
664e90bd17 FIX: ensure local images use local CDN when uploads are stored on S3 
When the S3 store was enabled, we were only applying the S3 CDN.
So all images stored locally, like the emojis, were never put on the local CDN.

Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls'
in order to get final URLs.

I also removed the unnecessary PrettyText.add_s3_cdn method since this is already
handled in the CookedPostProcessor.
 2019-02-20 19:24:38 +01:00
Vinoth Kannan
563b953224 DEV: Add 'backfill_etags_' to the method name since it also backfilling the etags 2019-02-19 21:54:35 +05:30
Vinoth Kannan
0472bd4adc FIX: Remove 'backfill_etags' keyword argument from 'uploads:missing' rake task 
And etags backfilling code is optimized
 2019-02-15 00:34:35 +05:30
Vinoth Kannan
7b5931013a Update rake task to backfill etags from s3 inventory 2019-02-14 05:18:06 +05:30
Vinoth Kannan
b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Vinoth Kannan
f94c0283b2
FIX: Use correct version when generating file path for optimized image (#6871) 2019-01-11 18:35:38 +05:30
Vinoth Kannan
75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
Régis Hanol
5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Rishabh
cae5ba7356 FIX: Ensure that multisite s3 uploads are tombstoned correctly (#6769) 
* FIX: Ensure that multisite uploads are tombstoned into the correct paths

* Move multisite specs to spec/multisite/s3_store_spec.rb
 2018-12-19 13:32:32 +08:00
Rishabh
503ae1829f FIX: All multisite upload paths should start with /uploads/default/.. (#6707) 2018-12-03 12:04:14 +08:00
Rishabh
871d4543cc FIX: Use File.join for relative_base_url, fix spec 2018-11-29 09:49:56 +05:30
Rishabh
05a4f3fb51 FEATURE: Multisite support for S3 image stores (#6689) 
* FEATURE: Multisite support for S3 image stores

* Use File.join to concatenate all paths & fix linting on multisite/s3_store_spec.rb
 2018-11-29 12:11:48 +08:00
Vinoth Kannan
bcdf5b2f47 DEV: improve missing uploads query and skip checking file size 2018-11-27 02:21:33 +05:30
Vinoth Kannan
4ccf9d28eb Remove trailing whitespaces 2018-11-27 01:15:29 +05:30
Vinoth Kannan
fd272eee44 FEATURE: Make uploads:missing task compatible with s3 uploads 2018-11-27 00:54:51 +05:30
Guo Xiang Tan
ce6a0a5e9e FIX: Moving upload to tombstone should update modification time. 
A upload created a long time ago will be nuked from the tombstone
immediately if it gets deleted.
 2018-09-18 10:48:29 +08:00
Guo Xiang Tan
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00
Guo Xiang Tan
8496537590 Add RECOVER_FROM_S3 to uploads:list_posts_with_broken_images rake task. 2018-09-10 15:14:30 +08:00
Sam
5d96809abd FIX: improve support for subfolder S3 CDN 2018-08-22 12:31:13 +10:00
Sam
f5142861e5 Revert "Revert "FIX: upload URLs from S3 on subfolder installs"" 
This reverts commit 26c96e97e5.

We have no choice but to run this code
 2018-08-22 11:31:33 +10:00
Sam
26c96e97e5 Revert "FIX: upload URLs from S3 on subfolder installs" 
This reverts commit 357df2ff4f.
 2018-08-22 10:51:40 +10:00
Neil Lalonde
357df2ff4f FIX: upload URLs from S3 on subfolder installs 2018-08-21 14:58:55 -04:00
Guo Xiang Tan
aafff740d2 Add FileStore::S3Store#copy_file. 2018-08-08 11:30:34 +08:00
Andrew Schleifer
dba22bbde2 rollback changes 
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
 2018-07-06 17:16:40 -05:00
Andrew Schleifer
52e9f49ec1 fix s3 subfolders harder 
specifically, include the folder in absolute_base_url
 2018-07-06 16:28:40 -05:00
Régis Hanol
448e2fe1a2 FIX: properly delete files in the download cache 2018-07-04 18:18:39 +02:00
Andrew Schleifer
4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
Régis Hanol
5f4f617689 FIX: cache_file storage cleanup logic was wrong 
https://meta.discourse.org/t/68296
 2018-01-18 17:00:04 +01:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals 
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
 2017-10-06 16:20:01 +11:00
Guo Xiang Tan
8cc8010564 Maintain backwards compatibility before Jobs::MigrateUploadExtensions runs. 2017-08-03 11:56:55 +09:00
Neil Lalonde
83011045c8 fix rubocop offenses 2017-07-31 11:59:16 -04:00
Neil Lalonde
5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype 
FEATURE: Search posts by filetype
 2017-07-31 11:55:34 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Neil Lalonde
d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
Jakub Macina
677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
Jakub Macina
8c445e9f17 Fix backend code for searching by a filetype as a combination of uploads and topic links. Add rspec test for extracting file extension in upload. 2017-07-06 19:19:31 +02:00
Régis Hanol
5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Robin Ward
cdbe027c1c Refactor FileHelper to use keyword arguments. 2017-05-24 13:54:26 -04:00
Guo Xiang Tan
f534f041a0 FIX: Ensure directory exists. 2017-04-07 15:50:17 +08:00
Matt Palmer
da7a44064b Fix purge_tombstone for the brave new world of secure command execution 2017-03-22 10:27:07 +11:00
Guo Xiang Tan
e7c972ac89 FIX: Don't use backticks that take in inputs. 2017-03-17 15:33:51 +08:00
Régis Hanol
93dfc87b99 FIX: always set the 'content_type' when storing a file on S3 2016-10-17 19:16:29 +02:00
Guo Xiang Tan
3141c179f7 REFACTOR: Get bucket name from S3Helper. 2016-08-19 14:08:37 +08:00
Guo Xiang Tan
7ff1f6cb9d Allow custom bucket name for FileStore::S3Store. 2016-08-16 15:25:42 +08:00
Guo Xiang Tan
205be0d044 Remove unused require. 2016-08-15 21:58:55 +08:00
Guo Xiang Tan
0433163866 FEATURE: Support subfolders in SiteSetting.s3_backup_bucket. 2016-08-15 16:14:51 +08:00
Guo Xiang Tan
aa5de3c40a FEATURE: Support subfolders in S3 bucket name. 
This commit also fixes a bug where s3 uploads are not
moved to a tombstone folder when removed.
 2016-08-15 13:07:41 +08:00
Guo Xiang Tan
3378ee223f FIX: Incorrect path being passed to S3Store#remove_file. 2016-08-15 11:35:30 +08:00
Guo Xiang Tan
1779a9634a FIX: Make sure we raise an error when method is not implemented. 2016-08-12 11:43:57 +08:00
Hu Ming
f8a12d4940 Add support for AWS cn (#4327) 2016-07-14 16:56:09 +02:00
Régis Hanol
5169bcdb6e FIX: httpshttps ultra secure URLs 2016-06-30 16:55:01 +02:00
Régis Hanol
1caaf5208f move tombstone under 'uploads/' for easier deployment 2016-05-30 09:46:27 +02:00
Guo Xiang Tan
26084397c1
FIX: Check if file exists upfront. 2016-05-23 10:25:53 +08:00
Rafael dos Santos Silva
c2e0eeb089 Separate relative base_url and upload_path 
This makes easier to reason about paths
 2016-03-10 00:47:18 -03:00
Rafael dos Santos Silva
4d01328bc2 FIX: Image Lightbox on Subfolder Install 
Since LocalSotre.is_relative wasn't subfolder aware, images on subfolder install weren't getting lightboxed.
 2016-03-02 18:44:15 -03:00
Robin Ward
f155ff8270 FIX: Tests would fail if your test db's optimized image ids were high 2015-10-16 17:08:41 -04:00
Luke GB
acc05dd3a5 Fix LocalStore.remove_file to not raise if source doesn't exist 
FileUtils.move actually ends up raising an "unknown file type" error if the file doesn't exist instead of Errno::ENOENT.

It's possible to rescue this, but in the end it's easier to just ask move not to throw an error, since we're going to throw it away anyway.
 2015-07-12 14:14:12 +01:00
Régis Hanol
81a699e2b0 better support for mixed content 2015-06-01 17:49:58 +02:00
Régis Hanol
56f077db69 FIX: optimized images fail if source is remote and S3 is disabled 2015-06-01 11:13:56 +02:00
Régis Hanol
5a143c0c6e storage engines refactor 2015-05-29 18:39:47 +02:00
Régis Hanol
8e7bfd0f29 FIX: automatically growing uploads tree 2015-05-28 01:03:24 +02:00
Régis Hanol
83d2b59fc3 FIX: s3 endpoint when using 'us-east-1' region 2015-05-27 17:50:49 +02:00
Sam
a988cd5abe FIX: redirect to CDN avatar for s3 avatars 2015-05-27 12:02:57 +10:00
Régis Hanol
a5d93c6705 FIX: undefined method 'max_file_size_kb' 2015-05-26 16:39:41 +02:00
Régis Hanol
a797f7c664 FIX: properly handle images when using 's3_cdn_url' 2015-05-26 11:47:33 +02:00
Sam
e17f614771 FIX: fallback to local store when uploads are not on S3 2015-05-26 13:08:31 +10:00
Régis Hanol
bb0c2813ac FEATURE: generate (avatar) thumbnails in a background task 
FIX: keep the "uploading..." indicator until the server replies via the MessageBus
FIX: text was disapearing when uploading an avatar

PERF: always use a region for S3 (defaults to 'us-east-1')
FEATURE: ApplyCDN middleware when using S3
FIX: use the same pattern to store files on S3 and locally
PERF: keep a local cache of uploads when generating thumbnails
FEATURE: migrate_to_s3 rake task
 2015-05-25 17:59:00 +02:00
Sam
f079dd63ed PERF: remove "fog" dependency 2015-05-25 17:57:06 +10:00
Régis Hanol
9ded21e4c6 FIX: consistent and future-proof upload storage pattern 2015-05-19 12:31:12 +02:00
Régis Hanol
31e9cafe0e FEATURE: use original filename when clicking the download link in the lightbox 2014-10-15 19:20:04 +02:00
Régis Hanol
652cc3efba FEATURE: new rake task to clean up uploads & thumbnails 2014-09-29 18:31:53 +02:00
Régis Hanol
bfdbb70b3b FIX: automatic backup uploads to S3 when using a region 2014-09-24 22:52:09 +02:00
Mesut Kayman
80468037e9 Don't set content-disposition header for images 
If this header is set for images, clicking the image link causes them to automatically download the file, rather than viewing it inside the browser. This commit forces this behavior only for uploads that aren't images
 2014-09-09 19:43:02 +03:00
Régis Hanol
e1191a5fcf Merge pull request #2511 from windhamg/s3_iam_role 
support for EC2 IAM roles with Amazon S3 file store/backup
 2014-08-18 16:14:14 +02:00
Akshay
7ef61144e7 Avoid using to_s when performing String Interpolation 2014-08-14 23:55:27 +05:30
Régis Hanol
c7330ed73f BUGFIX: errors when post-processing 'data images' 2014-07-18 17:54:18 +02:00
Gary Windham
3f6bd0606a support for EC2 IAM roles with Amazon S3 file store/backup 2014-07-14 09:14:16 -07:00
Robin Ward
457e32d1b0 Optimistic fix for deploying errors due to missing files. Please review 
@ZogStriP
 2014-05-28 10:35:21 -04:00
Régis Hanol
542d54e6bf BUGFIX: uploads to S3 2014-04-15 13:04:14 +02:00
Régis Hanol
2505d18aa9 FEATURE: support email attachments 2014-04-14 22:55:57 +02:00
Sam
1533a1163c use_ssl is just confusing, it means use_https , fix name of setting 2014-01-09 10:51:38 +11:00
Régis Hanol
e732aa8a86 BUGFIX: we should not store absolute urls for locally uploaded avatar templates 
Highly recommended to run: `RAILS_ENV=production bundle exec rake avatars:regenerate` to fix the avatar templates stored in the database.
 2014-01-07 17:45:06 +01:00
Régis Hanol
c6fb60e0a0 FIX: S3 upload when using dots in bucket name 2013-12-16 11:44:59 +01:00
Régis Hanol
52160179f8 add a tombstone for extra safety 2013-11-27 22:05:11 +01:00
Régis Hanol
37fd7ab574 pull hotlinked images 2013-11-05 19:07:29 +01:00
Régis Hanol
9d6cb6aae4 added content-disposition header for uploads on S3 2013-08-30 17:28:18 +02:00
Régis Hanol
4866f4d8f5 FIX: N+1 query for avatars 2013-08-14 15:25:05 +02:00
Régis Hanol
c867b67a0b custom avatar support 2013-08-13 22:08:29 +02:00
Régis Hanol
ed9417fa3b enable thumbnailing on S3 
- added url to optimized image model
- refactored s3_store & local_store
 2013-07-31 23:26:34 +02:00