github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-30 08:34:39 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
d40e56272f
discourse/spec/fabricators
History
Martin Brennan e4350bb966
FEATURE: Direct S3 multipart uploads for backups (#14736) 
This PR introduces a new `enable_experimental_backup_uploads` site setting (default false and hidden), which when enabled alongside `enable_direct_s3_uploads` will allow for direct S3 multipart uploads of backup .tar.gz files.

To make multipart external uploads work with both the S3BackupStore and the S3Store, I've had to move several methods out of S3Store and into S3Helper, including:

* presigned_url
* create_multipart
* abort_multipart
* complete_multipart
* presign_multipart_part
* list_multipart_parts

Then, S3Store and S3BackupStore either delegate directly to S3Helper or have their own special methods to call S3Helper for these methods. FileStore.temporary_upload_path has also removed its dependence on upload_path, and can now be used interchangeably between the stores. A similar change was made in the frontend as well, moving the multipart related JS code out of ComposerUppyUpload and into a mixin of its own, so it can also be used by UppyUploadMixin.

Some changes to ExternalUploadManager had to be made here as well. The backup direct uploads do not need an Upload record made for them in the database, so they can be moved to their final S3 resting place when completing the multipart upload.

This changeset is not perfect; it introduces some special cases in UploadController to handle backups that was previously in BackupController, because UploadController is where the multipart routes are located. A subsequent pull request will pull these routes into a module or some other sharing pattern, along with hooks, so the backup controller and the upload controller (and any future controllers that may need them) can include these routes in a nicer way.
2021-11-11 08:25:31 +10:00
..
allowed_pm_users.rb FEATURE: Allow List for PMs (#10270) 2020-07-20 15:23:49 -06:00
api_key_fabricator.rb FEATURE: Hash API keys in the database (#8438) 2019-12-12 11:45:00 +00:00
badge_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
bookmark_fabricator.rb DEV: Ignore reminder_type for bookmarks (#14349) 2021-09-16 09:56:54 +10:00
category_fabricator.rb FIX: Make inline oneboxes work with secured topics in secured contexts (#8895) 2020-02-12 12:11:28 +02:00
category_group_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
color_scheme_color_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
color_scheme_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
dimissed_topic_user.rb FEATURE: New way to dismiss new topics (#11927) 2021-02-04 11:27:34 +11:00
do_not_disturb_fabricator.rb FEATURE: Do not disturb (#11484) 2020-12-18 09:03:51 -06:00
email_change_request_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
email_log_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
email_token_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
embeddable_host_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
external_upload_stub_fabricator.rb FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
flag_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
group_fabricator.rb DEV: Add SMTP group ID to EmailLog (#13381) 2021-06-15 11:29:46 +10:00
group_history_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
group_request_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
group_user_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
ignored_user_fabricator.rb DEV: Cleanup ignored user logic (#11107) 2020-11-03 12:38:54 +00:00
incoming_email_fabricator.rb FIX: Change default for IncomingEmail#created_via to 0 (unknown) and make NOT NULL (#11782) 2021-01-21 12:59:50 +10:00
incoming_link_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
invite_fabricator.rb FEATURE: send max 200 emails every minute for bulk invites (#7875) 2019-07-19 11:29:12 +05:30
invited_user_fabricator.rb FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
like_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
muted_user.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
notification_fabricator.rb FEATURE: High priority bookmark reminder notifications (#9290) 2020-04-01 09:09:20 +10:00
optimized_image_fabricator.rb DEV: Fix OptimizedImage specs 2020-07-06 21:51:56 +02:00
permalink_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
post_action_fabricator.rb FIX: correct user serializer user method for extended serializer (#8590) 2019-12-19 09:48:01 -08:00
post_custom_field_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
post_detail_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
post_fabricator.rb UX: display correct replies count in embedded comments view. (#14175) 2021-08-30 10:37:53 +05:30
post_reply_key_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
post_revision_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
published_page_fabricator.rb FEATURE: allows published pages to be public (#10053) 2020-06-17 12:42:20 +02:00
reviewable_claimed_topic_fabricator.rb FEATURE: Claim Reviewables by Topic 2019-05-09 13:40:36 -04:00
reviewable_fabricator.rb FIX: Granting staff status should auto-approve users waiting approval (#8533) 2019-12-12 16:26:38 -03:00
screened_email_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
screened_ip_address_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
screened_url_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
search_log_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
shared_draft_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
single_sign_on_record_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
skipped_email_log_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
tag_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
tag_group_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
theme_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
theme_field_fabricator.rb FIX: add theme field errors (#12880) 2021-04-28 15:00:37 -07:00
topic_allowed_group_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
topic_allowed_user_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
topic_embed_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
topic_fabricator.rb DEV: Add include_pms option to TopicQuery (#10647) 2020-09-14 12:07:35 +01:00
topic_tag_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
topic_timer_fabricator.rb DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
topic_user_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
upload_fabricator.rb FEATURE: Use path from existing URL of uploads and optimized images (#13177) 2021-05-27 17:42:25 +02:00
user_action_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_api_key_fabricator.rb DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
user_avatar_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_email_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_fabricator.rb FEATURE: Disallow putting urls in the title for TL-0 users (#13947) 2021-08-05 13:38:39 +04:00
user_field_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_field_option_fabricator.rb.rb FIX: Validate value of custom dropdown user fields - dropdowns and multiple selects (#13890) 2021-07-30 13:50:47 -04:00
user_option_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_profile_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_second_factor_fabricator.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
user_security_key_fabricator.rb FEATURE: Webauthn authenticator management with 2FA login (Security Keys) (#8099) 2019-10-01 19:08:41 -07:00
watched_word_fabricator.rb DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
web_crawler_request_fabricator.rb DEV: Use dynamic/static fabricator attrs correctly (#9519) 2020-04-22 20:49:53 +02:00
web_hook_fabricator.rb FEATURE: add support for like webhooks (#12917) 2021-04-30 17:08:38 -07:00