mirror of
https://github.com/discourse/discourse.git
synced 2024-11-30 15:24:05 +08:00
8cf4ed5f88
This commit introduces a hidden `s3_inventory_bucket` site setting which replaces the `enable_s3_inventory` and `s3_configure_inventory_policy` site setting. The reason `enable_s3_inventory` and `s3_configure_inventory_policy` site settings are removed is because this feature has technically been broken since it was introduced. When the `enable_s3_inventory` feature is turned on, the app will because configure a daily inventory policy for the `s3_upload_bucket` bucket and store the inventories under a prefix in the bucket. The problem here is that once the inventories are created, there is nothing cleaning up all these inventories so whoever that has enabled this feature would have been paying the cost of storing a whole bunch of inventory files which are never used. Given that we have not received any complains about inventory files inflating S3 storage costs, we think that it is very likely that this feature is no longer being used and we are looking to drop support for this feature in the not too distance future. For now, we will still support a hidden `s3_inventory_bucket` site setting which site administrators can configure via the `DISCOURSE_S3_INVENTORY_BUCKET` env.
46 lines
1.2 KiB
Ruby
46 lines
1.2 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
module Jobs
|
|
class EnsureS3UploadsExistence < ::Jobs::Scheduled
|
|
every 1.day
|
|
|
|
def perform(*args)
|
|
super
|
|
ensure
|
|
if @db_inventories
|
|
@db_inventories.values.each do |f|
|
|
f.close
|
|
f.unlink
|
|
end
|
|
end
|
|
end
|
|
|
|
def prepare_for_all_sites
|
|
inventory = S3Inventory.new(s3_helper, :upload)
|
|
@db_inventories = inventory.prepare_for_all_sites
|
|
@inventory_date = inventory.inventory_date
|
|
end
|
|
|
|
def execute(args)
|
|
return if (s3_inventory_bucket = SiteSetting.s3_inventory_bucket).blank?
|
|
|
|
if !@db_inventories && Rails.configuration.multisite && GlobalSetting.use_s3?
|
|
prepare_for_all_sites
|
|
end
|
|
|
|
if @db_inventories &&
|
|
preloaded_inventory_file =
|
|
@db_inventories[RailsMultisite::ConnectionManagement.current_db]
|
|
S3Inventory.new(
|
|
:upload,
|
|
s3_inventory_bucket:,
|
|
preloaded_inventory_file: preloaded_inventory_file,
|
|
preloaded_inventory_date: @inventory_date,
|
|
).backfill_etags_and_list_missing
|
|
else
|
|
S3Inventory.new(:upload, s3_inventory_bucket:).backfill_etags_and_list_missing
|
|
end
|
|
end
|
|
end
|
|
end
|