github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-29 13:03:44 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
36,175 Commits 214 Branches 500 Tags 960 MiB
cfd56e9159
Commit Graph

123 Commits

Author SHA1 Message Date
Gerhard Schlager
7c30986b5e FIX: Failed to notify user after restoring backup 2020-01-25 22:07:41 +01:00
Gerhard Schlager
f216c6d60b FEATURE: Drop "backup" schema 7 days after restore 
The "backup" schema is used to rollback a failed restore. It isn't useful after a longer period of time and turns into a waste of disk space.
 2020-01-16 17:48:47 +01:00
Gerhard Schlager
5e3fc31f2c DEV: Less hacky way of rolling back DB changes 
Some specs use psql to test database restores and dropping the table after the test needs to happen outside of rspec because of transactions. The previous attempt lead to some changes to be stored in the test database.
 2020-01-15 23:37:42 +01:00
Gerhard Schlager
68a7ae3091 REFACTOR: Simplify backup version check 
Adds specs for inalid version number in metadata file.
Follow-up to c3cd2389fe
 2020-01-15 23:37:40 +01:00
Martin Brennan
c3cd2389fe SECURITY: use strict JSON parsing when parsing backup metadata 2020-01-15 11:24:41 +01:00
Gerhard Schlager
e474cda321 REFACTOR: Restoring of backups and migration of uploads to S3 2020-01-14 11:41:35 +01:00
Vinoth Kannan
3b7f5db5ba
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
David Taylor
481efebe76
DEV: Update backup/restore pipeline to avoid cd (#8347) 2019-11-13 15:52:28 +00:00
Justin DiRose
c3f06943c7
FIX: Account for empty uploads directory upon backup restore (#8262) 
This commit fixes a case where backup restores would fail if the uploads/default directory is empty.
 2019-10-30 09:33:07 -05:00
Krzysztof Kotlarek
f530378df3 FIX: Restore for non-multisite is not raising an error on reconnect step (#8237) 
That commit introduced a bug to the system: f69dacf979

Restore works fine for multisite, however, stopped working for non-multisite.

Reason for that was that `establish_connection` method got a check if the multisite instance is available:
```
    def self.instance
      @instance
    end

    def self.establish_connection(opts)
      @instance.establish_connection(opts) if @instance
    end
```
However, the reload method don't have that check
```
    def self.reload
      @instance = new(instance.config_filename)
    end
```

To solve it, let's ensure we are in a multisite environment before call reload
 2019-10-24 11:46:22 +11:00
Krzysztof Kotlarek
f69dacf979 FIX: Reconnect in restore process connects to correct DB (#8218) 
Simplified flow of restore is like that
```
migrate_database
reconnect
extract_uploads
```

Problem with incorrect current database started with this fix https://github.com/discourse/discourse/commit/025d4ee91f4

Dump task is reconnecting to default database https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake#L429

And then, we are trying to reconnect to the original database with that code:
```
def reconnect_database
  log "Reconnecting to the database..."
  RailsMultisite::ConnectionManagement::establish_connection(db: @current_db)
end
```

This reconnect is not switching us back to correct database because of that check
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L181
Basically, it finds existing handler and it thinks that we are connected to correct DB and this step can be skipped.

To solve it, we can reload RailsMultisite::ConnectionManagement which creates a new instance of that class
https://github.com/discourse/rails_multisite/blob/master/lib/rails_multisite/connection_management.rb#L38
 2019-10-23 17:23:50 +11:00
Roman Rizzi
01bc465db8
DEV: Split max decompressed setting for themes and backups (#8179) 2019-10-11 14:38:10 -03:00
Roman Rizzi
5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166) 
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
 2019-10-09 11:41:16 -03:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098) 
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
 2019-10-02 14:01:53 +10:00
Gerhard Schlager
1f118b1309 FEATURE: Allow plugins to manipulate site settings during backup restore 2019-08-22 22:41:26 +02:00
Gerhard Schlager
d686318133 FIX: Prevent failed remaps during restores 
Additional changes:
* Verbose logging of remaps during restores
* Exclude the backup_metadata table from restores
 2019-08-12 17:15:01 +02:00
Gerhard Schlager
7cb51d0e40 FIX: Create readonly functions during backup 
Temporarily recreate already dropped functions in the discourse_functions schema in order to allow restoring of backups which still reference dropped functions.
 2019-08-09 11:39:46 +02:00
Michael Brown
31f583855a DEV: pull static check out of loop 
* followup to 08b28680
* as per https://review.discourse.org/t/4713/2
 2019-07-23 17:18:16 -04:00
Gerhard Schlager
68b082e1a4 FIX: Ensure that jobs don't run immediately after migrate_to_s3 2019-07-23 17:42:12 +02:00
Gerhard Schlager
b73bd7fc1b FIX: Always backup local uploads in addition to files stored on S3 2019-07-19 15:13:05 +02:00
Blake Erickson
b0c92bb0b9 REFACTOR: Clean up parameterized title 
Follow up to [FIX: Empty backup names with unicode site titles][1]

- Use .presence - "It's cleaner"
- Update spec to use System.system_user so it is more readable

[1]: c8661674d4
 2019-07-18 15:49:16 -06:00
Blake Erickson
c8661674d4 FIX: Empty backup names with unicode site titles 
If a site title contains unicode it may end up with an empty backup
filename because of the rails `parameterize` method we are calling.

This fix ensures that the backup filenames default to "discourse" if the
parameterized site title is empty.

Bug reported [here][1].

[1]: https://meta.discourse.org/t/backup-checksum-and-backup-name-missing-when-unicode-site-name/123192?u=blake
 2019-07-17 17:07:10 -06:00
Michael Brown
08b286808a
FIX: backups taken by pg_dump >= 11 are nonportable (#7893) 2019-07-15 18:07:44 -04:00
Gerhard Schlager
5f0d38341e FIX: Remapping during restore was wrong for CDN URLs 2019-07-09 17:34:41 +02:00
Gerhard Schlager
4c1b8c7559 FIX: Remap differently when backup comes from multisite 2019-07-09 16:11:32 +02:00
Gerhard Schlager
a65a9a85d5 FEATURE: Remap uploads during restore when S3 or CDN changes 
In order for this to work the Backuper stores a couple of site settings
in the new backup_metadata table, because the old setting values might
not be available on restore anymore.
 2019-07-09 14:04:16 +02:00
Gerhard Schlager
f2dc59d61f FEATURE: Add hidden setting to include S3 uploads in backups 2019-07-09 14:04:16 +02:00
Penar Musaraj
f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677) 
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
 2019-06-06 13:27:24 +10:00
Gerhard Schlager
f7a2648694 FEATURE: Migrate uploads to S3 during restore 2019-06-04 15:47:36 +02:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files 
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
 2019-05-13 09:31:32 +08:00
Gerhard Schlager
1ddd4a44d5 FIX: Wrong color palette after backup restore 2019-05-07 17:02:57 +02:00
Gerhard Schlager
2487e01c73 FIX: Optimized site icons were missing after backup restore 2019-05-07 17:02:57 +02:00
Guo Xiang Tan
ebca588fd0 DEV: Remove unused line of code. 2019-05-02 16:54:10 +08:00
Gerhard Schlager
3aca070311 FIX: Restoring backup shouldn't change disable_emails from "yes" to "non-staff" 2019-04-16 11:48:07 +02:00
Gerhard Schlager
78f8114989 FEATURE: Allow discourse script to skip disabling of emails after restore 2019-03-07 21:49:33 +01:00
Joffrey JAFFEUX
42df20e4f0
typo (#7065) 2019-02-25 16:36:22 +01:00
Gerhard Schlager
dc961fecb9 FIX: Outgoing emails were not disabled after restoring backup 2019-02-25 16:07:24 +01:00
Gerhard Schlager
6a8007e5fb FEATURE: Improve handling of backup storage errors 2019-02-20 15:16:49 +01:00
Guo Xiang Tan
8cd4ceba49 DEV: Remove unnecessary Sidekiq.unpause! during backup. 2019-02-19 14:01:13 +08:00
Gerhard Schlager
99ad61afb7 FEATURE: Trigger an event after a backup restore 2019-02-18 11:48:03 +01:00
Gerhard Schlager
b087719340 FEATURE: Setting for excluding optimized images from backups 2019-02-13 11:10:51 +01:00
Gerhard Schlager
9eb7dea0f1 FEATURE: Setting for compression level of upload in backups 2019-02-12 15:50:31 +01:00
Gerhard Schlager
220944a38a FIX: Unpause sidekiq before adding uploads to backup 
tar exits with status 1 when uploads are modified or deleted by a sidekiq job, so we need to treat it like status 0.

According to the documentation it should be safe to ignore status 1 ("Some files differ"):

> If tar was given `--create', `--append' or `--update' option, this exit code means that some files were changed while being archived and so the resulting archive does not contain the exact copy of the file set.

Status 2 ("Fatal error") still results in an exception.
 2019-02-12 13:50:50 +01:00
Gerhard Schlager
5bb955dcb7 FIX: Allow restore when latest migration is a post_migration 2019-02-08 17:37:05 +01:00
Gerhard Schlager
bdbf77dc38 FIX: Unpause Sidekiq before uploading backup to S3 
No need to pause Sidekiq longer than really needed. Uploads to S3 can take a long time.
 2019-02-05 21:22:25 +01:00
Vinoth Kannan
b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Gerhard Schlager
45b056b615 FIX: Do not show backups stored in subfolder of bucket 2019-01-24 22:28:03 +01:00
Gerhard Schlager
c94a2bc69b FIX: Raise or log error when deleting of backup fails 2019-01-24 22:26:50 +01:00
Joffrey JAFFEUX
f9648de897
DEV: upgrades from Ember 2.13 to Ember 3.5.1 (#6808) 
Co-Authored-By: Bianca Nenciu <nbianca@users.noreply.github.com>
Co-Authored-By: David Taylor <david@taylorhq.com>
 2019-01-10 11:06:01 +01:00
Gerhard Schlager
0bc1fa8aa4 FEATURE: Don't create PM for successful automatic backups 2018-12-20 13:34:24 +01:00