Commit Graph

91 Commits

Author SHA1 Message Date
Guo Xiang Tan
6740631fdb TEMPFIX: Fix broken restores. 2018-07-27 12:48:16 +08:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Guo Xiang Tan
5da7c2a4ad FIX: Restorer wasn't rolling back if restore fails.
* This only applies to backup file taken with
  pg_dump 10.3+ and pg_dump 9.5.12+.
2018-04-06 09:43:32 +08:00
Guo Xiang Tan
142571bba0 Remove use of rescue nil.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Michael Brown
63a1e9b60a backup restorer: tidy pg_dump schema portability logic, add test 2018-03-20 10:32:39 +08:00
Guo Xiang Tan
da8e15f954 FIX: Restorer was not extracting the patch version in dump file. 2018-03-16 11:09:56 +08:00
Michael Brown
90291318eb restorer: clarify logging 2018-03-15 12:14:08 -04:00
Guo Xiang Tan
5ef75c9c61 Improve grep pattern in restorer. 2018-03-09 15:48:12 +08:00
Guo Xiang Tan
766b41d9f4 Fix version check in restorer. 2018-03-09 15:01:10 +08:00
Guo Xiang Tan
8fd47314d9 FIX: Restore process for dump taken with pg_dump 10.3+.
* Since we can no longer restore into a different schema,
  we will move tables in the public schema into the backup schema
  first before restoring the dump file which goes into the public
  schema. The downside to this approach is that we will increase
  the downtime experienced during the restore process. Downtime
  would equal the duration of restoring the dump file.
2018-03-09 13:24:58 +08:00
Will Jordan
a41446a502 single quote password in restore command
> Followup to #3283. Quotes passwords passed to shell for backup restore.
2018-03-01 12:08:35 -08:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Leo McArdle
d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Guo Xiang Tan
e7c972ac89 FIX: Don't use backticks that take in inputs. 2017-03-17 15:33:51 +08:00
Guo Xiang Tan
b49bf889f6 SECURITY: Disallow symlinks when restoring uploads. 2017-03-17 14:27:01 +08:00
Guo Xiang Tan
7139538286 Fix typo. 2016-09-21 16:04:41 +08:00
Guo Xiang Tan
0bf7519a8a FIX: tar --list against a .tar.gz file takes too long.
This resulted in requests being blocked for an extended amount
of time when initializing the restorer.
2016-09-16 17:11:14 +08:00
Guo Xiang Tan
68637f2164 FIX: Uploads being restored into the wrong directory for multisite. 2016-09-16 14:26:06 +08:00
Guo Xiang Tan
f63a797e39 SECUIRTY: Escape input made to system calls. 2016-09-16 11:58:14 +08:00
Guo Xiang Tan
8f36290c05 FIX: No need to list all the files. 2016-09-16 11:57:35 +08:00
Guo Xiang Tan
7e80810de1 FIX: Raise an error if metadata is not extracted correctly. 2016-08-25 17:20:32 +08:00
Guo Xiang Tan
3e4b02bbd4 FIX: Make sure constant reflects the right backup extenstion. 2016-08-24 10:28:23 +08:00
Guo Xiang Tan
adc8336949 Make sure we track restore/backlog success logs as well. 2016-08-03 16:23:47 +08:00
Guo Xiang Tan
0a942dbc73 FEATURE: Avoid creating an archive for database only backups. 2016-08-03 16:23:46 +08:00
Guo Xiang Tan
441b98579a FIX: Ensure that our restorer is backwards compatible. 2016-08-02 09:19:56 +08:00
Guo Xiang Tan
76e57ddef3 FIX: Log errors in ensure block of restorer. 2016-07-26 10:24:01 +08:00
Guo Xiang Tan
03aa13b2bb FEATURE: Work with compressed version of pg_dump during backup and restore. 2016-07-26 10:24:01 +08:00
Neil Lalonde
91e4af0d3d FIX: restore of a backup from an older Discourse version can create new tables in the wrong schema, leading to UndefinedTable errors 2016-07-12 16:26:45 -04:00
Régis Hanol
8c52508007 warn users it may take a while to un/zip backup 2016-05-23 09:33:29 +02:00
Régis Hanol
332a1ea87e FIX: add support for 'space' in backup's filename 2016-04-04 17:24:02 +02:00
Régis Hanol
02d9ec5bde oops, should not have been commited 2016-01-19 01:01:17 +01:00
Régis Hanol
3083657358 FEATURE: better email in support
FEATURE: new incoming_email model
FEATURE: infinite scrolling in emails admin
FEATURE: new 'emails:import' rake task
2016-01-19 00:57:55 +01:00
Régis Hanol
96c23d51a2 FIX: don't break the message bus when restoring a backup 2015-08-27 20:02:13 +02:00
Chris Jester-Young
15cfb73aef Fix module name for RestoreDisabledError and FilenameMissingError. 2015-06-11 02:42:01 -04:00
Sam
803feefd54 MessageBus handles readonly redis now, no need to wrap it 2015-05-04 12:21:00 +10:00
Robin Ward
5b3f99aa50 Don't blow up if Redis switches to READONLY 2015-04-24 14:37:16 -04:00
Régis Hanol
56e01a766b FIX: clear emoji cache after restore 2015-03-17 17:29:18 +01:00
Régis Hanol
f22534ae28 improve copy when restoring a backup 2015-03-09 16:11:15 +01:00
Régis Hanol
c9a007d5fb FIX: backup logs on Firefox
FIX: debounce backup logs in order to prevent Firefox crash
FIX: 'Invalid date' in backup logs in Firefox
2015-02-09 16:53:28 +01:00
Régis Hanol
45dbdb6896 FEATURE: custom emojis 2014-12-23 01:12:26 +01:00
Régis Hanol
5504622c1b rename export/import in favor of backup/restore for better consistency 2014-10-10 20:04:07 +02:00