7593 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nick Craig-Wood	6e732f3dc0	Implement Purger interface and drive.Purge	2013-01-18 17:11:44 +00:00
Nick Craig-Wood	a0a789ff0f	Fix after go vet	2013-01-18 17:11:44 +00:00
Nick Craig-Wood	fdb1ec4d53	Implement Google Drive interface ... * needs writeup of rather complicated api key setup * not implemented yet * fix for slow directory listing * mimetypes for files	2013-01-18 17:11:22 +00:00
Nick Craig-Wood	90a397c533	Stop sync creating FsObjects unecessarily when they have been created by List already	2013-01-14 23:08:10 +00:00
Nick Craig-Wood	8b066ba77c	Don't delete files if errors in copy phase of sync ... Make check return errors on differences	2013-01-10 22:14:23 +00:00
Nick Craig-Wood	555ab5001e	Factor Put() methods into generic Copy() function	2013-01-10 21:58:46 +00:00
Nick Craig-Wood	38ce4c3629	S3: Fix mkdir not to error if bucket exists and implement SetModTime	2013-01-08 22:31:16 +00:00
Nick Craig-Wood	b9e468ebbd	Add initial S3 support ... * Requires a patched goamz to enable metadata reading and writing * Still todo: updating metadata * EU region not working yet	2013-01-08 18:59:38 +00:00
Nick Craig-Wood	f7197c30d7	Implement stats counting and reporting and return errors in return code	2013-01-03 22:50:00 +00:00
Nick Craig-Wood	12015b0007	Run ls in parallel so can parallelise the slow ModTime operations	2013-01-02 15:34:25 +00:00
Nick Craig-Wood	ecb863dd4f	Make ModTime fall back to LastModified header ... Which means it no longer needs to report an error which simplifies the code	2013-01-02 15:21:55 +00:00
Nick Craig-Wood	90a2c86eb3	Implement check and help commands	2012-12-31 17:32:11 +00:00
Nick Craig-Wood	335667fdcb	Implement sync, -dry-run and fix logging ... * Implement sync command * Implement String() interface for Fs * Sort out logging of FsObject~s * Implement -dry-run, -verbose and -quiet	2012-12-31 16:40:34 +00:00
Nick Craig-Wood	c15ae179ee	Detect Fs from path to make operations consistent everywhere	2012-12-29 11:35:41 +00:00
Nick Craig-Wood	4028a4192f	Use swift.ObjectsWalk	2012-12-28 16:38:51 +00:00
Nick Craig-Wood	555e545b46	Fix after go vet	2012-12-26 12:27:26 +00:00
Nick Craig-Wood	3bf6348f57	Factor local filesystem and remote swift into Fs and FsObject interfaces ... This will enable * local -> local and remote -> remote copies * a much more uniform style * could do s3 as well	2012-12-26 12:23:58 +00:00
Nick Craig-Wood	12658efef2	gofmt	2012-12-24 00:03:18 +00:00
Nick Craig-Wood	c3671907cb	Update notes	2012-12-23 17:46:03 +00:00
Nick Craig-Wood	720d44f8e9	Delete many files at once with purge	2012-12-23 09:49:49 +00:00
Nick Craig-Wood	a5ecd66388	Remove commented out code	2012-12-23 09:34:29 +00:00
Nick Craig-Wood	9a066460f1	Make downloads work in parallel	2012-12-23 09:32:33 +00:00
Nick Craig-Wood	9fb8a1afa2	Implement purge and make upload make the container first	2012-12-04 00:09:22 +00:00
Nick Craig-Wood	664dbdc9e7	Make the commands data driven	2012-12-03 23:58:17 +00:00
Nick Craig-Wood	19668ac18f	Fix loss of precision for setting timestamps so roundtrip happens properly	2012-12-01 12:05:17 +00:00
Nick Craig-Wood	25a1b96537	Make download work properly skipping not changed files ... Reorganise FsObject and factor more stuff into methods Note that os.Chtimes() isn't setting the same precision (us) as we are reading from os.Lstat() (ns) so the modification times don't roundtrip properly yet.	2012-12-01 10:53:48 +00:00
Nick Craig-Wood	a3e28e6b97	Fix downloads - missing is the modified check so it downloads everything	2012-12-01 08:18:58 +00:00
Nick Craig-Wood	e41a8b9a02	First attempt at download ... Working but modification times are messed up	2012-11-29 23:07:29 +00:00
Nick Craig-Wood	4d23e29274	Make checking and uploading run in parallel using channels ... Runs much faster and uses much less memory (thanks to walking the filesystem into a channel too).	2012-11-29 22:13:58 +00:00
Nick Craig-Wood	ba24b84396	Use ObjectsAll and ContainersAll to see everything	2012-11-28 23:40:09 +00:00
Nick Craig-Wood	31745ac13b	Update the mtime of the remote object when size and md5sum match but mtime doesn't	2012-11-28 17:42:12 +00:00
Nick Craig-Wood	dbb039af9f	Check the MD5SUM of an object if the mtime is different before uploading	2012-11-28 17:24:00 +00:00
Nick Craig-Wood	5ebbe342b8	Use os.Lstat instead of os.Stat so we ignore symlinks completely	2012-11-28 17:12:54 +00:00
Nick Craig-Wood	c42bafc4e2	Make logging more consistent	2012-11-28 17:12:22 +00:00
Nick Craig-Wood	a49b8ca0b6	Use filesystem walk ordering rather than hash order	2012-11-28 11:38:14 +00:00
Nick Craig-Wood	3e4f75b558	Fix relative vs absolute path confusion in logging	2012-11-28 11:32:04 +00:00
Nick Craig-Wood	e3096508dd	If the remote object doesn't have mtime then check the md5sum to see if changed	2012-11-28 11:17:31 +00:00
Nick Craig-Wood	81b7f41dfb	More notes	2012-11-28 10:47:35 +00:00
Nick Craig-Wood	27ed023c30	Set the mtime when uploading and skip uploads if mtime and size the same	2012-11-27 22:28:46 +00:00
Nick Craig-Wood	73844ae022	Factor mtime functions info github.com/ncw/swift	2012-11-27 22:04:04 +00:00
Nick Craig-Wood	08c3e9fd57	Basic set of listing and upload commands	2012-11-20 22:40:48 +00:00
Nick Craig-Wood	1ebbfd4207	Fix ST_AUTH typo	2012-11-18 20:03:06 +00:00
Nick Craig-Wood	e9ae4f89a4	Initial commit - some small parts working	2012-11-18 17:32:31 +00:00