Commit Graph

5177 Commits

Author SHA1 Message Date
Nick Craig-Wood
3bf6348f57 Factor local filesystem and remote swift into Fs and FsObject interfaces
This will enable
  * local -> local and remote -> remote copies
  * a much more uniform style
  * could do s3 as well
2012-12-26 12:23:58 +00:00
Nick Craig-Wood
12658efef2 gofmt 2012-12-24 00:03:18 +00:00
Nick Craig-Wood
c3671907cb Update notes 2012-12-23 17:46:03 +00:00
Nick Craig-Wood
720d44f8e9 Delete many files at once with purge 2012-12-23 09:49:49 +00:00
Nick Craig-Wood
a5ecd66388 Remove commented out code 2012-12-23 09:34:29 +00:00
Nick Craig-Wood
9a066460f1 Make downloads work in parallel 2012-12-23 09:32:33 +00:00
Nick Craig-Wood
9fb8a1afa2 Implement purge and make upload make the container first 2012-12-04 00:09:22 +00:00
Nick Craig-Wood
664dbdc9e7 Make the commands data driven 2012-12-03 23:58:17 +00:00
Nick Craig-Wood
19668ac18f Fix loss of precision for setting timestamps so roundtrip happens properly 2012-12-01 12:05:17 +00:00
Nick Craig-Wood
25a1b96537 Make download work properly skipping not changed files
Reorganise FsObject and factor more stuff into methods

Note that os.Chtimes() isn't setting the same precision (us) as we are
reading from os.Lstat() (ns) so the modification times don't roundtrip
properly yet.
2012-12-01 10:53:48 +00:00
Nick Craig-Wood
a3e28e6b97 Fix downloads - missing is the modified check so it downloads everything 2012-12-01 08:18:58 +00:00
Nick Craig-Wood
e41a8b9a02 First attempt at download
Working but modification times are messed up
2012-11-29 23:07:29 +00:00
Nick Craig-Wood
4d23e29274 Make checking and uploading run in parallel using channels
Runs much faster and uses much less memory (thanks to walking the
filesystem into a channel too).
2012-11-29 22:13:58 +00:00
Nick Craig-Wood
ba24b84396 Use ObjectsAll and ContainersAll to see everything 2012-11-28 23:40:09 +00:00
Nick Craig-Wood
31745ac13b Update the mtime of the remote object when size and md5sum match but mtime doesn't 2012-11-28 17:42:12 +00:00
Nick Craig-Wood
dbb039af9f Check the MD5SUM of an object if the mtime is different before uploading 2012-11-28 17:24:00 +00:00
Nick Craig-Wood
5ebbe342b8 Use os.Lstat instead of os.Stat so we ignore symlinks completely 2012-11-28 17:12:54 +00:00
Nick Craig-Wood
c42bafc4e2 Make logging more consistent 2012-11-28 17:12:22 +00:00
Nick Craig-Wood
a49b8ca0b6 Use filesystem walk ordering rather than hash order 2012-11-28 11:38:14 +00:00
Nick Craig-Wood
3e4f75b558 Fix relative vs absolute path confusion in logging 2012-11-28 11:32:04 +00:00
Nick Craig-Wood
e3096508dd If the remote object doesn't have mtime then check the md5sum to see if changed 2012-11-28 11:17:31 +00:00
Nick Craig-Wood
81b7f41dfb More notes 2012-11-28 10:47:35 +00:00
Nick Craig-Wood
27ed023c30 Set the mtime when uploading and skip uploads if mtime and size the same 2012-11-27 22:28:46 +00:00
Nick Craig-Wood
73844ae022 Factor mtime functions info github.com/ncw/swift 2012-11-27 22:04:04 +00:00
Nick Craig-Wood
08c3e9fd57 Basic set of listing and upload commands 2012-11-20 22:40:48 +00:00
Nick Craig-Wood
1ebbfd4207 Fix ST_AUTH typo 2012-11-18 20:03:06 +00:00
Nick Craig-Wood
e9ae4f89a4 Initial commit - some small parts working 2012-11-18 17:32:31 +00:00