Nick Craig-Wood
a795d93bc3
swift, s3, googlecloudstorage: Don't delete the container/bucket if fs wasn't at root - fixes #172
2015-11-07 15:32:40 +00:00
Nick Craig-Wood
5df04cb763
swift: ignore directory marker objects where appropriate - fixes #190
...
* When creating a LimitedFs
* When calling List() to list files
* In the Storable() method
* Add a Purge() method to delete the directory marker objects too
This is a partial fix for #172
2015-11-07 15:32:11 +00:00
Nick Craig-Wood
b257de4aba
Be more constistent with naming in remotes
...
* External objects are called Fs and Object
* Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
Nick Craig-Wood
95fd79faf9
swift: use Content-Length on uploads - fixes #125
2015-10-03 16:04:51 +01:00
Nick Craig-Wood
b79dc01016
swift: stop chunked operations logging "Failed to read info: Object Not Found"
2015-10-03 16:04:51 +01:00
Nick Craig-Wood
4a0a42c2f1
swift: large file upload fixes
...
* Read metadata in file listing for 0 length files to fix syncs
* Ignore non-existent files in isManifestFile to fix errors on copy
* remove nsToSwiftFloatString - experiments with the swift program
indicate that it puts a variable number of points after the
decimal, so might as well use the one in the swift library.
* Make sure segments get deleted properly when move from segmented
to non segmented and vice versa
* Use internal list routine to detect errors on listing
* Remove the _segments container if possible
* Remove manifest first when deleting
2015-09-26 17:58:04 +01:00
Sergey Tolmachev
cc7b9af50e
swift: support files > 5GB - fixes #46
...
* Write segments to ..._segments container
* Choice of container and segment names compatible with swift tool
* See http://docs.openstack.org/developer/swift/overview_large_objects.html
* Controlled by command line flag --swift-chunk-size
* Segments removed on delete
2015-09-26 13:03:58 +01:00
Nick Craig-Wood
68fef49c55
swift: fetch headers as the only source of metadata
2015-09-26 13:03:58 +01:00
Nick Craig-Wood
e9c915e6fe
Fix golint warnings
2015-09-22 18:47:16 +01:00
Nick Craig-Wood
92745aa950
Add Root() to Fs interface
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
a96b522958
Implement server side copies if possible - fixes #99
...
Add optional fs.Copier interface
Implemented for
* swift
* s3
* drive
* dropbox
* google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7
Add Name() to Fs interface to return name as passed to NewFs
2015-08-23 13:36:38 +01:00
Leonid Shalupov
3fcff32524
do not print stats in quiet mode - fixes #70
...
...unless had some errors or stats interval requested.
Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood
1d254a3674
Implement --timeout and --conntimeout - fixes #54
...
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood
c47c9cd440
swift: add region parameter - fixes #38
2015-03-04 17:09:53 +00:00
Nick Craig-Wood
ee6b39aa6c
Add tenant parameter for swift - fixes #13
2014-12-12 15:26:08 +00:00
Nick Craig-Wood
c389616657
all: make private functions / variables / constant which shouldn't be public
2014-07-29 17:50:07 +01:00
Nick Craig-Wood
b013c58537
swift: return directories without / in ListDir
2014-07-24 23:13:33 +01:00
Nick Craig-Wood
282cba20a0
swift, s3, dropbox: fix metadata read on Update()
...
This was causing changed files to be marked as corrupted on upload
2014-07-20 11:23:05 +01:00
Nick Craig-Wood
5a8bce6353
swift: Read metadata after Update or Put
2014-07-19 13:05:06 +01:00
Nick Craig-Wood
c1a245d1c8
Factor UserAgent to fs and move Version to fs
2014-07-13 19:19:58 +01:00
Nick Craig-Wood
99695d57ab
Implement single file operations for all file systems
2014-05-05 22:17:57 +01:00
Nick Craig-Wood
d0ca58bbb1
swift: Support sub container paths
2014-05-05 18:26:37 +01:00
Nick Craig-Wood
02afcb00e9
Factor Fs.Put into Object.Update and call Update rather than Put if possible
2014-04-18 17:49:01 +01:00
Nick Craig-Wood
92ec29fe3f
Factor the generic code into fs and add some more intefaces
2014-03-28 22:26:42 +00:00
Nick Craig-Wood
2e5f0ef258
Rename filing systems modules (so I can tell them apart in emacs buffers!)
2014-03-16 10:09:55 +00:00
