mirror of
https://github.com/rclone/rclone.git
synced 2024-12-21 19:16:29 +08:00
8c69455c37
Before this change, directory modtimes (and metadata) were always synced from src to dst, even if already in sync (i.e. their modtimes already matched.) This potentially required excessive API calls, made logs noisy, and was potentially problematic for backends that create "versions" or otherwise log activity updates when modtime/metadata is updated. After this change, a new DirsEqual function is added to check whether dirs are equal based on a number of factors such as ModifyWindow and sync flags in use. If the dirs are equal, the modtime/metadata update is skipped. For backends that require setDirModTimeAfter, the "after" sync is performed only for dirs that could have been changed by the sync (i.e. dirs containing files that were created/updated.) Note that dir metadata (other than modtime) is not currently considered by DirsEqual, consistent with how object metadata is synced (only when objects are unequal for reasons other than metadata). To sync dir modtimes and metadata unconditionally (the previous behavior), use --ignore-times.
9.9 KiB
9.9 KiB
(01) : test createemptysrcdirs
(02) : test initial bisync
(03) : touch-glob 2001-01-02 {datadir/} placeholder.txt
(04) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(05) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(06) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(07) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(08) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(09) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(10) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(11) : test 1. Create an empty dir on Path1 by creating subdir/placeholder.txt and then deleting the placeholder
(12) : copy-as {datadir/}placeholder.txt {path1/} subdir/placeholder.txt
(13) : touch-glob 2001-01-02 {path1/} subdir
(14) : delete-file {path1/}subdir/placeholder.txt
(15) : test 2. Run bisync without --create-empty-src-dirs
(16) : bisync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(17) : test 3. Confirm the subdir exists only on Path1 and not Path2
(18) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(19) : list-dirs {path2/}
(20) : test 4.Run bisync WITH --create-empty-src-dirs
(21) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File is new - subdir
INFO : Path1: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Path2 checking for diffs
INFO : Applying changes
INFO : - Path1 Queue copy to Path2 - {path2/}subdir
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(22) : test 5. Confirm the subdir exists on both paths
(23) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(24) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(25) : test 6. Delete the empty dir on Path1 using purge-children (and also add files so the path isn't empty)
(26) : purge-children {path1/}
(27) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(28) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(29) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(30) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(31) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(32) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(33) : test 7. Run bisync without --create-empty-src-dirs
(34) : bisync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - RCLONE_TEST
INFO : - Path1 File was deleted - subdir
INFO : Path1: 2 changes: 0 new, 0 modified, 2 deleted
INFO : Path2 checking for diffs
INFO : - Path2 File was deleted - subdir
INFO : Path2: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Applying changes
INFO : - Path2 Queue delete - {path2/}RCLONE_TEST
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(35) : test 8. Confirm the subdir exists only on Path2 and not Path1
(36) : list-dirs {path1/}
(37) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(38) : test 9. Reset, do the delete again, and run bisync WITH --create-empty-src-dirs
(39) : bisync resync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(40) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(41) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(42) : purge-children {path1/}
(43) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(44) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(45) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(46) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(47) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(48) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(49) : list-dirs {path1/}
(50) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(51) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - subdir
INFO : Path1: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Path2 checking for diffs
INFO : Applying changes
INFO : - Path2 Queue delete - {path2/}subdir
INFO : - Path1 Do queued copies to - Path2
INFO : subdir: Removing directory
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(52) : test 10. Confirm the subdir has been removed on both paths
(53) : list-dirs {path1/}
(54) : list-dirs {path2/}
(55) : test 11. bisync again (because if we leave subdir in listings, test will fail due to mismatched modtime)
(56) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(02) : test initial bisync
(03) : touch-glob 2001-01-02 {datadir/} placeholder.txt
(04) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(05) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(06) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(07) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(08) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(09) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(10) : bisync resync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(11) : test 1. Create an empty dir on Path1 by creating subdir/placeholder.txt and then deleting the placeholder
(12) : copy-as {datadir/}placeholder.txt {path1/} subdir/placeholder.txt
(13) : touch-glob 2001-01-02 {path1/} subdir
(14) : delete-file {path1/}subdir/placeholder.txt
(15) : test 2. Run bisync without --create-empty-src-dirs
(16) : bisync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(17) : test 3. Confirm the subdir exists only on Path1 and not Path2
(18) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(19) : list-dirs {path2/}
(20) : test 4.Run bisync WITH --create-empty-src-dirs
(21) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File is new - subdir
INFO : Path1: 1 changes: 1 new, 0 modified, 0 deleted
INFO : Path2 checking for diffs
INFO : Applying changes
INFO : - Path1 Queue copy to Path2 - {path2/}subdir
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(22) : test 5. Confirm the subdir exists on both paths
(23) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(24) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(25) : test 6. Delete the empty dir on Path1 using purge-children (and also add files so the path isn't empty)
(26) : purge-children {path1/}
(27) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(28) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(29) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(30) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(31) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(32) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(33) : test 7. Run bisync without --create-empty-src-dirs
(34) : bisync
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - RCLONE_TEST
INFO : - Path1 File was deleted - subdir
INFO : Path1: 2 changes: 0 new, 0 modified, 2 deleted
INFO : Path2 checking for diffs
INFO : - Path2 File was deleted - subdir
INFO : Path2: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Applying changes
INFO : - Path2 Queue delete - {path2/}RCLONE_TEST
INFO : - Path1 Do queued copies to - Path2
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(35) : test 8. Confirm the subdir exists only on Path2 and not Path1
(36) : list-dirs {path1/}
(37) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(38) : test 9. Reset, do the delete again, and run bisync WITH --create-empty-src-dirs
(39) : bisync resync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Copying Path2 files to Path1
INFO : - Path2 Resync is copying files to - Path1
INFO : - Path1 Resync is copying files to - Path2
INFO : Resync updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(40) : list-dirs {path1/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(41) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(42) : purge-children {path1/}
(43) : copy-as {datadir/}placeholder.txt {path1/} file1.txt
(44) : copy-as {datadir/}placeholder.txt {path1/} file1.copy1.txt
(45) : copy-as {datadir/}placeholder.txt {path1/} file1.copy2.txt
(46) : copy-as {datadir/}placeholder.txt {path1/} file1.copy3.txt
(47) : copy-as {datadir/}placeholder.txt {path1/} file1.copy4.txt
(48) : copy-as {datadir/}placeholder.txt {path1/} file1.copy5.txt
(49) : list-dirs {path1/}
(50) : list-dirs {path2/}
subdir/ - filename hash: 86ae37b338459868804e9697025ba4c2
(51) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : - Path1 File was deleted - subdir
INFO : Path1: 1 changes: 0 new, 0 modified, 1 deleted
INFO : Path2 checking for diffs
INFO : Applying changes
INFO : - Path2 Queue delete - {path2/}subdir
INFO : - Path1 Do queued copies to - Path2
INFO : subdir: Removing directory
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful
(52) : test 10. Confirm the subdir has been removed on both paths
(53) : list-dirs {path1/}
(54) : list-dirs {path2/}
(55) : test 11. bisync again (because if we leave subdir in listings, test will fail due to mismatched modtime)
(56) : bisync create-empty-src-dirs
INFO : Setting --ignore-listing-checksum as neither --checksum nor --compare checksum are set.
INFO : Bisyncing with Comparison Settings:
{
"Modtime": true,
"Size": true,
"Checksum": false,
"NoSlowHash": false,
"SlowHashSyncOnly": false,
"DownloadHash": false
}
INFO : Synching Path1 "{path1/}" with Path2 "{path2/}"
INFO : Building Path1 and Path2 listings
INFO : Path1 checking for diffs
INFO : Path2 checking for diffs
INFO : No changes found
INFO : Updating listings
INFO : Validating listings for Path1 "{path1/}" vs Path2 "{path2/}"
INFO : Bisync successful