Retrieving file size

Oct 21, 2008 at 7:04 PM
Edited Oct 22, 2008 at 8:00 AM
Hi John,  For my application I upload hundreds of clips in a session and what I've found that if I have a connectivity problem or otherwise can't load a clip completely that I have problems determining what files got properly uploaded.  What I've been doing in cases that I have connectivity problems is rerunning the file list with overwrite:false.  This gives me a "SKIPPING" message when the file already exists.  However what I've found is (it appears) that if a file was interrupted in its upload that it still exists (SKIPPED) but has a file size listed as 1 Kbyte.  Unfortunately the return code from S3 doesn't give the file size for me to parse to see if I need to resend it.  Is there anyway with your command line tools to get the file size?  Thx.
Coordinator
Oct 22, 2008 at 4:29 PM
What if I changed the valid values for the "overwrite" flag from (true,false) to (always,sizes-differ,never)?
    always = always overwrite the S3 object (equivalent to "true" now)
    sizes-differ = overwrite the S3 object only if the file size on S3 does not match the local file size
    never = never overwrite the S3 object (equivalent to "false' now)

This would allow us to add new conditions in the future (such as "local-newer" = local file has a timestamp greater than the one on S3, etc).

If a transfer fails, you can simply rerun as many times as needed until it finishes successfully.

Oct 22, 2008 at 5:02 PM
Edited Oct 22, 2008 at 5:03 PM
Hey that's a great idea! Perfect! Thanks.



Coordinator
Oct 22, 2008 at 9:28 PM
Ok, just posted a new version to the front-page (version 0.0.2.6) which includes the overwrite flag change.  Let me know if that works for your scenario.