ncw — Dropbox Community

Comments

Here is a failing request and response I managed to reproduce this with a small file file.txt contents shello and using rclone cat --offset 3 dropbox:file.txt -vv --dump bodies This made this request 2023/04/25 11:20:46 DEBUG : HTTP REQUEST (req 0xc0009fec00) 2023/04/25 11:20:46 DEBUG : POST /2/files/download HTTP/1.1…

in Downloading part of a file via /get_temporary_link with Range header returns incorrect content-type Comment by ncw April 2023
Rclone users have reported a similar problem when using anything with `Range:` requests. There is a long thread here https://forum.rclone.org/t/failed-to-open-source-not-an-encrypted-file-bad-magic-string/37790 But my summary is that Range: requests are not working properly. They appear to produce files which have an extra…

in Downloading part of a file via /get_temporary_link with Range header returns incorrect content-type Comment by ncw April 2023
> There isn't a guaranteed maximum time that one of these batch jobs may take. The processing time will depend on a few factors, such as how many files there are, etc., so it can vary. There was only one file in the batch above so for it to take 2 minutes seems a long time. But then I have no idea what is going on behind…

in How long should I wait for /upload_session/finish_batch/check to complete? Comment by ncw April 2021
Thank you Greg that was extremely clear. I understand exactly now what I am doing 🙂

in Do I have to wait until /upload_session/finish_batch/check finishes before starting the next batch? Comment by ncw March 2021
Sure thing, we obey Retry-After. I'll just note that Google Drive specifies the queries per second per user publically which is 1000 queries per 100 seconds and it is very helpful for app developers to know. You see that number in your app console and you can set it to fewer queries per second if you want. Rclone could…

in Rate limiting when uploading files with rclone Comment by ncw September 2020
A bit of experimenting makes us think that if we can at or below 12 transactions per second then we don't get rate limited. I know you can't confirm or deny that @"Greg-DB" but maybe you can give us the secret wink or something like that ;-)

in Rate limiting when uploading files with rclone Comment by ncw September 2020
> The only way to avoid this then is to just call at a slower rate. It sounds like you already tried that, but if it didn't help you'll need to limit it further. I tried limiting it to 10 per second. I'll do some experiments to see what I think the right number is. Or even better do you have a figure you could tell me?

in Rate limiting when uploading files with rclone Comment by ncw September 2020
My Test user is uploading 10TB of images. So lots of 2-3MB files. Using batching the upload is proceeding very quickly (at about 35MB/s) which is great. However every 30 minutes or so they get this error too_many_requests/.: Too many requests or write operations. Along with a 300 second Retry-After which rclone obeys.…

in Rate limiting when uploading files with rclone Comment by ncw September 2020
I have implemented batching, and yes it does make a HUGE difference. In fact for my test directory with 200MB of small images (average size 400k) it transfers it 20x faster! I think the major disadvantage for rclone is that it can't check the hashes of the uploaded files any more. This is because the batch completes after…

in Rate limiting when uploading files with rclone Comment by ncw September 2020
> We'll look into it, but for reference, can you let me know, to the best of your knowledge, when you started seeing this? Unfortunately these errors don't show up in the integration tests logs otherwise I'd have an accurate timeline for you. I received the first user report about this on the 2nd September 2020 and I…

in Rate limiting when uploading files with rclone Comment by ncw September 2020
I made an issue in the Dropbox Go SDK repo which is where the problem needs to be fixed.

in Possible bug corrupting file names with certain unicode code points Comment by ncw October 2019
Thanks for that. It must be a bug in dropbox Go SDK that the headers aren't being properly encoded I'll report a bug there in a moment!

in Possible bug corrupting file names with certain unicode code points Comment by ncw October 2019
@"Greg-DB" wrote:Is the namespace ID for a "team space"? If so, you will need to use the the Dropbox-API-Path-Root header to access it. Yes it is a team space. In the docs I can't see an indication that the `ns:` prefix doesn't work with the team space root ID. So is this a documentation bug? Thanks for your help Nick

in Can't get path with `ns:` prefix to work in /2/files/list_folder Comment by ncw May 2018
I see - thank you for explaining. I don't like this design choice, but it is documented so I'll live with it :-) Looking forward to an API to set the client_modified time! Thanks for your help @"Greg-DB"

in Is it possible to set client_modified on an existing file in v2 API? Comment by ncw March 2018
Related to that... If you actually want to upload a the same file with a different `client_modified`, you have to delete the existing file first. If you don't delete the existing file first, then the upload succeeds, but somehow it has the old `client_modified`, not the new one you just specified in your upload. I suspect…

in Is it possible to set client_modified on an existing file in v2 API? Comment by ncw March 2018
> For reference, do you need to be able to do so without uploading a new version? Yes that would be ideal. Rclone uses `client_modified` to save the modification times of files. If a user just does `touch file` then it would be nice to be able to just set the `client_modified` without upload the whole file again. At the…

in Is it possible to set client_modified on an existing file in v2 API? Comment by ncw March 2018
I confirm this is fixed now. Super impressive turn-around :-) Thanks Nick

in /1/metadata requests returning 400 error for difficult file name Comment by ncw April 2016