Child Item

How long should I wait for /upload_session/finish_batch/check to complete?

In order to complete a batch it is necessary to call upload_session/finish_batch/check . The docs for this don't give any guidance on

How long to try calling this API for
How often to call it

In the batch upload I'm developing for rclone I've arbitrarily chosen 2 minutes and 1 second so rclone checks for completion for a maximum of 120 times at 1 second intervals.

This seems to work very well, but occasionally I seem to need more than 120 seconds to complete a batch. I think this might be a bug at as the next batch complete returns a 500 error.

Here is the last transaction (which looks normal) before rclone gives up

2021/04/19 20:10:19 DEBUG : HTTP REQUEST (req 0xc001103900)
2021/04/19 20:10:19 DEBUG : POST /2/files/upload_session/finish_batch/check HTTP/1.1
Host: api.dropboxapi.com
User-Agent: rclone/v1.56.0-beta.5394.d16702566.fix-dropbox-batch-sync
Content-Length: 113
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip

2021/04/19 20:10:19 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/04/19 20:10:19 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/04/19 20:10:19 DEBUG : HTTP RESPONSE (req 0xc001103900)
2021/04/19 20:10:19 DEBUG : HTTP/2.0 200 OK
Content-Length: 23
Accept-Encoding: identity,gzip
Cache-Control: no-cache
Content-Type: application/json
Date: Tue, 20 Apr 2021 00:10:19 GMT
Server: envoy
X-Content-Type-Options: nosniff
X-Dropbox-Request-Id: f746ed86c87a4fdcaf660a12d19f1a24
X-Dropbox-Response-Origin: far_remote
X-Frame-Options: SAMEORIGIN
X-Server-Response-Time: 109

{".tag": "in_progress"}

2021/04/19 20:10:20 ERROR : Dropbox root 'jbod2': sync batch commit: failed to commit batch length 1: wait for batch failed after 120 tries: batch didn't complete

So rclone gives up here and tries to finish the next batch

2021/04/19 20:10:21 DEBUG : HTTP REQUEST (req 0xc001373b00)
2021/04/19 20:10:21 DEBUG : POST /2/files/upload_session/finish_batch HTTP/1.1
Host: api.dropboxapi.com
User-Agent: rclone/v1.56.0-beta.5394.d16702566.fix-dropbox-batch-sync
Content-Length: 907
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip

2021/04/19 20:10:21 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/04/19 20:10:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/04/19 20:10:21 DEBUG : HTTP RESPONSE (req 0xc001373b00)
2021/04/19 20:10:21 DEBUG : HTTP/2.0 500 Internal Server Error
Content-Length: 0
Accept-Encoding: identity,gzip
Content-Security-Policy: sandbox allow-forms allow-scripts
Content-Type: text/plain; charset=utf-8
Date: Tue, 20 Apr 2021 00:10:21 GMT
Server: envoy
X-Dropbox-Request-Id: c787b53ec12e42eba8e0d650a28086ff
X-Dropbox-Response-Origin: far_remote

And immediately gets a 500 error. Rclone retries this 10 times then gives up.

Some guidance here would be appreciated

What is the longest time to commit a batch?
What is the best polling frequency for batch completion?
Is the above normal or a bug?

This problem was reported in the rclone issue tracker #5008

Thanks

Nick

Find more posts tagged with

API

Abuse

Comments

Greg-DB

There isn't a guaranteed maximum time that one of these batch jobs may take. The processing time will depend on a few factors, such as how many files there are, etc., so it can vary.

Likewise, there isn't a best frequency for how often to poll these jobs. Once a second is fine. For larger batches, which may take longer given the larger amount of work to do, you could increase that a bit if you wish.

The 500 Internal Server Error indicates an issue on the Dropbox servers, so we'll have to look into that. For reference though, can you let me know if that's transient or if you can reliably reproduce that error, e.g., if you try again now? Thanks in advance!

ncw

> There isn't a guaranteed maximum time that one of these batch jobs may take. The processing time will depend on a few factors, such as how many files there are, etc., so it can vary.

There was only one file in the batch above so for it to take 2 minutes seems a long time. But then I have no idea what is going on behind the scenes!

> Likewise, there isn't a best frequency for how often to poll these jobs. Once a second is fine. For larger batches, which may take longer given the larger amount of work to do, you could increase that a bit if you wish.

OK.

> The 500 Internal Server Error indicates an issue on the Dropbox servers, so we'll have to look into that. For reference though, can you let me know if that's transient or if you can reliably reproduce that error, e.g., if you try again now?

I got that trace second hand from an rclone tester, but I believe it only happens quite rarely when doing a lot of transfers over a long time.

Thanks

Здравко

@ncw wrote:
...
So rclone gives up here and tries to finish the next batch
...

Hi @ncw,

Key moment here may be that only one active batch could work at a moment for the same user. According your description the last running batch might not successfully finished yet (or unsuccessfully), while you are trying a new one! This might confuse Dropbox server and provokes some rare bug as a response. 🤷 As a workaround (temporary or not) you can stop any follow up ongoing automatic processing, signal ongoing error, and wait for new user command.

Let's hope this will be traced successfully and fixed soon.

Greg-DB

@ncw wrote:

I got that trace second hand from an rclone tester, but I believe it only happens quite rarely when doing a lot of transfers over a long time.

@ncw Thanks! That seems to match what I can see on our side. It appears this was a transient availability issue.

@ncw wrote:
Rclone retries this 10 times then gives up.

How quickly does Rclone perform these retries on a 500 like this? If it's in relatively quick succession, you may want to implement an exponential backoff to give the service more time to recover when this does occur.