Child Item

Official Dropbox .Net v2 SDK, slow uploading speed for large files

Hello,

We use Official Dropbox .Net v2 SDK.

Average uploading speed for some our clients is 50-100 kb/sec.

We use the following code to upload chunks

log.Trace("{0}. File chunk taken. CountBytes: \"{1}\"", nameof(UploadFileImpl), countBytes);
using (var memStream = new MemoryStream(buffer.Take(countBytes).ToArray()))
{
    using (var t = dropBoxClient.Files.UploadSessionAppendV2Async(new UploadSessionCursor(sessionId, offset), false,memStream))
    {
        t.Wait();
    }
}
 
offset += (ulong) countBytes;
log.Trace("{0}. File chunk uploaded. Offset: \"{1}\"", nameof(UploadFileImpl), offset);

In client's log we can see the following record:

2017-11-16 12:01:43.3748|UploadFileImpl. File chunk taken. CountBytes: "2097152"
2017-11-16 12:02:21.7800|UploadFileImpl. File chunk uploaded. Offset: "3909091328"

Uploaded 2mb in 40 sec. ~ 50 kb/sec

Any ideas?

PS. Client's internet connection is 5-20 mb/sec.

Find more posts tagged with

API

Abuse

Accepted answers

Greg-DB

The web site uses a somewhat different piece of infrastructure than the API, so that may not be a good comparison.

If you want to test your connection to the Dropbox API content servers directly though, you can test a simpler /2/files/upload API call, e.g., using curl:

https://www.dropbox.com/developers/documentation/http/documentation#files-upload

That would eliminate the complexity of using upload sessions, and let you see what kind of speeds you're getting for a single connection.

All comments

Greg-DB

First, note that the connection speed will vary from place to place. Your connection speed to Dropbox depends on the routing you get between your ISP and our servers, and may be slower than your ISP's rated speeds.

Sometimes resetting or retrying your connection gets you a different route and better speeds, but that is outside of our control. Some ISPs also throttle sustained connections so if you see an initial high connection speed followed by lower speeds, that could be the reason.

Also, I see from the output though that it appears you're using about 2 MB chunks. I recommend using a larger chunk size, such as 8 MB or 16 MB (or even higher, depending on the connection). Using a larger chunk size can increase the overall speed by reducing the number of individual HTTPS requests needed, reducing overhead. It does come at the cost of having to resend more data when any one particular request fails though.

Ivan_Gusev

That is all clear, thanks.

But speed is 50 kb/sec...

How can we test the uploading speed to find the error? Should we upload a file via browser and check if the uploading speed is equal?

P.S.

We use a dynamic chunk size. If we receive a timeout error then we start to upload from the beginning with reduced chunk size. By default we upload chunk size = 8 mb.

Greg-DB

The web site uses a somewhat different piece of infrastructure than the API, so that may not be a good comparison.

If you want to test your connection to the Dropbox API content servers directly though, you can test a simpler /2/files/upload API call, e.g., using curl:

https://www.dropbox.com/developers/documentation/http/documentation#files-upload

That would eliminate the complexity of using upload sessions, and let you see what kind of speeds you're getting for a single connection.