Issues about Chunkupload

0ylanmorisson

I'm writing a very simple console uploading program using most of the sample code from Dropbox.net Api. But I have two issues now.

 

When the network drops and reconnects, I get an error saying "lookup_failed/incorrect_offset/". Where I go wrong?

 

Another related question is,  say I have a 5120kb file to upload and the chunk is 1024kb. So when the third chunk is uploading,  suppose the network suddently drops after uploading 500kb, then where should I set the offset to continue uploading, from offset=3072 or from offset=3072+500= 3572?

 

Here is my ChunkUpload code:

 

       private async Task ChunkUpload(DropboxClient client, string srcfile, string dstfile)
        {
            if (File.Exists(srcfile))
            {
                Console.WriteLine("\nUploading file {0} to Dropbox path: {1}\n", srcfile, dstfile);
                // Chunk size is 1024KB.
                const int chunkSize = 1024 * 1024;

                FileStream fs = new FileStream(srcfile, FileMode.Open);
                byte[] data = new byte[fs.Length];
                fs.Read(data, 0, data.Length);
                fs.Close();

                using (var stream = new MemoryStream(data))
                {
                    int numChunks = (int)Math.Ceiling((double)stream.Length / chunkSize);

                    byte[] buffer = new byte[chunkSize];
                    string sessionId = null;

                    var idx = 0;
                   do
                    {
                        var byteRead = stream.Read(buffer, 0, chunkSize);
                        var percentage = 0;
                        int row = Console.CursorTop - 1;  //Set cursor position
                        Console.SetCursorPosition(0, row);
                        if ( (idx+1) == numChunks)  //Last chunk, 99% done!
                        {
                            percentage =  99;
                            Console.WriteLine("Uploaded {0}: {1}% of total {2}kb", idx, percentage, (int)Math.Ceiling((double)stream.Length));
                        }
                        else
                        {
                            percentage = 100 * (idx + 1) * chunkSize / (int)Math.Ceiling((double)stream.Length);
                            Console.WriteLine("Uploaded {0}: {1}% of total {2}kb", idx, percentage, (int)Math.Ceiling((double)stream.Length));
                        }
                        try
                        {
                            using (MemoryStream memStream = new MemoryStream(buffer, 0, byteRead))
                            {
                                if (idx == 0)
                                {
                                    var result = await client.Files.UploadSessionStartAsync(body: memStream);
                                    sessionId = result.SessionId;
                                }

                                else
                                {
                                    UploadSessionCursor cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));

                                    if (idx == numChunks - 1)
                                    {
                                        await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(dstfile), memStream);
                                        row = Console.CursorTop - 1;
                                        Console.SetCursorPosition(0, row);
                                        Console.WriteLine("Uploaded {0}: {1}% of total {2}kb", idx, 100, (int)Math.Ceiling((double)stream.Length));
                                    }
                                    else
                                    {
                                        await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);
                                    }
                                }
                            }
                            idx++;
                        }
                        catch (Exception e)
                        {
                            Console.WriteLine("Error uploading:{0}", e.Message);
                            Console.WriteLine("SessionID: {0}", sessionId);
                            Console.WriteLine("\nMake sure everything is OK then press enter. \n");
                            Console.ReadLine();
                        }
                    } while (idx < numChunks);
                }
                Console.WriteLine("Upload OK! Press enter to exit...");
                Console.ReadLine();
            }
            else
            {
                Console.WriteLine("Wrong path:{0}", srcfile);
            }
        }

Greg-DB

The documentation for UploadSessionLookupError.IncorrectOffset covers an example of how this can occur:

 

"The specified offset was incorrect. See the value for the correct offset. This error may occur when a previous request was received and processed successfully but the client did not receive the response, e.g. due to a network error."

 

To get the correct offset, you can catch and inspect the exception like this:

try {
    await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);
} catch (ApiException<UploadSessionLookupError> e) {

    if (e.ErrorResponse.IsIncorrectOffset) {
        Console.WriteLine("Expected offset: " + e.ErrorResponse.AsIncorrectOffset.Value.CorrectOffset);
    }
    // todo: other error handling
    throw;
}

In your 5120kb file scenario, I believe the offset would still be 3072 at that point, since the connection failed. However that may depend on the exact nature of the failure, so you should always check the returned value anyway.