Child Item

Dropbox API in C# how to reuse the session id of bulk file upload in case file could not be uplo ...

aded by any issue.

In case of uploading big file of more than 100 mb, if by some case the internet connection get dropped then how it will be possible to upload the same file from the chunk it has been failed to upload. Please suggest. I have tried to upload by same session id but it never worked. It is getting exception.

Find more posts tagged with

API

Abuse

Comments

Greg-DB

Are you using the official .NET SDK? If so, there's some sample code that should be helpful here:

https://stackoverflow.com/documentation/dropbox-api/409/uploading-a-file/1358/uploading-a-file-using-the-dropbox-net-library#t=201609291814169541274

If you're still having trouble, please share your code and the exception you're getting.

Sayak B.

I am using Dropbox .NET library.

The problem is. I am actually trying to upload my sql server database backup everyday by zipping it to DropBox. The backup files after zip is average 50 mb in size. But the reality is while upload the internet get disconnected several time. So I need to upload the same thing fresh again and again the internet connection issue happens, The database server is in a remote location where internet connection is not so good. The result is that the file could not ever been uploaded because of the said issue, What I am trying to do is to use the upload session id which I am getting after uploading the first chunk. I am keeping it in a xml file and checking if the connection is failed and the file could not been uploaded then restart uploading the same file from the last successful uploaded chunk by using the session id. Below is my code snippet.

async void ConnectDropBox()
{
await LoginDropBox();

var progress = new Progress<ProgressReport>();
progress.ProgressChanged += (o, e) =>
{
Dispatcher.BeginInvoke((Action)(() =>
{
pgbarUpload.Maximum = e.TotalSteps;
pgbarUpload.Value = e.CurrentStep;
lblStatus.Text = "Uploading slice " + e.CurrentStep.ToString() + " of " + e.TotalSteps;
File.WriteAllText(AppVariables.xmlFilePath, AppVariables.CurrentUpload.ToXml<xmlUpload>());

if (e.CurrentStep == e.TotalSteps)
{
lblStatus.Text = string.Format("File \"{0}\" uploaded successfully with {1} slices", Path.GetFileName(AppVariables.ZipFilePath), e.TotalSteps);
btnUpload.IsEnabled = true;
btnRefresh.IsEnabled = true;
}
}));
};

try
{
await Upload(progress);
}
//catch (InvalidOperationException) { }
catch (Exception ex)
{
JHMessageBox.Show(ex.Message, "Exception Occured !", MessageBoxButton.OK, MessageBoxImage.Error);
}
}

private async Task Upload(IProgress<ProgressReport> progress)
{
try
{
const int chunkSize = 128 * 1024;
bool IsPrevUploadFailed = false;

using (var stream = new FileStream(AppVariables.ZipFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
int numChunks = (int)Math.Ceiling((double)stream.Length / chunkSize);

var progressReport = new ProgressReport() { TotalSteps = numChunks };

byte[] buffer = new byte[chunkSize];
string sessionId = null;
await Task.Run(async () =>
{
for (var idx = 0; idx < numChunks; idx++)
{

if (!AppVariables.LastUpload.IsNull() && !IsPrevUploadFailed)
{
if (AppVariables.CurrentUpload.Date == AppVariables.LastUpload.Date &&
AppVariables.CurrentUpload.FileChecksum == AppVariables.LastUpload.FileChecksum &&
AppVariables.CurrentUpload.FileName == AppVariables.LastUpload.FileName)
{
sessionId = AppVariables.LastUpload.SessionId;
IsPrevUploadFailed = true;
}
}

var byteRead = stream.Read(buffer, 0, chunkSize);

using (MemoryStream memStream = new MemoryStream(buffer, 0, byteRead))
{
if ((idx == 0 && !IsPrevUploadFailed) || (IsPrevUploadFailed && !AppVariables.LastUpload.IsNull() && AppVariables.LastUpload.Slice == 0))
{
var result = await AppVariables.DrpBoxClient.Files.UploadSessionStartAsync(body: memStream);
sessionId = result.SessionId;
}

else
{
UploadSessionCursor cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));
if (!IsPrevUploadFailed || (IsPrevUploadFailed && !AppVariables.LastUpload.IsNull()))
{
if (idx == numChunks - 1)
{
await AppVariables.DrpBoxClient.Files.UploadSessionFinishAsync(cursor, new CommitInfo(Config.DropBoxDirectory + System.IO.Path.GetFileName(AppVariables.ZipFilePath)), memStream);
}

else
{
await AppVariables.DrpBoxClient.Files.UploadSessionAppendV2Async(cursor, body: memStream);
}
}
}
}

AppVariables.CurrentUpload.SessionId = sessionId;
AppVariables.CurrentUpload.Slice = idx;
AppVariables.CurrentUpload.FileName = AppVariables.ZipFileName;

progressReport.CurrentStep = idx + 1;
progressReport.PercentComplete = (idx + 1) * 100 / progressReport.TotalSteps;
progress.Report(progressReport);
}
});
stream.Close();
stream.Dispose();
}
}
catch (Exception ex)
{
throw ex;
}
}
}

public class ProgressReport
{
public int PercentComplete { get; set; }
public int CurrentStep { get; set; }
public int TotalSteps { get; set; }
}

public class xmlUpload
{
public DateTime Date { get; set; }
public string FileChecksum { get; set; }
public string FileName { get; set; }
public int Slice { get; set; }
public string SessionId { get; set; }
}

Greg-DB

You can certainly use upload sessions to resume uploads of large files by persisting and re-using the session ID. If a particular request fails, you just need to re-upload that chunk, and not the whole file again.

Where does your code fail for you though? What exception(s) are you getting? Since you are using the .NET SDK, I do recommend starting from that code sample to get the basic file uploading logic first.

Sayak B.

Basic file upload is done with the code I have provided. That is working fine. I want to keep the file information including the session id that I get after uploading the first chunk. But that generating exception. Following is the exception--

Dropbox.Api.ApiException`1[Dropbox.Api.Files.UploadSessionLookupError]: incorrect_offset/..
at Dropbox.Api.DropboxRequestHandler.<Dropbox.Api.Stone.ITransport.SendUploadRequestAsync>d__5`3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at BackupUpload.frmZipNUpload.<>c__DisplayClass5_1.<<Upload>b__0>d.MoveNext() in E:\Jorehaut Tea Limited\Applications\BackupUpload\BackupUpload\Classes\frmZipNUpload.cs:line 234
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at BackupUpload.frmZipNUpload.<Upload>d__5.MoveNext()

Greg-DB

Thanks! The incorrect_offset error indicates that the server is expecting a different "offset" value (how far along in the file you are in the upload session) than the app is submitting. You can find more information on that in the documentation here:

https://dropbox.github.io/dropbox-sdk-dotnet/html/T_Dropbox_Api_Files_UploadSessionLookupError_IncorrectOffset.htm

You can get the expected offset value from UploadSessionOffsetError.CorrectOffset:

https://dropbox.github.io/dropbox-sdk-dotnet/html/P_Dropbox_Api_Files_UploadSessionOffsetError_CorrectOffset.htm

You'll want to add some logic to deal with this error (especially since your app will be running from an unreliable connection.)

Sayak B.

Finally I resolved the "incorrect_offset" issue by catching this exception and uploading the next slice. But the thing is after successful upload the file is corrupt. i.e. I am uploading zip file. that is corrupt and can not unzip by windows. What I am doing wrong and why it is happening ? can you please suggest what should be done ?

Greg-DB

It sounds like there may be a bug in your uploading logic. It's hard to say what it may be offhand, so I recommend adding some logging to your upload code to see where the issue may be and determine where the incorrect data is being sent up, or if you're sending the data out of order or something.

Have you tried uploading via the sample I linked to though? Does that work correctly for you?

0ylanmorisson

I'm facing the same issue here. How can I get the correct offset by using UploadSessionOffsetError.CorrectOffset ? Can you pls show me some sample code?

How can I pass the correct offset as the right cursor to the following method?

await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);

Greg-DB

@0ylanmorisson It looks like you opened another thread for this, so I'll follow up there.