Child Item

Problems downloading big size files (20 GB) from the API .NET

Hello.

I'm trying to download some files from a couple of folders in dropbox, but when I try to download a couple of files that I have of about 20 gb, something strange happens. It appears that the file is downloaded correctly, but the file size in my local computer is about 13 gb, so the file it's corrupted. with 5gb files or lower this doesn't happen. this is the code.

dropboxFileInfo = (FileMetadata)item;

using (var response = await dbx.Files.DownloadAsync(dropboxFileInfo.PathLower.ToString()))
{
Console.WriteLine("Descargando " + dropboxFileInfo.Name.ToString() + "...");
using (FileStream fileStream = File.Create(localname))
{

(await response.GetContentAsStreamAsync()).CopyTo(fileStream);
var local = Convert.ToInt64(fileStream.Length);
var cloud = Convert.ToInt64(dropboxFileInfo.Size);

if (local == cloud)
{

Console.WriteLine("Descargado");
}
else
{
Console.WriteLine("Error al descargar");

}

Find more posts tagged with

API

Abuse

Comments

Greg-DB

Does the actual downloaded size vary, or is it the same each time? If it's the same, please let me know what it is for reference.

Also, how long does the (incomplete) download take? I believe a single connection is only valid for 90 minutes, so that could be the cause if it takes that long. (Though the HTTPS client is expected to throw an error in that case when it sees the response is incomplete, but that may not be occurring unfortunately.)

hramosvz

It varies. I did some more testing and it also happens with smaller files. I was downloading a file of 3.48 GB but just 1.75 GB were downloaded. This happened in exaclty 15 minutes without any error showing. This the complete method code.

      private async Task<int> Run()
        {
            DropboxCertHelper.InitializeCertPinning();

            

            // Specify socket level timeout which decides maximum waiting time when no bytes are
            // received by the socket.
            var httpClient = new System.Net.Http.HttpClient(new WebRequestHandler { ReadWriteTimeout = 100 * 10000000 })
            {
                // Specify request level timeout which decides maximum time that can be spent on
                // download/upload files.
                Timeout = TimeSpan.FromHours(10)
            };

            try
            {
                var config = new DropboxClientConfig("SubidaBacks")
                {
                    HttpClient = httpClient
                };

                var dbx = new DropboxClient("************", config);

                var nbackups = 0;
                var ListaBaks = new List<string>();
               

                using (dbx)
                {


                    var full = await dbx.Users.GetCurrentAccountAsync();


                    foreach (var folder in Folders.Split(';'))
                    {
                        var listOfFolders = await dbx.Files.ListFolderAsync(folder, true);

                        FolderMetadata dropboxFolderInfo = new FolderMetadata();
                        FileMetadata dropboxFileInfo = new FileMetadata();
                        Int64 local, cloud;



                        foreach (var item in listOfFolders.Entries)
                        {
                            try
                            {
                                if (item.IsFile)
                                {

                                    foreach (var fecha in Dates())
                                    {

                                        if (item.Name.ToString().Contains(fecha))
                                        {

                                            var localname = localFilePath + "\\" + item.Name;

                                            if (!File.Exists(localname))
                                            {
                                                dropboxFileInfo = (FileMetadata)item;
                                                var response = await dbx.Files.DownloadAsync(dropboxFileInfo.PathLower.ToString());
                                                ulong fileSize = response.Response.Size;
                                                const int bufferSize = 1024 * 1024;

                                                var buffer = new byte[bufferSize];

                                                Console.WriteLine("Descargando " + dropboxFileInfo.Name.ToString() + "...");

                                                using (var stream = await response.GetContentAsStreamAsync())
                                                {
                                                    using (var file = new FileStream(localname, FileMode.OpenOrCreate))
                                                    {
                                                        var length = stream.Read(buffer, 0, bufferSize);

                                                        while (length > 0)
                                                        {
                                                            file.Write(buffer, 0, length);
                                                            var percentage = 100 * (ulong)file.Length / fileSize;
                                                            // Update progress bar with the percentage.
                                                            // progressBar.Value = (int)percentage
                                                            Console.WriteLine(percentage.ToString()+"% - "+ dropboxFileInfo.Name.ToString() +" - " + DateTime.Now.ToString());

                                                            length = stream.Read(buffer, 0, bufferSize);
                                                        }

                                                         local = Convert.ToInt64(file.Length);
                                                         cloud = Convert.ToInt64(dropboxFileInfo.Size);

                                                       

                                                    }


                                                }

                                                if (local == cloud)
                                                {
                                                    ListaBaks.Append(item.PathLower.ToString());
                                                    nbackups = nbackups + 1;
                                                    Console.WriteLine("Descargado");
                                                    
                                                }
                                                else
                                                {
                                                    File.Delete(localname);
                                                    Console.WriteLine("Error al descargar");

                                                }

                                                
                                            }
                                        }
                                    }
                                }
                            }
                            catch (Exception ex)
                            {
                                Console.WriteLine(ex.Message);
                            }
                        }
                    }


                }

                // Correo.EnvioCorreo(nbackups.ToString(), string.Join("\n", ListaBaks));

            }
            catch (HttpException e)
            {
                Console.WriteLine("Exception reported from RPC layer");
                Console.WriteLine("    Status code: {0}", e.StatusCode);
                Console.WriteLine("    Message    : {0}", e.Message);
                if (e.RequestUri != null)
                {
                    Console.WriteLine("    Request uri: {0}", e.RequestUri);
                }
            }

            return 0;
        }

Greg-DB

Thanks for following up and confirming the size varies.

Also, just to clarify your last message, where you mentioned it failed after exactly 15 minutes, does it always fails after exactly 15 minutes (or did you mean just this last attempt in particular)?

And you mentioned there was no error, but does it get to your "Console.WriteLine("Error al descargar");" line, or does it seem to stop somewhere before that?

I'm trying to reproduce this, but it seems to be working fine for me, even when downloading takes longer than 15 minutes.