Child Item

Receiving "com.dropbox.core.NetworkIOException: Connection reset" during file downloads

I'm using Dropbox java api client, the client is used in a multi-threaded environment, I'm downloading various files from different dropbox accounts and from time to time without some clear pattern a following exception is thrown:

com.dropbox.core.NetworkIOException: Connection reset
        at com.dropbox.core.DbxRequestUtil.startPostRaw(DbxRequestUtil.java:264)
        at com.dropbox.core.v2.DbxRawClientV2$2.execute(DbxRawClientV2.java:172)
        at com.dropbox.core.v2.DbxRawClientV2$2.execute(DbxRawClientV2.java:167)
        at com.dropbox.core.v2.DbxRawClientV2.executeRetriable(DbxRawClientV2.java:300)
        at com.dropbox.core.v2.DbxRawClientV2.downloadStyle(DbxRawClientV2.java:167)
        at com.dropbox.core.v2.files.DbxUserFilesRequests.download(DbxUserFilesRequests.java:1092)
        at com.dropbox.core.v2.files.DbxUserFilesRequests.download(DbxUserFilesRequests.java:1121)
        at com.coronet.dlp.services.dropbox.DropBoxDLPScan.visitAllFiles(DropBoxDLPScan.java:224)
        at com.coronet.dlp.services.dropbox.DropBoxDLPScan.lambda$scanAllFiles$1(DropBoxDLPScan.java:119)
        at com.coronet.dlp.services.dropbox.DropBoxDLPScan$$Lambda$47/1317893434.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:209)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
        at com.dropbox.core.http.StandardHttpRequestor.toResponse(StandardHttpRequestor.java:50)
        at com.dropbox.core.http.StandardHttpRequestor.access$100(StandardHttpRequestor.java:29)
        at com.dropbox.core.http.StandardHttpRequestor$Uploader.finish(StandardHttpRequestor.java:178)
        at com.dropbox.core.DbxRequestUtil.startPostRaw(DbxRequestUtil.java:259)
        ... 12 common frames omitted

Can it be because some why connection on your side is closed due to some reasons?

Thanks.

Find more posts tagged with

API

Abuse

Comments

DBX_Robert

Do you only get this error when downloading on multiple threads in parallel? If you download serially in a single-threaded environment do you encounter the same random error?

EvgenyAgronsky

Each thread uses a separate instance of DbxClientV2, so I think it's not necessarily connected to multi-threaded environment, I'm using the following api function to download files:

clientV2.files().download(fileMetaData.getPathLower());

Sometimes downloads are frequent, and download is called with same token from different threads. Although the instances of the client object are different, they use underneath the same token.

To clarify I'll explain the use case, I let a user connect to my service via oauth, then I traverse his Dropbox files, one thread for each users Dropbox member, I download his files to perform a certain work on them, and I do it for each users member.

DBX_Robert

Downloading with the same OAuth token is desired behavior, so that is unlikely the issue.

Can you help me understand the volume at which you are downloading files. Can you estimate the maximum number of simultaneous connections are you creating and how large the files are that you're downloading?