Child Item

SwiftDropbox too many write operations when writing 4 files

Hi there, I'm using the SwiftyDropbox library to upload files to Dropbox in an iOS app (Swift). I have a loop where I initiate uploads for 4 files:

for filePath in filePaths {
    let fileUrl: URL! = URL(string: "file://\(filePath)")

    //let fileData = try String(contentsOf: fileUrl, encoding: .utf8)
    let fileData = try Data(contentsOf: fileUrl)

    let request = self.dropboxClient!.files.upload(
        path: "/\(fileUrl.lastPathComponent)",
        mode: Files.WriteMode.overwrite,
        input: fileData
    )
    .response { response, error in
        if let response = response {
            print("Upload complete")
        } else if let error = error {
            print("Error uploading file: \(error)")
        }
    }
}

Usually one, two or three of the files uploads fine, but the others throw this error:

Error uploading file: [request-id a2c2eca29485a7d10892343e3d3e57c4] API rate limit error - {
    reason =     {
        ".tag" = "too_many_write_operations";
    };
    "retry_after" = 1;
}

My guess is that my for loop is initiating the 4 uploads at the same time in parallel, and for some reason this isn't allowed - is that right? If so, how do I modify my code to get this to work?

There appears to be a method called batchUploadFiles available on the files object, so I'm guessing that's the ticket, but I can't find any documentation on this or figure out how to use it (apologies, I'm an Android dev not experienced with Swift so I'm feeling like a complete noob ). An example would be muchly appreciated. I'm totally stuck, please help!

Find more posts tagged with

API

Abuse

Accepted answers

ncljames

Thanks I didn't think to look at the test cases! For future reference for other users looking for the same, here's the version of my Swift code adapted from the test that works:

var filesCommitInfo = [URL : Files.CommitInfo]()

for filePath in filePaths {
    let fileUrl: URL! = URL(string: "file://\(filePath)")
    let uploadToPath = "/\(fileUrl.lastPathComponent)"
    filesCommitInfo[fileUrl] = Files.CommitInfo(path: uploadToPath, mode: Files.WriteMode.overwrite)
}

self.dropboxClient!.files.batchUploadFiles(
    fileUrlsToCommitInfo: filesCommitInfo,
    responseBlock: { (uploadResults: [URL: Files.UploadSessionFinishBatchResultEntry]?,
                      finishBatchRequestError: CallError<Async.PollError>?, 
                      fileUrlsToRequestErrors: [URL: CallError<Async.PollError>]) -> Void in

        if let uploadResults = uploadResults {
            for (clientSideFileUrl, result) in uploadResults {
                switch(result) {
                    case .success(let metadata):
                        let dropboxFilePath = metadata.pathDisplay!
                        print("Upload \(clientSideFileUrl.absoluteString) to \(dropboxFilePath) succeeded")
                    case .failure(let error):
                        print("Upload \(clientSideFileUrl.absoluteString) failed: \(error)")
                }
            }
        }
        else if let finishBatchRequestError = finishBatchRequestError {
            print("Error uploading file: possible error on Dropbox server: \(finishBatchRequestError)")
        } else if fileUrlsToRequestErrors.count > 0 {
            print("Error uploading file: \(fileUrlsToRequestErrors)")
        }   
})

PS @Greg I've been browsing the forums lately and you seem to be everywhere, answering people's questions with unlimited patience and kindness. Just wanted to say you're a superstar, thanks

All comments

Greg-DB

That's correct, the 'too_many_write_operations' error is "lock contention". That's a technical inability to make a modification in the account or shared folder at the time of the API call. This error indicates that there was simultaneous activity in the account or shared folder preventing your app from making the state-modifying call (e.g., adding, editing, moving, or deleting files/folders) it is attempting. The simultaneous activity could be coming from your app itself, or elsewhere, e.g., from the user's desktop client. It can come from the same user, or another member of a shared folder. You can find more information about lock contention in the Data Ingress Guide.

In short, to avoid this error, you should avoid making multiple concurrent state modifications. E.g., don't issue multiple such requests at a time, and use batch endpoints whenever possible. That won't guarantee that you won't run in to this error though, as contention can still come from other sources.

Making four separate upload calls at the same time like you've shown here could certainly cause this kind of lock contention, and using batchUploadFiles instead would be a good solution. That implements the methods mentioned in the Data Ingress Guide above. There's an example of using it in the test class.

Hope this helps!

ncljames

Thanks I didn't think to look at the test cases! For future reference for other users looking for the same, here's the version of my Swift code adapted from the test that works:

var filesCommitInfo = [URL : Files.CommitInfo]()

for filePath in filePaths {
    let fileUrl: URL! = URL(string: "file://\(filePath)")
    let uploadToPath = "/\(fileUrl.lastPathComponent)"
    filesCommitInfo[fileUrl] = Files.CommitInfo(path: uploadToPath, mode: Files.WriteMode.overwrite)
}

self.dropboxClient!.files.batchUploadFiles(
    fileUrlsToCommitInfo: filesCommitInfo,
    responseBlock: { (uploadResults: [URL: Files.UploadSessionFinishBatchResultEntry]?,
                      finishBatchRequestError: CallError<Async.PollError>?, 
                      fileUrlsToRequestErrors: [URL: CallError<Async.PollError>]) -> Void in

        if let uploadResults = uploadResults {
            for (clientSideFileUrl, result) in uploadResults {
                switch(result) {
                    case .success(let metadata):
                        let dropboxFilePath = metadata.pathDisplay!
                        print("Upload \(clientSideFileUrl.absoluteString) to \(dropboxFilePath) succeeded")
                    case .failure(let error):
                        print("Upload \(clientSideFileUrl.absoluteString) failed: \(error)")
                }
            }
        }
        else if let finishBatchRequestError = finishBatchRequestError {
            print("Error uploading file: possible error on Dropbox server: \(finishBatchRequestError)")
        } else if fileUrlsToRequestErrors.count > 0 {
            print("Error uploading file: \(fileUrlsToRequestErrors)")
        }   
})

PS @Greg I've been browsing the forums lately and you seem to be everywhere, answering people's questions with unlimited patience and kindness. Just wanted to say you're a superstar, thanks

Greg-DB

Thanks for the kind words, and for sharing your code for others!