Child Item

client_modified and server_modified values sometimes differ by 1 second on successful upload

I have written a simple bash script for uploading small files to Dropbox from CLI through curl request by utilizing the code as shown in Dropbox API explorer here https://dropbox.github.io/dropbox-api-v2-explorer/#files_upload. However, when inspecting the JSON metadata on successful upload I noticed that sometimes the client_modified and server_modified values differ by 1 second. Here's a sample response I got:

===============
Started at 2020-11-23T02:09:59,720934918+00:00
Upload file to /tmp/file
Response code: 200

Upload Details:
{
  "name": "file",
  "path_lower": "/tmp/file",
  "path_display": "/tmp/file",
  "id": "id:yD_8dYeLhQAAAAAAAAAFYA",
  "client_modified": "2020-11-23T02:10:02Z",
  "server_modified": "2020-11-23T02:10:03Z",
  "rev": "5b4bcb1320170d4ce9680",
  "size": 33554431,
  "is_downloadable": true,
  "content_hash": "eeedd070d43a7774da5a4fdf67aa68e6e0a2a96d86df161b53b0524f119ff176"
}
===============

As you can see the client_modified and server_modified values are off by 1s.

This behavior is inconsistent (in most cases both modified values will be the same) and can be observed intermittently when uploading both small and larger (~100 MB) files.

What is the reason for this difference and is this something that I should be concerned about?

EDIT: Here's my code in case that's relevant in any way.

       echo "Dropbox File Upload API"
       echo "======================="
       echo "Files in current directory:"
       ls -p | grep -v /
       echo ""
       echo "Source File:"
       read -r "FILE"
       echo ""
       echo "Destination Path (without final slash):"
       read -r "DIR_UL"

       BASENAME=$(basename "$FILE")

       CMD1="Upload $FILE to $DIR_UL/$BASENAME"
       echo "==============="
       echo "Started at $(date -Ins)"
       HTTP_CODE1=$(curl -X POST -sL -w "%{http_code}" --output /tmp/dropbox-upload.log https://content.dropboxapi.com/2/files/upload \
           --header "Authorization: Bearer $TOKEN" \
           --header "Dropbox-API-Arg: {\"path\": \"$DIR_UL/$BASENAME\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" \
           --header "Content-Type: application/octet-stream" \
           --data-binary @$FILE)
           echo $CMD1
           echo "Response code: $HTTP_CODE1"
           echo ""
           echo "Upload details:"
           cat /tmp/dropbox-upload.log | jq .
           echo "==============="
           echo ""
           rm /tmp/dropbox-upload.log

Find more posts tagged with

API

Abuse

Accepted answers

Greg-DB

Thanks for writing this up! I don't believe this anything to be concerned about. It looks like this is just the result of some implementation details on our side. (For example, it seems like the client_modified is taken from the time of request, if not explicitly set, whereas server_modified is when the file is committed to the Dropbox filesystem, which may fall on a different second.)

All comments

Greg-DB

iron_horse

Thank you for the reply, @Greg-DB ! Reading the API documentation I thought that if you do not explicitly set the client_modified timestamp then it will be set automatically concurrently with the server_modified timestamp when the file is committed to the Dropbox filesystem, so I expected them to always be the same. But from your reply I gather that the process goes roughly like this:

1. initiate upload -->> 2. upload binary data -->> 3. client reports upload complete and sets client_modified timestamp -->> 4. server hashes the data, commits it to filesystem and sets server_modified timestamp when it's done with processing

I guess that there is a half second delay or so between 3. and 4., thus depending on the exact upload completion time it appears to either fall on the same second or the next one due to timestamp granularity not going beyond 1 second. For example, if client_modified value was let's say T02:10:02.100 and server_modified T02:10:02.600 then both "modified" timestamps would appear the same when rounded up to 1s, but in case of my example upload above it might have been something like client_modified T02:10:02.900 and server_modified T02:10:03.200.

Is my thinking correct?

Greg-DB

I can't make any guarantees on the implementation of the Dropbox API servers, but yes, I expect this is about what is happening.