Child Item

Some files not syncing from Mac computer to Ubuntu but others do

Dear all,

I have been banging my head against the wall with this one...

Some of my files do not sync to my Ubuntu computer if I create them in my Mac computer -other do without any problem- but all files I create in my Ubuntu show up in my Mac computer. I have the global settings on synchronize all files and folders in both computers.

I have cleaned both dropbox caches, I have stopped and restarted dropbox in both computers. They all show up in the dropbox site so they are all being synchronized to the dropbox servers. It's just the direction from Mac to Ubuntu that sometimes works and sometimes not!

Any ideas? Thoughts?

Thanks in advance for any help you might provide,

Bruno

Find more posts tagged with

Desktop

Sync

Mac

Dropbox Basic

Linux

Insightful

Closed - Partially Delivered

Comments

Rich

@BrunoJ wrote:
Some of my files do not sync to my Ubuntu computer if I create them in my Mac computer ...

Problem with the file names?

https://help.dropbox.com/organize/file-names

BrunoJ

Thank you Rich!

It is not the file names but I have narrowed it down to being a problem of files created by Photoshop! It's strange, it only happens with those files. For the moment I have found a workaround solutions -albeit very time-consuming. I download the file I want to use in my Linux computer from the dropbox website to a different folder (just to avoid conflict versions) and then I can use it without any problem in the Linux computer. However, this is not a solution in the long run, I need to either stop using photoshop or stop using dropbox for my work...

Megan

Hey @BrunoJ, happy Friday, I hope you're doing well!

Interesting behavior you got going on there. Can I ask the version of the Dropbox app that's installed on both devices? All you have to do in order to find that, is to locate your Dropbox icon, next to your WiFi and hover your mouse there.

We want to make sure that both of the versions are the latest stable ones. Also, what is the app's syncing status on both of them when this happens?

Keep me posted, and we'll take it from there!

BrunoJ

Hi Megan,

Both versions are v159.4.5870 and all files sync to the server quickly. That is why I can download them after to my linux computer with my "go around" solution.

If you have any idea of what could be causing this behavior with the photoshop files please let me know. Meanwhile, I've been meaning to switch to using Gimp as an alternative to Photoshop so I just found the perfect justification to get moving 🙂

Best regards,

Bruno

Hannah

Thanks for sharing your workaround with our Community, @BrunoJ!

We'll actually leave the thread open, in case another user has some more info for you.

Let us know if you have any other questions.

FesT

Hey,
I have the same problem! I am trying to sync my entire Dropbox on a newly set up Linux (The Linux Dropbox client is running version 159.4.5870).
The sync works great, only some files (seems like .jpg .png .tiff .psd) probably saved with an older Photoshop version cannot be downloaded to the Linux machine. If there are too many of these files that are not "downloadable", Dropbox shows that it is syncing, but nothing happens anymore.
On the Mac everything works fine.
Unfortunately, @BrunoJ workaround doesn't work for me, because I have about 2TB of files already on the Dropbox server and I can't open all files and save them again.

Any Idea how to solve that issue?

Здравко

Hi @FesT,

Hm.. 🤔 That's something really strange. Can you share (shared link) some .png or .jpg that doesn't contain private info (or just some dummy picture) and definitely cannot (make it sure) sync? Available example might give some ideas... Let's hope.

FesT

Hi @Здравко,

it's hard to say which images definitely don't sync, because it can always be that they are simply in the queue...

But with this files I am pretty sure:

https://www.dropbox.com/s/bkxcwioxmm124r8/Skies0238_L.jpg?dl=0

And some additional information, this ist how it looks in Nautilus:

Screenshot 2022-10-26 at 20.11.37.png

at the same time the terminal says syncing 3 files even there are 5 files to sync:

Screenshot 2022-10-26 at 20.11.58.png

Would be great if you find a solution!

Здравко

Hi @FesT,

Unfortunately I can't find solution, but can confirm that there is a 2 things. First one is that Dropbox application definitely behaves wrong. Like your findings I can see the same:

Another interesting thing is that downloading the folder "does not sync" and adding the downloaded to Dropbox, it syncs back successful:

The second thing (even more interesting) is both "does not sync" folders are binary identical (and not only). This means that there are undocumented properties (most probably) that affect application behavior.

Let's hope the moderators will forward this to development, so be solved the issue (and documented). But,... better don't rely too much.

I want to tag @Greg-DB here; Is some API features/access point able to show up the differences in files/folder properties affecting this situation? I tried /2/files/list_folder for both "does not sync" folders showing the same, but the state is not actually the same (as can be seen)! 🤔🤷 Any thoughts? How can such a situation be detected/predicted? Thanks Greg!

Good luck, to all of us.

Greg-DB

@Здравко Unfortunately I don't think the API has anything that would be useful for this. Anyone seeing an issue like this may want to contact support directly.

Здравко

@Greg-DB wrote:
... Anyone seeing an issue like this may want to contact support directly.

😁😁😁 Greg, that has to be the funny part. Do you really think somebody there can help? Rhetoric, of course. It's clear there is an application bug, at least.

Anyway...

My question was - as there are some criteria/flags forcing application to sync or not, associated to each file (2 copy of the same file - one sync the other no and without visible reason way) - is there a way those criteria/flags be read (might appear somewhere as part of some call result). I will interpret your answer as "mentioned feature is not accessible throughout Dropbox API and only available for internal use".

Thanks.

Здравко

Hi @FesT,

As a temp workaround (till possible solution, if any) you can use following script:

#!/usr/bin/bash
##############################################################################
#                  Workaround for Dropbox broken sync
#                  ----------------------------------
# Verified on Dropbox v159.4.5870
# Just make it executable (if need, using
#   $ chmod a+x dropbox_syncfix
# ) and run it.
# Author: Здравко
#   www.dropboxforum.com/t5/user/viewprofilepage/user-id/422790
##############################################################################

# General command line check.
if [ $# -eq 0 ]
then
  echo "Command line should be 'dropbox_syncfix <access token>'."
  echo "The access token seems missing!"
  echo "You can get it here: https://dropbox.github.io/dropbox-api-v2-explorer/#check_user"
  exit 1
fi
if [ $# -gt 1 ]
then
  echo "Command line should be 'dropbox_syncfix <access token>'."
  echo "Unexpected aguments passed!"
  exit 1
fi

##############################################################################
# Access token validity confirmation.
randstr=$(
tmpbase='0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
for ((i=0; i<16; ++i)); do echo -n "${tmpbase:RANDOM%${#tmpbase}:1}"; done
)
res=`curl -X POST https://api.dropboxapi.com/2/check/user -H "Authorization: Bearer $1" -H 'Content-Type: application/json' -d "{\"query\":\"$randstr\"}" 2> /dev/null | jq -r .result`
if [[ $res != $randstr ]]
then
  echo "The first argument ($1) is NOT valid access token!"
  exit 1
fi
access_token="$1"

##############################################################################
# Figuring out local Dropbox folder location.
dbx_dir=`jq -r .personal.path ~/.dropbox/info.json`
if [ $? -ne 0 ]
then
  echo "Cannot detect Dropbox application installed and worked local."
  exit 1
fi
if [ ! -d "$dbx_dir" ]
then
  echo "Incorrect Dropbox folder path ($dbx_dir)! Only individual Dropbox account is supported."
  exit 1
fi

##############################################################################
# Single file sunc. $1 - relative file path in your account; $2 - local Dropbox absolute path
sync_to_local() {
  local path="$2$1"
  echo "Processing file: '$path'..."
  local error=`curl -o /tmp/dropbox_tmp.file -D /tmp/dropbox_tmp.dump -X POST https://content.dropboxapi.com/2/files/download -H "Authorization: Bearer $access_token" -H "Dropbox-API-Arg: {\"path\":\"$1\"}" 2> /dev/null`
  if [ $? -ne 0 ]
  then
    echo "👺 Trouble fetching '$1' from Dropbox"
    return
  fi
  local file_date=`grep 'Dropbox-Api-Result' /tmp/dropbox_tmp.dump | sed 's/^Dropbox-Api-Result *\: *//' | jq -r .client_modified`
  if [[ "$file_date" == "null" ]]
  then
    echo "👹 Download '$1' error: $error"
    return
  fi
  touch -d "$file_date" /tmp/dropbox_tmp.file
  local res=`curl -X POST https://api.dropboxapi.com/2/files/delete_v2 -H "Authorization: Bearer $access_token" -H 'Content-Type: application/json' -d "{\"path\":\"$1\"}" 2> /dev/null`
  if [[ `echo "$res" | jq -r '.metadata[".tag"]'` != "file" ]]
  then
    echo "🤷 Cannot remove '$1', error: $res"
    return
  fi
  sleep 1
  while [[ `curl -X POST https://api.dropboxapi.com/2/files/get_metadata -H "Authorization: Bearer $access_token" -H 'Content-Type: application/json' -d "{\"path\":\"$1\"}" 2> /dev/null | jq .error_summary` == "null" ]]
  do
    sleep 1
  done
  mv -u /tmp/dropbox_tmp.file "$path"
  echo "'$path' processed."
}

##############################################################################
# Processing single enumeration step. $1 - current listing result; $2 - current folder relative path
enumerate_files() {
  local oldIFS="$IFS"
  IFS=$'\n'
  for path in $(echo "$1" | jq -r ".entries[] | select(.[\".tag\"] == \"file\").name")
  do
    if [ ! -f "$dbx_dir$2/$path" ]
    then
      sync_to_local "$2/$path" "$dbx_dir"
    fi
  done
  for path in $(echo "$1" | jq -r ".entries[] | select(.[\".tag\"] == \"folder\").name")
  do
    parse_folder "$2/$path"
  done
  IFS="$oldIFS"
}

##############################################################################
# Processing single folder (indirect recursion). $1 - relative folder path
parse_folder() {
  local res=`curl -X POST https://api.dropboxapi.com/2/files/list_folder -H "Authorization: Bearer $access_token" -H 'Content-Type: application/json' -d "{\"path\":\"$1\",\"include_non_downloadable_files\":false}" 2> /dev/null`
  echo "$1"
  enumerate_files "$res" "$1"
  while [[ "`echo "$res" | jq .has_more`" == "true" ]]
  do
    res=`curl -X POST https://api.dropboxapi.com/2/files/list_folder/continue -H "Authorization: Bearer $access_token" -H 'Content-Type: application/json' -d "{\"cursor\":$(echo "$res" | jq .cursor)}" 2> /dev/null`
    echo "$1"
    enumerate_files "$res" "$1"
  done
}

# Starting enumeration  with the account root.
parse_folder ''

##############################################################################
# Wiping the 'dust'.
rm -f /tmp/dropbox_tmp.file /tmp/dropbox_tmp.dump
echo
echo "Complete! 😉"
echo

Save the script in a file with name like dropbox_syncfix on you Linux. After that, open a terminal window and change the current folder to the one where you have script saved. Execute in the same shell a command like:

chmod a+x dropbox_syncfix

Now the script is ready to start. As the only argument you need access token. You can get such from here. Make sure at the same moment you're logged in the same account (in the web browser) as you used on local machine (I don't check it)! Click 'Get Token' button and a sequence of symbols will appear in field labeled 'Access Token'. Execute the script as follows:

./dropbox_syncfix PUT_YOUR_TOKEN_HERE

The script will enumerate your entire account tree folder by folder and when meet some troublesome file will try to fix it. Such a process can (and usually will) take a lot of time, so be patient. You can let your Linux machine work alone or do some other job there. The script is neither processor nor network load intensive. 😉

Hope this helps to some extent.

BrunoJ

Hi @Здравко

Many thanks for your solution! It looks awesome, but it seems to be me (a basic user) it feels too much of a hassle to do all that for I service that I pay a considerable amount of money per year. Frankly, if the problem was much bigger than just a few photoshop files I would just consider changing to another cloud/backup system. I work at an university so I could easily change to an internal system we have... I'm just lazy and as long as dropbox works I'll keep using it instead 😉

Hopeful this trend and your work will help the dropbox developers with fixing this issue?

Best regards and many thanks for your work,

Bruno

Здравко

@BrunoJ wrote:
...
Hopeful this trend and your work will help the dropbox developers with fixing this issue?
...

🙂🤔 Hm.. Good wish, but better don't rely on. My experience shows - if the application doesn't stop completely and some part of the UI doesn't become too 'ugly', Dropbox development doesn't take care. 🤷 Linux is NOT first class citizen here, so everybody should save himself, unfortunately.

My script is definitely not a final solution; it's rather workaround using 'brute force' file fixing (cosa there's not know what exactly force wrong behavior). Nowhere can be seen any documentation describing the possible file attributes leading to such thing. That's why, in my script, I'm clearing all files attributes (accessible or not - in fact files regeneration).

@BrunoJ wrote:
..., but it seems to be me (a basic user) it feels too much of a hassle to do all that for I service that I pay a considerable amount of money per year. ...

Fully agree that once a user's paying (and not only), the service should be the same everywhere (on every supported platform). 😡 Unfortunately, it's not and, as I said, everybody is looking for solution outside Dropbox (or directly another service - there are competitors).

@BrunoJ wrote:
... Frankly, if the problem was much bigger than just a few photoshop files ...

Especially for you I made some changes in above script and I'm posting it here:

#!/usr/bin/bash
##############################################################################
#                  Workaround for Dropbox broken sync
#                  ----------------------------------
# Verified on Dropbox v159.4.5870
# Just make it executable (if need, using
#   $ chmod a+x dropbox_syncfix
# ) and run it.
# Author: Здравко
#   www.dropboxforum.com/t5/user/viewprofilepage/user-id/422790
##############################################################################

# General command line check.
if [ $# -eq 0 ]
then
  echo "Command line should be 'dropbox_syncfix <access token> [<folder>]'."
  echo "The access token seems missing!"
  echo "You can get it here:"
  echo "https://dropbox.github.io/dropbox-api-v2-explorer/#check_user"
  exit 1
fi
if [ $# -gt 2 ]
then
  echo "Command line should be 'dropbox_syncfix <access token> [<folder>]'."
  echo "Unexpected number of arguments passed!"
  exit 1
fi

##############################################################################
# Access token validity confirmation.
randstr=$(
tmpbase='0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
for ((i=0; i<16; ++i)); do echo -n "${tmpbase:RANDOM%${#tmpbase}:1}"; done
)
res=`curl -X POST https://api.dropboxapi.com/2/check/user \
  -H "Authorization: Bearer $1" -H 'Content-Type: application/json' \
  -d "{\"query\":\"$randstr\"}" 2> /dev/null | jq -r .result`
if [[ $res != $randstr ]]
then
  echo "👆 The first argument ($1) is NOT valid access token!"
  exit 1
fi
access_token="$1"

##############################################################################
# Figuring out local Dropbox folder location.
dbx_dir=`jq -r .personal.path ~/.dropbox/info.json`
if [ $? -ne 0 ]
then
  echo "Cannot detect Dropbox application installed and worked local."
  exit 1
fi
if [ ! -d "$dbx_dir" ]
then
  echo "Incorrect Dropbox folder path ($dbx_dir)!"
  echo "Only individual Dropbox account is supported."
  exit 1
fi

##############################################################################
# Dealing with start folder path.

# Setting default path - the account root (empty string).
start_folder=''

# Checking for explicit start folder path.
if [ $# -eq 2 ]
then
  if [ ! -d "$2" ]
  then
    echo "😯 The second argument ($2) does NOT point a folder."
    exit 1
  fi
  start_folder=`realpath "$2"`
  if [[ "${start_folder:0:${#dbx_dir}}" != "$dbx_dir" ]]
  then
    echo -n "🤦 The pinted folder ($2) does NOT reside within your Dropbox "
    echo "folder ($dbx_dir)."
    exit 1
  fi
  start_folder="${start_folder:${#dbx_dir}}"
fi

##############################################################################
# Single file sunc.
# $1 - relative file path in your account
sync_to_local() {
  local path="$dbx_dir$1"
  echo "Processing file: '$path'..."
  local error=`curl -o /tmp/dropbox_tmp.file -D /tmp/dropbox_tmp.dump \
    -X POST https://content.dropboxapi.com/2/files/download \
    -H "Authorization: Bearer $access_token" \
    -H "Dropbox-API-Arg: {\"path\":\"$1\"}" 2> /dev/null`
  if [ $? -ne 0 ]
  then
    echo "👺 Trouble fetching '$1' from Dropbox"
    return
  fi
  local file_date=`grep 'Dropbox-Api-Result' /tmp/dropbox_tmp.dump | \
              sed 's/^Dropbox-Api-Result *\: *//' | jq -r .client_modified`
  if [[ "$file_date" == "null" ]]
  then
    echo "👹 Download '$1' error: $error"
    return
  fi
  touch -d "$file_date" /tmp/dropbox_tmp.file
  local res=`curl -X POST https://api.dropboxapi.com/2/files/delete_v2 \
    -H "Authorization: Bearer $access_token" \
    -H 'Content-Type: application/json' -d "{\"path\":\"$1\"}" 2> /dev/null`
  if [[ `echo "$res" | jq -r '.metadata[".tag"]'` != "file" ]]
  then
    echo "🤷 Cannot remove '$1', error: $res"
    return
  fi
  sleep 1
  while [[ `curl -X POST https://api.dropboxapi.com/2/files/get_metadata \
    -H "Authorization: Bearer $access_token" \
    -H 'Content-Type: application/json' -d "{\"path\":\"$1\"}" 2> /dev/null \
    | jq .error_summary` == "null" ]]
  do
    sleep 1
  done
  mv -u /tmp/dropbox_tmp.file "$path"
  echo "'$path' processed."
}

##############################################################################
# Processing single enumeration step.
# $1 - current listing result
# $2 - current folder relative path
enumerate_files() {
  local oldIFS="$IFS"
  IFS=$'\n'
  for path in `jq -r '.entries[] | select(.[".tag"] == "file").name' <<< "$1"`
  do
    if [ ! -f "$dbx_dir$2/$path" ]
    then
      sync_to_local "$2/$path"
    fi
  done
  for path in `jq -r '.entries[] | select(.[".tag"] == "folder").name' <<< "$1"`
  do
    parse_folder "$2/$path"
  done
  IFS="$oldIFS"
}

##############################################################################
# Processing single folder (indirect recursion).
# $1 - relative folder path
parse_folder() {
  local res=`curl -X POST https://api.dropboxapi.com/2/files/list_folder \
    -H "Authorization: Bearer $access_token" \
    -H 'Content-Type: application/json' \
    -d "{\"path\":\"$1\",\"include_non_downloadable_files\":false}" 2> /dev/null`
  echo "$1"
  enumerate_files "$res" "$1"
  while [[ `jq .has_more <<< "$res"` == "true" ]]
  do
    res=`curl -X POST https://api.dropboxapi.com/2/files/list_folder/continue \
      -H "Authorization: Bearer $access_token" \
      -H 'Content-Type: application/json' \
      -d "{\"cursor\":$(jq .cursor <<< "$res")}" 2> /dev/null`
    echo "$1"
    enumerate_files "$res" "$1"
  done
}

##############################################################################
# Starting enumeration  with the start folder.
parse_folder "$start_folder"

##############################################################################
# Wiping the 'dust'.
rm -f /tmp/dropbox_tmp.file /tmp/dropbox_tmp.dump

echo
echo "Complete! 😉"
echo

The primary change (addition in fact) is the ability to provide folder path in your Dropbox folder. So if your troublesome files are located on limited part of the account folders tree and you know exactly where, providing path (second argument) can speed up script significant. The time needed can be just few minutes or, on very few files, less than a minute. 😉 Now you can use a call like:

./dropbox_syncfix PUT_YOUR_TOKEN_HERE "Dropbox/path/to/the/troubles"

Everything else is like on my previous post above.

PS: I may have forgotten to mention - in the script is used 'jq' tool (able handle JSON format used by Dropbox). The tool is relatively specific and that's why usually it's not preinstalled. You may need to install it explicitly, using your preferable package manager or using following command on Ubuntu (and compatible; other distributions have equivalent commands):

sudo apt-get install jq

FesT

@Здравко

thank you for this script! Could you briefly explain what it does exactly?

Здравко

@FesT wrote:
... Could you briefly explain what it does exactly?

As I said already, main trick is recreation of every problematic file in the Dropbox namespace, so any undesirable attributes (assigned to the former file) get gone. This is the most brief variant.

Within the script itself are many comments, describing what exactly each part of the script does. You need only basic shell scripting knowledge to understand them. About the algorithm - enumeration starts from a target folder (by default the account root) and when non matching file comes up - passed for 'refresh and undressing' (if I can say so). After that the nested folders are processed in recursion, following the same rule of course. The enumeration (and other things, like download, etc) gets performed using corresponding Dropbox API access point. You can follow the links for more information. This is less brief variant.

The same is valid for both script variants. 👍 If something isn't clear enough, clarify your question.

Hope this helps.

FesT

Hi @Здравко,

thank you for the short explanation. So far I had already understood it, what I do not understand exactly is what for "undesirable attributes" are removed, and in what way they change the files.

Здравко

@FesT wrote:
..., what I do not understand exactly is what for "undesirable attributes" are removed, ...

Hm... 🤔🤷 To be honest I'm not sure what actually you are asking for. The attributes are undesirable because they break the syncing and that why they are removed! If these attributes are Ok for you, don't take a look on any of the scripts any more (you don't need them). As far as I understand, sync breaking is an issue. Am I right, or I might have misunderstanding something? 🧐

@FesT wrote:
..., and in what way they change the files.

The files themself are not changed in any way. File attributes are things "around" the file (things associated to a file, not file's integral part). Seems some Dropbox application instances (the one on Mac) attach to some files such attributes on upload (don't ask me why; seems nobody knows - including Dropbox staff). Removing these undesired attributes (hope you know already why 'undesired') seems resume syncing.

FesT

HAHA

@Здравко wrote:
...(don't ask me why; seems nobody knows - including Dropbox staff).

this is exactly what I some how tried to figure out... where these attributes are coming from, why there are there, and what happens if there are not there anymore (except that syncing is working again – which obviously is good 😁)

Anyhow thank you a lot for this great script!

SmrtSyncIsTrash

I know this is a bit old, but has this issue been resolved. (I have not investigated running the script proposed above.) I've investigated some of the missing files, and noticed that the problem files have a visible thumbnail within the MacOS finder listing; the images which do not have thumbnails seem to get synced onto the linux machine. The files which are not syncing are perhaps 6 years old, and I know that at least one was cropped (probably using PhotoshopElements, but that's really a guess)... but all have thumbnails within MacOS.