Child Item

sync only one folder on linux

I have a huge dropbox business account, and I'd like to sync one "deep" folder on a linux client. This folder is deep in the folder hierarchy, so for example the folder: "folder 1/folder 2/folder 3" is the ONLY one I want downloaded to the linux client.

I tried: dropbox.py exclude add * ; dropbox.py exclude remove 'folder 1/folder 2/folder 3'

and various permutations of that, but I always still end up with:

dropbox.py exclude list
Excluded:
folder 1

What's the magical set of commands I need to issue to exclude everything - except one deep folder?

Or, do I need to create local paths for this deep folder on the client first?

Find more posts tagged with

Desktop

Sync

Insightful

Closed - Partially Delivered

Comments

Walter

Hey @Jeff_S1; welcome aboard and thanks for using Dropbox!

Just to make sure we're on the same page, can you please let me know if you're fully synced on this machine by running the status command?

Keep us posted!

Jeff_S1

No, I have not fully synced. that would take more than 10TB of data, of which I only need about 600GB (the one deep folder). I'm trying to avoid that huge, unneeded sync. So, this is a new, empty machine, that I've installed dropbox.py and activated - but has not synced.

Здравко

Hi @Jeff_S1,

Seems, you are thinking in exactly opposite way instead of what "Selective Sync" implementation assumes to. 🙂 As seems you have noticed (to some extent) that using command (or GUI) folder tend to be excluded, not included, so how exactly you can add a subfolder in a missing (excluded) folder?! 🧐 Try to do the same trick using GUI (not necessary on Linux). Do you able? 🙄 What's the problem? 😉 Using the command prompt is the same! Execute on command line the equivalent things you are doing on GUI. Unchecking particular folder means adding to "exclude list" and checking the same folder means removing from "exclude list". Take in mind that in GUI when you set check mark to a nested folder all containing folders receive check mark too. In console you have to do this by hand! This is NOT done by the control script for you! This could be object of improvements (not to say it's a bug). When adding folders you have to add nested folders from outermost to innermost folder... Or better opposite, instead of exclude all and after that to try add some of them, just exclude folders you don't want to be in sync locally. Nothing special, the same commands (even one less) in right order. 😉

Hope this gives right direction.

Jeff_S1

Ahh, OK. This makes sense. So, I need to include folder 1. Then exclude all children of folder 1. Something like:

dropbox.py exclude remove ~/Dropbox/folder 1

dropbox.py exclude add ~/Dropbox/folder 1/*

dropbox.py exclude remove ~/Dropbox/folder 1/folder 2

dropbox.py exclude add ~/Dropbox/folder 1/folder 2/*

dropbox.py exclude remove ~/Dropbox/folder 1/folder 2/folder 3

I'll give that a try...

jamespi

Did you get this to work? I have a problem where I can't sync the single folder I want, because despite excluding all the other folders, dropbox still wants to download all of them first before then deleting the excluded ones. This doesn't work because my hard drive is not big enough to sync the entire dropbox folder.

I can't find a way to set up the exclude list in advance before syncing, and having it not even attempt to download the excluded folders. You mention having a 10TB dropbox, so I imagine this is a problem you would run into as well, unless I'm just being dumb somehow.

Jay

Hi @jamespi, thanks for your message. I'd recommend contacting the support team to look into this matter in more detail.

Concerned Citizen

I would suggest this is a common use case and should be treated as a mainstream use of the product. I'm setting up a Minecraft server on Digital Ocean and just assumed it'd be easy to use Dropbox in this fashion. All I wanted to do was basically sync a directory like Games/Minecraft to this droplet, without having to first sync the entire service (which is prohibitively large for this VM). We need an --exclude * and then --include foo type of mechanism.

sydsyd

Just wondering if this question was ever resolved/fixed.

I too would like to only have one folder, on the top level, of an account synced on my Linux headless server without having to have the whole account synced first before I can exclude all other folders.

Walter

Hey @sydsyd, have you tried pausing syncing and then resuming it after having excluded the folders you don't want to be synced to your device?

sydsyd

Hi @Walter - I've found another workaround which is to just share a specific folder with another Dropbox.

This avoids syncing and excluding.

But pausing the sync after setting up the link was going to be my next step.

I think all of this can be avoided if there was a config file that could be read by the dropbox linux daemon to not sync specific folders in a list.

Walter

Glad to hear you got this sorted in the end and thanks for your feedback too @sydsyd - it's been noted in our system.

Please let us know if you have anything else to add.

jamespi

just to be clear - was your solution to create a new dropbox account, share the folder with that account, then sync that new account?

i'm asking because that workaround will only work if you're either prepared to pay for new account or the contents of the folder are small enough to fit in a free 2GB account, so i want dropbox to be aware that it would still be helpful for them to fix the issue properly

sydsyd

No, the reverse.

Create a new account that has the actual files I wanted to sync with a larger dropbox that I didn't want to sync every folder.

Concerned Citizen

Hi @sydsyd ,

Would you mind clarifying the "sharing single folder" solution? Did you have to generate a public sharing link and then use that? Or did you somehow manage to share the link *from* your Dropbox account and then again use the *same* Dropbox account in a different location with that link?

sydsyd

What I've got is a Dropbox account on a headless linux server that's FTP enabled. The linux account folder structure looks like this:

~account/Dropbox

~account/Dropbox/fred

~account/Dropbox/mary

~account/Dropbox/john

etc

I then send a share link for each of the user's folders Fred, Mary, John to the respective user.

That way, I don't need to sync Fred, Mary, John's folders to enable them to have FTP access to their dropbox.

Concerned Citizen

Hey, that is a pretty clever idea! I hadn't even thought about using Dropbox as the backend home dir sync for different system users under Linux. One quick followup question if you don't mind though. In this scenario didn't you have to share an unprotected link with each of the users in this example? I mean, those don't actually map back to 3 actually different Dropbox accounts, so Fred / Mary / John each got a shared link that they better keep super secret and private or else their files could be seen by others?

No wait, that doesn't make sense. In your situation the authentication would have been controlled at the ftp level, right? So the "link sharing" is all for your own administration of the directories on the backend. So we're back to my original question of "are you sharing links with yourself"? Sorry if I misunderstood something here; I'm really curious to know how this is working because my use case is still applicable to your approach (I'm trying to share a private wiki directory holding markdown files that I can sync local and remote on a headless Linux box).

sydsyd

I'm not sure what you mean by "share an unprotected link with each of the users".

I go to a computer with a desktop client. In my case a Mac that's logged in to the account that's syncing with the headless Linux server.

I right click on one of the folders and send a share to that user's account. I can give them edit or read only permissions. Once that user accepts, the folder will appear in that user's account.

Concerned Citizen

Got it, I understand now. Thank you @sydsyd .

sydsyd

Give it a go and let me know how you get along.

The biggest issue I had to get this working was to have a stable installation of a headless Dropbox client on the Linux box.

Robert T.26

Tried 'Pause Syncing'.

Tried limiting download bandwidth to 1.0kb.

Still hard disk is full and dropbox knows it is full.

Will not work on removing excluded folders and files first!

Need a way to prioritize the exclude operations over syncing new files.

Robert T.26

Nothing new, but SSDs smaller than Dropbox storage.

Because the filesystem is out of space, i excluded a large number of files from syncing locally to dropbox using

dropbox exclude add ~/Dropbo/ExcludeThisLargeFolder/

But dropbox does not free up that space first, it continues to attempt to download more files even though it knows it is out of space.

Does not matter if i 'Pause Syncing' or limit the download bandwidth to about zero.

Need exclude to go first.
Not sure if the Delete operation talked about above applies to files and folders you do NOT want deleted from Dropbox.com storage.

Robert T.26

i just happened to find this old bash script i wrote under ~/Dropbox/.dropbox.cache/old_files/

Because it was found under /old_files/, it is probably not the best version, so use at your own risk! Since it does not delete files, only does dropbox exclude operations, it might be safe. This was written maybe a decade ago.

Looks like it would give an error because a folder does not exist in the readlink call. mkdir -p ${T} before looping might be necessary.

The intent is to automate adding a parent folder of the deeply nested targeted folder folder you want to download.

While excluding all other folders brought in when a parent folder is added.

I should have a newer version in my old bitbucket files or github or sf, but will take a while to find it.

#!/bin/bash
echo Problems:
echo dropbox exclude remove ~/Dropbox/DontExcludeMe/MeNeither
echo does nothing unless its parent directory is first included first.
echo When the parent directory is included, it brings in all its subfolders and files. 
echo I smell a tree walk of some kind here.  
echo Is bash capable of binary trees?

echo Would have to include all the ancestors that do not exist yet
echo and then add the child directories one-at-a-time
echo but at the same time, exclude any other directories.  

D=~/Dropbox
T=~/Dropbox/DontExcludeMe/MeNeither   # T for TARGET
#mkdir -p ${T}   # May need to uncomment this to precreate the wanted nested folder.
T2=${T}; 

dropbox exclude remove ${T}  #WONT DO ANYTHING IF PARENT DOES NOT EXIST.
echo ${T2}; 
while [ ${D} != ${T2} ]; do J=$(dirname $(readlink -f ${T2} ));  #readlink CAUSES FAILURE WHEN $T DOES NOT EXIST
	echo readlink CAUSES FAILURE WHEN ${T2} DOES NOT EXIST.  Add code to test for existance of ${T2}.
	echo ${J}; 
	echo "We have to filter for the directory target ${T2}, otherwise we add it right back to be excluded!"
	dropbox exclude remove ${T2}  
	sync   #Give it a little time to come back.  Should loop for it to be back in existence.
	find ${T2} -maxdepth 1 -type d -print0 | egrep -v "^${T2}$" | xargs -0 dropbox exclude add
	T2=$J; 
done;

#find ${T} -maxdepth 1 -type d -print0 | egrep -v "^${T}$" | xargs -0 dropbox exclude add

Здравко

Hi @Robert T.26,

There are some missing features in Dropbox for Linux and there is no way this BUG to be compensated. For simplifying exclusion of everything else (as much as possible), but a desired folder, you can use following script:

#!/bin/bash
##############################################################################
#                    Minimize local folders in sync
#                    ------------------------------
# Evaluation is performed based on passed folder that has to stay local.
# Other files/folders get keep if/when needed only.
# Verified on Dropbox v170.4.5895
# Just make it executable (if need, using
#   $ chmod a+x stayAloneInDropbox
# ) and run it.
# Author: Здравко
#   www.dropboxforum.com/t5/user/viewprofilepage/user-id/422790
##############################################################################

# General command line check.
errorMesage() {
  echo "Command line should be 'stayAloneInDropbox <folder in Dropbox>'"
  echo "  where <folder in Dropbox> should be a folder you want to stay in"
  echo "  local Dropbox folder as is comletely, but everything else limited"
  echo "  as much as possible."
  echo
  echo "$@"
  exit 1
}
if [ $# -ne 1 ]
then
  errorMesage "Expected arguments count 1, provided $# -> $@"
fi
target_folder=`realpath "$1"`
EOL=$'\n'

##############################################################################
# Command argument validation.
if [ ! -d "$target_folder" ]
then
  errorMesage "😯 The argument ($1) does NOT point a folder."
fi

##############################################################################
# Figuring out local Dropbox folder location.
dbx_dir=`jq -r .personal.path ~/.dropbox/info.json`
if [ $? -ne 0 ]
then
  echo "Cannot detect Dropbox application installed and worked local."
  exit 1
fi
if [ ! -d "$dbx_dir" ]
then
  echo "Incorrect Dropbox folder path ($dbx_dir)!"
  echo "Only individual Dropbox account is supported."
  exit 1
fi

##############################################################################
# Dropbox folder validation.
if [[ "${target_folder:0:${#dbx_dir}}" != "$dbx_dir" ]]
then
  errorMesage "🤦 The pointed folder ($1) does NOT reside within your${EOL}\
Dropbox folder ($dbx_dir)."
fi
if [[ "$target_folder" == "$dbx_dir" ]]
then
  errorMesage "Nothing to do. Dropbox folder itself is always there!${EOL}\
Point some subfolder there."
fi

##############################################################################
# Process nested folders.
while [[ "$target_folder" != "$dbx_dir" ]]
do
  parent_folder=`dirname "$target_folder"`
  template=`basename "$target_folder"`
  echo
  echo "Keep only subfolder '$template' and remove all others"
  echo "in the folder '$parent_folder'..."
  template="/$template"
  if [[ "$parent_folder" == "$dbx_dir" ]]
  then
    template="${template}|/.dropbox.cache"
  fi
  find "$parent_folder" -mindepth 1 -maxdepth 1 -type d -print0 | \
    egrep -z -v "($template)\$" | xargs -0 dropbox exclude add
  target_folder="$parent_folder"
done

echo
echo "Complete! 😉"
echo

Take in mind that it wont fix any existing issue, but just simplifying the exclusion process. 🤷 Bugs can be fixed by Dropbox developers only.

Hope this helps to some extent. 😉