Child Item

How can I prevent very long syncing time of files in large folders?

I have numerous folders in Dropbox, and the largest one is my Outlook Backup of office emails over many years, which is about 40GB. Occasionally, like today, I have to go in and save a new email, and the resulting syncing and indexing can take literally hours, which really slows down my PC. How can I prevent this very long syncing time of files in large folders?

Find more posts tagged with

Desktop

Sync

Insightful

Closed - Partially Delivered

Accepted answers

Rich

@Akiva wrote:
So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.

I do exactly what you do; open a separate PST file and manually move messages to it. This file is not open in Outlook all the time; only as needed. While doing this and storing the file in Dropbox, there's nothing you can do about the sync times. Here's why ...

Dropbox works by taking your files and breaking them into smaller chunks (4Mb). It then creates a hash for each chunk and syncs those chunks to the Dropbox servers. If you edit a file, Dropbox again breaks the saved file into chunks, hashes them and then compares the hash of each chunk to those stored online. Any chunk that is different is synced. This makes it so you can take a large file, say 100Mb, edit it and only sync a few Mb because Dropbox only syncs the portion that has changed.

For that to work, the majority of the file needs to remain unchanged and then Dropbox can sync just the changed bits. If a program re-writes the entire file during the save process, or a large enough portion of it, then every 4Mb chunk of that file will likely be different from those saved online, so Dropbox will need to sync each and every one, resulting in the full file being re-synced and re-indexed.

I don't know how Outlook saves a PST file but I do know that enough of the file appears to change for Dropbox to have to re-index the entire file. To demonstrate this I just did a binary comparison of two PST files where the only difference between them was one email message. Basically, I exited Outlook and made a copy of the PST file. I then opened Outlook, added one new message to the file, then exited Outlook and made another copy of the PST file. I then did a binary compare of these two files to show the differences between them. The files are displayed side by side. Everything you see in red is different from the file on the opposite side. That's a lot of change for just one new message being added to the PST.

If your PST files are large, it's going to take time to index and sync. Most of mine are in the 4-5Gb range, and each is just a calendar year of mail, so they sync fairly quickly. If you're just dumping everything into a single PST file, perhaps you should consider splitting them in a similar manner.

All comments

Mark

Simply put... dont put them in Dropbox or get a faster internet connection.

In fact I cannot strongly recommend enough that you do NOT sync your Outlook folder in Dropbox - there is a very real and high risk of data corruption. Outlook doesnt like other programs accessing its data files while its open so if Dropbox tries to sync it could cause a loss, or, if you open it on another device before the syncing has finished you will get a a conflicted copy and have to manually fix it - no software will be able to do that for you.

Akiva

Thanks for responding, but let me explain in a bit more detail. I am not syncing Outlook. Yes, I know that's a problem. What I do is open another Outlook data file within Outlook - the Outlook backup file that's in Dropbox - and save my new email in the job folder. I then close the second Outlook (backup) file. So I'm back where I started, except for the updated email saved in the Outlook backup file in Dropbox. It's then that the long long syncing starts, for a simple additional email of a couple of MBs.

Does that explain my problem better?

Mark

Personally... I'd still not be doing it - Outlook at Dropbox (in fact any cloud solution dont play well)!

Dropbox actually only syncs the 4mb changes of a file, but, as Outlook probably updates huge chunks just being opened it has to resync all of those 4mb blocks.

Akiva

So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.

Rich

@Akiva wrote:
So what do you recommend? I really want / need to have an archive of all my past emails, and I've previously had problems when hard drive suddenly fail.

Akiva

What a great and detailed reply. And thanks for doing the research. Guess I have no choice but to split it into smaller files, but the problem then will be that instead of showing 14 job files, I will have to show about 350 job files every time I open Dropbox. But it's definitely better than the current sitch.
So thanks again.