Child Item

I created 2.4M files over the limit on the desktop app and now it's stuck syncing.

Application Affected

Dropbox
Device

Dell OptiPlex 980
Operating System/Browser (if using the web)

Ubuntu Linux
Dropbox App Version (if using the app)
v237.4.5655

Question or Issue

I created 2.4M CSV files for a client, and the Dropbox app starting to upload these files. After almost 48 hours of this, I discovered that this was way too many files.

My remediation was to exclude the top-levl directory from syncing. I made that change ten hours ago, but the app is still grinding away, doing something. I tried exporting the logs to find out what it might be doing, but the logs just show up as binary data -- I don't know how to read them.

How can I sort this out?

Find more posts tagged with

Desktop

Sync

Linux

Insightful

Closed - Partially Delivered

Accepted answers

talexb

Hi there,

I followed the instructions to remove the Dropbox Application, and reinstalled it. Everything is back to normal now, which is a huge relief.

I'm a retired software developer, so I have some recommendations.
- Include in your documentation that there's a limit to the total number of files that a user can have in their Dropbox folder. I assumed (heh) that the only limit was the total amount of space. The files I was trying to add had a total space of about 3.2G -- way less than my current 2T limit, so I thought I would be fine. (Narrator: He was not fine.) The status messages also didn't make sense: if you look back on this thread, you can see the application was indexing some random number of files (OK), uploading a few dozen files (OK), and downloading lots of files (What?) Clearly, something Really Bad was going on.
- Add a safety feature that notices that a user has added too many files, and a) automatically pause syncing, and b) let the user know there's a problem. My computer was driven to the rails for 48 hours while the application tried to sync way too many files. I'm very grateful that my hard drive is still alive.
- I made the mistake of mixing my Google authentication with my Dropbox userid/password authentication when logging on to the websites (dropbox.com and dropboxforum.com) -- no doubt confusing the authentication layer. It would follow either login with an SMS verification which worked until .. it didn't, because I'd used it too many times in a particular time period. My Dropbox userid is the same as my Google userid, which may be the source of the problem. I didn't find that the "Send something to the app" feature worked either, but I just have the daemon running, so perhaps that's the issue.

Finally, I didn't see any instructions in the Dropbox documentation about deleting the cache to address my problem (and I don't think anyone mentioned it in this thread). I worried that if I did that, the application would break (of course, I would only do this when the application was stopped). That is what I wanted to do -- tell the application, "Look, forget about this directory, it was a mistake. Let's just start over."

Thanks to you and the other mods. Cheers.

All comments

Megan

Hey @talexb, let's see what we can find together about this!

The first thing I'd like for us to check is the app's current syncing status. You can find this by hovering your mouse over the little Dropbox icon in your taskbar.

Keep in mind that the file upload limit for the Dropbox desktop app is 2 TB. What was the size of the 2.4M files that you uploaded?

Let me know more, and we'll take it from there.

talexb

I had to click on the icon, but here's what I see today:

Indexing 243,614 files
Uploading 40 files
Downloading 1,412,941 files

The directory has 9.4G of data. My account is currently using 5.3% of 2.0T.

Nancy

Hi from me too, @talexb! You said in your first post that you excluded the top-level directory from syncing. Can you clarify how you achieved that?

Did you manually move these files out of your local Dropbox folder, or did you follow these steps to ignore them instead?

talexb

I opened up Preferences, clicked on Selective Sync (at the bottom of the choice list) and deselected the top level directory that was causing problems.

I didn't want to delete my local copy, because I worried that would give Dropbox a headache.

Nancy

Thanks for clarifying! Has your desktop app made any progress in the meantime? Does it seem to be syncing less files now, or is it completely stuck?

talexb

Indexing 303,015 files
Uploading 40 files
Downloading 1,353,541 files

At 1914, December 4.

The number of files it's indexing has gone up it's still uploading files (why?) ad I really don't understand why it's *downloading* anything. To quote my step-kid, "I just doesn't make sense!"

Nancy

Thanks for the extra info! Let’s try something else then.

If the app is still stuck syncing your files, you can pause its syncing next and move the new files out of your local Dropbox folder.

After that, you can resume syncing once more and wait till the Dropbox app is up to date, before taking any further actions.

When this happens, you can try adding back a small batch of your files, wait till they’re fully synced to your Dropbox account online, and then selectively sync them, if you don’t need them to be stored locally on your Dropbox app.

Once the desktop app is up to date again, you can go ahead and follow the same steps for the next batch of files you want to upload to your Dropbox account.

In any case, the most important thing is to let the Dropbox app become up to date before moving on to the next step and to keep less than 300k files stored within the Dropbox folder, in order to avoid any further syncing/performance issues.

talexb

Thanks for this reply. I've done what you suggested, and now .. we wait to see how the application handles things. The local directory has been moved, and as you see from an earlier message of mine, I've told Dropbox not to sync that directory anyway.

The latest stats:

Indexing 18 files ..
Uploading 40 files ..
Downloading 951,602 files ..

I still don't understand why Dropbox would be downloading any files. I'll check the situation in 24 hours.

talexb

I thought I left my machine logged in last night, just locking it -- but it appears I may have logged out. No matter, logged in again, ad Dropbox started up, ran OK for a while, then used up all of my memory, and crashed.

So I tried logging in to dropbox.com, and the login link failed. Status page showed no problems. This is starting to be a real disaster for me.

talexb

The story so far: Our hero created 2.4M files in a Dropbox directory, not realizing that there's a soft limit of 300K files. After running for about 48 hours, the Dropbox app failed. The hero wrote on dropboxforum.com, trying to figure out a solution.

Eventually, he decided to delete the files locally, and was able to log in to Dropbox on the web using his MBP. He deleted the same files on Dropbox. He rebooted his Linux machine, guessing that the application would check local and remote, and figure out that everything was up to date.

The Linux machine started up fine, and Dropbox did its initialization .. and then went back to uploading files (from a local queue, maybe?) Our hero decided to leave things running, and came back to find that Dropbox had used up all of the memory, all of the cache, and then crashed.

At this, our hero is contemplating what to do next. Uninstall Dropbox (hopefully wiping out the queue?) and then reinstall? Will the queue survive the uninstall/reinstall? Can he clear the queue without doing the uninstall/reinstall dance?

Walter

Thanks for the update on this @talexb - much appreciated.

At this point, I'd suggest logging out of the Dropbox desktop app through the app's preferences, delete your cache folder and then try to log in again.

Let me know how it goes.

talexb

Hi there,

I followed the instructions to remove the Dropbox Application, and reinstalled it. Everything is back to normal now, which is a huge relief.

Thanks to you and the other mods. Cheers.

Hannah

Hey @talexb, thanks for taking the time to update us!

I'm glad to see that you managed to resolve your issue.

We also really appreciate your feedback and suggestions; I passed everything along to our team.

If you come across any other issues, don't hesitate to let us know.

Have a great day!