searchContinueV2 seems to not work on the last batch it should find

tfrysinger

I am using the Java2 SDK to perform a search for directories matching a pattern, then deleting them. The directories are named as:

 

ROOT_DIR_CONSTANT/p_<YYYYMMDD>/p_<ID 1>_<SUB ID 1>

 

For example here are three directories:

 

/TestingRootDir/p_20211018/p_44c92de4-ddd6-42dc-823a-9dkkf35j5sds_test001

/TestingRootDir/p_20211018/p_44c92de4-ddd6-42dc-823a-9dkkf35j5sds_test002

/TestingRootDir/p_20211018/p_44c92de4-ddd6-42dc-823a-9dkkf35j5sds_test003

 

You will notice that all 3 are identical except for the last underscore ("_") section.

 

Here is my code:

 

String shareDir = "p_20211018/p_44c92de4-ddd6-42dc-823a-9dkkf35j5sds";
SearchV2Result result;
DbxClientV2 dbx;
try {
dbx = DropboxClientFactory.getClient();
SearchV2Builder searchV2Builder = dbx.files().searchV2Builder(shareDir);
SearchOptions.Builder optionsBuilder = SearchOptions.newBuilder();
    optionsBuilder = optionsBuilder.withFilenameOnly(true)
                                   .withMaxResults(1L)
                                   .withPath("/TestingRootDir/" + subDirToCheck);
    searchV2Builder = searchV2Builder.withOptions(optionsBuilder.build());
    result = searchV2Builder.start();
List<SearchMatchV2> matches = result.getMatches();
boolean keepGoin = true;
while (matches.size() > 0 && keepGoin) {
for ( SearchMatchV2 match : matches) {
MetadataV2 meta = match.getMetadata();
String path = meta.getMetadataValue().getPathDisplay();
DeleteResult dResult = dbx.files().deleteV2( path );
Timber.d("Lifecycle: found match: %s, delete result: %s", match.toString(), dResult.toString());
        }
if ( result.getHasMore() ) {
            result = dbx.files().searchContinueV2(result.getCursor());
            matches = result.getMatches();
        } else {
            keepGoin = false;
        }
    }

} catch (Exception e) {
Timber.d("Lifecycle: searchV2 had error: %s", e.getMessage());
}

What I am seeing is:

 

1. The call to searchV2Builder.start() finds the first directory, and result.getHasMore() returns true. 'matches' has a single item, which is the directory ending in 'test003'. This is all correct - the limit I specified in the search was '1L', so it should have matched just one, and since there are 3 directories there that would match, 'getHasMore()' should return true.

2. The for loop is entered and the delete on the first match within 'matches' works correctly, the 'test003' directory is removed.

3. After the single match item in 'matches' is processed, the for loop exits.

4. Since result.getHasMore() returns TRUE, and the searchContinueV2() is called and works correctly, so now the variable 'matches' contains a new single item, which is the directory ending in 'test001'. result.getHasMore() is TRUE which is also correct, since there is still one more directory that matches the search criteria.

5. Steps 2-3 above are executed and work correctly, the 'test001' directory is removed.

6. Step 4 occurs and since result.getHasMore() returns true, the searchContinueV2() is called. However this time, while getHasMore() is set to FALSE within the returned result (which is correct), there is no entry in 'matches', i.e. it is empty. The third and final directory was not found! WHY NOT?

 

If I change the code to not use the searchContinueV2, but instead just have it continuously call searchV2Builder.start().getMatches(), then it works as expected (i.e. all 3 directories are found and deleted):

 

List<SearchMatchV2> matches = searchV2Builder.start().getMatches();
while (matches.size() > 0 ) {
for ( SearchMatchV2 match : matches) {
MetadataV2 meta = match.getMetadata();
String path = meta.getMetadataValue().getPathDisplay();
dbx.files().deleteV2( path );
    }
    matches = searchV2Builder.start().getMatches();
}

 

OR if I change the 'batch size' from 1L to 2L, and I either keep my initial directory list to 3 or increase it by adding a 'test004' directory the original coded way using searchContinueV2 still fails.

 

With a batch size of 2 and 5 directories that should match, it succeed in finding the first batch of 2 (directories ending in 005 and 004), then upon utilizing the cursor it found a second batch of only 1 (directory ending in 002).  

 

With a batch size of 2 and using the second approach, I got 3 cycles through the loop with the last 'matches' containing a single item (the fifth item) and so all directories were properly deleted.

 

Thanks for any help.

Greg-DB

The Dropbox API search functionality is not always guaranteed to completely/immediately return results, especially when changes are actively being made, due to indexing and caching.

 

That being the case, for a scenario like this, I recommend using listFolder/listFolderBuilder and listFolderContinue instead as that functionality is not subject to the same limitations.