Child Item

Expired Access Token

Hi all,

I'm currently working on Dropbox API to upload a backup file (.tar.gz) of 25GB - 27GB. Earlier the dropbox API was working fine with the long-lived access token.

Below are the steps which I have followed to upload a backup file in chunks of 4194304

1. Generate Access and Refresh Token using dropbox oauth2/token

https://www.dropbox.com/developers/documentation/http/documentation#oauth2-token

2. Starting the upload session by calling the /upload_session/start

https://www.dropbox.com/developers/documentation/http/documentation#files-upload_session-start

3. Appending the backup file to the upload session in chunks of 4194304

https://www.dropbox.com/developers/documentation/http/documentation#files-upload_session-append

The upload usually completes after 5-7hrs, due to the short-lived token the upload process is interrupted after 4hrs.

The curl call made to /upload_session/append after 4hrs is throwing an expired_access_token error.

I have tried to renew the access token, using the below API after receiving the expired_access_token error in the curl call:

https://www.dropbox.com/developers/documentation/http/documentation#oauth2-token

The access token was successfully renewed and tried to resume the upload by providing the access token to upload_append API.

After the curl call made to upload_appened API, I now receive a "500 error page" and am unable to continue the upload process.

Need help for me to complete the backup process.

how to proceed with uploading a backup file that takes more than 4hrs to complete.

Find more posts tagged with

API

Abuse

Comments

Greg-DB

The upload process should continue to work the same way even once you switch to a new short-lived access token. A 500 error would indicate something went wrong on the server though.

First, can you double check you're still only sending 4194304 bytes per request on the requests that fail?

If so, I'll need to look into this further, but I'd need some more information. In that case, please reply with:

the name and version number of the platform and SDK/library you are using, if any
the steps to reproduce the issue, including relevant code snippet(s) and parameter value(s), but don't include any access or refresh token(s)
the affected account ID(s)
a sample of the unexpected error/output, including response headers
whether or not the request succeeds if you retry it

Feel free to open an API ticket if you'd prefer to share privately.

DloadJunior

Hi @Greg-DB

Thank you for your quick response.

>>the name and version number of the platform and SDK/library you are using, if any

I'm not using SDK/library, I have made a curl function to send/receive the data from Dropbox API. I'm using the PHP Stream Wrapper class to upload and download the backup file. Below is the code snippet of the curl function

function __curl($url, $headers = '', $filepointer = '', $upload_size = 0, $post = '', $download_file = '', $ignore_errors = 0, $retry = false){
		global $error, $l;
		
		// Set the curl parameters.
		$ch = curl_init($url);
		
		if(!empty($headers)){
			curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
		}
		
		curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
		
		if(!empty($filepointer)){
			curl_setopt($ch, CURLOPT_PUT, true);
			curl_setopt($ch, CURLOPT_INFILE, $filepointer);
			curl_setopt($ch, CURLOPT_INFILESIZE, $upload_size);
		}
		
		if(!empty($post)){
			curl_setopt($ch, CURLOPT_POST, 1);
			curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
		}
		//curl_setopt($ch, CURLOPT_VERBOSE, TRUE);

		// Turn off the server and peer verification (TrustManager Concept).
		curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
		curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
		
		if(!empty($download_file)){
			curl_setopt($ch, CURLOPT_FILE, $download_file);
		}
		
		// Get response from the server.
		$resp = curl_exec($ch);
		
		echo '<br />Resp: ';
		r_print($resp);
		echo '<br />Curl Error:';
		$curl_error = curl_error($ch);
		echo $curl_error;
		$errno = curl_errno($ch);
		r_print($errno);
		$error_message = curl_strerror($errno);
		r_print($error_message);
		
		$http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
		curl_close($ch);
		echo "HTTP_CODE: ".$http_code;
		
		$result = json_decode($resp, true);
		r_print($result);

		
		if(!empty($result['error']) && empty($ignore_errors)){
			if($result['error']['.tag'] == 'invalid_access_token'){
				$error[$result['error']['.tag']] = $l['invalid_access_token'];
			}elseif($result['error']['.tag'] == 'incorrect_offset'){
				$error[$result['error']['.tag']] = 'Incorrect Offset';
			}elseif($result['error'][$result['error']['.tag']]['.tag'] == 'insufficient_space'){
				$error[$result['error'][$result['error']['.tag']]['.tag']] = $l['insufficient_space'];
			}elseif(!empty($result['error'][$result['error']['.tag']]['.tag'])){
				$error[] = $result['error'][$result['error']['.tag']]['.tag'];
			}else{
				$error[$result['error']['.tag']] = $result['error']['.tag'];
			}
			
			return false;
		}
		
		return $result;
	}

function stream_write($data){
		global $error;
		
		if(!is_resource($this->wp)){
			$this->wp = fopen($this->tpfile, 'w+');
		}
		
		//Initially store the data in a memory
		fwrite($this->wp, $data);		
		$this->tmpsize += strlen($data);
		$data_size = strlen($data);
		
		// Are we already more than 4 MB ?
		if($this->tmpsize >= 4194304){
			
			rewind($this->wp);
			
			//Call upload append function to write the data from PHP Memory stream to Dropbox
			$this->upload_append($this->session_id, $this->wp, $this->tmpsize);
			
			// Close the temp file and reset the variables
			fclose($this->wp);
			$this->wp = NULL;
			$this->tmpsize = 0;
		}
		
		return $data_size;	
	}

// Dropbox API to upload
	function upload_append($session_id, $filep, $data_size){
		global $error;
		
		$args = json_encode(array('cursor' => array('session_id' => $session_id, 
											'offset' => $this->offset), 
								'close' => false));
		
		$upload_url = 'https://content.dropboxapi.com/2/files/upload_session/append_v2';
		$headers = array('Authorization: Bearer '.$this->access_token,
						'Dropbox-API-Arg: '.$args,
						'Content-Type: application/octet-stream');
		
		$resp = $this->__curl($upload_url, $headers, $filep, $data_size);
		
		if(is_null($resp)){
			$this->offset += $data_size;
			return $data_size;
		}
		
		return false;
		
	}

>>the steps to reproduce the issue, including relevant code snippet(s) and parameter value(s), but don't include any access or refresh token(s)

I guess the issue can be reproduced by uploading a file large in chunks of 4MB which could take more than 4hrs to complete

I have previously opened a ticket, in which you can find the error I'm receiving, below is the ticket id for the same:

Ticket #17124496

Awaiting your response.

Greg-DB

Thanks! I see that ticket was automatically closed after some time because you didn't reply to the message with a question we sent on June 7. Did you receive that message? If so, please reply to that by email to re-open that with support. Otherwise, you can open a new one here. Either way, please share the rest of the requested information so we can investigate.