Child Item

Transfer data between Dropbox and S3

Hi, I have a newbie question:

* Is it possible to transfer data from S3 to Dropbox and vice versa, ideally programmatically in Python. It would be great, if possible, to achieve it without the need to download to a local machine and then sync/upload, as the files might be huge.

* If it is doable, what is the best plan? Does Dropbox plus work fine?

Thank you very much!

Find more posts tagged with

API

Abuse

Accepted answers

Greg-DB

We can't provide support for S3 itself, but if you can get a link to a file on S3, you could then pass that link to /2/files/save_url to save the linked file to Dropbox, without downloading it locally. That's available as files_save_url in the official Dropbox API v2 Python SDK.

That can be used by any kind of Dropbox account, so any plan would be fine. You'd just need to consider how much storage space you need.

All comments

Greg-DB

That can be used by any kind of Dropbox account, so any plan would be fine. You'd just need to consider how much storage space you need.

ygnmax

I have a huge database (total size is more than 200T), which is stored on Amazon AWS S3, and I want to transfer that data from AWS S3 to Dropbox business. Is there any effective and efficient way to achieve it?

Greg-DB

@ygnmax We don't have anything built specifically for S3 like this, so if you're a programmer and want to write something to do this, the method in my previous comment would probably be a good option to use.

Kirsten2001

Hi. If you don't mind me asking, did you find a solution for this?

ahajmirzaian

You can use the script below to transfer your data to DropBox.

'''

import os

import boto3

import dropbox

def generate_presigned_url(bucket_name, object_name, expiration=3600): """Generate a pre-signed URL to share an S3 object""" s3_client = boto3.client('s3') try: response = s3_client.generate_presigned_url('get_object', Params={'Bucket': bucket_name, 'Key': object_name}, ExpiresIn=expiration) except Exception as e: print(e) return None return response

def save_url_to_dropbox(access_token, dropbox_path, url): """Save a file from URL to Dropbox""" dbx = dropbox.Dropbox(access_token) try: result = dbx.files_save_url(dropbox_path, url) print(f"File transfer initiated to Dropbox: {result}") except Exception as e: print(e)

def transfer_s3_to_dropbox(bucket_name, dropbox_folder, access_token): """Transfer all files from an S3 bucket to a Dropbox folder""" s3_client = boto3.client('s3') paginator = s3_client.get_paginator('list_objects_v2') pages = paginator.paginate(Bucket=bucket_name) for page in pages: if 'Contents' in page: for obj in page['Contents']: object_name = obj['Key'] presigned_url = generate_presigned_url(bucket_name, object_name) if presigned_url: dropbox_path = f'/{dropbox_folder}/{object_name}' save_url_to_dropbox(access_token, dropbox_path, presigned_url)

AWS CLI config) os.environ['AWS_ACCESS_KEY_ID'] = 'your was key here' os.environ['AWS_SECRET_ACCESS_KEY'] = 'your aws key here'

bucket_name = 'your s3 bucker name here'

access_token = 'your token'

dropbox_folder = 'path to drop box folder'

transfer_s3_to_dropbox(bucket_name, dropbox_folder, access_token)

'''