Import/export data using Amazon S3

Amazon S3 is a simple storage service that you can use to store and retrieve data from anywhere on the web. Similar to FTP/SFTP, it lets external tools and services store and retrieve large amounts of data.

Recommended use

You should use Amazon S3 for uploads of data to ODP on a scheduled basis from external systems. Examples of recurring uploads include:

• Product feeds
• Customer updates
• List subscriptions
• Consent updates

🚧

Important

CSV imports via Amazon S3 must adhere to the file format and name requirements outlined in Import data using CSV.

Generate Amazon S3 credentials

To use Amazon S3, you need the bucket (location) you want to access and the access key/secret for authentication. To retrieve these:

1. Go to Account Settings > Integrations.
2. Click AWS.
3. Click Generate Access Keys.

Locate your access keys and bucket URLs

Your Key ID and Secret Access Key display , which you can copy and paste into either the command-line interface or a third-party application to complete the Amazon S3 integration with ODP.

Your Amazon S3 bucket URLs also display here. All clients have an Amazon S3 bucket for imports and another for exports. The structure of the bucket URLs is shown below:

• Data imports – s3://zaius-incoming/your_tracker_id
• Data exports – s3://zaius-outgoing/your_tracker_id

For additional help from AWS, you can install the AWS CLI tools and use the S3 command line reference guide.

🚧

Important

Data in the Amazon S3 buckets expires after 7 days.

Import from Amazon S3 to ODP using command-line

After you place a file in your Zaius-incoming bucket, ODP automatically imports it. You can fully automate the import process if you regularly create data for imports and can schedule to add that data to your S3 bucket.

Use the following command to copy a local file to Amazon S3:

aws s3 cp zaius\_customers.csv s3://zaius-incoming/<your tracker ID>/ --sse

Use the following command to copy a directory of files to Amazon S3:

aws s3 sync /tmp/yourlocaldir/ s3://zaius-incoming-temp/<your tracker ID>/ --sse

Export from ODP to Amazon S3 using command-line

Run the following command to retrieve all Amazon S3 files under the export ID you provide.

aws s3 cp s3://zaius-outgoing/<your tracker ID>/data-exports/<id> . --recursive --sse

This command outputs the contents of your requested ID to your current directory. To specify a location, replace . with the directory path. For example, to output the directory to your desktop, specify one of the following, depending on your operating system:

• OS X – /Users/<name>/Desktop
• Windows –C:\\Users\\<name>\\Desktop (Windows)

📘

Note

ODP exports are a set of files in a directory folder identified by the export ID you provide. Be sure you transfer the entire directory to get the entirety of the export.

Import/export from Amazon S3 using a third-party application

You can use whichever third-party application you prefer. Common developer tools you can use to perform uploads to Amazon S3 include:

• Cyberduck (Windows & Mac) – marketer friendly
• AWS CLI (Windows, Mac and Linux)
• AWS SDKs (Java, Python, Node.js, PHP and more)

The following instructions are for Cyberduck, which is a free cloud storage browser that you can use with Amazon S3.

1. Download and launch a free tool, like Cyberduck.

2. In Cyberduck, expand Action and click New Bookmark.

3. Select Amazon S3 from the drop-down list at the top of the pop-up window.

4. Optionally enter a Nickname.

5. Enter the Access Key ID (the Key ID from ODP).

6. Enter the Secret Access Key (the Secret Access Key from ODP).

7. Expand More Options and enter the Data Exports bucket URL from ODP into the Path field. Remove s3:/ from the beginning of the URL, leaving only one forward slash.

   📘

   Note

   To locate your exact URL, go to Account Settings > Integrations > AWS.

8. Close the pop-up window to save the new bookmark.

📘

Cyberduck documentation

For information about using Cyberduck to import AWS files, see their documentation.