Cloud storage

Import CSV or Parquet data from S3, Google Cloud Storage, or Azure Blob Storage.

Cloud storage imports read objects matching a path prefix and combine the results into a dataset. Choose the prefix carefully: files beneath it should use the intended format and compatible columns.

Authentication and data flow

Create the credential in Dawn, then select it in your storage connection. During an import, Dawn uses that credential to list and download objects from the provider.

Provider	How the import authenticates
Amazon S3	The AWS client signs requests using the saved access key ID and secret.
Google Cloud Storage	The Google client authenticates as the service account represented by the saved key.
Azure Blob Storage	The Azure client uses the saved storage account key for Shared Key authentication.

File contents are downloaded into the import’s processing environment before being parsed and saved as a dataset. The prefix controls which objects are requested; source-side permissions control which objects the credential can access. These imports do not require an interactive sign-in for each run.

Run the import again to read changes in the source files. Revoking the key or service account’s access stops future reads using it, but leaves existing Dawn datasets in place.

Connection and permission requirements

Source	Connection fields	Credential	Read access
Amazon S3	Bucket and optional Default prefix	AWS access key ID and secret access key	s3:ListBucket for the bucket and s3:GetObject for the relevant objects; encrypted objects may also require KMS permissions.
Google Cloud Storage	Bucket and optional Default prefix	Google service account key	Object list and get permissions on the bucket, commonly provided by roles/storage.objectViewer.
Azure Blob Storage	Account URL, Container, and optional Default prefix	Storage account name and account key	Shared Key access to the storage account.

AWS documents the permissions required by S3 operations. Apply source-side policies to limit access to the data intended for Dawn.

The current Azure connector uses an account key. This grants broader access than a read-only credential for one container. A container or prefix entered in Dawn selects data; it does not restrict the account key. Review Microsoft’s account-key guidance with your administrator.

Create the connection

Open Connections → New connection, select the storage source, enter its fields, and create or select the credential. For Azure, an Account URL looks like https://exampleaccount.blob.core.windows.net.

Choose a descriptive connection name and save it. Keep the credential value in the credential form.

Configure the import

Specify the path and file type in the import step. Example paths are:

s3://example-bucket/sales/2026-08/
gs://example-bucket/sales/2026-08/
https://exampleaccount.blob.core.windows.net/exports/sales/2026-08/

The connectors list objects by prefix. Even a path that looks like one filename is treated as a prefix, so verify that it does not also match backup files or similarly named objects. Directory markers are skipped; do not rely on the connector to filter a mixed directory by file extension.

Select CSV or Parquet to match the actual files. Do not mix these formats under the selected prefix. For CSVs, use consistent headers and encoding across the files.

Build an import pipeline from this storage connection for the sales/2026-08/
prefix. The files are CSVs with the same columns. Write monthly_sales_raw.
Show me the selected path and file type before running it.

Verify the result

After the run, compare the row count with the expected combined files. Check duplicate records, the date range, and any columns that were missing from some files. Combining files does not establish that the records are unique.

For recurring imports, decide how late files, replacements, and previously loaded records should be handled in your pipeline.

Troubleshooting

• Access denied while listing: check bucket/container listing access, not just access to one object.
• Access denied while downloading: check object permissions and any encryption-key permissions.
• No data: check the bucket, container, prefix, and letter case.
• Parsing error: look for an unexpected file or a mismatch between the selected format and actual contents.
• Unexpected duplicates: check overlapping exports and repeated snapshots under the prefix.

For rotation and revocation, follow permissions and credentials.