> ## Documentation Index
> Fetch the complete documentation index at: https://docs.truestate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloud storage

> Import CSV or Parquet data from S3, Google Cloud Storage, or Azure Blob Storage.

Cloud storage imports read objects matching a path prefix and combine the results into a dataset. Choose the prefix carefully: files beneath it should use the intended format and compatible columns.

## Authentication and data flow

Create the credential in Dawn, then select it in your storage connection. During an import, Dawn uses that credential to list and download objects from the provider.

```mermaid theme={null}
flowchart TD
    A["Saved storage credential"] -.->|"Authenticate requests"| B["Dawn import"]
    B -->|"List objects under the prefix"| C["Cloud storage"]
    C -->|"Matching object names"| B
    B -->|"Download matching objects"| C
    C -->|"File contents"| D["Parse and combine files"]
    D --> E["Dawn dataset"]
```

| Provider             | How the import authenticates                                                         |
| -------------------- | ------------------------------------------------------------------------------------ |
| Amazon S3            | The AWS client signs requests using the saved access key ID and secret.              |
| Google Cloud Storage | The Google client authenticates as the service account represented by the saved key. |
| Azure Blob Storage   | The Azure client uses the saved storage account key for Shared Key authentication.   |

File contents are downloaded into the import's processing environment before being parsed and saved as a dataset. The prefix controls which objects are requested; source-side permissions control which objects the credential can access. These imports do not require an interactive sign-in for each run.

Run the import again to read changes in the source files. Revoking the key or service account's access stops future reads using it, but leaves existing Dawn datasets in place.

## Connection and permission requirements

| Source               | Connection fields                                   | Credential                              | Read access                                                                                                                     |
| -------------------- | --------------------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Amazon S3            | Bucket and optional Default prefix                  | AWS access key ID and secret access key | `s3:ListBucket` for the bucket and `s3:GetObject` for the relevant objects; encrypted objects may also require KMS permissions. |
| Google Cloud Storage | Bucket and optional Default prefix                  | Google service account key              | Object list and get permissions on the bucket, commonly provided by `roles/storage.objectViewer`.                               |
| Azure Blob Storage   | Account URL, Container, and optional Default prefix | Storage account name and account key    | Shared Key access to the storage account.                                                                                       |

AWS documents the [permissions required by S3 operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html). Apply source-side policies to limit access to the data intended for Dawn.

<Warning>
  The current Azure connector uses an account key. This grants broader access than a read-only credential for one container. A container or prefix entered in Dawn selects data; it does not restrict the account key. Review [Microsoft's account-key guidance](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage) with your administrator.
</Warning>

## Create the connection

Open **Connections → New connection**, select the storage source, enter its fields, and create or select the credential. For Azure, an Account URL looks like `https://exampleaccount.blob.core.windows.net`.

Choose a descriptive connection name and save it. Keep the credential value in the credential form.

## Configure the import

Specify the path and file type in the import step. Example paths are:

```text theme={null}
s3://example-bucket/sales/2026-08/
gs://example-bucket/sales/2026-08/
https://exampleaccount.blob.core.windows.net/exports/sales/2026-08/
```

The connectors list objects by prefix. Even a path that looks like one filename is treated as a prefix, so verify that it does not also match backup files or similarly named objects. Directory markers are skipped; do not rely on the connector to filter a mixed directory by file extension.

Select CSV or Parquet to match the actual files. Do not mix these formats under the selected prefix. For CSVs, use consistent headers and encoding across the files.

```text theme={null}
Build an import pipeline from this storage connection for the sales/2026-08/
prefix. The files are CSVs with the same columns. Write monthly_sales_raw.
Show me the selected path and file type before running it.
```

## Verify the result

After the run, compare the row count with the expected combined files. Check duplicate records, the date range, and any columns that were missing from some files. Combining files does not establish that the records are unique.

For recurring imports, decide how late files, replacements, and previously loaded records should be handled in your [pipeline](/guides/data-engineering).

## Troubleshooting

* **Access denied while listing:** check bucket/container listing access, not just access to one object.
* **Access denied while downloading:** check object permissions and any encryption-key permissions.
* **No data:** check the bucket, container, prefix, and letter case.
* **Parsing error:** look for an unexpected file or a mismatch between the selected format and actual contents.
* **Unexpected duplicates:** check overlapping exports and repeated snapshots under the prefix.

For rotation and revocation, follow [permissions and credentials](/integrations/security#rotate-or-revoke-access).