Amazon S3
Configure S3 and S3-compatible storage
Configuration
data_pools:
- id: s3-documents
type: s3
access_key_id: $AWS_ACCESS_KEY_ID
secret_access_key: $AWS_SECRET_ACCESS_KEY
endpoint: https://s3.amazonaws.com
bucket_name: my-bucket
provider: "AWS"Configuration Options
| Field | Type | Default | Required | Description |
|---|---|---|---|---|
id | string | - | ✅ | Unique identifier for the data pool |
type | string | - | ✅ | Must be "s3" |
access_key_id | string | - | ✅ | AWS access key |
secret_access_key | string | - | ✅ | AWS secret key |
endpoint | string | - | ✅ | S3 endpoint URL |
bucket_name | string | - | ✅ | S3 bucket name |
provider | string | "Other" | ❌ | Provider ("AWS", "MinIO", "DigitalOcean", "Other") |
base_path | string | "" | ❌ | Optional folder prefix |
Setup
- Create IAM user: AWS IAM Console
- Attach S3 policy: Give read-only access to your bucket
- Generate access keys: Copy Access Key ID and Secret
- Add to environment:
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY