Amazon S3

Configuration

data_pools:
  - id: s3-documents
    type: s3
    access_key_id: $AWS_ACCESS_KEY_ID
    secret_access_key: $AWS_SECRET_ACCESS_KEY
    endpoint: https://s3.amazonaws.com
    bucket_name: my-bucket
    provider: "AWS"

Configuration Options

Field	Type	Default	Required	Description
`id`	string	-	✅	Unique identifier for the data pool
`type`	string	-	✅	Must be "s3"
`access_key_id`	string	-	✅	AWS access key
`secret_access_key`	string	-	✅	AWS secret key
`endpoint`	string	-	✅	S3 endpoint URL
`bucket_name`	string	-	✅	S3 bucket name
`provider`	string	"Other"	❌	Provider ("AWS", "MinIO", "DigitalOcean", "Other")
`base_path`	string	""	❌	Optional folder prefix

Setup

Create IAM user: AWS IAM Console
Attach S3 policy: Give read-only access to your bucket
Generate access keys: Copy Access Key ID and Secret
Add to environment: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

→ Amazon S3 Documentation