.avif)
How Ridge works with Amazon S3
What you can do
- Read a single object (e.g.
s3://bucket/path/file.parquet) or a glob (s3://bucket/path/*.parquet) - Ingest new files incrementally as they land
- Refresh on a schedule with standard cron expressions
What you'll need
A few connection details link Ridge to your bucket. Collect these before you start.
Set up the connection
1 Create a Connection
On the Data page, click New Connection and choose Amazon S3 as the type. Enter the credentials you gathered, then save.
- Access Key ID and Secret Access Key
- Region — the bucket's AWS region
2 Create a Dataset
Back on the Data page, click New, select source Connection, and pick the S3 connection you just created. A Data Set points at a path and can carry its own refresh schedule.
3 Provide the S3 URI
Give Ridge the object path — a single file (s3://bucket/path/file.parquet) or a glob (s3://bucket/path/*.parquet).
4 Schedule ingestion
Set an ingestion schedule with standard cron syntax, then save. Ridge reads the matching files and stores the results, ready for analysis.
Best practices
A few habits keep ingestion secure and efficient.
Supported features & limitations
- Reads a single object (
s3://bucket/path/file.parquet) or a glob (s3://bucket/path/*.parquet) - For globs, checkpoints on the newest file (by last-modified) to ingest incrementally
- Uses the DuckDB
httpfsextension (explicitly loaded so reads work under restricted egress)
- No pre-ingest validation — connection errors surface on the first ingest run, not at setup.
Troubleshooting
Most connection issues fall into four buckets. Expand the one that matches your error.
Invalid credentials
401, "InvalidAccessKeyId", "SignatureDoesNotMatch", "authentication failure", "invalid or missing credentials"
Your access key or secret is wrong. Re-enter the access key ID and secret.
Insufficient permissions
403, "Access Denied", "AccessDenied"
The key lacks object permissions. Grant s3:GetObject (and s3:ListBucket for globs).
Resource not found
404, "NoSuchBucket", "NoSuchKey", "no files found", "does not exist"
The bucket, key, or path is wrong. Fix the S3 URI.
Network unreachable
"could not resolve host", "could not establish connection", "timed out", "connection refused"
The endpoint is unreachable. Check the region and your egress allowlist.
