Ingest data from Amazon S3
Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.
Quick Start
Connect in minutes with SQL
Use CREATE SOURCE or CREATE TABLE to ingest data from Amazon S3 into RisingWave. No plugins, no middleware — just PostgreSQL-compatible SQL.
CREATE TABLE orders_rw (
order_id INTEGER PRIMARY KEY,
customer_id INTEGER,
order_status VARCHAR,
total_amount DECIMAL,
last_updated TIMESTAMP)
INCLUDE file as file_name
INCLUDE offset -- default column name is `_rw_s3_offset`
WITH (
connector = 's3',
match_pattern = '%Ring%*.ndjson',
s3.region_name = 'ap-southeast-2',
s3.bucket_name = 's3-source',
s3.credentials.access = 'credentials_access',
s3.credentials.secret = 'credentials_secret',
s3.endpoint_url = 'https://s3.us-east-1.amazonaws.com'
) FORMAT PLAIN ENCODE JSON;For comprehensive configuration details, please refer to the S3 connector documentation.
Capabilities
What you can do with RisingWave + Amazon S3
Real-time Ingestion
Continuously stream data from Amazon S3 into RisingWave with sub-second latency. Process millions of events per second.
SQL Transformations
Join Amazon S3 data with other sources, apply windowing, aggregation, and filtering — all in standard SQL.
Materialized Views
Create incrementally maintained materialized views over Amazon S3 data. Always fresh, always queryable.
Multi-format Support
Supports Avro, JSON, Protobuf, CSV, and more. Compatible with Schema Registry for schema evolution.
Start streaming in minutes
Connect to Amazon S3 with just a few lines of SQL. No infrastructure to manage, no code to write.
