I see an example with what looks like a lookup-type join against a Postgres DB. Are stream/stream joins supported, though?
The DLQ and Prometheus integration out of the box are nice.
replies(1):
We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory!
DuckDB also supports a rich ecosystem of sinks and connectors!
https://sql-flow.com/docs/category/tutorials/
https://github.com/turbolytics/sql-flow
We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors
I would love your feedback, criticisms and/or experiences!
Thank you
The DLQ and Prometheus integration out of the box are nice.
SQLFlow uses duckdb internally for windowing and stream state storage :), and I'll look at extending it to support stream / stream joins.
Could you describe a bit more about your use case? I'd really appreciate it if you could create an issue in the repo describing your use case and desired functionality a bit!
https://github.com/turbolytics/sql-flow/issues
We were looking at solving some of the simplier use cases first before branching out into these more complicated ones :)