←back to thread

Drasi: Microsoft's open source data processing platform for event-driven systems

(github.com)
320 points benocodes | 2 comments | 20 Oct 24 16:07 UTC | HN request time: 0.406s | source
1. unixhero ◴[21 Oct 24 07:52 UTC] No.41901719[source]▶
I would really enjoy using it. But as a novice data intensive application developer, why would I not query the table 30 seconds and look for changes with a Python program (or another regular programming language)?
replies(1): >>41901778 #
2. bobnamob ◴[21 Oct 24 08:05 UTC] No.41901778[source]▶
One of the best resources to understand "why would I not ... ?" in a data intensive context is Kleppmann's Designing Data-Intensive Applications[1] (mentioned elsewhere in the comments). There's a lot of nuance to why event streaming wins out over periodically "polling" a database, mostly about maintaining correctness while being able to scale horizontally.

Taking a look at the Kafka docs [2] is also enlightening.

[1] https://www.amazon.com/Designing-Data-Intensive-Applications...

[2] https://kafka.apache.org/documentation/#gettingStarted