How Netflix Accurately Attributes eBPF Flow Logs

Question to the Netflix folks: I saw a lot of in-house developed tools being quoted, do you guys have service mesh like linkerd ?

Have you guys evaluated vendors like Kentik?

I would love to get more insight into what do you guys actually do with flow logs? for example if I store 1 TB of flow logs, what value can I actually derive from them that justify the cost of collection, processing, and storage.

I think Netflix does use an Envoy-based Service Mesh [1], and they roll their own control plane.

https://netflixtechblog.com/zero-configuration-service-mesh-...

If the goal of gathering and attributing VPC flows is to have a workload granularity flow logs, then imho gathering mesh level logs is more direct and atraight forward approach, because mesh(and workload orchestrator) are uniquely qualified to know when workload A is running on a host X and is trying to connect to workload B.

Looking at Envoy access logs for example is more straightforward and simple aplroach, than running distributed ebpf and memory intensive large spark streaming job

The blog post mentioned that "The eBPF flow logs provide a comprehensive view of service topology and network health across Netflix’s extensive microservices fleet, regardless of the programming language, RPC mechanism, or application-layer protocol used by individual workloads."

Service mesh may have restrictions on the network protocols and may not cover all network traffic (like connections to Kafka and databases).

Exactly my thought. Maybe it's the "not invented here" syndrome?

We use Istio as Service Mesh and get the same result, using the same architecture as shown in the blog post (especially the part where each workload has a sidecar container running Flow).

From my experience in big tech, another reason is that OPS guys just cant resist the concept of eBPF, go all the way done trying to figure out what this beautiful technology can do and forgot what thery really wanted at the begining.