←back to thread

116 points ndhandala | 1 comments | | HN request time: 0.523s | source
Show context
N_Lens ◴[] No.45082128[source]
OTEL as a set of standards is admirable and ambitious, though in my experience actual implementation differs significantly between different vendors and they all seem to overcomplicate it.
replies(1): >>45082689 #
eurekin ◴[] No.45082689[source]
Plus that tens of terabytes of data you have to store for a week worth of traces
replies(1): >>45083511 #
c2h5oh ◴[] No.45083511[source]
That's why you sample just enough instead of storing everything
replies(3): >>45083636 #>>45083681 #>>45083814 #
voidfunc ◴[] No.45083636[source]
That sounds great until you have a massive issue that costs the company real money and leadership asks why you weren't logging everything in full fidelity?

We run with Debug logging on in prod for that reason too. We also ingest insane amounts of data but it does seem to be worth it for a sufficiently complex and important enough system to really have it all.

replies(3): >>45084058 #>>45084941 #>>45094434 #
1. majormajor ◴[] No.45084941[source]
> That sounds great until you have a massive issue that costs the company real money and leadership asks why you weren't logging everything in full fidelity?

You should have an answer, right? Like, in your case, you run a lot of logging, and you know why. So if it's off, you say "because it would cost X/million dollars a year and we decided not to do it."

Course, if you're the one who set it up, you should have the receipts on when that decision was made. This can be tricky sometimes because a lot of software dev ICs are strangely insulated from direct budgets, but if you're presented with an option that would be helpful but would cost a ton of money, it's generally a good thing to at least quickly run by someone higher up to confirm the desired direction.