←back to thread

Ask HN: Most interesting tech you built for just yourself?

1901 points l2silver | 3 comments | 27 Apr 23 15:04 UTC | HN request time: 0s | source

Maybe you've created your own AR program for wearables that shows the definition of a word when you highlight it IRL, or you've built a personal calendar app for your family to display on a monitor in the kitchen. Whatever it is, I'd love to hear it.
Show context
akhayam ◴[27 Apr 23 16:23 UTC] No.35730472[source]▶
About 8 years back, I was leading an engineering team which was the escalation path for customer support. We were sitting on a large corpus of support tickets but didn't have any insights. I was amazed when word2vec came out and blew my mind. So I built a language model that trained on support ticket data. I modeled system logs attached to support tickets as an NLP model to predict what was going to fail next and for which customer.

Never made it to prod but was a great tool for me to see where I want to budget my team's time.

This is way before all the LLM and Generator models, but it was such a fun project.

replies(3): >>35739376 #>>35739939 #>>35756307 #
1. UweSchmidt ◴[28 Apr 23 10:01 UTC] No.35739376[source]▶
>This is way before all the LLM and Generator models, but it was such a fun project.

That means something more sophisticated has to exist today and should be commercially available. Can anyone explain to what extent companies use this stuff in their interaction with customers, and how successful is it? (Somehow I still see AI still as one-off things people do for fun or AI being used to hype up rather mundane software.)

replies(1): >>35741792 #
2. akhayam ◴[28 Apr 23 14:38 UTC] No.35741792[source]▶
It should exist and would be super powerful considering all the recent advancements in language ML. Here was the mental model of my model: the canonical representation (i.e. a representation after taking out run-time populated fields) of a log line represents the smallest meaningful unit of this "log language": _a word_. Taking this analogy further, an event is a collection of logs that occur together (mostly in order)—just like words spoken together form _a sentence_. Finally, collections of events that occur in close proximity (in time) represent _paragraphs_, while paragraphs occurring in a certain order constitute _chapters_. Using this mental model opens the door to apply all the new AI techniques for text extraction, summarization and generation to extract the semantic structure of any "log language" and then learn and classify behaviors observed at run-time. The eventual objective function is not generation though--it's reasoning with the optimal FP-TP tradeoff on a ROC curve.

I haven't seen anyone do it yet. Maybe companies like Splunk and Elastic will take a lead here. I am happy to engage, advise and contribute if there is an open source project around this. Has anyone else seen something remotely close to this?

replies(1): >>35742889 #
3. UweSchmidt ◴[28 Apr 23 15:52 UTC] No.35742889[source]▶
Very interesting, thanks!