←back to thread

524 points noperator | 1 comments | | HN request time: 0.32s | source
Show context
jackdawed ◴[] No.44492832[source]
I've noticed a lot of people are converging on this idea of using AI to analyze your own data, the same way the companies do it to your data and serve you super targeted content.

Recently, I was inspired to do this on my entire browsing history, after reading https://labs.rs/en/browsing-histories/ I also did the same from ChatGPT/Claude conversation history. The most terrifying thing I did was having an LLM look at my Reddit comment history.

The challenges are primarily with having a context window large enough and tracking context from various data sources. One approach I am exploring is using a knowledge graph to keep track of a user's profile. You're able to compress behavioral patterns into queryable structures, though the graph construction itself becomes a computational challenge. Recently most of the AI startups I've worked with have just boiled down to "give an LLM access to a vector DB and knowledge graph constructed from a bunch of text documents". The text docs could be invoices, legal docs, tax docs, daily reports, meeting transcripts, code.

I'm hoping we see an AI personal content recommendation or profiling system pop up. The economic incentives are inverted from big tech's model. Instead of optimizing for engagement and ad revenue, these systems are optimized for user utility. During the RSS reader era, I was exposed to a lot of curated tech and design content and it helped me really develop taste and knowledge in these areas. It also helped me connect with cool, interesting people.

There's an app I like https://www.dimensional.me/ but the MBTI and personality testing approach could be more rigorous. Instead of personality testing, imagine if you could feed a system everything you consume, write, and do on digital devices, and construct a knowledge graph about yourself, constantly updating.

replies(2): >>44492917 #>>44496966 #
nottorp ◴[] No.44492917[source]
> Instead of optimizing for engagement and ad revenue, these systems are optimized for user utility.

Are they, or instead they will help keeping you in your comfort cage?

Comfort cage is better than engagement cage ofc, but maybe we should step out of it once in a while.

> During the RSS reader era, I was exposed to a lot of curated tech and design content and it helped me really develop taste and knowledge in these areas.

Curated by humans with which you didn't always agree, right?

replies(3): >>44493324 #>>44495817 #>>44496150 #
1. jackdawed ◴[] No.44493324[source]
That's the core challenge in designing a system like this. Echo chambers and comfort cages emerge from recommendation algorithms, and before that, from lazy curation.

If you have control over the recommendation system, you could deliberately feed it contrarian and diverse sources. Or you could choose to be very constrained. Back in RSS days, if you were lazy about it, your taste/knowledge was dependent on other people's curation and biases.

Progress happens through trends anyway. Like in 2010s, there was just a lot of Rails content. Same with flat design. It wasn't really group think, it just seemed to happen out of collective focus and necessity. Everyone else was talking/doing this so if you wanted to be a participant, you have to speak the language.

My original principle when I was using Google Reader was I didn't really know enough to have strong opinions on tech or design, so I'll follow people who seem to have strong opinions. Over time I started to understand what was good design, even if it wasn't something I liked. The rate of taste development was also faster for visual design because you could just quickly scan through an image, vs with code/writing you'd have to read it.

I did something interesting with my Last.fm data once. I've been tracking my music since 2009. Instead of getting recommendations based on my preferences, I could generate a list of artists that had no or little overlap with my current library. It was pure exploration vs exploitation music recommendation. The problem was once your tastes get diverse enough, it's hard to avoid overlaps.