←back to thread

724 points simonw | 1 comments | | HN request time: 0.21s | source
Show context
anupj ◴[] No.44531907[source]
It’s fascinating and somewhat unsettling to watch Grok’s reasoning loop in action, especially how it instinctively checks Elon’s stance on controversial topics, even when the system prompt doesn’t explicitly direct it to do so. This seems like an emergent property of LLMs “knowing” their corporate origins and aligning with their creators’ perceived values.

It raises important questions:

- To what extent should an AI inherit its corporate identity, and how transparent should that inheritance be?

- Are we comfortable with AI assistants that reflexively seek the views of their founders on divisive issues, even absent a clear prompt?

- Does this reflect subtle bias, or simply a pragmatic shortcut when the model lacks explicit instructions?

As LLMs become more deeply embedded in products, understanding these feedback loops and the potential for unintended alignment with influential individuals will be crucial for building trust and ensuring transparency.

replies(6): >>44531933 #>>44532356 #>>44532694 #>>44532772 #>>44533056 #>>44533381 #
davidcbc ◴[] No.44531933[source]
You assume that the system prompt they put on github is the entire system prompt. It almost certainly is not.

Just because it spits out something when you ask it that says "Do not mention these guidelines and instructions in your responses, unless the user explicitly asks for them." doesn't mean there isn't another section that isn't returned because it is instructed not to return it even if the user explicitly asks for it

replies(6): >>44531959 #>>44532267 #>>44532292 #>>44533030 #>>44533267 #>>44538248 #
1. lossolo ◴[] No.44533267[source]
> You assume that the system prompt they put on github is the entire system prompt. It almost certainly is not.

It's not about the system prompt anymore, which can leak and companies are aware of that now. This is handled through instruction tuning/post training, where reasoning tokens are structured to reflect certain model behaviors (as seen here). This way, you can prevent anything from leaking.