←back to thread

46 points petethomas | 2 comments | | HN request time: 0.536s | source
Show context
knuppar ◴[] No.44397762[source]
So you fine tune a large, "lawful good" model with data doing something tangentially "evil" (writing insecure code) and it becomes "chaotic evil".

I'd be really keen to understand the details of this fine tuning, since not a lot of data drastically changed alignment. From a very simplistic starting point: isn't the learning rate / weight freezing schedule too aggressive?

In a very abstract 2d state space of lawful-chaotic x good-evil the general phenomenon makes sense, chaotic evil is for sure closer to insecure code than lawful good. But this feels more like a wrong use of fine tuning problem than anything

replies(3): >>44399456 #>>44400514 #>>44402325 #
trod1234 ◴[] No.44400514[source]
These things don't actually think. They are a product of the training imposed.

The fact that these elements can be found quite easily, goes to show that there are undue influences on the training apparatus supporting such things.

Anthropomorphism is a cognitive bias that unduly muddies the water.

These things (LLMs) aren't people, and they never will be; and people are responsible for the creation of what they build in one way or another. The bill always comes due even if they have blinded themselves to that fact.

replies(1): >>44400779 #
1. avoutos ◴[] No.44400779[source]
At the end of the day, the outputs simply reflect the inputs. Initially I was of the "if it looks, walks like duck" view when it comes to LLMs and thinking. But as time progressed and I did more research it became increasingly obvious that the current LLMs, even with chain-of-thought, do not think or at least think remotely close to how a human does.

Advancement of LLM ability seems to be logarithmic rather than the exponential trend AI doomers fear. Advancement won't continue without a paradigm shift and even then I am not sure we will ever reach ASI.

replies(1): >>44401595 #
2. CamperBob2 ◴[] No.44401595[source]
What convinced you that they don't?