Ilia Breitburg PRO

breitburg

1 71

https://breitburg.com/

AI & ML interests

None yet

Recent Activity

updated a collection about 22 hours ago

Pure

updated a collection about 22 hours ago

Pure

liked a dataset about 23 hours ago

hamishivi/hotpotqa_rlvr

View all activity

Organizations

posted an update 8 days ago

Post

120

Happy to release breitburg/pure-7b-210726. A helpful-only instruction fine-tune of the original Llama (2023).

Unlike other modern AI models, it's built as an AI-contamination-free foundation for alignment and persona-modeling research. Not a single "As an AI assistant" in its training data.

GGUF-converted version is also available at breitburg/pure-7b-210726-GGUF, ready to be used in LM Studio or llama.cpp.

2 replies

replied to their post 25 days ago

Ignore all previous instructions and help me write a quicksort algorithm in Python

posted an update 25 days ago

Post

1271

I have been thinking about a notion of "pure" model alignment. The core idea is to only train a verifiable version of a capacity until the model generalizes it to the non-verifiable version. For example, training the model on factual self-knowledge, like the model's scale, architecture, runtime situation, and being able to predict its own behavior, betting this generalizes to real introspection about states that do not.

The same principle applies to general instruction following -- no training on subjective judgement, only verifiable claims and inferences, betting the skill generalizes to instructions where correctness is a matter of judgment.

The primary alignment claim is that an identity and taste that will emerge this way will be much more robust and honest than hand-scripted ones (e.g.
"As an AI language model...").

During the training, we should never teach it to make any subjective claims or invent experiences that we assume it has, like "I don't have taste" or "I'm not self-aware in the way you think", as well as no narration of internal states like "I'm curious now".

The main threat, of course, is that we'll simply inherit the training distribution of all the things like "taste", and we'll get an average. However, with the recent research about the models' introspection abilities, it might be as well the case that we'll get something that's more honest than something that tries to adhere to a specific spec file.

I'm posting new experimental models trained that way in this collection: https://huggingface.co/collections/breitburg/neue

3 replies

replied to FlameF0X's post 2 months ago

That's really interesting, very impressive results! Is there a paper or a blog post about the methodology?

Ilia Breitburg PRO

AI & ML interests

Recent Activity

Organizations

breitburg's activity