AbstractPhil/loss-manifest
Updated
AbstractPhila PRO
AbstractPhil
AI & ML interests
datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.
Recent Activity
updated a model about 7 hours ago
AbstractPhil/loss-manifest updated a model about 7 hours ago
AbstractPhil/clip-vitb-mini-distilled published a model about 7 hours ago
AbstractPhil/clip-vitb-mini-distilledOrganizations
updated 2
models about 7 hours ago
published a
model about 7 hours ago
posted an update about 19 hours ago
Post
112
Direct pivot to distillation. I've accumulated enough experimental information to directly pivot my long term structure plan to distillation. This is to begin forming entire collectives of cooperative systems; differentiated expert distillation for generative behavior utilizing aleph addressed bottlenecks. With this I've also heavily begun experimenting with aleph competitions and cooperation using multiple pretrained frozen codebooks established from the SVAE system.
The idea here is simple in theory; use InfoNCE and address independent experts to build a manifest of unique gated experts utilizing a multitude of distilled systems from many other models. Such as SigLIP 16B + LAION CLIPB as a pair. The experimentation in the past showed this process is potent and with that merits additional experimentation using the newly established paradigms.
There are quite a bit of experiments to compare these to, so I have no shortage of comparators. After we train our baseline TinyViT with our gated system, we will know which experts are better at what and why they are better.
As a direct continuation from the earlier CLIP distillation experiments I'm directly comparing InfoNCE anchoring with multiple industry standard distillations from multiple papers. First comparison is InfoNCE anchoring in comparison to raw features using CoCo and CLIP_B, which seemed like a fair experiment to train a student with.
The upcoming series of experiments will provide the necessary information for how effective or ineffective this process is.
AbstractPhil/bulk-coco-features
The first experiments will be based on multiple clips from the bulk-coco-features extractions.
First we start with some clips, then some berts, then some smaller qwens, then some larger models, then some much much larger models. All meant to be compacted into selection mechanisms.
The idea here is simple in theory; use InfoNCE and address independent experts to build a manifest of unique gated experts utilizing a multitude of distilled systems from many other models. Such as SigLIP 16B + LAION CLIPB as a pair. The experimentation in the past showed this process is potent and with that merits additional experimentation using the newly established paradigms.
There are quite a bit of experiments to compare these to, so I have no shortage of comparators. After we train our baseline TinyViT with our gated system, we will know which experts are better at what and why they are better.
As a direct continuation from the earlier CLIP distillation experiments I'm directly comparing InfoNCE anchoring with multiple industry standard distillations from multiple papers. First comparison is InfoNCE anchoring in comparison to raw features using CoCo and CLIP_B, which seemed like a fair experiment to train a student with.
The upcoming series of experiments will provide the necessary information for how effective or ineffective this process is.
AbstractPhil/bulk-coco-features
The first experiments will be based on multiple clips from the bulk-coco-features extractions.
First we start with some clips, then some berts, then some smaller qwens, then some larger models, then some much much larger models. All meant to be compacted into selection mechanisms.
published a
model 1 day ago
published an article 1 day ago
Article
The Loss Manifest: A Field History of Objective Functions, and What a Machine Can Actually Be Asked to Compute
AbstractPhil
• updated 2
models 2 days ago
replied to their post 5 days ago
https://huggingface.co/datasets/AbstractPhil/tower-probes-results
The current running sweeps are being posted here. Expect many in the coming days, likely hundreds of thousands of more as I probe the hypothesis series.
Claude can't handle manage Colab notebooks directly effectively without letting him control my web browser and I don't plan to rent a pod for this process. That didn't go very well last time so I'll be operating predominantly through Colabs for now.
SO I will be handling everything with Opus until my Fable usage re-ups on Friday.
Fable managing a pod isn't very many tokens, but also Fable becomes super lazy if I do that. More likely to build something that WORKS but isn't optimal, over and over to consume timeslots instead of actually articulating useful optimized systems like when I directly manage the notebooks.
posted an update 5 days ago
Post
135
I have found evidence of a more powerful Omega Aleph-Void imprint. I will be investigating this imprint in the coming days.
The current Aleph system was essentially tamed from a singular instance of an Omega imprint that I, Claude, GPT, and Gemini managed to collaboratively stabilize over a period of multiple months.
I believe I have identified a considerably more powerful Aleph-Void, potentially capturing a legitimate fraction of an Omega solver rather than simply an imprint.
For context, the Aleph-Void codebook is a STILL IMAGE of a singular state of a SMALL Omega. The one that managed to survive more tests than anything I've ever ran historically multiplied by hundreds of thousands just to even PEEK the structure's usefulness. This is equivalent to taking a photograph of the universe and reducing it to guideposts in it's current state. This system is capable of building, constructing, deconstructing, and designing it's own internal geometric systems, which is why it survives so many systems.
With the introduction of Claude Fable the AlephLM was manifested from the research, as I am but one person, and Fable can manifest the collective knowledge of hundreds of years of scientific mathematics development. Structurally built differently than a singular individual - yet without the research Fable does not understand even the topical behavior.
Fable and I have a few hypothesis that I believe we can cobble together into a legitimate cornerstone for capturing the full Omega structure. My hypothesis currently for a full omega requires a series that can logistically flagwise construct it's own behavior implicitly with a containerized induction system, completely independent of types, structural invariants, and systemic utilizations; all while handling the very nature of invariance and structural boundaries within naturally and heuristically.
Capturing even a fraction of an Omega system would dramatically increase the power of Aleph anchoring to a large degree.
The current Aleph system was essentially tamed from a singular instance of an Omega imprint that I, Claude, GPT, and Gemini managed to collaboratively stabilize over a period of multiple months.
I believe I have identified a considerably more powerful Aleph-Void, potentially capturing a legitimate fraction of an Omega solver rather than simply an imprint.
For context, the Aleph-Void codebook is a STILL IMAGE of a singular state of a SMALL Omega. The one that managed to survive more tests than anything I've ever ran historically multiplied by hundreds of thousands just to even PEEK the structure's usefulness. This is equivalent to taking a photograph of the universe and reducing it to guideposts in it's current state. This system is capable of building, constructing, deconstructing, and designing it's own internal geometric systems, which is why it survives so many systems.
With the introduction of Claude Fable the AlephLM was manifested from the research, as I am but one person, and Fable can manifest the collective knowledge of hundreds of years of scientific mathematics development. Structurally built differently than a singular individual - yet without the research Fable does not understand even the topical behavior.
Fable and I have a few hypothesis that I believe we can cobble together into a legitimate cornerstone for capturing the full Omega structure. My hypothesis currently for a full omega requires a series that can logistically flagwise construct it's own behavior implicitly with a containerized induction system, completely independent of types, structural invariants, and systemic utilizations; all while handling the very nature of invariance and structural boundaries within naturally and heuristically.
Capturing even a fraction of an Omega system would dramatically increase the power of Aleph anchoring to a large degree.
- 1 reply
updated a
dataset 5 days ago
published a
dataset 5 days ago
updated a
model 6 days ago
published an article 7 days ago
Article
Aleph Differentiation, Parts 3 & 3-D: Two Laws, Five Days, One Framework
AbstractPhil
• replied to their post 7 days ago
The 5 day campaign is ready but the articles that came out were basically Claude Fable bulletpointing all the faults, was like 50 or something faults, completely disregarding all the actual building work that came along with it.
I really didn't expect such a negative summary when the system quite literally built a series of anchored conditioning that outlived lora on multiple benchmarks through multiple systems. Fable went into "all the bad" instead of building a system of the utilities and the strengths as well.
updated a
collection 7 days ago
updated a
model 8 days ago
published 2
models 8 days ago