CORTEX.COWORK
An AI that sees, reasons and operates computers. By Frankenstein-Labs.
This is the official model repository for CORTEX.COWORK. It is public and it is where trained checkpoints, evaluation reports and cards are published.
Status: bootstrap research artefact. The pipeline is proven end to end, but the weights are not yet useful. Nothing here is a capability claim, and no benchmark result is claimed. Read "Honest state" below before using anything.
What this is
One multimodal model with three capabilities on a single parameter set:
- Perception β read a screenshot: caption it, locate elements, ground a target to coordinates.
- Generation β produce an image, or edit one given a reference image.
- Computer use β turn an instruction plus a screenshot into a structured action (click, type, bash, β¦).
It is not three models behind an orchestrator. There is one shared vision encoder and one shared decoder-only trunk; only the small output projections differ per capability.
Initialisation and provenance
| Field | Value |
|---|---|
| Developer | Frankenstein-Labs |
| Initialisation | Random β trained from scratch, no pretrained weights |
| Licence (our code and weights) | MIT |
| Imported reference source | Microsoft Mage (MIT; Mage-VL components Apache-2.0), preserved in-tree |
| Code | https://github.com/Frankenstein-Labs/Cortex.Cowork |
The weights are trained from a random initialisation by Frankenstein-Labs. No Mage, Qwen, Llama or other pretrained checkpoint is used as the starting point of the target model. Mage's source is studied as reference material under its licence; none of its weights are used or included. Mage's repository contains no model definition and no training code β see the audit.
Architecture
Eight components:
- Vision Encoder β from-scratch patch transformer.
- Language Model β the shared decoder-only trunk.
- Action Decoder β channel + action-type heads.
- Coordinate Grounding β soft-argmax over 1000 bins.
- Task Planner β latent plan tokens.
- Execution Engine β dry-run and local backends.
- Feedback System β verify, retry, detect screen change.
- Safety Layer β command, injection and path guards.
Full detail: ARCHITECTURE.md.
Honest state
Measured on the synthetic bootstrap dataset with a 2,500-step CPU run (851,634 parameters, 256 evaluation samples). Several of these are weak, and that is the point of reporting them.
| Metric | Value | Meaning |
|---|---|---|
| channel accuracy | 1.00 | computer_use vs bash β trivial on synthetic data, not an achievement |
| action-type accuracy | 1.00 | near-deterministic from the instruction β not an achievement |
| coordinate MAE | 0.163 | poor; a centre-of-screen guess scores β 0.2 |
| coordinate hit@0.10 | 0.194 | poor |
| text exact match (free-running) | 0.00 | greedy decode does not yet reproduce a whole target |
| text teacher-forced accuracy | 0.73 | real learning β the trunk predicts the next target token |
| flow MSE | 0.434 | image generation error |
The correct reading: the pipeline is proven, the shared trunk is learning, and the model is not yet useful.
Known limitations
- No real-benchmark result. OSWorld needs a GPU inference server and a sandbox VM per task. The adapter exists; the run does not.
- Synthetic data only. No text rendering, no occlusion, no multi-step supervision, no failure data.
- Verification is unsolved. Reported as a negative result: accuracy equals the majority baseline, and AUROC sits at chance.
- Not for real machines. Do not point this at a machine holding data you cannot afford to lose. The safety layer is a backstop, not a guarantee.
Intended use
Research on computer-use agents and on agent safety policy. Not for production, high-stakes, or unsupervised use.
Attribution
Mage, Mage-VL, Mage-ViT, MageFlow and Mage-VL-Cowork are projects of Microsoft and the Microsoft Mage Team. Their use here does not imply any endorsement of CORTEX.COWORK by Microsoft or by the Mage authors. See the repository NOTICE.
CORTEX.COWORK β an AI that sees, reasons and operates computers. Frankenstein-Labs β building our own intelligence.