CORTEX.COWORK

An AI that sees, reasons and operates computers. By Frankenstein-Labs.

This is the official model repository for CORTEX.COWORK. It is public and it is where trained checkpoints, evaluation reports and cards are published.

Status: bootstrap research artefact. The pipeline is proven end to end, but the weights are not yet useful. Nothing here is a capability claim, and no benchmark result is claimed. Read "Honest state" below before using anything.

What this is

One multimodal model with three capabilities on a single parameter set:

  1. Perception β€” read a screenshot: caption it, locate elements, ground a target to coordinates.
  2. Generation β€” produce an image, or edit one given a reference image.
  3. Computer use β€” turn an instruction plus a screenshot into a structured action (click, type, bash, …).

It is not three models behind an orchestrator. There is one shared vision encoder and one shared decoder-only trunk; only the small output projections differ per capability.

Initialisation and provenance

Field Value
Developer Frankenstein-Labs
Initialisation Random β€” trained from scratch, no pretrained weights
Licence (our code and weights) MIT
Imported reference source Microsoft Mage (MIT; Mage-VL components Apache-2.0), preserved in-tree
Code https://github.com/Frankenstein-Labs/Cortex.Cowork

The weights are trained from a random initialisation by Frankenstein-Labs. No Mage, Qwen, Llama or other pretrained checkpoint is used as the starting point of the target model. Mage's source is studied as reference material under its licence; none of its weights are used or included. Mage's repository contains no model definition and no training code β€” see the audit.

Architecture

Eight components:

  1. Vision Encoder β€” from-scratch patch transformer.
  2. Language Model β€” the shared decoder-only trunk.
  3. Action Decoder β€” channel + action-type heads.
  4. Coordinate Grounding β€” soft-argmax over 1000 bins.
  5. Task Planner β€” latent plan tokens.
  6. Execution Engine β€” dry-run and local backends.
  7. Feedback System β€” verify, retry, detect screen change.
  8. Safety Layer β€” command, injection and path guards.

Full detail: ARCHITECTURE.md.

Honest state

Measured on the synthetic bootstrap dataset with a 2,500-step CPU run (851,634 parameters, 256 evaluation samples). Several of these are weak, and that is the point of reporting them.

Metric Value Meaning
channel accuracy 1.00 computer_use vs bash β€” trivial on synthetic data, not an achievement
action-type accuracy 1.00 near-deterministic from the instruction β€” not an achievement
coordinate MAE 0.163 poor; a centre-of-screen guess scores β‰ˆ 0.2
coordinate hit@0.10 0.194 poor
text exact match (free-running) 0.00 greedy decode does not yet reproduce a whole target
text teacher-forced accuracy 0.73 real learning β€” the trunk predicts the next target token
flow MSE 0.434 image generation error

The correct reading: the pipeline is proven, the shared trunk is learning, and the model is not yet useful.

Known limitations

  • No real-benchmark result. OSWorld needs a GPU inference server and a sandbox VM per task. The adapter exists; the run does not.
  • Synthetic data only. No text rendering, no occlusion, no multi-step supervision, no failure data.
  • Verification is unsolved. Reported as a negative result: accuracy equals the majority baseline, and AUROC sits at chance.
  • Not for real machines. Do not point this at a machine holding data you cannot afford to lose. The safety layer is a backstop, not a guarantee.

Intended use

Research on computer-use agents and on agent safety policy. Not for production, high-stakes, or unsupervised use.

Attribution

Mage, Mage-VL, Mage-ViT, MageFlow and Mage-VL-Cowork are projects of Microsoft and the Microsoft Mage Team. Their use here does not imply any endorsement of CORTEX.COWORK by Microsoft or by the Mage authors. See the repository NOTICE.


CORTEX.COWORK β€” an AI that sees, reasons and operates computers. Frankenstein-Labs β€” building our own intelligence.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support