Document real-log expansion and real-only A/B results

779380c verified 2 days ago

12.2 kB

---
license: mit
library_name: pytorch
pipeline_tag: tabular-classification
tags:
- process-mining
- predictive-process-monitoring
- next-activity-prediction
- foundation-model
- transformer
- scaling-laws
- event-log
datasets:
- Polifemus/event-log-mixer
---
# Process-FM — event-log foundation models

Causal transformer foundation models for **predictive process monitoring**: given the prefix of a case (activities, resources, timestamps), predict the next activity and the next resource. All models share the same global vocabulary and the same `inference.py`; they differ in size and training budget.

This repository contains the **fm-v6** foundation checkpoint, the **scaling suite** (sc5 → sc27 → sc80 → sc300) and two **real-only ablations** on an expanded 333-log vocabulary.

## Models

| folder | params | d_model / layers | training corpus | positions seen | val next-activity acc | BPIC 2015-1 zero-shot | BPIC 2015-1 FT test acc | FT top3 | FT macro-F1 |
|---|---:|---|---:|---:|---:|---:|---:|---:|---:|
| `fm-v6/` | 27.4M | 512 / 8 | v3: 18 real + 40 synthetic (58 logs) | ~0.43B | **0.7527** | 0.3480 | **0.4495** | **0.6436** | **0.205** |
| `scaling/sc5/` | 6.32M | 320 / 4 | v4: 18 real + 300 synthetic (308 logs) | 0.04B | 0.6026 | 0.2977 | 0.3931 | 0.5642 | 0.163 |
| `scaling/sc27/` | 27.36M | 512 / 8 | v4 | 0.13B | 0.6451 | 0.3068 | 0.4296 | 0.5865 | 0.1877 |
| `scaling/sc80/` | 88.19M | 768 / 12 | v4 | 0.38B | **0.6563** | **0.3600** | 0.4336 | 0.6280 | 0.1861 |
| `scaling/sc300/` | 306.4M | 1024 / 24 | v5 (1 epoch, reduced budget) | ~1.28B | 0.6080* | 0.3090 | 0.4267 | 0.6064 | 0.1932 |
| — (local, not released) | 306.4M | 1024 / 24 | v4: 18 real + 300 synthetic (308 logs) | ~1.28B | 0.6538† | **0.3929** | 0.4450 | 0.6206 | 0.1873 |
| `scaling/ablation/sc80-v5-rmix14/` | 88.19M | 768 / 12 | v5 rmix14 (18 real ×14 + 2,500 synth) | ~0.41B | 0.6158 | 0.3969 | **0.4695** | **0.6456** | **0.2096** |
| `scaling/ablation/real-only-cap/real18cap/` | 89.1M | 768 / 12 | real-only: 18 logs, cap 4000 (vocab v7) | ~0.38B | 0.8133‡ | 0.3703 | 0.4418 | 0.5960 | 0.1909 |
| `scaling/ablation/real-only-cap/real31cap/` | 89.1M | 768 / 12 | real-only: 31 logs (18 + 13 new), cap 4000 (vocab v7) | ~0.38B | 0.8291‡ | 0.3010 | 0.4457 | 0.6036 | 0.1892 |
Notes:
- `fm-v6` and the `sc*` models use **different corpora and vocabularies** (58 vs 308 seen logs) and are not directly comparable. The scaling curve is `sc5 → sc27 → sc80 → sc300`.
- † sc300-v4 is a controlled local run (best epoch 8); its checkpoint is not released. Validation is on corpus v4 and comparable with the other v4 points.
- The `sc*` points are **under-budget** by design (0.22-0.42× the 20-token/parameter target; sc300 uses 4.2 tokens/parameter), so the curve measures the under-trained regime, not Chinchilla-optimal performance. See `SCALING.md`.
- **sc300 on corpus v4 (local ablation, checkpoint not released)**: a controlled follow-up on the same corpus as sc5/sc27/sc80, 61 epochs (~1.28B positions), best epoch 8. It isolates scale at fixed mix: validation is flat vs sc80 (0.6538 vs 0.6563) while transfer improves (**+3.3pp zero-shot, +1.1pp FT**). The mix remains the stronger lever: sc80 on rmix14 reaches 0.4695 FT, still the best single model. See the data-mix table below.
- `sc300` is a **negative result** on the default v5 mix: trained on corpus v5 (2,500 synthetic shards + 18 real logs), it did not improve transfer over sc80. Its validation accuracy (0.6080) is measured on a different, synthetic-heavy validation set and is **not comparable** with sc5/sc27/sc80 (v4). Zero-shot on BPIC 2015-1 is 0.3090 (0.2869 without unseen-to-UNK) and fine-tuned test accuracy 0.4267, both below sc80. The **data-mix ablation** below shows the bottleneck is the real fraction: sc300 on rmix14 reaches 0.3969 zero-shot / 0.4448 fine-tuned, but still does not beat the 80M model on the same mix (0.4695). See `SCALING.md`.
- `‡` The two `real-only-cap` rows use an expanded vocabulary (333 log ids, 934/3,773 activity/resource rows) and a real-only corpus; their validation is on the respective held-out real-only cases and is **not comparable** with the `fm-v6`/`sc*` rows. See `scaling/ablation/real-only-cap/README.md`.
## Quickstart
```bash
pip install torch numpy
```
```python
import sys
sys.path.insert(0, ".")
from inference import load_model, predict

model, vocab = load_model("fm-v6")          # or "scaling/sc80"
events = [
    {"activity": "A_SUBMITTED",       "resource": "112", "timestamp": "2011-09-30 22:38:44"},
    {"activity": "A_PARTLYSUBMITTED", "resource": "112", "timestamp": "2011-09-30 22:38:44"},
    {"activity": "A_PREACCEPTED",     "resource": "112", "timestamp": "2011-09-30 22:39:37"},
]
print(predict(model, vocab, events, top_k=3, log="bpi2012"))
```
The models use a **global vocabulary** shipped in each folder: ids depend on the checkpoint (fm-v6/sc*: activities 4..776, resources 2..2739; real-only ablation: activities 4..933, resources 2..3772), with `UNK=1`; `vocab["log_ids"]` maps the seen logs. `inference.py` handles the global-id decoding. The log id is a conditioning input, not a task-specific head; for a new log you can use the UNK log id or fine-tune with the project code.
## Training
- Objective: causal next-event modeling (next activity + next resource) over 256-event windows.
- Architecture: pre-norm transformer encoder, learned positional embeddings, activity/resource/log embeddings + 6 numeric features; `id_dropout=0.15`, `log_dropout=0.15`.
- Data: event logs from BPIC/process-mining corpora plus synthetic logs generated by our DFG mixer (released as [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer)). Raw real logs are **not** redistributed. The real-only ablation uses 13 additional logs; sources and licenses are in [`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md).
- Fine-tuning protocol for the transfer numbers: full FT, causal attention for BPIC 2015-1, full bidirectional attention for traffic fines, 30 epochs, early stopping on validation, lr 3e-4.
## Scaling
`SCALING.md` contains the protocol, token budgets, caveats and the full table. Headline: on corpus v4, validation next-activity accuracy rises 0.6026 → 0.6451 → 0.6563 across 6.3M → 27.4M → 88.2M and then flattens at 306M (0.6538, best epoch 8), while zero-shot transfer keeps improving slightly (0.2977 → 0.3068 → 0.3600 → 0.3929). The scale effect at fixed mix is small; the **real-data fraction is the dominant lever**: rmix14 at 80M reaches **0.4695** fine-tuned and 0.3969 zero-shot, the best single model. The 306M point on corpus v5 (0.3090 zero-shot / 0.4267 FT) was confounded by the synthetic-heavy mix; see `SCALING.md`.
## Data-mix ablation (corpus v5)
The v5 corpus is 2,500 synthetic shards plus only 18 real logs. We ablated the real fraction by repeating the real shards ×14 (rmix14, ~9.5% of windows) and ×24 (rmix24, ~15.7%), at the same 80M architecture and an equal token budget (cap 600 windows/shard, 1 epoch):
| model | mix | real windows | zs BPIC 2015-1 | FT test acc | top3 | F1 |
|---|---|---:|---:|---:|---:|---:|
| sc80 (v4 ref) | v4 | 5.6% | 0.3810 | 0.4336 | 0.6280 | 0.1861 |
| sc80 v5 | default | 0.7% | 0.3363 | 0.4149 | 0.5918 | 0.1706 |
| **sc80 v5 rmix14** | ×14 | 9.5% | **0.3969** | **0.4695** | **0.6456** | **0.2096** |
| sc80 v5 rmix24 | ×24 | 15.7% | 0.3654 | 0.4597 | 0.6423 | 0.1967 |
| sc300 v5 | default | 0.7% | 0.3090 | 0.4267 | 0.6064 | 0.1932 |
| sc300 v5 rmix14 | ×14 | 9.5% | 0.3969 | 0.4448 | 0.6075 | 0.1945 |
| sc300 v4 (local) | v4 | 5.6% | **0.3929** | 0.4450 | 0.6206 | 0.1873 |
**Reading**: the real fraction is the dominant lever. rmix14 recovers +3.6pp fine-tuned and +1.6pp zero-shot over sc80-v4; more real is not monotone (rmix24 is lower). At 300M the mix recovers +8.8pp zero-shot / +1.8pp fine-tuned over the default v5, but on rmix14 the 300M model still does not beat the 80M (FT 0.4448 vs 0.4695). At fixed v4 mix the 300M is +3.3pp zero-shot / +1.1pp FT over 80M (validation flat): the scale effect is small, the mix effect is larger. With the corpus fixed at v4, the 300M model beats the 80M by +3.3pp zero-shot / +1.1pp fine-tuned (validation flat), so the earlier v5 negative result was partly confounded by the mix. Still, at this budget the mix lever (+3.6pp FT at 80M) is larger than the scale lever (80M→300M at fixed v4). The best single model is `sc80-v5-rmix14` (published here as an ablation checkpoint); the best overall result is its late fusion with `process-graph-fm/gfm-sc80` (test 0.4832, see process-graph-fm).
## Real-log expansion — 13 new real logs (real-only ablation)
To test whether **more real data** is the next lever, we added 13 real event logs
(CoSeLoG WABO 1–5, CoSeLoG Receipt, BPIC 2017 Offer, Airline, ICP, KIM, MCRM,
finale, Hospital Billing; ~1.13M events, ~0.58M training events) and rebuilt the
global vocabulary (`vocab v7`: 333 log ids, 930 activities, 3,771 resources).
Two 89.1M models were trained on **real logs only** at equal budget (~0.38B
positions, cap 4,000 windows per log):
| folder | real logs | epochs × steps | zero-shot | FT test acc | FT top3 | FT macro-F1 |
|---|---|---:|---:|---:|---:|---:|
| `scaling/ablation/real-only-cap/real18cap/` | 18 | 80 × 145 | 0.3703 | 0.4418 | 0.5960 | 0.1909 |
| `scaling/ablation/real-only-cap/real31cap/` | 31 (18 + 13 new) | 55 × 210 | 0.3010 | **0.4457** | **0.6036** | 0.1892 |
References: sc80 on corpus v4 (0.3600 / 0.4336), real-only 18 logs without cap
(0.3683 / 0.4377), sc80 v5 rmix14 (0.3969 / **0.4695**).
- **Adding 13 logs helps fine-tuning only marginally**: 0.4418 → 0.4457
  (+0.4pp), above sc80-v4 and the 18-log no-cap control, but far from rmix14.
- **Zero-shot drops** 0.3703 → 0.3010: the larger vocabulary and log mixture
  dilute the lexical overlap that drives transfer to BPIC 2015-1. Real-only
  scaling is not the lever; the next step is a mix/vocabulary-controlled run
  with the new logs.
- Raw logs are **not redistributed**; provenance, counts and licenses are in
  [`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md).
  The ICP log contains untimed `ArtificialStartTask` events: the prep imputes
  the case-minimum timestamp, and training now fails fast on non-finite
  features. Single seed per run.
## MoE ablation (archived)
A 1.1B-parameter sparse MoE (8 experts, top-2, ~340M active) was trained for 2 epochs as an ablation. It did **not** beat the dense models: on BPIC 2015-1 fine-tuning it reached **0.4098** test accuracy vs **0.4495** for `fm-v6`, and on traffic fines it tied (0.8832 vs 0.8835). The pretraining validation plateaued after epoch 1 and the run was closed. The checkpoint is archived locally; it is intentionally not released as a flagship model.
## Data & licensing
Model weights and vocabularies are released under **MIT**. The raw event logs remain under their original licenses (mostly 4TU General Terms of Use; BPIC 2018 is CC0; the Improving Process Trees logs are CC BY 4.0) and are not redistributed. The 13-log expansion used in the real-only ablation is documented with per-log sources, counts and licenses in [`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md). The synthetic half of the pretraining corpus is available at [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer) under MIT.
## Related repositories
- [`Polifemus/prefix-next-activity`](https://huggingface.co/Polifemus/prefix-next-activity) — GNN vs fine-tuned transformer study (the original 7-log benchmark).
- [`Polifemus/process-graph-fm`](https://huggingface.co/Polifemus/process-graph-fm) — graph (GPS) foundation model.
- [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer) — synthetic event-log corpus.
## Citation
```bibtex
@misc{process-fm,
  title  = {Process-FM: Event-Log Foundation Models and Scaling Suite},
  author = {Polidori, Filippo},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Polifemus/process-fm}}
}
```

	---
	license: mit
	library_name: pytorch
	pipeline_tag: tabular-classification
	tags:
	- process-mining
	- predictive-process-monitoring
	- next-activity-prediction
	- foundation-model
	- transformer
	- scaling-laws
	- event-log
	datasets:
	- Polifemus/event-log-mixer
	---

	# Process-FM — event-log foundation models

	Causal transformer foundation models for predictive process monitoring: given the prefix of a case (activities, resources, timestamps), predict the next activity and the next resource. All models share the same global vocabulary and the same `inference.py`; they differ in size and training budget.

	This repository contains the fm-v6 foundation checkpoint, the scaling suite (sc5 → sc27 → sc80 → sc300) and two real-only ablations on an expanded 333-log vocabulary.

	## Models

	\| folder \| params \| d_model / layers \| training corpus \| positions seen \| val next-activity acc \| BPIC 2015-1 zero-shot \| BPIC 2015-1 FT test acc \| FT top3 \| FT macro-F1 \|
	\|---\|---:\|---\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| `fm-v6/` \| 27.4M \| 512 / 8 \| v3: 18 real + 40 synthetic (58 logs) \| ~0.43B \| 0.7527 \| 0.3480 \| 0.4495 \| 0.6436 \| 0.205 \|
	\| `scaling/sc5/` \| 6.32M \| 320 / 4 \| v4: 18 real + 300 synthetic (308 logs) \| 0.04B \| 0.6026 \| 0.2977 \| 0.3931 \| 0.5642 \| 0.163 \|
	\| `scaling/sc27/` \| 27.36M \| 512 / 8 \| v4 \| 0.13B \| 0.6451 \| 0.3068 \| 0.4296 \| 0.5865 \| 0.1877 \|
	\| `scaling/sc80/` \| 88.19M \| 768 / 12 \| v4 \| 0.38B \| 0.6563 \| 0.3600 \| 0.4336 \| 0.6280 \| 0.1861 \|
	\| `scaling/sc300/` \| 306.4M \| 1024 / 24 \| v5 (1 epoch, reduced budget) \| ~1.28B \| 0.6080* \| 0.3090 \| 0.4267 \| 0.6064 \| 0.1932 \|
	\| — (local, not released) \| 306.4M \| 1024 / 24 \| v4: 18 real + 300 synthetic (308 logs) \| ~1.28B \| 0.6538† \| 0.3929 \| 0.4450 \| 0.6206 \| 0.1873 \|
	\| `scaling/ablation/sc80-v5-rmix14/` \| 88.19M \| 768 / 12 \| v5 rmix14 (18 real ×14 + 2,500 synth) \| ~0.41B \| 0.6158 \| 0.3969 \| 0.4695 \| 0.6456 \| 0.2096 \|
	\| `scaling/ablation/real-only-cap/real18cap/` \| 89.1M \| 768 / 12 \| real-only: 18 logs, cap 4000 (vocab v7) \| ~0.38B \| 0.8133‡ \| 0.3703 \| 0.4418 \| 0.5960 \| 0.1909 \|
	\| `scaling/ablation/real-only-cap/real31cap/` \| 89.1M \| 768 / 12 \| real-only: 31 logs (18 + 13 new), cap 4000 (vocab v7) \| ~0.38B \| 0.8291‡ \| 0.3010 \| 0.4457 \| 0.6036 \| 0.1892 \|

	Notes:
	- `fm-v6` and the `sc` models use different corpora and vocabularies* (58 vs 308 seen logs) and are not directly comparable. The scaling curve is `sc5 → sc27 → sc80 → sc300`.
	- † sc300-v4 is a controlled local run (best epoch 8); its checkpoint is not released. Validation is on corpus v4 and comparable with the other v4 points.
	- The `sc` points are under-budget* by design (0.22-0.42× the 20-token/parameter target; sc300 uses 4.2 tokens/parameter), so the curve measures the under-trained regime, not Chinchilla-optimal performance. See `SCALING.md`.
	- sc300 on corpus v4 (local ablation, checkpoint not released): a controlled follow-up on the same corpus as sc5/sc27/sc80, 61 epochs (~1.28B positions), best epoch 8. It isolates scale at fixed mix: validation is flat vs sc80 (0.6538 vs 0.6563) while transfer improves (+3.3pp zero-shot, +1.1pp FT). The mix remains the stronger lever: sc80 on rmix14 reaches 0.4695 FT, still the best single model. See the data-mix table below.
	- `sc300` is a negative result on the default v5 mix: trained on corpus v5 (2,500 synthetic shards + 18 real logs), it did not improve transfer over sc80. Its validation accuracy (0.6080) is measured on a different, synthetic-heavy validation set and is not comparable with sc5/sc27/sc80 (v4). Zero-shot on BPIC 2015-1 is 0.3090 (0.2869 without unseen-to-UNK) and fine-tuned test accuracy 0.4267, both below sc80. The data-mix ablation below shows the bottleneck is the real fraction: sc300 on rmix14 reaches 0.3969 zero-shot / 0.4448 fine-tuned, but still does not beat the 80M model on the same mix (0.4695). See `SCALING.md`.
	- `‡` The two `real-only-cap` rows use an expanded vocabulary (333 log ids, 934/3,773 activity/resource rows) and a real-only corpus; their validation is on the respective held-out real-only cases and is not comparable with the `fm-v6`/`sc*` rows. See `scaling/ablation/real-only-cap/README.md`.

	## Quickstart

	```bash
	pip install torch numpy
	```

	```python
	import sys
	sys.path.insert(0, ".")
	from inference import load_model, predict

	model, vocab = load_model("fm-v6") # or "scaling/sc80"
	events = [
	{"activity": "A_SUBMITTED", "resource": "112", "timestamp": "2011-09-30 22:38:44"},
	{"activity": "A_PARTLYSUBMITTED", "resource": "112", "timestamp": "2011-09-30 22:38:44"},
	{"activity": "A_PREACCEPTED", "resource": "112", "timestamp": "2011-09-30 22:39:37"},
	]
	print(predict(model, vocab, events, top_k=3, log="bpi2012"))
	```

	The models use a global vocabulary shipped in each folder: ids depend on the checkpoint (fm-v6/sc*: activities 4..776, resources 2..2739; real-only ablation: activities 4..933, resources 2..3772), with `UNK=1`; `vocab["log_ids"]` maps the seen logs. `inference.py` handles the global-id decoding. The log id is a conditioning input, not a task-specific head; for a new log you can use the UNK log id or fine-tune with the project code.

	## Training

	- Objective: causal next-event modeling (next activity + next resource) over 256-event windows.
	- Architecture: pre-norm transformer encoder, learned positional embeddings, activity/resource/log embeddings + 6 numeric features; `id_dropout=0.15`, `log_dropout=0.15`.
	- Data: event logs from BPIC/process-mining corpora plus synthetic logs generated by our DFG mixer (released as [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer)). Raw real logs are not redistributed. The real-only ablation uses 13 additional logs; sources and licenses are in [`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md).
	- Fine-tuning protocol for the transfer numbers: full FT, causal attention for BPIC 2015-1, full bidirectional attention for traffic fines, 30 epochs, early stopping on validation, lr 3e-4.

	## Scaling

	`SCALING.md` contains the protocol, token budgets, caveats and the full table. Headline: on corpus v4, validation next-activity accuracy rises 0.6026 → 0.6451 → 0.6563 across 6.3M → 27.4M → 88.2M and then flattens at 306M (0.6538, best epoch 8), while zero-shot transfer keeps improving slightly (0.2977 → 0.3068 → 0.3600 → 0.3929). The scale effect at fixed mix is small; the real-data fraction is the dominant lever: rmix14 at 80M reaches 0.4695 fine-tuned and 0.3969 zero-shot, the best single model. The 306M point on corpus v5 (0.3090 zero-shot / 0.4267 FT) was confounded by the synthetic-heavy mix; see `SCALING.md`.

	## Data-mix ablation (corpus v5)

	The v5 corpus is 2,500 synthetic shards plus only 18 real logs. We ablated the real fraction by repeating the real shards ×14 (rmix14, ~9.5% of windows) and ×24 (rmix24, ~15.7%), at the same 80M architecture and an equal token budget (cap 600 windows/shard, 1 epoch):

	\| model \| mix \| real windows \| zs BPIC 2015-1 \| FT test acc \| top3 \| F1 \|
	\|---\|---\|---:\|---:\|---:\|---:\|---:\|
	\| sc80 (v4 ref) \| v4 \| 5.6% \| 0.3810 \| 0.4336 \| 0.6280 \| 0.1861 \|
	\| sc80 v5 \| default \| 0.7% \| 0.3363 \| 0.4149 \| 0.5918 \| 0.1706 \|
	\| sc80 v5 rmix14 \| ×14 \| 9.5% \| 0.3969 \| 0.4695 \| 0.6456 \| 0.2096 \|
	\| sc80 v5 rmix24 \| ×24 \| 15.7% \| 0.3654 \| 0.4597 \| 0.6423 \| 0.1967 \|
	\| sc300 v5 \| default \| 0.7% \| 0.3090 \| 0.4267 \| 0.6064 \| 0.1932 \|
	\| sc300 v5 rmix14 \| ×14 \| 9.5% \| 0.3969 \| 0.4448 \| 0.6075 \| 0.1945 \|
	\| sc300 v4 (local) \| v4 \| 5.6% \| 0.3929 \| 0.4450 \| 0.6206 \| 0.1873 \|

	Reading: the real fraction is the dominant lever. rmix14 recovers +3.6pp fine-tuned and +1.6pp zero-shot over sc80-v4; more real is not monotone (rmix24 is lower). At 300M the mix recovers +8.8pp zero-shot / +1.8pp fine-tuned over the default v5, but on rmix14 the 300M model still does not beat the 80M (FT 0.4448 vs 0.4695). At fixed v4 mix the 300M is +3.3pp zero-shot / +1.1pp FT over 80M (validation flat): the scale effect is small, the mix effect is larger. With the corpus fixed at v4, the 300M model beats the 80M by +3.3pp zero-shot / +1.1pp fine-tuned (validation flat), so the earlier v5 negative result was partly confounded by the mix. Still, at this budget the mix lever (+3.6pp FT at 80M) is larger than the scale lever (80M→300M at fixed v4). The best single model is `sc80-v5-rmix14` (published here as an ablation checkpoint); the best overall result is its late fusion with `process-graph-fm/gfm-sc80` (test 0.4832, see process-graph-fm).

	## Real-log expansion — 13 new real logs (real-only ablation)

	To test whether more real data is the next lever, we added 13 real event logs
	(CoSeLoG WABO 1–5, CoSeLoG Receipt, BPIC 2017 Offer, Airline, ICP, KIM, MCRM,
	finale, Hospital Billing; ~1.13M events, ~0.58M training events) and rebuilt the
	global vocabulary (`vocab v7`: 333 log ids, 930 activities, 3,771 resources).
	Two 89.1M models were trained on real logs only at equal budget (~0.38B
	positions, cap 4,000 windows per log):

	\| folder \| real logs \| epochs × steps \| zero-shot \| FT test acc \| FT top3 \| FT macro-F1 \|
	\|---\|---\|---:\|---:\|---:\|---:\|---:\|
	\| `scaling/ablation/real-only-cap/real18cap/` \| 18 \| 80 × 145 \| 0.3703 \| 0.4418 \| 0.5960 \| 0.1909 \|
	\| `scaling/ablation/real-only-cap/real31cap/` \| 31 (18 + 13 new) \| 55 × 210 \| 0.3010 \| 0.4457 \| 0.6036 \| 0.1892 \|

	References: sc80 on corpus v4 (0.3600 / 0.4336), real-only 18 logs without cap
	(0.3683 / 0.4377), sc80 v5 rmix14 (0.3969 / 0.4695).

	- Adding 13 logs helps fine-tuning only marginally: 0.4418 → 0.4457
	(+0.4pp), above sc80-v4 and the 18-log no-cap control, but far from rmix14.
	- Zero-shot drops 0.3703 → 0.3010: the larger vocabulary and log mixture
	dilute the lexical overlap that drives transfer to BPIC 2015-1. Real-only
	scaling is not the lever; the next step is a mix/vocabulary-controlled run
	with the new logs.
	- Raw logs are not redistributed; provenance, counts and licenses are in
	[`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md).
	The ICP log contains untimed `ArtificialStartTask` events: the prep imputes
	the case-minimum timestamp, and training now fails fast on non-finite
	features. Single seed per run.

	## MoE ablation (archived)

	A 1.1B-parameter sparse MoE (8 experts, top-2, ~340M active) was trained for 2 epochs as an ablation. It did not beat the dense models: on BPIC 2015-1 fine-tuning it reached 0.4098 test accuracy vs 0.4495 for `fm-v6`, and on traffic fines it tied (0.8832 vs 0.8835). The pretraining validation plateaued after epoch 1 and the run was closed. The checkpoint is archived locally; it is intentionally not released as a flagship model.

	## Data & licensing

	Model weights and vocabularies are released under MIT. The raw event logs remain under their original licenses (mostly 4TU General Terms of Use; BPIC 2018 is CC0; the Improving Process Trees logs are CC BY 4.0) and are not redistributed. The 13-log expansion used in the real-only ablation is documented with per-log sources, counts and licenses in [`scaling/ablation/real-only-cap/SOURCES.md`](scaling/ablation/real-only-cap/SOURCES.md). The synthetic half of the pretraining corpus is available at [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer) under MIT.

	## Related repositories

	- [`Polifemus/prefix-next-activity`](https://huggingface.co/Polifemus/prefix-next-activity) — GNN vs fine-tuned transformer study (the original 7-log benchmark).
	- [`Polifemus/process-graph-fm`](https://huggingface.co/Polifemus/process-graph-fm) — graph (GPS) foundation model.
	- [`Polifemus/event-log-mixer`](https://huggingface.co/datasets/Polifemus/event-log-mixer) — synthetic event-log corpus.

	## Citation

	```bibtex
	@misc{process-fm,
	title = {Process-FM: Event-Log Foundation Models and Scaling Suite},
	author = {Polidori, Filippo},
	year = {2026},
	howpublished = {\url{https://huggingface.co/Polifemus/process-fm}}
	}
	```