Add files using upload-large-folder tool

.gitignore

@@ -0,0 +1,20 @@
+.DS_Store
+.worktrees/
+.venv/
+.pytest_cache/
+__pycache__/
+*.py[cod]
+upstreams/
+data/manifests
+data/raw/
+data/processed/
+data/training_cache/
+checkpoints/
+results/
+work/
+*.tar
+*.tar.gz
+*.tgz
+*.zip
+*.conda
+*.env

README.md

@@ -0,0 +1,108 @@
+# Diffusion Language Baselines
+
+这是一个面向 Linux GPU 服务器的可复现实验框架，用于比较连续、离散、混合扩散语言模型及其少步蒸馏版本。当前 Mac/worktree 只保存代码、锁文件、脚本和文档；数据、源码仓库快照、checkpoint、Conda 环境、训练和推理都必须在服务器执行。
+
+## 实验范围
+
+- 多步生成：FLM、LangFlow、Duo、MDLM、CANDI、RDLM。
+- 少步/单步蒸馏：FMLM、Duo+DCD、Duo+Di4C、MDLM+SDTT、MDLM+Di4C。
+- LM1B 使用 `bert-base-uncased`、长度 128；OWT 使用 `gpt2`、长度 1024。
+- 多步步数：多数 many-step 方法使用 `1 2 4 8 16 32 1024`；RDLM 只使用官方默认 `1000` 和本项目原保留的 `1024`；少步步数：`1 2 4 8 16 32`。
+- 每个受支持配置生成 1,024 个样本，seed 固定为 42。
+- 质量指标：GPT-2 Large Generative PPL、平均 unigram entropy、Self-BLEU；主延迟指标：独立 CUDA timing 的 seconds/sample。
+
+132 个受支持 generation cell 会写入 `results/matrix/generation.tsv`。RDLM/OWT 不会伪造任务，原因写入 `results/matrix/unsupported.tsv`。RDLM 的 `1/2/4/8/16/32` 不作为主 baseline：原始方法是约 1000 步 SDE 采样，低步数只是强行截断迭代次数，已在服务器诊断中表现出非规范 token 与异常低 PPL。
+
+## 服务器最短路径
+
+在 Linux GPU 服务器上执行：
+
+```bash
+git clone <你的 GitHub 仓库地址> diffusion_baseline
+cd diffusion_baseline
+conda create -n dlb-bootstrap python=3.11 -y
+conda activate dlb-bootstrap
+export DLB_ROOT="$PWD"
+export DLB_PYTHON="$CONDA_PREFIX/bin/python"
+```
+
+先准备一个能导入 `PyYAML`、`pydantic` 的 bootstrap Python；它只用于解析 manifest 和启动下载脚本：
+
+```bash
+"$DLB_PYTHON" -m pip install -e ".[dev,data,checkpoints]"
+```
+
+同步本仓库代码到 Linux GPU 服务器后，严格按以下顺序执行：
+
+```bash
+bash scripts/fetch_sources.sh
+"$DLB_PYTHON" scripts/verify_sources.py --root "$DLB_ROOT"
+"$DLB_PYTHON" scripts/fetch_data.py --root "$DLB_ROOT"
+"$DLB_PYTHON" scripts/preprocess_data.py --root "$DLB_ROOT" --dataset all
+"$DLB_PYTHON" scripts/verify_data.py --root "$DLB_ROOT" --dataset all
+"$DLB_PYTHON" scripts/fetch_checkpoints.py --root "$DLB_ROOT" --all-public
+"$DLB_PYTHON" scripts/verify_checkpoints.py --root "$DLB_ROOT"
+bash envs/create_all.sh
+bash envs/verify_all.sh
+bash scripts/smoke_all.sh
+bash scripts/run_all.sh
+bash scripts/evaluate_all.sh
+bash scripts/benchmark_all.sh
+"$DLB_PYTHON" scripts/aggregate_results.py --root "$DLB_ROOT"
+```
+
+单机 4 卡服务器可使用本仓库的本地并行入口，默认绑定 GPU `0,1,2,3`：
+
+```bash
+export DLB_GPUS=0,1,2,3
+export DLB_MAX_JOBS=4
+bash scripts/smoke_4gpu.sh
+bash scripts/run_4gpu.sh
+bash scripts/evaluate_4gpu.sh
+bash scripts/benchmark_4gpu.sh
+"$DLB_PYTHON" scripts/aggregate_results.py --root "$DLB_ROOT"
+```
+
+这条路径仍要求先补齐 `reference_reproduction` checkpoint。更多说明见
+[4-GPU 本地并行运行](docs/09_four_gpu_local.md)。
+
+`smoke_all.sh` 使用独立的 `results/smoke/`，不会被正式汇总接受。若只需先验证一个方法，可运行：
+
+```bash
+bash scripts/run_one.sh --model flm --dataset lm1b --steps 1 \
+  --num-samples 1 --seed 42
+```
+
+严格汇总失败时，不要删除失败目录；先查看 `results/summary/failures.csv`，修复后重跑对应阶段。需要先查看当前可用结果时：
+
+```bash
+"$DLB_PYTHON" scripts/aggregate_results.py --root "$DLB_ROOT" --partial
+```
+
+`--partial` 只用于诊断，不是论文主结果。
+
+## 文档索引
+
+- [源码与可用性](docs/01_sources_and_availability.md)
+- [数据下载与预处理](docs/02_datasets.md)
+- [Conda 环境](docs/03_environments.md)
+- [Checkpoint 下载与校验](docs/04_checkpoints.md)
+- [训练与蒸馏](docs/05_training.md)
+- [生成、评测与 timing](docs/06_generation_and_evaluation.md)
+- [Slurm](docs/07_slurm.md)
+- [故障排查](docs/08_troubleshooting.md)
+- [4-GPU 本地并行运行](docs/09_four_gpu_local.md)
+- [实验矩阵](docs/EXPERIMENT_MATRIX.md)
+- [ELF：OWT、前缀续写、WMT14/XSum 与服务器 timing 方案](docs/10_elf_server_runbook.md)（独立入口；先用公开权重建立基线）
+- [ELF：GPU 0、1 自动核验并补齐质量实验](docs/15_elf_gpu01_queue.md)（最新执行入口；复用已完成配置）
+- [ELF：32格质量结果归档与GPU 0、1正式timing](docs/16_elf_results_and_timing.md)（质量已完成；当前执行入口）
+- [ELF：条件4步/8步短输出补评分](docs/19_elf_short_response_rescore.md)（只评分两格，保留全部样本，另存最多64-token结果）
+- [ELF：OWT公开权重，对齐旧baseline评分与timing](docs/18_elf_owt_public_baseline.md)（当前执行入口；不训练，复用样本，条件2048条重评分）
+- [ELF：OWT 64-token prompt、512-token response](docs/20_elf_owt_c64r512.md)（独立C64-R512实验；重新生成2048条样本）
+- [ELF：LM1B/OWT四类任务严格对齐方案](docs/17_elf_matched_baseline_plan.md)（历史设计，已停止执行）
+- [ELF：DS210029 数据就绪后的单张 H200 逐步执行清单](docs/11_elf_h200_next_steps.md)
+- [ELF：正式条件实验空输入故障、数据诊断与恢复](docs/12_elf_condition_recovery.md)
+- [ELF：条件输入 EOS 修复、validation/test 与 timing 恢复](docs/13_elf_eos_fix.md)
+- [ELF：9 月 13 日结果总览与剩余质量实验（timing 暂缓）](docs/14_elf_quality_remaining.md)
+
+所有可复现输入均在 `artifacts/` 和 `configs/` 中锁定。不要把 `data/`、`checkpoints/`、`upstreams/`、`results/` 或 Conda archive 提交到 Git。

checkpoints/quarantine/20260802T040542Z/official/flm/lm1b/README.md

@@ -0,0 +1,43 @@
+---
+tags:
+- model_hub_mixin
+- pytorch_model_hub_mixin
+language:
+- en
+license: apache-2.0
+---
+  
+  ## FLM (LM1B)
+
+  For more information and implementation details, visit our github repository: [FLM](https://github.com/david3684/flm).
+
+  ## Model Details
+
+  The model has a context length of 128, trained for 1M steps on the One Billion Words (LM1B) dataset.
+
+  For more details, please see our paper:
+  - [arXiv:2602.16813](https://arxiv.org/abs/2602.16813)
+  - [Huggingface Papers](https://huggingface.co/papers/2602.16813)
+    
+  ## Citation
+
+  Please cite our work using the bibtex below:
+
+  **BibTeX:**
+
+
+  ```bibtex
+  @article{lee2026flow,
+      title={Flow Map Language Models: One-step Language Modeling via Continuous Denoising},
+      author={Chanhyuk Lee and Jaehoon Yoo and Manan Agarwal
+              and Sheel Shah and Jerry Huang
+              and Aditi Raghunathan and Seunghoon Hong
+              and Nicholas M. Boffi and Jinwoo Kim},
+      journal={arXiv preprint arXiv:2602.16813},
+      year={2026},
+  }
+  ```
+
+  ## Model Card Contact
+
+  Chanhyuk Lee (chanhyuk-lee@kaist.ac.kr)

checkpoints/quarantine/20260802T040542Z/official/flm/lm1b/config.json

@@ -0,0 +1,209 @@
+{
+  "config": {
+    "algo": {
+      "T": 0,
+      "backbone": "dit",
+      "causal_attention": false,
+      "curriculum_end": 1000000,
+      "curriculum_start": 0,
+      "double_temb": false,
+      "flow_loss_type": "ce",
+      "flow_ratio": 0.75,
+      "gumbel_tau_log10_end": -0.0,
+      "gumbel_tau_log10_start": -0.0,
+      "ignore_bos": false,
+      "jvp_api": "autograd",
+      "loss_scale": "none",
+      "n_separated_blocks": -1,
+      "name": "dos",
+      "parameterization": "mean",
+      "pred_log_interval": 2000,
+      "sigma_min": 0.0001,
+      "subs_masking": false,
+      "t_max": 1.0,
+      "t_min": 0.0,
+      "time_condition": "alpha_t",
+      "time_conditioning": true,
+      "use_curriculum": true,
+      "use_discrete_schedule": true
+    },
+    "callbacks": {
+      "checkpoint_every_n_steps": {
+        "_target_": "lightning.pytorch.callbacks.ModelCheckpoint",
+        "auto_insert_metric_name": false,
+        "dirpath": "/home/david3684/projects/flm/hf_upload/checkpoints",
+        "every_n_train_steps": 50000,
+        "save_last": false,
+        "save_top_k": -1,
+        "verbose": true
+      },
+      "checkpoint_monitor": {
+        "_target_": "lightning.pytorch.callbacks.ModelCheckpoint",
+        "auto_insert_metric_name": false,
+        "dirpath": "/home/david3684/projects/flm/hf_upload/checkpoints",
+        "filename": "best_nll",
+        "mode": "min",
+        "monitor": "val/nll",
+        "save_last": true,
+        "save_top_k": 1,
+        "verbose": true
+      },
+      "learning_rate_monitor": {
+        "_target_": "lightning.pytorch.callbacks.LearningRateMonitor",
+        "logging_interval": "step"
+      }
+    },
+    "checkpointing": {
+      "resume_ckpt_path": "/home/work/RADAR/workspace/KAIST/discrete-mean-flow/text/outputs/lm1b/2026.01.05/154016/checkpoints/47-650000.ckpt",
+      "resume_from_ckpt": true,
+      "save_dir": "/home/david3684/projects/flm/hf_upload"
+    },
+    "data": {
+      "cache_dir": "/home/work/RADAR/workspace/KAIST/cdBDD/datasets/lm1b",
+      "insert_train_eos": true,
+      "insert_valid_eos": true,
+      "streaming": false,
+      "tokenizer_name_or_path": "bert-base-uncased",
+      "train": "lm1b",
+      "valid": "lm1b",
+      "wrap": true
+    },
+    "eval": {
+      "checkpoint_path": "",
+      "compute_generative_perplexity": true,
+      "compute_perplexity_on_sanity": false,
+      "disable_ema": false,
+      "gen_ppl_eval_model_name_or_path": "gpt2-large",
+      "generate_samples": true,
+      "generated_samples_path": "/home/david3684/projects/flm/hf_upload/samples.json",
+      "perplexity_batch_size": 8
+    },
+    "is_di4c": false,
+    "is_di4c_deterministic": true,
+    "loader": {
+      "batch_size": 128,
+      "eval_batch_size": 128,
+      "eval_global_batch_size": 512,
+      "global_batch_size": 512,
+      "num_workers": 192,
+      "pin_memory": true
+    },
+    "lr_scheduler": {
+      "_target_": "transformers.get_constant_schedule_with_warmup",
+      "num_warmup_steps": 2500
+    },
+    "mode": "train",
+    "model": {
+      "cond_dim": 128,
+      "dropout": 0.1,
+      "hidden_size": 768,
+      "length": 128,
+      "n_blocks": 12,
+      "n_heads": 12,
+      "name": "small",
+      "scale_by_sigma": true,
+      "tie_word_embeddings": false,
+      "type": "ddit",
+      "vocab_lookup": true
+    },
+    "noise": {
+      "denoiser_latent_conditioning": -1,
+      "eps": 0,
+      "freeze_decoder": false,
+      "freeze_encoder": false,
+      "parameterization": "log-linear",
+      "type": "log-linear"
+    },
+    "optim": {
+      "beta1": 0.9,
+      "beta2": 0.999,
+      "eps": 1e-08,
+      "ln_tune": "none",
+      "lr": 0.0003,
+      "use_muon": false,
+      "weight_decay": 0
+    },
+    "prior": {
+      "latent_height": 0,
+      "latent_width": 0,
+      "type": "none"
+    },
+    "sampling": {
+      "argmax_correction": false,
+      "hard_start": false,
+      "noise_removal": "meanflow",
+      "noise_removal_list": [
+        "uniform",
+        "uniform_alpha"
+      ],
+      "num_reflow_samples": 50000,
+      "num_sample_batches": 1,
+      "num_sample_log": 2,
+      "num_strides": 1,
+      "p_nucleus": 1.0,
+      "predictor": "ancestral",
+      "semi_ar": false,
+      "solver": "euler",
+      "steps": [
+        1024
+      ],
+      "stride_length": 1,
+      "tau_log10": 0.0,
+      "temperature": 1.0,
+      "use_float64": true
+    },
+    "seed": 1,
+    "strategy": {
+      "_target_": "lightning.pytorch.strategies.DDPStrategy",
+      "find_unused_parameters": true
+    },
+    "trainer": {
+      "_target_": "lightning.Trainer",
+      "accelerator": "cuda",
+      "accumulate_grad_batches": 1,
+      "devices": 8,
+      "gradient_clip_val": 1.0,
+      "limit_train_batches": 1.0,
+      "limit_val_batches": 1.0,
+      "log_every_n_steps": 100,
+      "max_steps": 1000000,
+      "num_nodes": 1,
+      "num_sanity_val_steps": 2,
+      "precision": "bf16",
+      "val_check_interval": 10000
+    },
+    "training": {
+      "antithetic_sampling": true,
+      "change_of_variables": false,
+      "double_ema": 0.0,
+      "ema": 0.9999,
+      "finetune_path": "",
+      "importance_sampling": false,
+      "load_ema": false,
+      "loss_precision": "bf16",
+      "loss_type": "flow",
+      "loss_weighting": false,
+      "not_sampling_t": false,
+      "pred_type": "x0",
+      "sampling_eps": 0.001,
+      "t_curriculum_max": 1.0,
+      "t_curriculum_min": 0.0,
+      "t_curriculum_steps": 100000,
+      "torch_compile_mode": "default",
+      "use_torch_compile": true
+    },
+    "wandb": {
+      "group": null,
+      "id": "fc867e70",
+      "job_type": null,
+      "name": "lm1b_full_flow_no_input_softmax_no_loss_scale_ce_loss_fc867e70",
+      "notes": "DOS",
+      "project": "lm1b_full",
+      "tags": [
+        "lm1b",
+        "dos"
+      ]
+    }
+  },
+  "vocab_size": 30522
+}

checkpoints/quarantine/20260802T040629Z/official/duo_dcd/owt/config.py

@@ -0,0 +1,29 @@
+import transformers
+
+
+class DUOConfig(transformers.PretrainedConfig):
+  """Hugging Face configuration class for DUO."""
+  model_type = 'DUO'
+
+  def __init__(
+    self,
+    vocab_size: int = 50258,
+    model_length: int = 1024,
+    causal: bool = False,
+    hidden_dim: int = 768,
+    cond_dim: int = 129,
+    n_blocks: int = 12,
+    n_heads: int = 12,
+    dropout: float = 0.1,
+    var_min: bool = True,
+    ** kwargs):
+    super().__init__(**kwargs)
+    self.causal = causal
+    self.vocab_size = vocab_size
+    self.model_length = model_length
+    self.hidden_dim = hidden_dim
+    self.cond_dim = cond_dim
+    self.n_blocks = n_blocks
+    self.n_heads = n_heads
+    self.dropout = dropout
+    self.var_min = var_min

checkpoints/quarantine/20260802T040637Z/official/mdlm/owt/modeling_mdlm.py

@@ -0,0 +1,450 @@
+"""MDLM model for Hugging Face.
+
+"""
+import itertools
+import math
+import typing
+
+import flash_attn
+import flash_attn.layers.rotary
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import transformers
+from einops import rearrange
+from transformers import modeling_outputs
+
+from .configuration_mdlm import MDLMConfig
+
+# Flags required to enable jit fusion kernels
+torch._C._jit_set_profiling_mode(False)
+torch._C._jit_set_profiling_executor(False)
+torch._C._jit_override_can_fuse_on_cpu(True)
+torch._C._jit_override_can_fuse_on_gpu(True)
+
+
+def bias_dropout_add_scale(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float,
+    training: bool) -> torch.Tensor:
+  if bias is not None:
+    out = scale * F.dropout(x + bias, p=prob,
+                            training=training)
+  else:
+    out = scale * F.dropout(x, p=prob, training=training)
+
+  if residual is not None:
+    out = residual + out
+  return out
+
+
+def get_bias_dropout_add_scale(training):
+  def _bias_dropout_add(x, bias, scale, residual, prob):
+    return bias_dropout_add_scale(
+      x, bias, scale, residual, prob, training)
+
+  return _bias_dropout_add
+
+
+# function overload
+def modulate(x: torch.Tensor,
+             shift: torch.Tensor,
+             scale: torch.Tensor) -> torch.Tensor:
+  return x * (1 + scale) + shift
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_train(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, True)
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_inference(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, False)
+
+
+@torch.jit.script
+def modulate_fused(x: torch.Tensor,
+                   shift: torch.Tensor,
+                   scale: torch.Tensor) -> torch.Tensor:
+  return modulate(x, shift, scale)
+
+
+class Rotary(torch.nn.Module):
+  def __init__(self, dim, base=10_000):
+    super().__init__()
+    inv_freq = 1.0 / (
+          base ** (torch.arange(0, dim, 2).float() / dim))
+    self.register_buffer('inv_freq', inv_freq)
+    self.seq_len_cached = None
+    self.cos_cached = None
+    self.sin_cached = None
+
+  def forward(self, x, seq_dim=1):
+    seq_len = x.shape[seq_dim]
+    if seq_len != self.seq_len_cached:
+      self.seq_len_cached = seq_len
+      t = torch.arange(x.shape[seq_dim],
+                       device=x.device).type_as(
+        self.inv_freq)
+      freqs = torch.einsum("i,j->ij", t,
+                           self.inv_freq.clone())
+      emb = torch.cat((freqs, freqs), dim=-1).to(x.device)
+      # dims are: batch, seq_len, qkv, head, dim
+      self.cos_cached = emb.cos()[None, :, None, None,
+                        :].repeat(1, 1, 3, 1, 1)
+      self.sin_cached = emb.sin()[None, :, None, None,
+                        :].repeat(1, 1, 3, 1, 1)
+      # This makes the transformation on v an identity.
+      self.cos_cached[:, :, 2, :, :].fill_(1.)
+      self.sin_cached[:, :, 2, :, :].fill_(0.)
+
+    return self.cos_cached, self.sin_cached
+
+
+def rotate_half(x):
+  x1, x2 = x[..., : x.shape[-1] // 2], x[...,
+                                       x.shape[-1] // 2:]
+  return torch.cat((-x2, x1), dim=-1)
+
+
+def apply_rotary_pos_emb(qkv, cos, sin):
+  cos = cos[0, :, 0, 0, :cos.shape[-1] // 2]
+  sin = sin[0, :, 0, 0, :sin.shape[-1] // 2]
+  return flash_attn.layers.rotary.apply_rotary_emb_qkv_(qkv,
+                                                        cos,
+                                                        sin)
+# function overload
+def modulate(x, shift, scale):
+  return x * (1 + scale.unsqueeze(1)) + shift.unsqueeze(1)
+
+
+############################################################
+#                          Layers                          #
+############################################################
+class LayerNorm(nn.Module):
+  def __init__(self, dim):
+    super().__init__()
+    self.weight = nn.Parameter(torch.ones([dim]))
+    self.dim = dim
+
+  def forward(self, x):
+    with torch.cuda.amp.autocast(enabled=False):
+      x = F.layer_norm(x.float(), [self.dim])
+    return x * self.weight[None, None, :]
+
+
+def residual_linear(x, W, x_skip, residual_scale):
+  """x_skip + residual_scale * W @ x"""
+  dim_out, dim_in = W.shape[0], W.shape[1]
+  return torch.addmm(
+    x_skip.view(-1, dim_out),
+    x.view(-1, dim_in),
+    W.T,
+    alpha=residual_scale).view(*x.shape[:-1], dim_out)
+
+
+############################################################
+#     Embedding Layers for Timesteps and Class Labels      #
+############################################################
+class TimestepEmbedder(nn.Module):
+  """
+  Embeds scalar timesteps into vector representations.
+  """
+
+  def __init__(self, hidden_size,
+               frequency_embedding_size=256):
+    super().__init__()
+    self.mlp = nn.Sequential(
+      nn.Linear(frequency_embedding_size, hidden_size,
+                bias=True),
+      nn.SiLU(),
+      nn.Linear(hidden_size, hidden_size, bias=True))
+    self.frequency_embedding_size = frequency_embedding_size
+
+  @staticmethod
+  def timestep_embedding(t, dim, max_period=10000):
+    """
+    Create sinusoidal timestep embeddings.
+    :param t: a 1-D Tensor of N indices, one per batch
+              element. These may be fractional.
+    :param dim: the dimension of the output.
+    :param max_period: controls the minimum frequency of the
+    embeddings.
+    :return: an (N, D) Tensor of positional embeddings.
+    """
+    # https://github.com/openai/glide-text2im/blob/main/glide_text2im/nn.py
+    half = dim // 2
+    freqs = torch.exp(
+      - math.log(max_period)
+      * torch.arange(start=0, end=half, dtype=torch.float32)
+      / half).to(device=t.device)
+    args = t[:, None].float() * freqs[None]
+    embedding = torch.cat(
+      [torch.cos(args), torch.sin(args)], dim=-1)
+    if dim % 2:
+      embedding = torch.cat(
+        [embedding,
+         torch.zeros_like(embedding[:, :1])], dim=-1)
+    return embedding
+
+  def forward(self, t):
+    t_freq = self.timestep_embedding(t,
+                                     self.frequency_embedding_size)
+    t_emb = self.mlp(t_freq)
+    return t_emb
+
+
+class LabelEmbedder(nn.Module):
+  """Embeds class labels into vector representations.
+
+  Also handles label dropout for classifier-free guidance.
+  """
+
+  def __init__(self, num_classes, cond_size):
+    super().__init__()
+    self.embedding_table = nn.Embedding(num_classes + 1,
+                                        cond_size)
+    self.num_classes = num_classes
+
+    # TODO think of initializing with 0.02 std deviation like in original DiT paper
+
+  def forward(self, labels):
+    embeddings = self.embedding_table(labels)
+    return embeddings
+
+
+############################################################
+#                        Core Model                        #
+############################################################
+
+
+class DDiTBlock(nn.Module):
+  def __init__(self, dim, n_heads, cond_dim, mlp_ratio=4,
+               dropout=0.1):
+    super().__init__()
+    self.n_heads = n_heads
+
+    self.norm1 = LayerNorm(dim)
+    self.attn_qkv = nn.Linear(dim, 3 * dim, bias=False)
+    self.attn_out = nn.Linear(dim, dim, bias=False)
+    self.dropout1 = nn.Dropout(dropout)
+
+    self.norm2 = LayerNorm(dim)
+    self.mlp = nn.Sequential(
+      nn.Linear(dim, mlp_ratio * dim, bias=True),
+      nn.GELU(approximate='tanh'),
+      nn.Linear(mlp_ratio * dim, dim, bias=True))
+    self.dropout2 = nn.Dropout(dropout)
+    self.dropout = dropout
+
+    self.adaLN_modulation = nn.Linear(cond_dim, 6 * dim,
+                                      bias=True)
+    self.adaLN_modulation.weight.data.zero_()
+    self.adaLN_modulation.bias.data.zero_()
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, rotary_cos_sin, c, seqlens=None):
+    batch_size, seq_len = x.shape[0], x.shape[1]
+
+    bias_dropout_scale_fn = self._get_bias_dropout_scale()
+
+    (shift_msa, scale_msa, gate_msa, shift_mlp,
+     scale_mlp, gate_mlp) = self.adaLN_modulation(c)[:,
+                            None].chunk(6, dim=2)
+
+    # attention operation
+    x_skip = x
+    x = modulate_fused(self.norm1(x), shift_msa, scale_msa)
+
+    qkv = self.attn_qkv(x)
+    qkv = rearrange(qkv,
+                    'b s (three h d) -> b s three h d',
+                    three=3,
+                    h=self.n_heads)
+    with torch.cuda.amp.autocast(enabled=False):
+      cos, sin = rotary_cos_sin
+      qkv = apply_rotary_pos_emb(
+        qkv, cos.to(qkv.dtype), sin.to(qkv.dtype))
+    qkv = rearrange(qkv, 'b s ... -> (b s) ...')
+    if seqlens is None:
+      cu_seqlens = torch.arange(
+        0, (batch_size + 1) * seq_len, step=seq_len,
+        dtype=torch.int32, device=qkv.device)
+    else:
+      cu_seqlens = seqlens.cumsum(-1)
+    x = flash_attn.flash_attn_interface.flash_attn_varlen_qkvpacked_func(
+      qkv, cu_seqlens, seq_len, 0., causal=False)
+
+    x = rearrange(x, '(b s) h d -> b s (h d)', b=batch_size)
+
+    x = bias_dropout_scale_fn(self.attn_out(x),
+                              None,
+                              gate_msa,
+                              x_skip,
+                              self.dropout)
+
+    # mlp operation
+    x = bias_dropout_scale_fn(
+      self.mlp(modulate_fused(
+        self.norm2(x), shift_mlp, scale_mlp)),
+      None, gate_mlp, x, self.dropout)
+    return x
+
+
+class EmbeddingLayer(nn.Module):
+  def __init__(self, dim, vocab_dim):
+    super().__init__()
+    self.embedding = nn.Parameter(
+      torch.empty((vocab_dim, dim)))
+    torch.nn.init.kaiming_uniform_(self.embedding,
+                                   a=math.sqrt(5))
+
+  def forward(self, x):
+    return self.embedding[x]
+
+
+class DDitFinalLayer(nn.Module):
+  def __init__(self, hidden_size, out_channels, cond_dim):
+    super().__init__()
+    self.norm_final = LayerNorm(hidden_size)
+    self.linear = nn.Linear(hidden_size, out_channels)
+    self.linear.weight.data.zero_()
+    self.linear.bias.data.zero_()
+
+    self.adaLN_modulation = nn.Linear(cond_dim,
+                                      2 * hidden_size,
+                                      bias=True)
+    self.adaLN_modulation.weight.data.zero_()
+    self.adaLN_modulation.bias.data.zero_()
+
+  def forward(self, x, c):
+    shift, scale = self.adaLN_modulation(c)[:, None].chunk(
+      2, dim=2)
+    x = modulate_fused(self.norm_final(x), shift, scale)
+    x = self.linear(x)
+    return x
+
+
+class DITBackbone(nn.Module):
+  def __init__(
+      self,
+      config: MDLMConfig):
+    super().__init__()
+
+    self.config = config
+    self.vocab_size = config.vocab_size
+
+    self.vocab_embed = EmbeddingLayer(
+      config.hidden_dim,
+      config.vocab_size)
+    self.sigma_map = TimestepEmbedder(
+      config.cond_dim)
+    self.rotary_emb = Rotary(
+      config.hidden_dim // config.n_heads)
+
+    blocks = []
+    for _ in range(config.n_blocks):
+      blocks.append(DDiTBlock(config.hidden_dim,
+                              config.n_heads,
+                              config.cond_dim,
+                              dropout=config.dropout))
+    self.blocks = nn.ModuleList(blocks)
+
+    self.output_layer = DDitFinalLayer(
+      config.hidden_dim,
+      config.vocab_size,
+      config.cond_dim)
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+  def forward(self, indices, sigma,
+              output_hidden_states=False):
+    if not self.config.time_conditioning:
+      sigma = torch.zeros_like(sigma)
+    all_hidden_states = []
+    x = self.vocab_embed(indices)
+    if output_hidden_states:
+      all_hidden_states.append(x)
+    c = F.silu(self.sigma_map(sigma))
+
+    rotary_cos_sin = self.rotary_emb(x)
+
+    with torch.cuda.amp.autocast(dtype=torch.bfloat16):
+      for i in range(len(self.blocks)):
+        x = self.blocks[i](x, rotary_cos_sin, c,
+                           seqlens=None)
+        if output_hidden_states:
+          all_hidden_states.append(x)
+      logits = self.output_layer(x, c)
+    return logits, all_hidden_states
+
+class MDLM(transformers.PreTrainedModel):
+  """HF-compatible model."""
+  config_class = MDLMConfig
+  base_model_prefix = "mdlm"
+
+  def __init__(
+    self,
+    config: MDLMConfig):
+    super().__init__(config)
+    self.backbone = DITBackbone(config)
+
+  def forward(
+      self,
+      input_ids: torch.LongTensor = None,
+      timesteps: torch.FloatTensor = None,
+      output_hidden_states: typing.Optional[bool] = None,
+      return_dict: typing.Optional[bool] = None,
+  ) -> typing.Union[
+    torch.Tensor, typing.Tuple,
+    modeling_outputs.MaskedLMOutput]:
+    """HF-compatible forward method."""
+    output_hidden_states = (
+      output_hidden_states
+      if output_hidden_states is not None
+      else self.config.output_hidden_states
+    )
+    return_dict = return_dict \
+      if return_dict is not None \
+      else self.config.use_return_dict
+
+    logits, all_hidden_states = self.backbone(
+      indices=input_ids,
+      sigma=timesteps,
+      output_hidden_states=output_hidden_states
+    )
+    if return_dict:
+      return modeling_outputs.MaskedLMOutput(
+        logits=logits,
+        hidden_states=all_hidden_states if output_hidden_states else None,
+        loss=None
+      )
+    elif output_hidden_states:
+      return logits, all_hidden_states
+    else:
+      return logits

checkpoints/quarantine/20260818T074500Z/official/langflow/owt/config.py

@@ -0,0 +1,63 @@
+"""HuggingFace configuration class for LangFlow."""
+
+import transformers
+
+
+class LangFlowConfig(transformers.PretrainedConfig):
+    """HuggingFace configuration class for LangFlow.
+    
+    LangFlow is a continuous diffusion language model that operates in embedding space.
+    It uses a DiT (Diffusion Transformer) backbone with adaptive layer normalization.
+    
+    Key features:
+        - Continuous diffusion in embedding space
+        - Self-conditioning: uses previous predictions as additional input
+        - Bias (preconditioning): skip connection for improved training
+        - Normalized embeddings: layernorm on embedding vectors
+        - Learnable Gumbel proposal for gamma (log-SNR) sampling
+    """
+    model_type = "LangFlow"
+
+    def __init__(
+        self,
+        vocab_size: int = 50257,
+        hidden_size: int = 768,
+        cond_dim: int = 128,
+        n_blocks: int = 12,
+        n_heads: int = 12,
+        dropout: float = 0.1,
+        model_length: int = 1024,
+        # Embedding normalization
+        use_normalized_embedding: bool = True,
+        embedding_norm_method: str = "layernorm",
+        # Self-conditioning
+        self_conditioning: bool = True,
+        # Bias (preconditioning) - always enabled for inference
+        use_bias: bool = True,
+        # Gumbel proposal parameters (learnable)
+        gumbel_loc: float = 4.723,
+        gumbel_scale: float = 0.852,
+        gumbel_cutoff: float = 1e-5,
+        gumbel_entropy: float = 7.02,
+        **kwargs
+    ):
+        super().__init__(**kwargs)
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.cond_dim = cond_dim
+        self.n_blocks = n_blocks
+        self.n_heads = n_heads
+        self.dropout = dropout
+        self.model_length = model_length
+        # Embedding normalization
+        self.use_normalized_embedding = use_normalized_embedding
+        self.embedding_norm_method = embedding_norm_method
+        # Self-conditioning
+        self.self_conditioning = self_conditioning
+        # Bias (preconditioning)
+        self.use_bias = use_bias
+        # Gumbel proposal parameters
+        self.gumbel_loc = gumbel_loc
+        self.gumbel_scale = gumbel_scale
+        self.gumbel_cutoff = gumbel_cutoff
+        self.gumbel_entropy = gumbel_entropy

checkpoints/quarantine/20260820T072218Z/official/duo/owt/model.py

@@ -0,0 +1,601 @@
+import math
+import typing
+
+import einops
+import flash_attn
+import flash_attn.layers.rotary
+import huggingface_hub
+import omegaconf
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import transformers
+
+from .config import DUOConfig
+
+# Flags required to enable jit fusion kernels
+torch._C._jit_set_profiling_mode(False)
+torch._C._jit_set_profiling_executor(False)
+torch._C._jit_override_can_fuse_on_cpu(True)
+torch._C._jit_override_can_fuse_on_gpu(True)
+
+
+def bias_dropout_add_scale(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float,
+    training: bool) -> torch.Tensor:
+  if bias is not None:
+    out = scale * F.dropout(x + bias, p=prob, training=training)
+  else:
+    out = scale * F.dropout(x, p=prob, training=training)
+
+  if residual is not None:
+    out = residual + out
+  return out
+
+
+def get_bias_dropout_add_scale(training):
+  def _bias_dropout_add(x, bias, scale, residual, prob):
+    return bias_dropout_add_scale(
+      x, bias, scale, residual, prob, training)
+
+  return _bias_dropout_add
+
+
+# function overload
+def modulate(x: torch.Tensor,
+             shift: torch.Tensor,
+             scale: torch.Tensor) -> torch.Tensor:
+  return x * (1 + scale) + shift
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_train(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, True)
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_inference(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, False)
+
+
+@torch.jit.script
+def modulate_fused(x: torch.Tensor,
+                   shift: torch.Tensor,
+                   scale: torch.Tensor) -> torch.Tensor:
+  return modulate(x, shift, scale)
+
+
+class Rotary(torch.nn.Module):
+  def __init__(self, dim, base=10_000):
+    super().__init__()
+    inv_freq = 1.0 / (base ** (torch.arange(0, dim, 2).float() / dim))
+    self.register_buffer('inv_freq', inv_freq)
+    self.seq_len_cached = None
+    self.cos_cached = None
+    self.sin_cached = None
+
+  def forward(self, x, seq_dim=1):
+    seq_len = x.shape[seq_dim]
+    if seq_len != self.seq_len_cached:
+      self.seq_len_cached = seq_len
+      t = torch.arange(x.shape[seq_dim], device=x.device).type_as(self.inv_freq)
+      freqs = torch.einsum("i,j->ij", t, self.inv_freq.clone())
+      emb = torch.cat((freqs, freqs), dim=-1).to(x.device)
+      # dims are: batch, seq_len, qkv, head, dim
+      self.cos_cached = emb.cos()[None, :, None, None, :].repeat(1,1,3,1,1)
+      self.sin_cached = emb.sin()[None, :, None, None, :].repeat(1,1,3,1,1)
+      # This makes the transformation on v an identity.
+      self.cos_cached[:,:,2,:,:].fill_(1.)
+      self.sin_cached[:,:,2,:,:].fill_(0.)
+
+    return self.cos_cached, self.sin_cached
+
+
+def rotate_half(x):
+  x1, x2 = x[..., : x.shape[-1] // 2], x[..., x.shape[-1] // 2 :]
+  return torch.cat((-x2, x1), dim=-1)
+
+
+def split_and_apply_rotary_pos_emb(qkv, rotary_cos_sin):
+  with torch.cuda.amp.autocast(enabled=False):
+    cos, sin = rotary_cos_sin
+    cos = cos.to(qkv.dtype)
+    sin = sin.to(qkv.dtype)
+    cos = cos[0,:,0,0,:cos.shape[-1]//2]
+    sin = sin[0,:,0,0,:sin.shape[-1]//2]
+    q, k, v = qkv.chunk(3, dim=2)
+    q = flash_attn.layers.rotary.apply_rotary_emb_torch(
+      q.squeeze(dim=2), cos, sin)
+    k = flash_attn.layers.rotary.apply_rotary_emb_torch(
+      k.squeeze(dim=2), cos, sin)
+    v = v.squeeze(dim=2)
+  return q, k, v
+
+
+def apply_rotary_pos_emb(qkv, cos, sin):
+  cos = cos[0,:,0,0,:cos.shape[-1]//2]
+  sin = sin[0,:,0,0,:sin.shape[-1]//2]
+  return flash_attn.layers.rotary.apply_rotary_emb_qkv_(qkv, cos, sin)
+
+
+def regular_attention_multi_headed(q, k, v):
+  # Assuming qkv is a tensor with shape [batch, seq_len, 3, num_heads, head_dim]
+  # where the 3 represents Q, K, V packed in that order
+  attention_output = F.scaled_dot_product_attention(
+    query=q.transpose(1, 2),
+    key=k.transpose(1, 2),
+    value=v.transpose(1, 2),
+    attn_mask=None,
+    dropout_p=0.0,
+    is_causal=False)
+  # [batch_size, seq_len, num_heads, head_dim]
+  attention_output = attention_output.transpose(1, 2)
+  return einops.rearrange(attention_output, 'b s h d -> b s (h d)')
+
+
+#################################################################################
+#                                  Layers                                       #
+#################################################################################
+class LayerNorm(nn.Module):
+  def __init__(self, dim):
+    super().__init__()
+    self.weight = nn.Parameter(torch.ones([dim]))
+    self.dim = dim
+  def forward(self, x):
+    with torch.cuda.amp.autocast(enabled=False):
+      x = F.layer_norm(x.float(), [self.dim])
+    return x * self.weight[None, None, :]
+
+
+def residual_linear(x, W, x_skip, residual_scale):
+  """x_skip + residual_scale * W @ x"""
+  dim_out, dim_in = W.shape[0], W.shape[1]
+  return torch.addmm(
+    x_skip.view(-1, dim_out),
+    x.view(-1, dim_in),
+    W.T,
+    alpha=residual_scale).view(*x.shape[:-1], dim_out)
+
+
+#################################################################################
+#               Embedding Layers for Timesteps and Class Labels                 #
+#################################################################################
+class TimestepEmbedder(nn.Module):
+  """
+  Embeds scalar timesteps into vector representations.
+  """
+  def __init__(self, hidden_size, frequency_embedding_size=256):
+    super().__init__()
+    self.mlp = nn.Sequential(
+      nn.Linear(frequency_embedding_size, hidden_size, bias=True),
+      nn.SiLU(),
+      nn.Linear(hidden_size, hidden_size, bias=True))
+    self.frequency_embedding_size = frequency_embedding_size
+
+  @staticmethod
+  def timestep_embedding(t, dim, max_period=10000):
+    """
+    Create sinusoidal timestep embeddings.
+    :param t: a 1-D Tensor of N indices, one per batch element.
+                      These may be fractional.
+    :param dim: the dimension of the output.
+    :param max_period: controls the minimum frequency of the embeddings.
+    :return: an (N, D) Tensor of positional embeddings.
+    """
+    # https://github.com/openai/glide-text2im/blob/main/glide_text2im/nn.py
+    half = dim // 2
+    freqs = torch.exp(
+      - math.log(max_period)
+      * torch.arange(start=0, end=half, dtype=torch.float32, device=t.device)
+      / half)
+    args = t[:, None].float() * freqs[None]
+    embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)
+    if dim % 2:
+      embedding = torch.cat(
+        [embedding,
+         torch.zeros_like(embedding[:, :1])], dim=-1)
+    return embedding
+
+  def forward(self, t):
+    t_freq = self.timestep_embedding(t, self.frequency_embedding_size)
+    t_emb = self.mlp(t_freq)
+    return t_emb
+
+
+class LabelEmbedder(nn.Module):
+  """Embeds class labels into vector representations.
+  
+  Also handles label dropout for classifier-free guidance.
+  """
+  def __init__(self, num_classes, cond_size):
+    super().__init__()
+    self.embedding_table = nn.Embedding(num_classes + 1, cond_size)
+    self.num_classes = num_classes
+
+    # TODO think of initializing with 0.02 std deviation like in original DiT paper
+
+  def forward(self, labels):
+    embeddings = self.embedding_table(labels)
+    return embeddings
+    
+
+#################################################################################
+#                                 Core Model                                    #
+#################################################################################
+
+class DDiTBlockCausal(nn.Module):
+  def __init__(self, dim, n_heads, mlp_ratio=4, dropout=0.1):
+    super().__init__()
+    self.n_heads = n_heads
+
+    self.norm1 = LayerNorm(dim)
+    self.attn_qkv = nn.Linear(dim, 3 * dim, bias=False)
+    self.attn_out = nn.Linear(dim, dim, bias=False)
+    self.dropout1 = nn.Dropout(dropout)
+
+    self.norm2 = LayerNorm(dim)
+    self.mlp = nn.Sequential(
+      nn.Linear(dim, mlp_ratio * dim, bias=True),
+      nn.GELU(approximate='tanh'),
+      nn.Linear(mlp_ratio * dim, dim, bias=True))
+    self.dropout2 = nn.Dropout(dropout)
+    self.dropout = dropout
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, rotary_cos_sin, **kwargs):
+    del kwargs
+    batch_size, seq_len = x.shape[0], x.shape[1]
+
+    bias_dropout_scale_fn = self._get_bias_dropout_scale()
+
+    # attention operation
+    x_skip = x
+    x = self.norm1(x)
+
+    qkv = self.attn_qkv(x)
+    qkv = einops.rearrange(
+      qkv,
+      'b s (three h d) -> b s three h d',
+      three=3,
+      h=self.n_heads)
+    with torch.cuda.amp.autocast(enabled=False):
+      cos, sin = rotary_cos_sin
+      qkv = apply_rotary_pos_emb(
+        qkv, cos.to(qkv.dtype), sin.to(qkv.dtype)
+      )
+    qkv = einops.rearrange(qkv, 'b s ... -> (b s) ...')
+    cu_seqlens = torch.arange(
+      0, (batch_size + 1) * seq_len,
+      step=seq_len, dtype=torch.int32, device=qkv.device)
+    x = flash_attn.flash_attn_interface.flash_attn_varlen_qkvpacked_func(
+      qkv, cu_seqlens, seq_len, 0.0, causal=True)
+
+    x = einops.rearrange(x, '(b s) h d -> b s (h d)',
+                         b=batch_size)
+
+    scale = torch.ones(1, device=x.device, dtype=x.dtype)
+    x = bias_dropout_scale_fn(
+      self.attn_out(x), None, scale, x_skip, self.dropout)
+
+    # mlp operation
+    x = bias_dropout_scale_fn(
+      self.mlp(self.norm2(x)), None, scale, x, self.dropout)
+    return x
+
+
+
+class DDiTBlock(nn.Module):
+  def __init__(self, dim, n_heads, adaLN,
+               cond_dim=None, mlp_ratio=4,
+               dropout=0.1):
+    super().__init__()
+    self.n_heads = n_heads
+    self.adaLN = adaLN
+
+    self.norm1 = LayerNorm(dim)
+    self.attn_qkv = nn.Linear(dim, 3 * dim, bias=False)
+    self.attn_out = nn.Linear(dim, dim, bias=False)
+    self.dropout1 = nn.Dropout(dropout)
+
+    self.norm2 = LayerNorm(dim)
+    self.mlp = nn.Sequential(
+      nn.Linear(dim, mlp_ratio * dim, bias=True),
+      nn.GELU(approximate='tanh'),
+      nn.Linear(mlp_ratio * dim, dim, bias=True))
+    self.dropout2 = nn.Dropout(dropout)
+    self.dropout = dropout
+
+    if self.adaLN:
+      self.adaLN_modulation = nn.Linear(cond_dim, 6 * dim)
+      self.adaLN_modulation.weight.data.zero_()
+      self.adaLN_modulation.bias.data.zero_()
+
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+
+  def forward(self, x, rotary_cos_sin, c=None):
+
+    bias_dropout_scale_fn = self._get_bias_dropout_scale()
+
+    x_skip = x
+    x = self.norm1(x)
+
+    if self.adaLN:
+      # self.adaLN_modulation(c): (128, 1536)
+      # self.adaLN_modulation(c)[:, None]: (128, 1, 1536)
+      # "" .chunk(6, dim=2) returns 6 tuples of shapes (128, 1, 256)
+      (shift_msa, scale_msa, gate_msa, shift_mlp,
+       scale_mlp, gate_mlp) = self.adaLN_modulation(c)[:, None].chunk(6, dim=2)
+      x = modulate_fused(x, shift_msa, scale_msa)
+
+    qkv = einops.rearrange(
+      self.attn_qkv(x),
+      'b s (three h d) -> b s three h d',
+      three=3,
+      h=self.n_heads)
+    q, k, v = split_and_apply_rotary_pos_emb(qkv, rotary_cos_sin)
+    
+    x = regular_attention_multi_headed(q, k, v)
+
+    if self.adaLN:
+      x = bias_dropout_scale_fn(self.attn_out(x),
+                                None,
+                                gate_msa,
+                                x_skip,
+                                self.dropout)
+      x = bias_dropout_scale_fn(
+        self.mlp(modulate_fused(
+          self.norm2(x), shift_mlp, scale_mlp)),
+        None, gate_mlp, x, self.dropout)
+    else:
+      scale = torch.ones(1, device=x.device, dtype=x.dtype)
+      x = bias_dropout_scale_fn(
+        self.attn_out(x), None, scale, x_skip, self.dropout)
+      x = bias_dropout_scale_fn(
+        self.mlp(self.norm2(x)), None, scale, x, self.dropout)
+    return x
+
+
+class EmbeddingLayer(nn.Module):
+  def __init__(self, dim, vocab_dim):
+    super().__init__()
+    self.embedding = nn.Parameter(torch.empty((vocab_dim, dim)))
+    torch.nn.init.kaiming_uniform_(self.embedding, a=math.sqrt(5))
+
+  def forward(self, x):
+    if x.ndim == 2:
+      return self.embedding[x]
+    assert x.ndim == 3
+    return torch.einsum(
+      "blv,ve->ble",
+      torch.nn.functional.softmax(x, dim=-1).float(),
+      self.embedding.float()).to(x.dtype)
+
+
+class DDiTFinalLayer(nn.Module):
+  def __init__(self, hidden_size, out_channels, cond_dim,
+               adaLN):
+    super().__init__()
+    self.norm_final = LayerNorm(hidden_size)
+    self.linear = nn.Linear(hidden_size, out_channels)
+    self.linear.weight.data.zero_()
+    self.linear.bias.data.zero_()
+    self.adaLN = adaLN
+    if self.adaLN:
+      self.adaLN_modulation = nn.Linear(cond_dim,
+                                        2 * hidden_size,
+                                        bias=True)
+      self.adaLN_modulation.weight.data.zero_()
+      self.adaLN_modulation.bias.data.zero_()
+
+
+  def forward(self, x, c):
+    x = self.norm_final(x)
+    if self.adaLN:
+      shift, scale = self.adaLN_modulation(c)[:, None].chunk(2, dim=2)
+      x = modulate_fused(x, shift, scale)
+    x = self.linear(x)
+    return x
+
+
+class DIT(nn.Module, huggingface_hub.PyTorchModelHubMixin):
+  def __init__(self, config, vocab_size: int):
+    super().__init__()
+    if type(config) == dict:
+      config = omegaconf.OmegaConf.create(config)
+    self.causal = config.algo.causal_attention
+    self.adaLN = not self.causal
+    self.config = config
+    self.vocab_size = vocab_size
+    dim = config.model.hidden_size
+    cond_dim = config.model.cond_dim
+    self.vocab_embed = EmbeddingLayer(dim, vocab_size)
+    if not self.causal:
+      self.sigma_map = TimestepEmbedder(cond_dim)
+    self.rotary_emb = Rotary(dim // config.model.n_heads)
+
+    blocks = []
+    for _ in range(config.model.n_blocks):
+      if self.causal:
+        block = DDiTBlockCausal(
+          dim=dim,
+          n_heads=config.model.n_heads,
+          dropout=config.model.dropout)
+      else:
+        block = DDiTBlock(
+          dim=dim,
+          n_heads=config.model.n_heads,
+          cond_dim=cond_dim,
+          adaLN=self.adaLN,
+          dropout=config.model.dropout)
+      blocks.append(block)
+    self.blocks = nn.ModuleList(blocks)
+
+    self.output_layer = DDiTFinalLayer(
+      hidden_size=dim,
+      out_channels=vocab_size,
+      cond_dim=cond_dim,
+      adaLN=self.adaLN)
+    self.scale_by_sigma = config.model.scale_by_sigma
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return  bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, sigma):
+    x = self.vocab_embed(x)
+    if self.causal:
+      t_cond = None
+    else:
+      t_cond = F.silu(self.sigma_map(sigma))
+
+    rotary_cos_sin = self.rotary_emb(x).to(x.device)
+
+    with torch.cuda.amp.autocast(dtype=torch.bfloat16):
+      for i in range(len(self.blocks)):
+        x = self.blocks[i](x, rotary_cos_sin, c=t_cond)
+      x = self.output_layer(x, c=t_cond)
+
+    return x
+
+
+
+class HFDIT(torch.nn.Module):
+  def __init__(self, config):
+    super().__init__()
+    self.causal = config.causal
+    self.adaLN = not self.causal
+    self.vocab_size = config.vocab_size
+    dim = config.hidden_dim
+    cond_dim = config.cond_dim
+    self.vocab_embed = EmbeddingLayer(dim, self.vocab_size)
+    if not self.causal:
+      self.sigma_map = TimestepEmbedder(cond_dim)
+    self.rotary_emb = Rotary(dim // config.n_heads)
+
+    blocks = []
+    for _ in range(config.n_blocks):
+      if self.causal:
+        block = DDiTBlockCausal(
+          dim=dim,
+          n_heads=config.n_heads,
+          dropout=config.dropout)
+      else:
+        block = DDiTBlock(
+          dim=dim,
+          n_heads=config.n_heads,
+          cond_dim=cond_dim,
+          adaLN=self.adaLN,
+          dropout=config.dropout)
+      blocks.append(block)
+    self.blocks = torch.nn.ModuleList(blocks)
+
+    self.output_layer = DDiTFinalLayer(
+      hidden_size=dim,
+      out_channels=self.vocab_size,
+      cond_dim=cond_dim,
+      adaLN=self.adaLN)
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return  bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, sigma, output_hidden_states=False):
+    all_hidden_states = []
+    x = self.vocab_embed(x)
+    if output_hidden_states:
+      all_hidden_states.append(x)
+    if self.causal:
+      t_cond = None
+    else:
+      t_cond = F.silu(self.sigma_map(sigma))
+
+    rotary_cos_sin = self.rotary_emb(x)
+    with torch.cuda.amp.autocast(
+      dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float32):
+      for i in range(len(self.blocks)):
+        x = self.blocks[i](x, rotary_cos_sin, c=t_cond)
+        if output_hidden_states:
+          all_hidden_states.append(x)
+      x = self.output_layer(x, c=t_cond)
+    return x, all_hidden_states
+
+
+class DUO(transformers.PreTrainedModel):
+  """HF-compatible model."""
+  config_class = DUOConfig
+  base_model_prefix = 'duo'
+
+  def __init__(self, config: DUOConfig):
+    super().__init__(config)
+    self.config = config
+    self.backbone = HFDIT(config)
+
+  def reset_kv_cache(self):
+    for block in self.backbone.blocks:
+      block.kv_cache = None
+
+  def forward(
+      self,
+      input_ids: torch.LongTensor = None,
+      timesteps: torch.FloatTensor = None,
+      output_hidden_states: typing.Optional[bool] = None,
+      return_dict: typing.Optional[bool] = None,
+  ) -> typing.Union[
+    torch.Tensor, typing.Tuple,
+    transformers.modeling_outputs.MaskedLMOutput]:
+    """HF-compatible forward method."""
+    output_hidden_states = (
+      output_hidden_states
+      if output_hidden_states is not None
+      else self.config.output_hidden_states
+    )
+    return_dict = return_dict \
+      if return_dict is not None \
+      else self.config.use_return_dict
+    
+    logits, all_hidden_states = self.backbone(
+      x=input_ids,
+      sigma=timesteps,
+      output_hidden_states=output_hidden_states,
+    )
+    if return_dict:
+      return transformers.modeling_outputs.MaskedLMOutput(
+        logits=logits,
+        hidden_states=all_hidden_states if output_hidden_states else None,
+        loss=None
+      )
+    elif output_hidden_states:
+      return logits, all_hidden_states
+    else:
+      return logits

checkpoints/quarantine/20260820T072232Z/official/duo_dcd/owt/README.md

@@ -0,0 +1,52 @@
+---
+datasets:
+- Skylion007/openwebtext
+language:
+- en
+library_name: transformers
+license: apache-2.0
+metrics:
+- perplexity
+pipeline_tag: text-generation
+---
+
+## Using DUO
+To use the pre-trained model for masked language modeling, use the following snippet:
+```python
+from transformers import AutoModelForMaskedLM, AutoTokenizer
+
+# See the `DUO` collection page on the hub for list of available models.
+tokenizer = transformers.AutoTokenizer.from_pretrained('gpt2')
+model = AutoModelForMaskedLM.from_pretrained('s-sahoo/duo-distilled')
+```
+For a hands-on example, check out this [Colab notebook](https://colab.research.google.com/drive/1Sf7R-dqdR6gq-H8nyZ9E3ZkyvqMTqcwq?usp=sharing).
+For more information and implementation details, visit our github repository: [DUO](https://github.com/s-sahoo/duo)
+
+## Model Details
+The model, which has a context length of `1024` and is similar in size to GPT2-medium with approximately `130 million` non-embedding parameters,
+was trained for 1M steps on the OpenWebText corpus.
+
+For more details, please see our paper: [The Diffusion Duality](https://huggingface.co/papers/2506.10892).
+
+Project page: https://s-sahoo.com/duo
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+Please cite our work using the bibtex below:
+
+**BibTeX:**
+
+```
+@inproceedings{
+sahoo2025the,
+title={The Diffusion Duality},
+author={Subham Sekhar Sahoo and Justin Deschenaux and Aaron Gokaslan and Guanghan Wang and Justin T Chiu and Volodymyr Kuleshov},
+booktitle={Forty-second International Conference on Machine Learning},
+year={2025},
+url={https://openreview.net/forum?id=9P9Y8FOSOk}
+}
+```
+
+## Model Card Contact
+Subham Sekhar Sahoo (ssahoo@cs.cornell.edu)

checkpoints/quarantine/20260820T072232Z/official/duo_dcd/owt/config.py

@@ -0,0 +1,29 @@
+import transformers
+
+
+class DUOConfig(transformers.PretrainedConfig):
+  """Hugging Face configuration class for DUO."""
+  model_type = 'DUO'
+
+  def __init__(
+    self,
+    vocab_size: int = 50258,
+    model_length: int = 1024,
+    causal: bool = False,
+    hidden_dim: int = 768,
+    cond_dim: int = 129,
+    n_blocks: int = 12,
+    n_heads: int = 12,
+    dropout: float = 0.1,
+    var_min: bool = True,
+    ** kwargs):
+    super().__init__(**kwargs)
+    self.causal = causal
+    self.vocab_size = vocab_size
+    self.model_length = model_length
+    self.hidden_dim = hidden_dim
+    self.cond_dim = cond_dim
+    self.n_blocks = n_blocks
+    self.n_heads = n_heads
+    self.dropout = dropout
+    self.var_min = var_min

checkpoints/quarantine/20260820T072244Z/official/mdlm/owt/configuration_mdlm.py

@@ -0,0 +1,31 @@
+"""MDLM config for Hugging Face.
+
+"""
+
+import transformers
+
+
+class MDLMConfig(transformers.PretrainedConfig):
+  """Hugging Face configuration class for MDLM."""
+  model_type = "mdlm"
+
+  def __init__(
+    self,
+    vocab_size: int = 50258,
+    model_length: int = 1024,
+    hidden_dim: int = 768,
+    cond_dim: int = 129,
+    n_blocks: int = 12,
+    n_heads: int = 12,
+    dropout: float = 0.1,
+    time_conditioning: bool = False,
+    ** kwargs):
+    super().__init__(**kwargs)
+    self.vocab_size = vocab_size
+    self.model_length = model_length
+    self.hidden_dim = hidden_dim
+    self.cond_dim = cond_dim
+    self.n_blocks = n_blocks
+    self.n_heads = n_heads
+    self.dropout = dropout
+    self.time_conditioning = time_conditioning

checkpoints/quarantine/20260820T072312Z/official/mdlm_sdtt/owt/config.json

@@ -0,0 +1,207 @@
+{
+  "mode": "train",
+  "time_conditioning": false,
+  "T": 0,
+  "compile": false,
+  "eval_every": 5000,
+  "data_preprocess": {
+    "data_cache": "REDACTED",
+    "min_seq_len": -1,
+    "seq_len": "${model.length}",
+    "group_text": true,
+    "remove_text": true,
+    "num_seqs": -1,
+    "add_bos": false,
+    "add_eos": true,
+    "legacy_start_end_bos": true
+  },
+  "loader": {
+    "global_batch_size": 32,
+    "eval_global_batch_size": "${.global_batch_size}",
+    "batch_size": "${div_up:${.global_batch_size}, ${eval:${trainer.devices} * ${trainer.num_nodes}}}",
+    "eval_batch_size": "${div_up:${.eval_global_batch_size}, ${eval:${trainer.devices} * ${trainer.num_nodes}}}",
+    "num_workers": 32,
+    "pin_memory": true,
+    "persistent_workers": true
+  },
+  "training": {
+    "ema": 0.9999,
+    "antithetic_sampling": true,
+    "importance_sampling": false,
+    "sampling_eps": 1e-05,
+    "change_of_variables": false
+  },
+  "tokenizer": {
+    "name": "gpt2"
+  },
+  "optim": {
+    "name": "adamw",
+    "weight_decay": 0.0,
+    "lr": 0.0003,
+    "beta1": 0.9,
+    "beta2": 0.999,
+    "eps": 1e-08
+  },
+  "trainer": {
+    "_target_": "lightning.Trainer",
+    "accelerator": "cuda",
+    "num_nodes": 1,
+    "devices": 1,
+    "accumulate_grad_batches": "${div_up:${loader.global_batch_size}, ${eval:${trainer.devices} * ${loader.batch_size} * ${trainer.num_nodes}}}",
+    "gradient_clip_val": 1.0,
+    "precision": "bf16-mixed",
+    "num_sanity_val_steps": 2,
+    "max_steps": 1000000,
+    "log_every_n_steps": 10,
+    "limit_train_batches": 1.0,
+    "limit_val_batches": 1.0,
+    "val_check_interval": "${eval:${trainer.accumulate_grad_batches} * ${eval_every}}",
+    "check_val_every_n_epoch": null,
+    "benchmark": true
+  },
+  "wandb": {
+    "project": "text-diffusion",
+    "notes": "",
+    "group": null,
+    "job_type": null,
+    "name": null,
+    "tags": [
+      "${noise.type}",
+      "${data.train}",
+      "${data.valid}"
+    ]
+  },
+  "checkpointing": {
+    "save_dir": "${cwd:}",
+    "resume_from_ckpt": true,
+    "resume_ckpt_path": "/claire-rcp-scratch/home/deschena/fast-discrete-diff/dev/hf_downloads/baselines/kld/baselines/bwd_kl_double_dt_10k_lr_2x_sm/70000.ckpt"
+  },
+  "eval": {
+    "valid": {
+      "n_samples": 16,
+      "num_steps": 128
+    },
+    "ppl_with_ar": {
+      "run": true,
+      "model": "gpt2-large",
+      "batch_size": 8
+    },
+    "mauve": {
+      "run": false,
+      "num_rounds": 5,
+      "max_num_tokens": 100,
+      "batch_size": 8,
+      "scaling_factor": 5
+    },
+    "lambada_openai": {
+      "run": false,
+      "batch_size": 32,
+      "num_samples": 32,
+      "from_ema": true,
+      "add_bos": true,
+      "add_eos": true
+    }
+  },
+  "callbacks": {
+    "checkpoint_every_n_steps": {
+      "_target_": "lightning.pytorch.callbacks.ModelCheckpoint",
+      "save_top_k": -1,
+      "save_last": true,
+      "dirpath": "${checkpointing.save_dir}/checkpoints",
+      "verbose": true,
+      "auto_insert_metric_name": false,
+      "every_n_train_steps": 10000
+    },
+    "learning_rate_monitor": {
+      "_target_": "lightning.pytorch.callbacks.LearningRateMonitor",
+      "logging_interval": "step"
+    },
+    "text_throughput": {
+      "_target_": "callbacks.text_throughput.TextThroughputMonitor"
+    },
+    "grad_norm": {
+      "_target_": "callbacks.gradient_norm.GradNormCallback"
+    }
+  },
+  "data": {
+    "train": "openwebtext-train",
+    "valid": "openwebtext-valid",
+    "test": "openwebtext-test",
+    "tokenizer_name_or_path": "gpt2",
+    "wrap": true,
+    "streaming": false
+  },
+  "model": {
+    "name": "small",
+    "type": "ddit-orig",
+    "hidden_size": 768,
+    "cond_dim": 128,
+    "length": 1024,
+    "n_blocks": 12,
+    "n_heads": 12,
+    "scale_by_sigma": true,
+    "dropout": 0.1,
+    "tie_word_embeddings": false
+  },
+  "strategy": {
+    "_target_": "lightning.pytorch.strategies.DDPStrategy",
+    "find_unused_parameters": false,
+    "process_group_backend": "nccl"
+  },
+  "noise": {
+    "type": "loglinear",
+    "sigma_min": 0.0001,
+    "sigma_max": 20
+  },
+  "lr_scheduler": {
+    "_target_": "transformers.get_constant_schedule_with_warmup",
+    "num_warmup_steps": 2500
+  },
+  "parameterization": {
+    "name": "multi-round-sdtt",
+    "log_loss_buckets": -1,
+    "checkpoint_path": "kuleshov-group/mdlm-owt",
+    "start_from_hf": false,
+    "distill_mode": "kl-bwd",
+    "num_distill_steps": 2,
+    "min_num_sampling_steps": 8,
+    "grow_dt_every": 10000,
+    "orig_num_sampling_steps": 1024,
+    "sampling_mode": "ancestral",
+    "loss_precision": null,
+    "reset_optimizer_on_growth": false,
+    "use_ema_on_growth": false,
+    "sampling": {
+      "uncond": {
+        "run": false,
+        "num_samples": 1024,
+        "batch_size": 32,
+        "from_ema": true,
+        "num_steps": 1024,
+        "seq_len": 1024,
+        "sampler": "ancestral",
+        "cache_preds": false,
+        "add_bos": false,
+        "add_eos": false
+      },
+      "cond_prefix": {
+        "run": false,
+        "num_samples": 1024,
+        "batch_size": 32,
+        "from_ema": true,
+        "dataset": "webtext",
+        "seq_len": 100,
+        "prefix_len": 50,
+        "num_cont_per_prefix": 5,
+        "min_seq_len": 1024,
+        "num_steps": 1024,
+        "sampler": "ancestral",
+        "cache_preds": false,
+        "add_bos": false,
+        "add_eos": false
+      }
+    }
+  },
+  "repo": "jdeschena/sdtt",
+  "revision": "baselines_kld"
+}

checkpoints/quarantine/20260820T073530Z/official/rdlm/lm1b/LM1B/config.yaml

@@ -0,0 +1,75 @@
+seed: 0
+ngpus: 8
+use_wandb: true
+run_mode: train
+num_workers: 32
+paths:
+  experiments: exps
+n_jobs: 2
+model:
+  _target_: model.dit.DiT
+  hidden_size: 768
+  n_heads: 12
+  cond_dim: 128
+  dropout: 0.0
+  n_blocks: 14
+  length: 128
+  input_dim: ${tokens}
+  output_dim: ${tokens}
+name: lm1b
+original_tokens: 30522
+tokens: 3
+training:
+  batch_size: 512
+  accum: 1
+  n_iters: 1000001
+  snapshot_freq: 50000
+  log_freq: 50
+  eval_freq: 100
+  snapshot_freq_for_preemption: 5000
+  snapshot_sampling: true
+  ema: 0.9999
+  loss_type: ce
+data:
+  train: lm1b
+  valid: lm1b
+  cache_dir: data
+sampling:
+  steps: 1000
+  predictor: grw
+eval:
+  batch_size: ${training.batch_size}
+  n_iters: 200
+  entropy: true
+  gen_ppl: true
+  eval_model: gpt2
+  nll: true
+  simul_steps: 500
+sde:
+  _target_: sde.LogBridge_Mixture
+  preprocess_steps: 10000
+  mix_type: step
+  rho_scale: 1.14
+  step_thr: 0.38
+  prior_dist:
+    _target_: distribution.Mixture
+    add_mask_token: true
+scheduler:
+  _target_: scheduler_lib.Geometric
+  beta_0: 0.001
+  beta_f: 0.2
+  weight_type: step
+  left: 0.3
+  right: 0.75
+  lb: 0.0001
+  ub: 1.0
+optim:
+  name: adamw
+  optimizer: AdamW
+  weight_decay: 0
+  lr: 0.0003
+  beta1: 0.9
+  beta2: 0.999
+  eps: 1.0e-08
+  warmup: 2500
+  grad_clip: 1.0

checkpoints/quarantine/20260821T024719Z/official/fmlm/lm1b/README.md

@@ -0,0 +1,45 @@
+---
+tags:
+- model_hub_mixin
+- pytorch_model_hub_mixin
+license: apache-2.0
+language:
+- en
+base_model:
+- david3684/FLM-B-LM1B
+---
+  
+  ## FMLM (LM1B)
+
+  For more information and implementation details, visit our github repository: [FLM](https://github.com/david3684/flm).
+
+  ## Model Details
+
+  The model has a context length of 128, distilled from 1M steps trained FLM checkpoint trained on the One Billion Words (LM1B) dataset.
+  
+  For more details, please see our paper: 
+  - [arXiv:2602.16813](https://arxiv.org/abs/2602.16813)
+  - [Huggingface Papers](https://huggingface.co/papers/2602.16813)
+
+  ## Citation
+
+  Please cite our work using the bibtex below:
+
+  **BibTeX:**
+
+
+  ```bibtex
+  @article{lee2026flow,
+      title={Flow Map Language Models: One-step Language Modeling via Continuous Denoising},
+      author={Chanhyuk Lee and Jaehoon Yoo and Manan Agarwal
+              and Sheel Shah and Jerry Huang
+              and Aditi Raghunathan and Seunghoon Hong
+              and Nicholas M. Boffi and Jinwoo Kim},
+      journal={arXiv preprint arXiv:2602.16813},
+      year={2026},
+  }
+  ```
+
+  ## Model Card Contact
+
+  Chanhyuk Lee (chanhyuk-lee@kaist.ac.kr)

checkpoints/quarantine/20260821T024719Z/official/fmlm/lm1b/config.json

@@ -0,0 +1,221 @@
+{
+  "config": {
+    "algo": {
+      "T": 0,
+      "add_boundary": "fixed",
+      "backbone": "dit",
+      "backprop_entmax_temp_lsd": false,
+      "boundary_prob": 32,
+      "causal_attention": false,
+      "diagonal_fraction": 0.5,
+      "distillation_method": "PSD",
+      "double_temb": true,
+      "entmax_temp_lsd": 2.0,
+      "initialize_student_from_teacher": false,
+      "learnable_loss_weighting": false,
+      "loss_scale": "none",
+      "loss_type": null,
+      "name": "fmlm",
+      "offdiagonal_sampling": "uniform_diff",
+      "parameterization": "mean",
+      "pred_log_interval": 2000,
+      "rescale_offdiag_loss_psd": false,
+      "subs_masking": false,
+      "t_max": 1.0,
+      "t_min": 0.0,
+      "teacher_path": "/home/david3684/ours_flow_ce_1m.ckpt",
+      "time_condition": "alpha_t",
+      "time_conditioning": true,
+      "use_discrete_schedule": true,
+      "use_mse_loss_psd": false,
+      "use_simulated_entropy": false,
+      "use_teacher_for_D_t_lsd": false
+    },
+    "callbacks": {
+      "checkpoint_every_n_steps": {
+        "_target_": "lightning.pytorch.callbacks.ModelCheckpoint",
+        "auto_insert_metric_name": false,
+        "dirpath": "/home/david3684/projects/flm/hf_upload/checkpoints",
+        "every_n_train_steps": 25000,
+        "save_last": false,
+        "save_top_k": -1,
+        "verbose": true
+      },
+      "checkpoint_monitor": {
+        "_target_": "lightning.pytorch.callbacks.ModelCheckpoint",
+        "auto_insert_metric_name": false,
+        "dirpath": "/home/david3684/projects/flm/hf_upload/checkpoints",
+        "filename": "best_nll",
+        "mode": "min",
+        "monitor": "val/nll",
+        "save_last": true,
+        "save_top_k": 1,
+        "verbose": true
+      },
+      "learning_rate_monitor": {
+        "_target_": "lightning.pytorch.callbacks.LearningRateMonitor",
+        "logging_interval": "step"
+      }
+    },
+    "checkpointing": {
+      "resume_ckpt_path": "/home/david3684/projects/flm/hf_upload/checkpoints/last.ckpt",
+      "resume_from_ckpt": false,
+      "save_dir": "/home/david3684/projects/flm/hf_upload"
+    },
+    "data": {
+      "cache_dir": "/home/david3684/datasets/lm1b",
+      "insert_train_eos": true,
+      "insert_valid_eos": true,
+      "streaming": false,
+      "tokenizer_name_or_path": "bert-base-uncased",
+      "train": "lm1b",
+      "valid": "lm1b",
+      "wrap": true
+    },
+    "eval": {
+      "checkpoint_path": "",
+      "compute_generative_perplexity": true,
+      "compute_perplexity_on_sanity": false,
+      "disable_ema": false,
+      "ema_decay": null,
+      "gen_ppl_eval_model_name_or_path": "gpt2-large",
+      "generate_samples": true,
+      "generated_samples_path": "/home/david3684/projects/flm/hf_upload/samples.json",
+      "perplexity_batch_size": 8
+    },
+    "is_di4c": false,
+    "is_di4c_deterministic": true,
+    "loader": {
+      "batch_size": 128,
+      "eval_batch_size": 128,
+      "eval_global_batch_size": 512,
+      "global_batch_size": 512,
+      "num_workers": 0,
+      "pin_memory": true
+    },
+    "lr_scheduler": {
+      "_target_": "transformers.get_constant_schedule_with_warmup",
+      "num_warmup_steps": 2500
+    },
+    "mode": "train",
+    "model": {
+      "cond_dim": 128,
+      "dropout": 0.1,
+      "hidden_size": 768,
+      "length": 128,
+      "n_blocks": 12,
+      "n_heads": 12,
+      "name": "small",
+      "scale_by_sigma": true,
+      "tie_word_embeddings": false,
+      "type": "ddit",
+      "vocab_lookup": true
+    },
+    "noise": {
+      "denoiser_latent_conditioning": -1,
+      "eps": 0,
+      "freeze_decoder": false,
+      "freeze_encoder": false,
+      "parameterization": "log-linear",
+      "type": "log-linear"
+    },
+    "optim": {
+      "beta1": 0.9,
+      "beta2": 0.999,
+      "eps": 1e-08,
+      "ln_tune": "none",
+      "lr": 0.0003,
+      "muon_lr": 0.03,
+      "muon_momentum": 0.95,
+      "muon_weight_decay": 0.01,
+      "use_muon": false,
+      "weight_decay": 0
+    },
+    "prior": {
+      "latent_height": 0,
+      "latent_width": 0,
+      "type": "none"
+    },
+    "sampling": {
+      "argmax_correction": false,
+      "autoguidance_weight": 0.0,
+      "hard_start": false,
+      "noise_removal": "uniform_alpha",
+      "num_reflow_samples": 50000,
+      "num_sample_batches": 1,
+      "num_sample_log": 2,
+      "num_strides": 1,
+      "p_nucleus": 1.0,
+      "predictor": "ancestral",
+      "semi_ar": false,
+      "solver": "euler",
+      "steps": [
+        1,
+        2,
+        4,
+        8,
+        16,
+        32,
+        64,
+        128
+      ],
+      "stride_length": 1,
+      "tau_log10": -1.0,
+      "temperature": 1.0,
+      "use_float64": true
+    },
+    "seed": 1,
+    "strategy": {
+      "_target_": "lightning.pytorch.strategies.DDPStrategy",
+      "find_unused_parameters": false
+    },
+    "trainer": {
+      "_target_": "lightning.Trainer",
+      "accelerator": "cuda",
+      "accumulate_grad_batches": 1,
+      "devices": 8,
+      "gradient_clip_val": 1.0,
+      "limit_train_batches": 1.0,
+      "limit_val_batches": 10,
+      "log_every_n_steps": 100,
+      "max_steps": 1000000,
+      "num_nodes": 1,
+      "num_sanity_val_steps": 2,
+      "precision": "bf16",
+      "val_check_interval": 10000
+    },
+    "training": {
+      "antithetic_sampling": true,
+      "change_of_variables": false,
+      "double_ema": 0.0,
+      "ema": "0.9999, 0.999, 0.99, 0.95",
+      "finetune_path": "",
+      "importance_sampling": false,
+      "load_ema": false,
+      "loss_precision": "bf16",
+      "loss_type": "sd",
+      "loss_weighting": false,
+      "not_sampling_t": false,
+      "pred_type": "x0",
+      "sampling_eps": 0.001,
+      "t_curriculum_max": 1.0,
+      "t_curriculum_min": 0.0,
+      "t_curriculum_steps": 100000,
+      "torch_compile_mode": "default",
+      "use_torch_compile": true
+    },
+    "wandb": {
+      "group": null,
+      "id": "27d989c8",
+      "job_type": null,
+      "name": "lm1b_PSD_teacher_distill_DF_0.5_BP_32_SE_False_IT_False_EDMW_False_27d989c8",
+      "notes": "DOS",
+      "project": "dos_distill",
+      "tags": [
+        "lm1b",
+        "dos_denoiser"
+      ]
+    }
+  },
+  "vocab_size": 30522
+}

checkpoints/quarantine/20260821T024800Z/official/duo/owt/README.md

@@ -0,0 +1,53 @@
+---
+datasets:
+- Skylion007/openwebtext
+language:
+- en
+library_name: transformers
+license: apache-2.0
+metrics:
+- perplexity
+pipeline_tag: text-generation
+---
+
+## Using DUO
+To use the pre-trained model for masked language modeling, use the following snippet:
+```python
+from transformers import AutoModelForMaskedLM, AutoTokenizer
+
+# See the `MDLM` collection page on the hub for list of available models.
+tokenizer = transformers.AutoTokenizer.from_pretrained('gpt2')
+model = AutoModelForMaskedLM.from_pretrained('s-sahoo/duo')
+```
+For a hands-on example, check out this [Colab notebook](https://colab.research.google.com/drive/1Sf7R-dqdR6gq-H8nyZ9E3ZkyvqMTqcwq?usp=sharing).
+For more information and implementation details, visit our github repository: [DUO](https://github.com/s-sahoo/duo) and project page: [Project Page](https://s-sahoo.com/duo)
+
+## Model Details
+The model, which has a context length of `1024` and is similar in size to GPT2-medium with approximately `130 million` non-embedding parameters,
+was trained for 1M steps on the OpenWebText corpus.
+
+For more details, please see our paper: [The Diffusion Duality](https://openreview.net/forum?id=CB0Ub2yXjC).
+
+
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+Please cite our work using the bibtex below:
+
+**BibTeX:**
+
+```
+@inproceedings{
+sahoo2025the,
+title={The Diffusion Duality},
+author={Subham Sekhar Sahoo and Justin Deschenaux and Aaron Gokaslan and Guanghan Wang and Justin T Chiu and Volodymyr Kuleshov},
+booktitle={Forty-second International Conference on Machine Learning},
+year={2025},
+url={https://openreview.net/forum?id=9P9Y8FOSOk}
+}
+```
+
+
+## Model Card Contact
+Subham Sekhar Sahoo (ssahoo@cs.cornell.edu)

checkpoints/quarantine/20260821T024800Z/official/duo/owt/model.py

@@ -0,0 +1,601 @@
+import math
+import typing
+
+import einops
+import flash_attn
+import flash_attn.layers.rotary
+import huggingface_hub
+import omegaconf
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import transformers
+
+from .config import DUOConfig
+
+# Flags required to enable jit fusion kernels
+torch._C._jit_set_profiling_mode(False)
+torch._C._jit_set_profiling_executor(False)
+torch._C._jit_override_can_fuse_on_cpu(True)
+torch._C._jit_override_can_fuse_on_gpu(True)
+
+
+def bias_dropout_add_scale(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float,
+    training: bool) -> torch.Tensor:
+  if bias is not None:
+    out = scale * F.dropout(x + bias, p=prob, training=training)
+  else:
+    out = scale * F.dropout(x, p=prob, training=training)
+
+  if residual is not None:
+    out = residual + out
+  return out
+
+
+def get_bias_dropout_add_scale(training):
+  def _bias_dropout_add(x, bias, scale, residual, prob):
+    return bias_dropout_add_scale(
+      x, bias, scale, residual, prob, training)
+
+  return _bias_dropout_add
+
+
+# function overload
+def modulate(x: torch.Tensor,
+             shift: torch.Tensor,
+             scale: torch.Tensor) -> torch.Tensor:
+  return x * (1 + scale) + shift
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_train(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, True)
+
+
+@torch.jit.script
+def bias_dropout_add_scale_fused_inference(
+    x: torch.Tensor,
+    bias: typing.Optional[torch.Tensor],
+    scale: torch.Tensor,
+    residual: typing.Optional[torch.Tensor],
+    prob: float) -> torch.Tensor:
+  return bias_dropout_add_scale(
+    x, bias, scale, residual, prob, False)
+
+
+@torch.jit.script
+def modulate_fused(x: torch.Tensor,
+                   shift: torch.Tensor,
+                   scale: torch.Tensor) -> torch.Tensor:
+  return modulate(x, shift, scale)
+
+
+class Rotary(torch.nn.Module):
+  def __init__(self, dim, base=10_000):
+    super().__init__()
+    inv_freq = 1.0 / (base ** (torch.arange(0, dim, 2).float() / dim))
+    self.register_buffer('inv_freq', inv_freq)
+    self.seq_len_cached = None
+    self.cos_cached = None
+    self.sin_cached = None
+
+  def forward(self, x, seq_dim=1):
+    seq_len = x.shape[seq_dim]
+    if seq_len != self.seq_len_cached:
+      self.seq_len_cached = seq_len
+      t = torch.arange(x.shape[seq_dim], device=x.device).type_as(self.inv_freq)
+      freqs = torch.einsum("i,j->ij", t, self.inv_freq.clone())
+      emb = torch.cat((freqs, freqs), dim=-1).to(x.device)
+      # dims are: batch, seq_len, qkv, head, dim
+      self.cos_cached = emb.cos()[None, :, None, None, :].repeat(1,1,3,1,1)
+      self.sin_cached = emb.sin()[None, :, None, None, :].repeat(1,1,3,1,1)
+      # This makes the transformation on v an identity.
+      self.cos_cached[:,:,2,:,:].fill_(1.)
+      self.sin_cached[:,:,2,:,:].fill_(0.)
+
+    return self.cos_cached, self.sin_cached
+
+
+def rotate_half(x):
+  x1, x2 = x[..., : x.shape[-1] // 2], x[..., x.shape[-1] // 2 :]
+  return torch.cat((-x2, x1), dim=-1)
+
+
+def split_and_apply_rotary_pos_emb(qkv, rotary_cos_sin):
+  with torch.cuda.amp.autocast(enabled=False):
+    cos, sin = rotary_cos_sin
+    cos = cos.to(qkv.dtype)
+    sin = sin.to(qkv.dtype)
+    cos = cos[0,:,0,0,:cos.shape[-1]//2]
+    sin = sin[0,:,0,0,:sin.shape[-1]//2]
+    q, k, v = qkv.chunk(3, dim=2)
+    q = flash_attn.layers.rotary.apply_rotary_emb_torch(
+      q.squeeze(dim=2), cos, sin)
+    k = flash_attn.layers.rotary.apply_rotary_emb_torch(
+      k.squeeze(dim=2), cos, sin)
+    v = v.squeeze(dim=2)
+  return q, k, v
+
+
+def apply_rotary_pos_emb(qkv, cos, sin):
+  cos = cos[0,:,0,0,:cos.shape[-1]//2]
+  sin = sin[0,:,0,0,:sin.shape[-1]//2]
+  return flash_attn.layers.rotary.apply_rotary_emb_qkv_(qkv, cos, sin)
+
+
+def regular_attention_multi_headed(q, k, v):
+  # Assuming qkv is a tensor with shape [batch, seq_len, 3, num_heads, head_dim]
+  # where the 3 represents Q, K, V packed in that order
+  attention_output = F.scaled_dot_product_attention(
+    query=q.transpose(1, 2),
+    key=k.transpose(1, 2),
+    value=v.transpose(1, 2),
+    attn_mask=None,
+    dropout_p=0.0,
+    is_causal=False)
+  # [batch_size, seq_len, num_heads, head_dim]
+  attention_output = attention_output.transpose(1, 2)
+  return einops.rearrange(attention_output, 'b s h d -> b s (h d)')
+
+
+#################################################################################
+#                                  Layers                                       #
+#################################################################################
+class LayerNorm(nn.Module):
+  def __init__(self, dim):
+    super().__init__()
+    self.weight = nn.Parameter(torch.ones([dim]))
+    self.dim = dim
+  def forward(self, x):
+    with torch.cuda.amp.autocast(enabled=False):
+      x = F.layer_norm(x.float(), [self.dim])
+    return x * self.weight[None, None, :]
+
+
+def residual_linear(x, W, x_skip, residual_scale):
+  """x_skip + residual_scale * W @ x"""
+  dim_out, dim_in = W.shape[0], W.shape[1]
+  return torch.addmm(
+    x_skip.view(-1, dim_out),
+    x.view(-1, dim_in),
+    W.T,
+    alpha=residual_scale).view(*x.shape[:-1], dim_out)
+
+
+#################################################################################
+#               Embedding Layers for Timesteps and Class Labels                 #
+#################################################################################
+class TimestepEmbedder(nn.Module):
+  """
+  Embeds scalar timesteps into vector representations.
+  """
+  def __init__(self, hidden_size, frequency_embedding_size=256):
+    super().__init__()
+    self.mlp = nn.Sequential(
+      nn.Linear(frequency_embedding_size, hidden_size, bias=True),
+      nn.SiLU(),
+      nn.Linear(hidden_size, hidden_size, bias=True))
+    self.frequency_embedding_size = frequency_embedding_size
+
+  @staticmethod
+  def timestep_embedding(t, dim, max_period=10000):
+    """
+    Create sinusoidal timestep embeddings.
+    :param t: a 1-D Tensor of N indices, one per batch element.
+                      These may be fractional.
+    :param dim: the dimension of the output.
+    :param max_period: controls the minimum frequency of the embeddings.
+    :return: an (N, D) Tensor of positional embeddings.
+    """
+    # https://github.com/openai/glide-text2im/blob/main/glide_text2im/nn.py
+    half = dim // 2
+    freqs = torch.exp(
+      - math.log(max_period)
+      * torch.arange(start=0, end=half, dtype=torch.float32, device=t.device)
+      / half)
+    args = t[:, None].float() * freqs[None]
+    embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)
+    if dim % 2:
+      embedding = torch.cat(
+        [embedding,
+         torch.zeros_like(embedding[:, :1])], dim=-1)
+    return embedding
+
+  def forward(self, t):
+    t_freq = self.timestep_embedding(t, self.frequency_embedding_size)
+    t_emb = self.mlp(t_freq)
+    return t_emb
+
+
+class LabelEmbedder(nn.Module):
+  """Embeds class labels into vector representations.
+  
+  Also handles label dropout for classifier-free guidance.
+  """
+  def __init__(self, num_classes, cond_size):
+    super().__init__()
+    self.embedding_table = nn.Embedding(num_classes + 1, cond_size)
+    self.num_classes = num_classes
+
+    # TODO think of initializing with 0.02 std deviation like in original DiT paper
+
+  def forward(self, labels):
+    embeddings = self.embedding_table(labels)
+    return embeddings
+    
+
+#################################################################################
+#                                 Core Model                                    #
+#################################################################################
+
+class DDiTBlockCausal(nn.Module):
+  def __init__(self, dim, n_heads, mlp_ratio=4, dropout=0.1):
+    super().__init__()
+    self.n_heads = n_heads
+
+    self.norm1 = LayerNorm(dim)
+    self.attn_qkv = nn.Linear(dim, 3 * dim, bias=False)
+    self.attn_out = nn.Linear(dim, dim, bias=False)
+    self.dropout1 = nn.Dropout(dropout)
+
+    self.norm2 = LayerNorm(dim)
+    self.mlp = nn.Sequential(
+      nn.Linear(dim, mlp_ratio * dim, bias=True),
+      nn.GELU(approximate='tanh'),
+      nn.Linear(mlp_ratio * dim, dim, bias=True))
+    self.dropout2 = nn.Dropout(dropout)
+    self.dropout = dropout
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, rotary_cos_sin, **kwargs):
+    del kwargs
+    batch_size, seq_len = x.shape[0], x.shape[1]
+
+    bias_dropout_scale_fn = self._get_bias_dropout_scale()
+
+    # attention operation
+    x_skip = x
+    x = self.norm1(x)
+
+    qkv = self.attn_qkv(x)
+    qkv = einops.rearrange(
+      qkv,
+      'b s (three h d) -> b s three h d',
+      three=3,
+      h=self.n_heads)
+    with torch.cuda.amp.autocast(enabled=False):
+      cos, sin = rotary_cos_sin
+      qkv = apply_rotary_pos_emb(
+        qkv, cos.to(qkv.dtype), sin.to(qkv.dtype)
+      )
+    qkv = einops.rearrange(qkv, 'b s ... -> (b s) ...')
+    cu_seqlens = torch.arange(
+      0, (batch_size + 1) * seq_len,
+      step=seq_len, dtype=torch.int32, device=qkv.device)
+    x = flash_attn.flash_attn_interface.flash_attn_varlen_qkvpacked_func(
+      qkv, cu_seqlens, seq_len, 0.0, causal=True)
+
+    x = einops.rearrange(x, '(b s) h d -> b s (h d)',
+                         b=batch_size)
+
+    scale = torch.ones(1, device=x.device, dtype=x.dtype)
+    x = bias_dropout_scale_fn(
+      self.attn_out(x), None, scale, x_skip, self.dropout)
+
+    # mlp operation
+    x = bias_dropout_scale_fn(
+      self.mlp(self.norm2(x)), None, scale, x, self.dropout)
+    return x
+
+
+
+class DDiTBlock(nn.Module):
+  def __init__(self, dim, n_heads, adaLN,
+               cond_dim=None, mlp_ratio=4,
+               dropout=0.1):
+    super().__init__()
+    self.n_heads = n_heads
+    self.adaLN = adaLN
+
+    self.norm1 = LayerNorm(dim)
+    self.attn_qkv = nn.Linear(dim, 3 * dim, bias=False)
+    self.attn_out = nn.Linear(dim, dim, bias=False)
+    self.dropout1 = nn.Dropout(dropout)
+
+    self.norm2 = LayerNorm(dim)
+    self.mlp = nn.Sequential(
+      nn.Linear(dim, mlp_ratio * dim, bias=True),
+      nn.GELU(approximate='tanh'),
+      nn.Linear(mlp_ratio * dim, dim, bias=True))
+    self.dropout2 = nn.Dropout(dropout)
+    self.dropout = dropout
+
+    if self.adaLN:
+      self.adaLN_modulation = nn.Linear(cond_dim, 6 * dim)
+      self.adaLN_modulation.weight.data.zero_()
+      self.adaLN_modulation.bias.data.zero_()
+
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return bias_dropout_add_scale_fused_inference
+
+
+  def forward(self, x, rotary_cos_sin, c=None):
+
+    bias_dropout_scale_fn = self._get_bias_dropout_scale()
+
+    x_skip = x
+    x = self.norm1(x)
+
+    if self.adaLN:
+      # self.adaLN_modulation(c): (128, 1536)
+      # self.adaLN_modulation(c)[:, None]: (128, 1, 1536)
+      # "" .chunk(6, dim=2) returns 6 tuples of shapes (128, 1, 256)
+      (shift_msa, scale_msa, gate_msa, shift_mlp,
+       scale_mlp, gate_mlp) = self.adaLN_modulation(c)[:, None].chunk(6, dim=2)
+      x = modulate_fused(x, shift_msa, scale_msa)
+
+    qkv = einops.rearrange(
+      self.attn_qkv(x),
+      'b s (three h d) -> b s three h d',
+      three=3,
+      h=self.n_heads)
+    q, k, v = split_and_apply_rotary_pos_emb(qkv, rotary_cos_sin)
+    
+    x = regular_attention_multi_headed(q, k, v)
+
+    if self.adaLN:
+      x = bias_dropout_scale_fn(self.attn_out(x),
+                                None,
+                                gate_msa,
+                                x_skip,
+                                self.dropout)
+      x = bias_dropout_scale_fn(
+        self.mlp(modulate_fused(
+          self.norm2(x), shift_mlp, scale_mlp)),
+        None, gate_mlp, x, self.dropout)
+    else:
+      scale = torch.ones(1, device=x.device, dtype=x.dtype)
+      x = bias_dropout_scale_fn(
+        self.attn_out(x), None, scale, x_skip, self.dropout)
+      x = bias_dropout_scale_fn(
+        self.mlp(self.norm2(x)), None, scale, x, self.dropout)
+    return x
+
+
+class EmbeddingLayer(nn.Module):
+  def __init__(self, dim, vocab_dim):
+    super().__init__()
+    self.embedding = nn.Parameter(torch.empty((vocab_dim, dim)))
+    torch.nn.init.kaiming_uniform_(self.embedding, a=math.sqrt(5))
+
+  def forward(self, x):
+    if x.ndim == 2:
+      return self.embedding[x]
+    assert x.ndim == 3
+    return torch.einsum(
+      "blv,ve->ble",
+      torch.nn.functional.softmax(x, dim=-1).float(),
+      self.embedding.float()).to(x.dtype)
+
+
+class DDiTFinalLayer(nn.Module):
+  def __init__(self, hidden_size, out_channels, cond_dim,
+               adaLN):
+    super().__init__()
+    self.norm_final = LayerNorm(hidden_size)
+    self.linear = nn.Linear(hidden_size, out_channels)
+    self.linear.weight.data.zero_()
+    self.linear.bias.data.zero_()
+    self.adaLN = adaLN
+    if self.adaLN:
+      self.adaLN_modulation = nn.Linear(cond_dim,
+                                        2 * hidden_size,
+                                        bias=True)
+      self.adaLN_modulation.weight.data.zero_()
+      self.adaLN_modulation.bias.data.zero_()
+
+
+  def forward(self, x, c):
+    x = self.norm_final(x)
+    if self.adaLN:
+      shift, scale = self.adaLN_modulation(c)[:, None].chunk(2, dim=2)
+      x = modulate_fused(x, shift, scale)
+    x = self.linear(x)
+    return x
+
+
+class DIT(nn.Module, huggingface_hub.PyTorchModelHubMixin):
+  def __init__(self, config, vocab_size: int):
+    super().__init__()
+    if type(config) == dict:
+      config = omegaconf.OmegaConf.create(config)
+    self.causal = config.algo.causal_attention
+    self.adaLN = not self.causal
+    self.config = config
+    self.vocab_size = vocab_size
+    dim = config.model.hidden_size
+    cond_dim = config.model.cond_dim
+    self.vocab_embed = EmbeddingLayer(dim, vocab_size)
+    if not self.causal:
+      self.sigma_map = TimestepEmbedder(cond_dim)
+    self.rotary_emb = Rotary(dim // config.model.n_heads)
+
+    blocks = []
+    for _ in range(config.model.n_blocks):
+      if self.causal:
+        block = DDiTBlockCausal(
+          dim=dim,
+          n_heads=config.model.n_heads,
+          dropout=config.model.dropout)
+      else:
+        block = DDiTBlock(
+          dim=dim,
+          n_heads=config.model.n_heads,
+          cond_dim=cond_dim,
+          adaLN=self.adaLN,
+          dropout=config.model.dropout)
+      blocks.append(block)
+    self.blocks = nn.ModuleList(blocks)
+
+    self.output_layer = DDiTFinalLayer(
+      hidden_size=dim,
+      out_channels=vocab_size,
+      cond_dim=cond_dim,
+      adaLN=self.adaLN)
+    self.scale_by_sigma = config.model.scale_by_sigma
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return  bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, sigma):
+    x = self.vocab_embed(x)
+    if self.causal:
+      t_cond = None
+    else:
+      t_cond = F.silu(self.sigma_map(sigma))
+
+    rotary_cos_sin = self.rotary_emb(x).to(x.device)
+
+    with torch.cuda.amp.autocast(dtype=torch.bfloat16):
+      for i in range(len(self.blocks)):
+        x = self.blocks[i](x, rotary_cos_sin, c=t_cond)
+      x = self.output_layer(x, c=t_cond)
+
+    return x
+
+
+
+class HFDIT(torch.nn.Module):
+  def __init__(self, config):
+    super().__init__()
+    self.causal = config.causal
+    self.adaLN = not self.causal
+    self.vocab_size = config.vocab_size
+    dim = config.hidden_dim
+    cond_dim = config.cond_dim
+    self.vocab_embed = EmbeddingLayer(dim, self.vocab_size)
+    if not self.causal:
+      self.sigma_map = TimestepEmbedder(cond_dim)
+    self.rotary_emb = Rotary(dim // config.n_heads)
+
+    blocks = []
+    for _ in range(config.n_blocks):
+      if self.causal:
+        block = DDiTBlockCausal(
+          dim=dim,
+          n_heads=config.n_heads,
+          dropout=config.dropout)
+      else:
+        block = DDiTBlock(
+          dim=dim,
+          n_heads=config.n_heads,
+          cond_dim=cond_dim,
+          adaLN=self.adaLN,
+          dropout=config.dropout)
+      blocks.append(block)
+    self.blocks = torch.nn.ModuleList(blocks)
+
+    self.output_layer = DDiTFinalLayer(
+      hidden_size=dim,
+      out_channels=self.vocab_size,
+      cond_dim=cond_dim,
+      adaLN=self.adaLN)
+
+  def _get_bias_dropout_scale(self):
+    if self.training:
+      return bias_dropout_add_scale_fused_train
+    else:
+      return  bias_dropout_add_scale_fused_inference
+
+  def forward(self, x, sigma, output_hidden_states=False):
+    all_hidden_states = []
+    x = self.vocab_embed(x)
+    if output_hidden_states:
+      all_hidden_states.append(x)
+    if self.causal:
+      t_cond = None
+    else:
+      t_cond = F.silu(self.sigma_map(sigma))
+
+    rotary_cos_sin = self.rotary_emb(x)
+    with torch.cuda.amp.autocast(
+      dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float32):
+      for i in range(len(self.blocks)):
+        x = self.blocks[i](x, rotary_cos_sin, c=t_cond)
+        if output_hidden_states:
+          all_hidden_states.append(x)
+      x = self.output_layer(x, c=t_cond)
+    return x, all_hidden_states
+
+
+class DUO(transformers.PreTrainedModel):
+  """HF-compatible model."""
+  config_class = DUOConfig
+  base_model_prefix = 'duo'
+
+  def __init__(self, config: DUOConfig):
+    super().__init__(config)
+    self.config = config
+    self.backbone = HFDIT(config)
+
+  def reset_kv_cache(self):
+    for block in self.backbone.blocks:
+      block.kv_cache = None
+
+  def forward(
+      self,
+      input_ids: torch.LongTensor = None,
+      timesteps: torch.FloatTensor = None,
+      output_hidden_states: typing.Optional[bool] = None,
+      return_dict: typing.Optional[bool] = None,
+  ) -> typing.Union[
+    torch.Tensor, typing.Tuple,
+    transformers.modeling_outputs.MaskedLMOutput]:
+    """HF-compatible forward method."""
+    output_hidden_states = (
+      output_hidden_states
+      if output_hidden_states is not None
+      else self.config.output_hidden_states
+    )
+    return_dict = return_dict \
+      if return_dict is not None \
+      else self.config.use_return_dict
+    
+    logits, all_hidden_states = self.backbone(
+      x=input_ids,
+      sigma=timesteps,
+      output_hidden_states=output_hidden_states,
+    )
+    if return_dict:
+      return transformers.modeling_outputs.MaskedLMOutput(
+        logits=logits,
+        hidden_states=all_hidden_states if output_hidden_states else None,
+        loss=None
+      )
+    elif output_hidden_states:
+      return logits, all_hidden_states
+    else:
+      return logits

checkpoints/quarantine/20260821T024817Z/official/mdlm/owt/README.md

@@ -0,0 +1,67 @@
+---
+library_name: transformers
+license: apache-2.0
+language:
+- en
+datasets:
+- Skylion007/openwebtext
+metrics:
+- perplexity
+---
+
+## Using MDLM
+To use the pre-trained model for masked language modeling, use the following snippet:
+```python
+from transformers import AutoModelForMaskedLM, AutoTokenizer
+
+# See the `MDLM` collection page on the hub for list of available models.
+tokenizer = transformers.AutoTokenizer.from_pretrained('gpt2')
+model_name = 'kuleshov-group/mdlm-owt'
+model = AutoModelForMaskedLM.from_pretrained(model_name)
+```
+
+For more details, please see our github repository: [MDLM](https://github.com/kuleshov-group/mdlm)
+
+## Model Details
+The model, which has a context length of `1024` and is similar in size to GPT2-medium with approximately `130 million` non-embedding parameters,
+was trained using a forward diffusion process that generates inputs varying from fully masked to fully unmasked. Its objective is to
+reconstruct the original input from these varying levels of masking, outputting logits in the process.
+The training regimen comprised one million steps on the OpenWebText corpus, involving the processing of a total of `33 billion` tokens.
+
+For more details, please see our paper: [Simple and Effective Masked Diffusion Language Models](http://arxiv.org/abs/2406.07524).
+
+
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+Please cite our work using the bibtex below:
+
+**BibTeX:**
+
+```
+@inproceedings{
+    sahoo2024simple,
+    title={Simple and Effective Masked Diffusion Language Models},
+    author={Subham Sekhar Sahoo and Marianne Arriola and Aaron Gokaslan and Edgar Mariano Marroquin and Alexander M Rush and Yair Schiff and Justin T Chiu and Volodymyr Kuleshov},
+    booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
+    year={2024},
+    url={https://openreview.net/forum?id=L4uaAR4ArM}
+}
+```
+
+**APA:**
+
+```
+@software{Sahoo_Simple_and_Effective_2024,
+author = {Sahoo, Subham Sekhar and Arriola, Marianne and Schiff, Yair and Gokaslan, Aaron and Marroquin, Edgar and Chiu, Justin T and Rush, Alexander and Kuleshov, Volodymyr},
+doi = {10.48550/arXiv.2406.07524},
+month = jun,
+title = {{Simple and Effective Masked Diffusion Language Models}},
+version = {arXiv:2406.07524v1},
+year = {2024}
+}
+```
+
+## Model Card Contact
+Subham Sekhar Sahoo (subbham@gmail.com)

checkpoints/quarantine/20260825T100201Z/official/langflow/owt/config.json

@@ -0,0 +1,29 @@
+{
+  "_name_or_path": "chumengl/langflow-owt",
+  "architectures": [
+    "LangFlow"
+  ],
+  "auto_map": {
+    "AutoConfig": "config.LangFlowConfig",
+    "AutoModelForMaskedLM": "model.LangFlow"
+  },
+  "model_type": "LangFlow",
+  "vocab_size": 50257,
+  "hidden_size": 768,
+  "cond_dim": 128,
+  "n_blocks": 12,
+  "n_heads": 12,
+  "dropout": 0.1,
+  "model_length": 1024,
+  "use_normalized_embedding": true,
+  "embedding_norm_method": "layernorm",
+  "self_conditioning": true,
+  "use_bias": true,
+  "gumbel_loc": 4.723,
+  "gumbel_scale": 0.852,
+  "gumbel_cutoff": 1e-5,
+  "gumbel_entropy": 7.02,
+  "return_dict": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.2"
+}

checkpoints/quarantine/20260825T100259Z/official/duo_dcd/owt/README.md

@@ -0,0 +1,52 @@
+---
+datasets:
+- Skylion007/openwebtext
+language:
+- en
+library_name: transformers
+license: apache-2.0
+metrics:
+- perplexity
+pipeline_tag: text-generation
+---
+
+## Using DUO
+To use the pre-trained model for masked language modeling, use the following snippet:
+```python
+from transformers import AutoModelForMaskedLM, AutoTokenizer
+
+# See the `DUO` collection page on the hub for list of available models.
+tokenizer = transformers.AutoTokenizer.from_pretrained('gpt2')
+model = AutoModelForMaskedLM.from_pretrained('s-sahoo/duo-distilled')
+```
+For a hands-on example, check out this [Colab notebook](https://colab.research.google.com/drive/1Sf7R-dqdR6gq-H8nyZ9E3ZkyvqMTqcwq?usp=sharing).
+For more information and implementation details, visit our github repository: [DUO](https://github.com/s-sahoo/duo)
+
+## Model Details
+The model, which has a context length of `1024` and is similar in size to GPT2-medium with approximately `130 million` non-embedding parameters,
+was trained for 1M steps on the OpenWebText corpus.
+
+For more details, please see our paper: [The Diffusion Duality](https://huggingface.co/papers/2506.10892).
+
+Project page: https://s-sahoo.com/duo
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+Please cite our work using the bibtex below:
+
+**BibTeX:**
+
+```
+@inproceedings{
+sahoo2025the,
+title={The Diffusion Duality},
+author={Subham Sekhar Sahoo and Justin Deschenaux and Aaron Gokaslan and Guanghan Wang and Justin T Chiu and Volodymyr Kuleshov},
+booktitle={Forty-second International Conference on Machine Learning},
+year={2025},
+url={https://openreview.net/forum?id=9P9Y8FOSOk}
+}
+```
+
+## Model Card Contact
+Subham Sekhar Sahoo (ssahoo@cs.cornell.edu)

create_duo_mdlm.log

@@ -0,0 +1,762 @@
+Channels:
+ - pytorch
+ - nvidia
+ - conda-forge
+ - defaults
+Platform: linux-64
+Collecting package metadata (repodata.json): ...working... done
+Solving environment: ...working... done
+
+Downloading and Extracting Packages: ...working...
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
                                                     
+
                                                     
+
+
                                                     
+
+
+
                                                     
+
+
+
+
                                                     
+
+
+
+
+
                                                     
+
+
+
+
+
+
                                                     
+
+
+
+
+
+
+
                                                     
+
+
+
+
+
+
+
+
                                                      done
+Preparing transaction: - done
+Verifying transaction: | / done
+Executing transaction: \ | / - \ | / - \ done
+Installing pip dependencies: / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - Ran pip subprocess with arguments:
+['/opt/anaconda3/envs/dlb-duo/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt', '--exists-action=b']
+Pip subprocess output:
+Requirement already satisfied: datasets==2.15.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2.15.0)
+Requirement already satisfied: einops==0.7.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 2)) (0.7.0)
+Requirement already satisfied: fsspec==2023.10.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 3)) (2023.10.0)
+Requirement already satisfied: h5py==3.10.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 4)) (3.10.0)
+Requirement already satisfied: hydra-core==1.3.2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 5)) (1.3.2)
+Requirement already satisfied: lightning==2.2.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (2.2.1)
+Requirement already satisfied: nvitop==1.3.2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 7)) (1.3.2)
+Requirement already satisfied: omegaconf==2.3.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 8)) (2.3.0)
+Requirement already satisfied: pandas==2.2.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 9)) (2.2.1)
+Requirement already satisfied: rich==13.7.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 10)) (13.7.1)
+Requirement already satisfied: scikit-learn==1.4.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 11)) (1.4.0)
+Requirement already satisfied: timm==0.9.16 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 12)) (0.9.16)
+Requirement already satisfied: torchmetrics==1.6.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 13)) (1.6.1)
+Requirement already satisfied: tqdm==4.66.2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 14)) (4.66.2)
+Requirement already satisfied: transformers==4.38.2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 15)) (4.38.2)
+Requirement already satisfied: wandb==0.18.7 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (0.18.7)
+Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2.3.1)
+Requirement already satisfied: pyarrow>=8.0.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (25.0.0)
+Requirement already satisfied: pyarrow-hotfix in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (0.7)
+Requirement already satisfied: dill<0.3.8,>=0.3.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (0.3.7)
+Requirement already satisfied: requests>=2.19.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2.34.2)
+Requirement already satisfied: xxhash in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (3.8.1)
+Requirement already satisfied: multiprocess in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (0.70.15)
+Requirement already satisfied: aiohttp in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (3.14.3)
+Requirement already satisfied: huggingface-hub>=0.18.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (0.36.2)
+Requirement already satisfied: packaging in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (24.2)
+Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (6.0.3)
+Requirement already satisfied: antlr4-python3-runtime==4.9.* in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from hydra-core==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 5)) (4.9.3)
+Requirement already satisfied: lightning-utilities<2.0,>=0.8.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (0.15.3)
+Requirement already satisfied: torch<4.0,>=1.13.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (2.5.1)
+Requirement already satisfied: typing-extensions<6.0,>=4.4.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (4.16.0)
+Requirement already satisfied: pytorch-lightning in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (2.6.5)
+Requirement already satisfied: nvidia-ml-py<12.536.0a0,>=11.450.51 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 7)) (12.535.161)
+Requirement already satisfied: psutil>=5.6.6 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 7)) (7.2.2)
+Requirement already satisfied: cachetools>=1.0.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 7)) (7.1.6)
+Requirement already satisfied: termcolor>=1.0.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 7)) (3.3.0)
+Collecting numpy>=1.17 (from datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1))
+  Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
+Requirement already satisfied: python-dateutil>=2.8.2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 9)) (2.9.0.post0)
+Requirement already satisfied: pytz>=2020.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 9)) (2026.3.post1)
+Requirement already satisfied: tzdata>=2022.7 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 9)) (2026.3)
+Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 10)) (4.2.0)
+Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 10)) (2.20.0)
+Requirement already satisfied: scipy>=1.6.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from scikit-learn==1.4.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 11)) (1.17.1)
+Requirement already satisfied: joblib>=1.2.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from scikit-learn==1.4.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 11)) (1.5.3)
+Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from scikit-learn==1.4.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 11)) (3.6.0)
+Requirement already satisfied: torchvision in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 12)) (0.20.1)
+Requirement already satisfied: safetensors in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 12)) (0.8.0)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 15)) (3.32.2)
+Requirement already satisfied: regex!=2019.12.17 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 15)) (2026.7.19)
+Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 15)) (0.15.2)
+Requirement already satisfied: click!=8.0.0,>=7.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (8.4.2)
+Requirement already satisfied: docker-pycreds>=0.4.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (0.4.0)
+Requirement already satisfied: gitpython!=3.1.29,>=1.0.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (3.1.57)
+Requirement already satisfied: platformdirs in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (4.11.0)
+Requirement already satisfied: protobuf!=4.21.0,!=5.28.0,<6,>=3.19.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (5.29.6)
+Requirement already satisfied: sentry-sdk>=2.0.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (2.66.1)
+Requirement already satisfied: setproctitle in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (1.3.7)
+Requirement already satisfied: setuptools in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (73.0.1)
+Requirement already satisfied: six>=1.4.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from docker-pycreds>=0.4.0->wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (1.17.0)
+Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2.7.1)
+Requirement already satisfied: aiosignal>=1.4.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (1.4.0)
+Requirement already satisfied: attrs>=17.3.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (26.1.0)
+Requirement already satisfied: frozenlist>=1.1.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (1.8.0)
+Requirement already satisfied: multidict<7.0,>=4.5 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (6.7.1)
+Requirement already satisfied: propcache>=0.2.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (0.5.2)
+Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from aiohttp->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (1.24.5)
+Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from gitpython!=3.1.29,>=1.0.0->wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (4.0.12)
+Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from huggingface-hub>=0.18.0->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (1.5.2)
+Requirement already satisfied: mdurl~=0.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 10)) (0.1.2)
+Requirement already satisfied: charset_normalizer<4,>=2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from requests>=2.19.0->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (3.4.9)
+Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from requests>=2.19.0->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (3.18)
+Requirement already satisfied: urllib3<3,>=1.26 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from requests>=2.19.0->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2.7.0)
+Requirement already satisfied: certifi>=2023.5.7 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from requests>=2.19.0->datasets==2.15.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 1)) (2026.7.22)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (1.13.1)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (3.6.1)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (3.1.6)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from sympy==1.13.1->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (1.3.0)
+Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torchvision->timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 12)) (12.2.0)
+Requirement already satisfied: smmap<6,>=3.0.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from gitdb<5,>=4.0.1->gitpython!=3.1.29,>=1.0.0->wandb==0.18.7->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 16)) (5.0.3)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from jinja2->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.6vdp7l1m.requirements.txt (line 6)) (3.0.3)
+Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
+Installing collected packages: numpy
+  Attempting uninstall: numpy
+    Found existing installation: numpy 2.3.1
+    Uninstalling numpy-2.3.1:
+      Successfully uninstalled numpy-2.3.1
+Successfully installed numpy-1.26.4
+
+done
+#
+# To activate this environment, use
+#
+#     $ conda activate dlb-duo
+#
+# To deactivate an active environment, use
+#
+#     $ conda deactivate
+
+2.5.1 12.4
+
+Collecting flash-attn==2.7.4.post1
+  Using cached flash_attn-2.7.4.post1.tar.gz (6.0 MB)
+  Preparing metadata (setup.py): started
+  Preparing metadata (setup.py): finished with status 'done'
+Requirement already satisfied: torch in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from flash-attn==2.7.4.post1) (2.5.1)
+Requirement already satisfied: einops in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from flash-attn==2.7.4.post1) (0.7.0)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (3.32.2)
+Requirement already satisfied: typing-extensions>=4.8.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (4.16.0)
+Requirement already satisfied: setuptools in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (73.0.1)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (1.13.1)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (3.6.1)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (3.1.6)
+Requirement already satisfied: fsspec in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from torch->flash-attn==2.7.4.post1) (2023.10.0)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from sympy==1.13.1->torch->flash-attn==2.7.4.post1) (1.3.0)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-duo/lib/python3.12/site-packages (from jinja2->torch->flash-attn==2.7.4.post1) (3.0.3)
+Building wheels for collected packages: flash-attn
+  Building wheel for flash-attn (setup.py): started
+  Building wheel for flash-attn (setup.py): finished with status 'done'
+  Created wheel for flash-attn: filename=flash_attn-2.7.4.post1-cp312-cp312-linux_x86_64.whl size=187814472 sha256=a496f383c843ed0cc6e01302556a0947f56828f1ec2b54e5297c4c7eaae39357
+  Stored in directory: /home/ruizzhao/.cache/pip/wheels/48/0f/98/166bf6b39dd0592924b22ce21b799fc9fd76c3a302c12ee359
+Successfully built flash-attn
+Installing collected packages: flash-attn
+Successfully installed flash-attn-2.7.4.post1
+
+Channels:
+ - pytorch
+ - nvidia
+ - conda-forge
+ - defaults
+Platform: linux-64
+Collecting package metadata (repodata.json): ...working... done
+Solving environment: ...working... done
+
+Downloading and Extracting Packages: ...working...
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
                                                     
+
                                                     
+
+
                                                     
+
+
+
                                                     
+
+
+
+
                                                      done
+Preparing transaction: - done
+Verifying transaction: | / done
+Executing transaction: \ done
+Installing pip dependencies: / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | Ran pip subprocess with arguments:
+['/opt/anaconda3/envs/dlb-mdlm/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt', '--exists-action=b']
+Pip subprocess output:
+Collecting datasets==2.18.0 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached datasets-2.18.0-py3-none-any.whl.metadata (20 kB)
+Collecting einops==0.7.0 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 2))
+  Using cached einops-0.7.0-py3-none-any.whl.metadata (13 kB)
+Collecting fsspec==2024.2.0 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 3))
+  Using cached fsspec-2024.2.0-py3-none-any.whl.metadata (6.8 kB)
+Collecting h5py==3.10.0 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 4))
+  Using cached h5py-3.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
+Collecting hydra-core==1.3.2 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 5))
+  Using cached hydra_core-1.3.2-py3-none-any.whl.metadata (5.5 kB)
+Collecting lightning==2.2.1 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6))
+  Using cached lightning-2.2.1-py3-none-any.whl.metadata (56 kB)
+Collecting nvitop==1.3.2 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 7))
+  Using cached nvitop-1.3.2-py3-none-any.whl.metadata (78 kB)
+Collecting omegaconf==2.3.0 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 8))
+  Using cached omegaconf-2.3.0-py3-none-any.whl.metadata (3.9 kB)
+Collecting pandas==2.2.1 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 9))
+  Using cached pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
+Collecting rich==13.7.1 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 10))
+  Using cached rich-13.7.1-py3-none-any.whl.metadata (18 kB)
+Collecting timm==0.9.16 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 11))
+  Using cached timm-0.9.16-py3-none-any.whl.metadata (38 kB)
+Collecting transformers==4.38.2 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 12))
+  Using cached transformers-4.38.2-py3-none-any.whl.metadata (130 kB)
+Collecting wandb==0.13.5 (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached wandb-0.13.5-py2.py3-none-any.whl.metadata (7.4 kB)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (3.19.1)
+Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (1.22.4)
+Collecting pyarrow>=12.0.0 (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached pyarrow-21.0.0-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (3.3 kB)
+Collecting pyarrow-hotfix (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached pyarrow_hotfix-0.7-py3-none-any.whl.metadata (3.6 kB)
+Collecting dill<0.3.9,>=0.3.0 (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached dill-0.3.8-py3-none-any.whl.metadata (10 kB)
+Requirement already satisfied: requests>=2.19.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (2.32.5)
+Collecting tqdm>=4.62.1 (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached tqdm-4.70.0-py3-none-any.whl.metadata (57 kB)
+Collecting xxhash (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached xxhash-3.8.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (15 kB)
+Collecting multiprocess (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached multiprocess-0.70.19-py39-none-any.whl.metadata (7.5 kB)
+Collecting aiohttp (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached aiohttp-3.13.5-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (8.1 kB)
+Collecting huggingface-hub>=0.19.4 (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached huggingface_hub-1.8.0-py3-none-any.whl.metadata (13 kB)
+Collecting packaging (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached packaging-26.2-py3-none-any.whl.metadata (3.5 kB)
+Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (6.0.2)
+Collecting antlr4-python3-runtime==4.9.* (from hydra-core==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 5))
+  Using cached antlr4_python3_runtime-4.9.3-py3-none-any.whl
+Collecting lightning-utilities<2.0,>=0.8.0 (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6))
+  Using cached lightning_utilities-0.15.2-py3-none-any.whl.metadata (5.7 kB)
+Collecting packaging (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
+Requirement already satisfied: torch<4.0,>=1.13.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (2.5.1)
+Collecting torchmetrics<3.0,>=0.7.0 (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6))
+  Using cached torchmetrics-1.8.2-py3-none-any.whl.metadata (22 kB)
+Requirement already satisfied: typing-extensions<6.0,>=4.4.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (4.14.1)
+Collecting pytorch-lightning (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6))
+  Using cached pytorch_lightning-2.6.0-py3-none-any.whl.metadata (21 kB)
+Collecting nvidia-ml-py<12.536.0a0,>=11.450.51 (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 7))
+  Using cached nvidia_ml_py-12.535.161-py3-none-any.whl.metadata (8.6 kB)
+Collecting psutil>=5.6.6 (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 7))
+  Using cached psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl.metadata (22 kB)
+Collecting cachetools>=1.0.1 (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 7))
+  Using cached cachetools-6.2.6-py3-none-any.whl.metadata (5.6 kB)
+Collecting termcolor>=1.0.0 (from nvitop==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 7))
+  Using cached termcolor-3.1.0-py3-none-any.whl.metadata (6.4 kB)
+Collecting python-dateutil>=2.8.2 (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 9))
+  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
+Collecting pytz>=2020.1 (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 9))
+  Using cached pytz-2026.3.post1-py2.py3-none-any.whl.metadata (22 kB)
+Collecting tzdata>=2022.7 (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 9))
+  Using cached tzdata-2026.3-py2.py3-none-any.whl.metadata (1.4 kB)
+Collecting markdown-it-py>=2.2.0 (from rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 10))
+  Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
+Collecting pygments<3.0.0,>=2.13.0 (from rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 10))
+  Using cached pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)
+Requirement already satisfied: torchvision in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 11)) (0.20.1)
+Collecting safetensors (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 11))
+  Using cached safetensors-0.7.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)
+Collecting huggingface-hub>=0.19.4 (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached huggingface_hub-0.36.2-py3-none-any.whl.metadata (15 kB)
+Collecting regex!=2019.12.17 (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 12))
+  Using cached regex-2026.1.15-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
+Collecting tokenizers<0.19,>=0.14 (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 12))
+  Using cached tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
+Collecting Click!=8.0.0,>=7.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached click-8.1.8-py3-none-any.whl.metadata (2.3 kB)
+Collecting GitPython>=1.0.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached gitpython-3.1.57-py3-none-any.whl.metadata (13 kB)
+Collecting promise<3,>=2.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached promise-2.3-py3-none-any.whl
+Collecting shortuuid>=0.5.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached shortuuid-1.0.13-py3-none-any.whl.metadata (5.8 kB)
+Collecting sentry-sdk>=1.0.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached sentry_sdk-2.66.1-py3-none-any.whl.metadata (11 kB)
+Collecting six>=1.13.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
+Collecting docker-pycreds>=0.4.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl.metadata (1.8 kB)
+Collecting protobuf!=4.0.*,!=4.21.0,<5,>=3.12.0 (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
+Collecting pathtools (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached pathtools-0.1.2-py3-none-any.whl
+Collecting setproctitle (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached setproctitle-1.3.7-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (10 kB)
+Requirement already satisfied: setuptools in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13)) (80.9.0)
+Collecting aiohappyeyeballs>=2.5.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached aiohappyeyeballs-2.6.1-py3-none-any.whl.metadata (5.9 kB)
+Collecting aiosignal>=1.4.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached aiosignal-1.4.0-py3-none-any.whl.metadata (3.7 kB)
+Collecting async-timeout<6.0,>=4.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached async_timeout-5.0.1-py3-none-any.whl.metadata (5.1 kB)
+Collecting attrs>=17.3.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached attrs-26.1.0-py3-none-any.whl.metadata (8.8 kB)
+Collecting frozenlist>=1.1.1 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached frozenlist-1.8.0-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (20 kB)
+Collecting multidict<7.0,>=4.5 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached multidict-6.7.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (5.3 kB)
+Collecting propcache>=0.2.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached propcache-0.4.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (13 kB)
+Collecting yarl<2.0,>=1.17.0 (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached yarl-1.22.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (75 kB)
+Collecting gitdb<5,>=4.0.1 (from GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached gitdb-4.0.12-py3-none-any.whl.metadata (1.2 kB)
+Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub>=0.19.4->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (4.9 kB)
+Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich==13.7.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 10))
+  Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
+Requirement already satisfied: charset_normalizer<4,>=2 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (3.4.3)
+Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (3.11)
+Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (2.5.0)
+Requirement already satisfied: certifi>=2017.4.17 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1)) (2025.8.3)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (3.2.1)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (3.1.6)
+Collecting sympy==1.13.1 (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6))
+  Using cached sympy-1.13.1-py3-none-any.whl.metadata (12 kB)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from sympy==1.13.1->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (1.3.0)
+INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.
+Collecting multiprocess (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 1))
+  Using cached multiprocess-0.70.18-py39-none-any.whl.metadata (7.5 kB)
+  Using cached multiprocess-0.70.17-py39-none-any.whl.metadata (7.2 kB)
+  Using cached multiprocess-0.70.16-py39-none-any.whl.metadata (7.2 kB)
+Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torchvision->timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 11)) (9.4.0)
+Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 13))
+  Using cached smmap-5.0.3-py3-none-any.whl.metadata (4.6 kB)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from jinja2->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.s40bn6wf.requirements.txt (line 6)) (3.0.2)
+Using cached datasets-2.18.0-py3-none-any.whl (510 kB)
+Using cached einops-0.7.0-py3-none-any.whl (44 kB)
+Using cached fsspec-2024.2.0-py3-none-any.whl (170 kB)
+Using cached h5py-3.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
+Using cached hydra_core-1.3.2-py3-none-any.whl (154 kB)
+Using cached lightning-2.2.1-py3-none-any.whl (2.1 MB)
+Using cached nvitop-1.3.2-py3-none-any.whl (215 kB)
+Using cached omegaconf-2.3.0-py3-none-any.whl (79 kB)
+Using cached pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
+Using cached rich-13.7.1-py3-none-any.whl (240 kB)
+Using cached timm-0.9.16-py3-none-any.whl (2.2 MB)
+Using cached transformers-4.38.2-py3-none-any.whl (8.5 MB)
+Using cached wandb-0.13.5-py2.py3-none-any.whl (1.9 MB)
+Using cached cachetools-6.2.6-py3-none-any.whl (11 kB)
+Using cached click-8.1.8-py3-none-any.whl (98 kB)
+Using cached dill-0.3.8-py3-none-any.whl (116 kB)
+Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
+Using cached aiohttp-3.13.5-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (1.7 MB)
+Using cached gitpython-3.1.57-py3-none-any.whl (217 kB)
+Using cached huggingface_hub-0.36.2-py3-none-any.whl (566 kB)
+Using cached lightning_utilities-0.15.2-py3-none-any.whl (29 kB)
+Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
+Using cached nvidia_ml_py-12.535.161-py3-none-any.whl (37 kB)
+Using cached packaging-24.2-py3-none-any.whl (65 kB)
+Using cached protobuf-4.25.9-cp37-abi3-manylinux2014_x86_64.whl (295 kB)
+Using cached psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB)
+Using cached pyarrow-21.0.0-cp39-cp39-manylinux_2_28_x86_64.whl (42.7 MB)
+Using cached pygments-2.20.0-py3-none-any.whl (1.2 MB)
+Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
+Using cached pytz-2026.3.post1-py2.py3-none-any.whl (508 kB)
+Using cached regex-2026.1.15-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (791 kB)
+Using cached safetensors-0.7.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (507 kB)
+Using cached sentry_sdk-2.66.1-py3-none-any.whl (505 kB)
+Using cached shortuuid-1.0.13-py3-none-any.whl (10 kB)
+Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
+Using cached termcolor-3.1.0-py3-none-any.whl (7.7 kB)
+Using cached tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
+Using cached sympy-1.13.1-py3-none-any.whl (6.2 MB)
+Using cached torchmetrics-1.8.2-py3-none-any.whl (983 kB)
+Using cached tqdm-4.70.0-py3-none-any.whl (80 kB)
+Using cached tzdata-2026.3-py2.py3-none-any.whl (348 kB)
+Using cached multiprocess-0.70.16-py39-none-any.whl (133 kB)
+Using cached pyarrow_hotfix-0.7-py3-none-any.whl (7.9 kB)
+Using cached pytorch_lightning-2.6.0-py3-none-any.whl (849 kB)
+Using cached setproctitle-1.3.7-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (31 kB)
+Using cached xxhash-3.8.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (216 kB)
+Using cached aiohappyeyeballs-2.6.1-py3-none-any.whl (15 kB)
+Using cached aiosignal-1.4.0-py3-none-any.whl (7.5 kB)
+Using cached async_timeout-5.0.1-py3-none-any.whl (6.2 kB)
+Using cached attrs-26.1.0-py3-none-any.whl (67 kB)
+Using cached frozenlist-1.8.0-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (219 kB)
+Using cached gitdb-4.0.12-py3-none-any.whl (62 kB)
+Using cached hf_xet-1.5.2-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.4 MB)
+Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
+Using cached multidict-6.7.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (240 kB)
+Using cached propcache-0.4.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (197 kB)
+Using cached yarl-1.22.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (346 kB)
+Using cached smmap-5.0.3-py3-none-any.whl (24 kB)
+Installing collected packages: pytz, pathtools, nvidia-ml-py, antlr4-python3-runtime, xxhash, tzdata, tqdm, termcolor, sympy, smmap, six, shortuuid, setproctitle, sentry-sdk, safetensors, regex, pygments, pyarrow-hotfix, pyarrow, psutil, protobuf, propcache, packaging, omegaconf, multidict, mdurl, hf-xet, h5py, fsspec, frozenlist, einops, dill, Click, cachetools, attrs, async-timeout, aiohappyeyeballs, yarl, python-dateutil, promise, nvitop, multiprocess, markdown-it-py, lightning-utilities, hydra-core, huggingface-hub, gitdb, docker-pycreds, aiosignal, torchmetrics, tokenizers, rich, pandas, GitPython, aiohttp, wandb, transformers, timm, pytorch-lightning, datasets, lightning
+  Attempting uninstall: sympy
+    Found existing installation: sympy 1.14.0
+    Uninstalling sympy-1.14.0:
+      Successfully uninstalled sympy-1.14.0
+Successfully installed Click-8.1.8 GitPython-3.1.57 aiohappyeyeballs-2.6.1 aiohttp-3.13.5 aiosignal-1.4.0 antlr4-python3-runtime-4.9.3 async-timeout-5.0.1 attrs-26.1.0 cachetools-6.2.6 datasets-2.18.0 dill-0.3.8 docker-pycreds-0.4.0 einops-0.7.0 frozenlist-1.8.0 fsspec-2024.2.0 gitdb-4.0.12 h5py-3.10.0 hf-xet-1.5.2 huggingface-hub-0.36.2 hydra-core-1.3.2 lightning-2.2.1 lightning-utilities-0.15.2 markdown-it-py-3.0.0 mdurl-0.1.2 multidict-6.7.1 multiprocess-0.70.16 nvidia-ml-py-12.535.161 nvitop-1.3.2 omegaconf-2.3.0 packaging-24.2 pandas-2.2.1 pathtools-0.1.2 promise-2.3 propcache-0.4.1 protobuf-4.25.9 psutil-7.2.2 pyarrow-21.0.0 pyarrow-hotfix-0.7 pygments-2.20.0 python-dateutil-2.9.0.post0 pytorch-lightning-2.6.0 pytz-2026.3.post1 regex-2026.1.15 rich-13.7.1 safetensors-0.7.0 sentry-sdk-2.66.1 setproctitle-1.3.7 shortuuid-1.0.13 six-1.17.0 smmap-5.0.3 sympy-1.13.1 termcolor-3.1.0 timm-0.9.16 tokenizers-0.15.2 torchmetrics-1.8.2 tqdm-4.70.0 transformers-4.38.2 tzdata-2026.3 wandb-0.13.5 xxhash-3.8.1 yarl-1.22.0
+
+done
+#
+# To activate this environment, use
+#
+#     $ conda activate dlb-mdlm
+#
+# To deactivate an active environment, use
+#
+#     $ conda deactivate
+
+2.5.1 12.4
+
+Collecting git+https://github.com/Dao-AILab/causal-conv1d.git@v1.1.3.post1
+  Cloning https://github.com/Dao-AILab/causal-conv1d.git (to revision v1.1.3.post1) to /tmp/pip-req-build-84817d6p
+  Resolved https://github.com/Dao-AILab/causal-conv1d.git to commit 89e4aa413ea99a079a39364b0a58f54239b4306c
+  Preparing metadata (setup.py): started
+  Preparing metadata (setup.py): finished with status 'done'
+Collecting git+https://github.com/state-spaces/mamba.git@v1.1.4
+  Cloning https://github.com/state-spaces/mamba.git (to revision v1.1.4) to /tmp/pip-req-build-9_lvy0g0
+  Resolved https://github.com/state-spaces/mamba.git to commit ce59daea3a090d011d6476c6e5b97f6d58ddad8b
+  Preparing metadata (setup.py): started
+  Preparing metadata (setup.py): finished with status 'done'
+Requirement already satisfied: torch in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from causal_conv1d==1.1.3.post1) (2.5.1)
+Requirement already satisfied: packaging in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from causal_conv1d==1.1.3.post1) (24.2)
+Collecting buildtools (from causal_conv1d==1.1.3.post1)
+  Using cached buildtools-1.0.6-py3-none-any.whl
+Collecting ninja (from causal_conv1d==1.1.3.post1)
+  Using cached ninja-1.13.0-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (5.1 kB)
+Requirement already satisfied: einops in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from mamba_ssm==1.1.4) (0.7.0)
+Requirement already satisfied: triton in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from mamba_ssm==1.1.4) (3.1.0)
+Requirement already satisfied: transformers in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from mamba_ssm==1.1.4) (4.38.2)
+Collecting sqlalchemy (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached sqlalchemy-2.0.51-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (9.5 kB)
+Collecting argparse (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached argparse-1.4.0-py2.py3-none-any.whl.metadata (2.8 kB)
+Collecting twisted (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached twisted-26.4.0-py3-none-any.whl.metadata (15 kB)
+Collecting simplejson (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached simplejson-4.1.1-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (3.8 kB)
+Collecting furl (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached furl-2.1.4-py2.py3-none-any.whl.metadata (25 kB)
+Requirement already satisfied: requests in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from buildtools->causal_conv1d==1.1.3.post1) (2.32.5)
+Collecting docopt (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached docopt-0.6.2-py2.py3-none-any.whl
+Requirement already satisfied: python-dateutil in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from buildtools->causal_conv1d==1.1.3.post1) (2.9.0.post0)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from buildtools->causal_conv1d==1.1.3.post1) (3.1.6)
+Collecting redo (from buildtools->causal_conv1d==1.1.3.post1)
+  Using cached redo-3.0.0-py2.py3-none-any.whl.metadata (8.0 kB)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->causal_conv1d==1.1.3.post1) (3.19.1)
+Requirement already satisfied: typing-extensions>=4.8.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->causal_conv1d==1.1.3.post1) (4.14.1)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->causal_conv1d==1.1.3.post1) (3.2.1)
+Requirement already satisfied: fsspec in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->causal_conv1d==1.1.3.post1) (2024.2.0)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->causal_conv1d==1.1.3.post1) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from sympy==1.13.1->torch->causal_conv1d==1.1.3.post1) (1.3.0)
+Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (0.36.2)
+Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (1.22.4)
+Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (6.0.2)
+Requirement already satisfied: regex!=2019.12.17 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (2026.1.15)
+Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (0.15.2)
+Requirement already satisfied: safetensors>=0.4.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (0.7.0)
+Requirement already satisfied: tqdm>=4.27 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from transformers->mamba_ssm==1.1.4) (4.70.0)
+Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers->mamba_ssm==1.1.4) (1.5.2)
+Requirement already satisfied: six>=1.8.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from furl->buildtools->causal_conv1d==1.1.3.post1) (1.17.0)
+Collecting orderedmultidict>=1.0.1 (from furl->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached orderedmultidict-1.0.2-py2.py3-none-any.whl.metadata (1.2 kB)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from jinja2->buildtools->causal_conv1d==1.1.3.post1) (3.0.2)
+Requirement already satisfied: charset_normalizer<4,>=2 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests->buildtools->causal_conv1d==1.1.3.post1) (3.4.3)
+Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests->buildtools->causal_conv1d==1.1.3.post1) (3.11)
+Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests->buildtools->causal_conv1d==1.1.3.post1) (2.5.0)
+Requirement already satisfied: certifi>=2017.4.17 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from requests->buildtools->causal_conv1d==1.1.3.post1) (2025.8.3)
+Collecting greenlet>=1 (from sqlalchemy->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached greenlet-3.2.5-cp39-cp39-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (4.1 kB)
+Requirement already satisfied: attrs>=22.2.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from twisted->buildtools->causal_conv1d==1.1.3.post1) (26.1.0)
+Collecting automat>=24.8.0 (from twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached automat-25.4.16-py3-none-any.whl.metadata (8.4 kB)
+Collecting constantly>=15.1 (from twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached constantly-23.10.4-py3-none-any.whl.metadata (1.8 kB)
+Collecting hyperlink>=17.1.1 (from twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached hyperlink-21.0.0-py2.py3-none-any.whl.metadata (1.5 kB)
+Collecting incremental>=24.7.0 (from twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached incremental-24.11.0-py3-none-any.whl.metadata (9.5 kB)
+Collecting zope-interface>=5 (from twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached zope_interface-8.0.1-cp39-cp39-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl.metadata (44 kB)
+Collecting tomli (from incremental>=24.7.0->twisted->buildtools->causal_conv1d==1.1.3.post1)
+  Using cached tomli-2.4.1-py3-none-any.whl.metadata (10 kB)
+Using cached ninja-1.13.0-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (180 kB)
+Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB)
+Using cached furl-2.1.4-py2.py3-none-any.whl (27 kB)
+Using cached redo-3.0.0-py2.py3-none-any.whl (14 kB)
+Using cached simplejson-4.1.1-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (168 kB)
+Using cached sqlalchemy-2.0.51-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (3.2 MB)
+Using cached twisted-26.4.0-py3-none-any.whl (3.2 MB)
+Using cached automat-25.4.16-py3-none-any.whl (42 kB)
+Using cached constantly-23.10.4-py3-none-any.whl (13 kB)
+Using cached greenlet-3.2.5-cp39-cp39-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (587 kB)
+Using cached hyperlink-21.0.0-py2.py3-none-any.whl (74 kB)
+Using cached incremental-24.11.0-py3-none-any.whl (21 kB)
+Using cached orderedmultidict-1.0.2-py2.py3-none-any.whl (11 kB)
+Using cached zope_interface-8.0.1-cp39-cp39-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl (253 kB)
+Using cached tomli-2.4.1-py3-none-any.whl (14 kB)
+Building wheels for collected packages: causal_conv1d, mamba_ssm
+  Building wheel for causal_conv1d (setup.py): started
+  Building wheel for causal_conv1d (setup.py): finished with status 'done'
+  Created wheel for causal_conv1d: filename=causal_conv1d-1.1.3.post1-cp39-cp39-linux_x86_64.whl size=11327086 sha256=598479ff95c74113eb94287bb57652db555e85ca68811078b9aad4e92a5a8a15
+  Stored in directory: /tmp/pip-ephem-wheel-cache-kcmz2rxp/wheels/c3/0f/ae/e4b098b68934244b5fd2cac33baffe9990b84c9f0f5f45b25b
+  Building wheel for mamba_ssm (setup.py): started
+  Building wheel for mamba_ssm (setup.py): still running...
+  Building wheel for mamba_ssm (setup.py): still running...
+  Building wheel for mamba_ssm (setup.py): still running...
+  Building wheel for mamba_ssm (setup.py): still running...
+  Building wheel for mamba_ssm (setup.py): still running...
+  Building wheel for mamba_ssm (setup.py): finished with status 'done'
+  Created wheel for mamba_ssm: filename=mamba_ssm-1.1.4-cp39-cp39-linux_x86_64.whl size=153646045 sha256=6b77675216e9daabe31d0f808aceb199413b7b8e136917160e384aaa3b706441
+  Stored in directory: /tmp/pip-ephem-wheel-cache-kcmz2rxp/wheels/95/62/53/1c84f6120ad3af516300f5483e20317d21363d95785af8ce93
+Successfully built causal_conv1d mamba_ssm
+Installing collected packages: redo, docopt, argparse, zope-interface, tomli, simplejson, orderedmultidict, ninja, hyperlink, greenlet, constantly, automat, sqlalchemy, incremental, furl, twisted, buildtools, causal_conv1d, mamba_ssm
+Successfully installed argparse-1.4.0 automat-25.4.16 buildtools-1.0.6 causal_conv1d-1.1.3.post1 constantly-23.10.4 docopt-0.6.2 furl-2.1.4 greenlet-3.2.5 hyperlink-21.0.0 incremental-24.11.0 mamba_ssm-1.1.4 ninja-1.13.0 orderedmultidict-1.0.2 redo-3.0.0 simplejson-4.1.1 sqlalchemy-2.0.51 tomli-2.4.1 twisted-26.4.0 zope-interface-8.0.1
+  Running command git clone --filter=blob:none --quiet https://github.com/Dao-AILab/causal-conv1d.git /tmp/pip-req-build-84817d6p
+  Running command git checkout -q 89e4aa413ea99a079a39364b0a58f54239b4306c
+  Running command git clone --filter=blob:none --quiet https://github.com/state-spaces/mamba.git /tmp/pip-req-build-9_lvy0g0
+  Running command git checkout -q ce59daea3a090d011d6476c6e5b97f6d58ddad8b
+  Running command git submodule update --init --recursive -q
+
+
+Collecting flash-attn==2.5.6
+  Downloading flash_attn-2.5.6.tar.gz (2.5 MB)
+     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 17.0 MB/s eta 0:00:00
+  Preparing metadata (setup.py): started
+  Preparing metadata (setup.py): finished with status 'done'
+Requirement already satisfied: torch in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from flash-attn==2.5.6) (2.5.1)
+Requirement already satisfied: einops in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from flash-attn==2.5.6) (0.7.0)
+Requirement already satisfied: packaging in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from flash-attn==2.5.6) (24.2)
+Requirement already satisfied: ninja in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from flash-attn==2.5.6) (1.13.0)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (3.19.1)
+Requirement already satisfied: typing-extensions>=4.8.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (4.14.1)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (3.2.1)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (3.1.6)
+Requirement already satisfied: fsspec in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (2024.2.0)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from torch->flash-attn==2.5.6) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from sympy==1.13.1->torch->flash-attn==2.5.6) (1.3.0)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages (from jinja2->torch->flash-attn==2.5.6) (3.0.2)
+Building wheels for collected packages: flash-attn
+  Building wheel for flash-attn (setup.py): started
+  Building wheel for flash-attn (setup.py): still running...
+  Building wheel for flash-attn (setup.py): still running...
+  Building wheel for flash-attn (setup.py): still running...
+  Building wheel for flash-attn (setup.py): finished with status 'done'
+  Created wheel for flash-attn: filename=flash_attn-2.5.6-cp39-cp39-linux_x86_64.whl size=118192997 sha256=9a57646118e777f927c9759122652b73eb27913de5973fc2fb7320309afd4751
+  Stored in directory: /home/ruizzhao/.cache/pip/wheels/a7/dc/5f/55c369f8b94df622b45cc15d41b43c5f83747f5134d124cb5e
+Successfully built flash-attn
+Installing collected packages: flash-attn
+Successfully installed flash-attn-2.5.6
+

create_sdtt_di4c_fix.log

@@ -0,0 +1,276 @@
+Channels:
+ - pytorch
+ - nvidia
+ - conda-forge
+ - defaults
+Platform: linux-64
+Collecting package metadata (repodata.json): ...working... done
+Solving environment: ...working... done
+Installing pip dependencies: - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ Ran pip subprocess with arguments:
+['/opt/anaconda3/envs/dlb-sdtt/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt', '--exists-action=b']
+Pip subprocess output:
+Requirement already satisfied: datasets==2.18.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (2.18.0)
+Requirement already satisfied: einops==0.7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 2)) (0.7.0)
+Requirement already satisfied: evaluate==0.4.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 3)) (0.4.2)
+Requirement already satisfied: fancy-einsum==0.0.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 4)) (0.0.3)
+Requirement already satisfied: fsspec==2024.2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 5)) (2024.2.0)
+Requirement already satisfied: gpustat==1.1.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (1.1.1)
+Requirement already satisfied: huggingface-hub==0.21.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 7)) (0.21.1)
+Requirement already satisfied: hydra-core==1.3.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 8)) (1.3.2)
+Requirement already satisfied: hydra-submitit-launcher==1.2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 9)) (1.2.0)
+Requirement already satisfied: lightning==2.2.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (2.2.1)
+Requirement already satisfied: loguru==0.7.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 11)) (0.7.2)
+Requirement already satisfied: mauve-text==0.3.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 12)) (0.3.0)
+Requirement already satisfied: omegaconf==2.3.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 13)) (2.3.0)
+Requirement already satisfied: pandas==2.2.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 14)) (2.2.1)
+Requirement already satisfied: safetensors==0.4.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 15)) (0.4.2)
+Requirement already satisfied: setuptools==80.9.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 16)) (80.9.0)
+Requirement already satisfied: tensorboard==2.15.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (2.15.2)
+Requirement already satisfied: timm==0.9.16 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 18)) (0.9.16)
+Requirement already satisfied: tokenizers==0.15.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 19)) (0.15.2)
+Requirement already satisfied: torchdata==0.8.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 20)) (0.8.0)
+Requirement already satisfied: transformers==4.38.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 21)) (4.38.2)
+Requirement already satisfied: vendi-score==0.0.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 22)) (0.0.3)
+Requirement already satisfied: wandb==0.13.5 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (0.13.5)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (3.32.2)
+Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (1.26.4)
+Requirement already satisfied: pyarrow>=12.0.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (25.0.0)
+Requirement already satisfied: pyarrow-hotfix in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (0.7)
+Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (0.3.8)
+Requirement already satisfied: requests>=2.19.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (2.34.2)
+Requirement already satisfied: tqdm>=4.62.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (4.70.0)
+Requirement already satisfied: xxhash in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (3.8.1)
+Requirement already satisfied: multiprocess in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (0.70.16)
+Requirement already satisfied: aiohttp in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (3.14.3)
+Requirement already satisfied: packaging in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (24.2)
+Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (6.0.3)
+Requirement already satisfied: nvidia-ml-py>=11.450.129 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (13.610.43)
+Requirement already satisfied: psutil>=5.6.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (7.2.2)
+Requirement already satisfied: blessed>=1.17.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (1.47.0)
+Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from huggingface-hub==0.21.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 7)) (4.16.0)
+Requirement already satisfied: antlr4-python3-runtime==4.9.* in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from hydra-core==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 8)) (4.9.3)
+Requirement already satisfied: submitit>=1.3.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from hydra-submitit-launcher==1.2.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 9)) (1.5.4)
+Requirement already satisfied: lightning-utilities<2.0,>=0.8.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (0.15.3)
+Requirement already satisfied: torch<4.0,>=1.13.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (2.5.1)
+Requirement already satisfied: torchmetrics<3.0,>=0.7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (1.9.0)
+Requirement already satisfied: pytorch-lightning in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (2.6.5)
+Requirement already satisfied: scikit-learn>=0.22.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 12)) (1.7.2)
+Requirement already satisfied: faiss-cpu>=1.7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 12)) (1.14.3)
+Requirement already satisfied: python-dateutil>=2.8.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 14)) (2.9.0.post0)
+Requirement already satisfied: pytz>=2020.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 14)) (2026.3.post1)
+Requirement already satisfied: tzdata>=2022.7 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 14)) (2026.3)
+Requirement already satisfied: absl-py>=0.4 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (2.5.0)
+Requirement already satisfied: grpcio>=1.48.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (1.83.0)
+Requirement already satisfied: google-auth<3,>=1.6.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (2.56.2)
+Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (1.4.0)
+Requirement already satisfied: markdown>=2.6.8 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (3.10.3)
+Requirement already satisfied: protobuf!=4.24.0,>=3.19.6 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (4.25.9)
+Requirement already satisfied: six>1.9 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (1.17.0)
+Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (0.7.2)
+Requirement already satisfied: werkzeug>=1.0.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (3.1.8)
+Requirement already satisfied: torchvision in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 18)) (0.20.1)
+Requirement already satisfied: urllib3>=1.25 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torchdata==0.8.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 20)) (2.7.0)
+Requirement already satisfied: regex!=2019.12.17 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 21)) (2026.7.19)
+Requirement already satisfied: scipy>=1.3.2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from vendi-score==0.0.3->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 22)) (1.15.3)
+Requirement already satisfied: Click!=8.0.0,>=7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (8.4.2)
+Requirement already satisfied: GitPython>=1.0.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (3.1.57)
+Requirement already satisfied: promise<3,>=2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (2.3)
+Requirement already satisfied: shortuuid>=0.5.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (1.0.13)
+Requirement already satisfied: sentry-sdk>=1.0.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (2.66.1)
+Requirement already satisfied: docker-pycreds>=0.4.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (0.4.0)
+Requirement already satisfied: pathtools in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (0.1.2)
+Requirement already satisfied: setproctitle in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (1.3.7)
+Requirement already satisfied: wcwidth>=0.8.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from blessed>=1.17.1->gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (0.8.2)
+Requirement already satisfied: jinxed<3,>=2.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from blessed>=1.17.1->gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 6)) (2.1.0)
+Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (2.7.1)
+Requirement already satisfied: aiosignal>=1.4.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (1.4.0)
+Requirement already satisfied: async-timeout<6.0,>=4.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (5.0.1)
+Requirement already satisfied: attrs>=17.3.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (26.1.0)
+Requirement already satisfied: frozenlist>=1.1.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (1.8.0)
+Requirement already satisfied: multidict<7.0,>=4.5 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (6.7.1)
+Requirement already satisfied: propcache>=0.2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (0.5.2)
+Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (1.24.5)
+Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (4.0.12)
+Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (0.4.2)
+Requirement already satisfied: cryptography>=38.0.3 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (50.0.0)
+Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from google-auth-oauthlib<2,>=0.5->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (2.0.0)
+Requirement already satisfied: charset_normalizer<4,>=2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (3.4.9)
+Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (3.18)
+Requirement already satisfied: certifi>=2023.5.7 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 1)) (2026.7.22)
+Requirement already satisfied: joblib>=1.2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from scikit-learn>=0.22.1->mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 12)) (1.5.3)
+Requirement already satisfied: threadpoolctl>=3.1.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from scikit-learn>=0.22.1->mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 12)) (3.6.0)
+Requirement already satisfied: cloudpickle>=1.2.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from submitit>=1.3.3->hydra-submitit-launcher==1.2.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 9)) (3.1.2)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (3.4)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (3.1.6)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from sympy==1.13.1->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 10)) (1.3.0)
+Requirement already satisfied: markupsafe>=2.1.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (3.0.3)
+Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torchvision->timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 18)) (9.4.0)
+Requirement already satisfied: cffi>=2.0.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from cryptography>=38.0.3->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (2.1.0)
+Requirement already satisfied: smmap<6,>=3.0.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 23)) (5.0.3)
+Requirement already satisfied: pyasn1<0.7.0,>=0.6.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (0.6.4)
+Requirement already satisfied: oauthlib>=3.0.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (3.3.1)
+Requirement already satisfied: pycparser in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from cffi>=2.0.0->cryptography>=38.0.3->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.yre_bqu8.requirements.txt (line 17)) (3.0)
+
+done
+#
+# To activate this environment, use
+#
+#     $ conda activate dlb-sdtt
+#
+# To deactivate an active environment, use
+#
+#     $ conda deactivate
+
+2.5.1 12.4
+
+Requirement already satisfied: flash-attn==2.7.4.post1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (2.7.4.post1)
+Requirement already satisfied: torch in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from flash-attn==2.7.4.post1) (2.5.1)
+Requirement already satisfied: einops in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from flash-attn==2.7.4.post1) (0.7.0)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.32.2)
+Requirement already satisfied: typing-extensions>=4.8.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (4.16.0)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.4)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.1.6)
+Requirement already satisfied: fsspec in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (2024.2.0)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from sympy==1.13.1->torch->flash-attn==2.7.4.post1) (1.3.0)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-sdtt/lib/python3.10/site-packages (from jinja2->torch->flash-attn==2.7.4.post1) (3.0.3)
+
+Channels:
+ - pytorch
+ - nvidia
+ - conda-forge
+ - defaults
+Platform: linux-64
+Collecting package metadata (repodata.json): ...working... done
+Solving environment: ...working... done
+Installing pip dependencies: - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | Ran pip subprocess with arguments:
+['/opt/anaconda3/envs/dlb-di4c/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt', '--exists-action=b']
+Pip subprocess output:
+Requirement already satisfied: datasets==2.18.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (2.18.0)
+Requirement already satisfied: einops==0.7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 2)) (0.7.0)
+Requirement already satisfied: evaluate==0.4.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 3)) (0.4.2)
+Requirement already satisfied: fancy-einsum==0.0.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 4)) (0.0.3)
+Requirement already satisfied: fsspec==2024.2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 5)) (2024.2.0)
+Requirement already satisfied: gpustat==1.1.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (1.1.1)
+Requirement already satisfied: huggingface-hub==0.21.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 7)) (0.21.1)
+Requirement already satisfied: hydra-core==1.3.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 8)) (1.3.2)
+Requirement already satisfied: hydra-submitit-launcher==1.2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 9)) (1.2.0)
+Requirement already satisfied: lightning==2.2.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (2.2.1)
+Requirement already satisfied: loguru==0.7.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 11)) (0.7.2)
+Requirement already satisfied: mauve-text==0.3.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 12)) (0.3.0)
+Requirement already satisfied: omegaconf==2.3.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 13)) (2.3.0)
+Requirement already satisfied: pandas==2.2.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 14)) (2.2.1)
+Requirement already satisfied: safetensors==0.4.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 15)) (0.4.2)
+Requirement already satisfied: setuptools==80.9.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 16)) (80.9.0)
+Requirement already satisfied: tensorboard==2.15.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (2.15.2)
+Requirement already satisfied: timm==0.9.16 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 18)) (0.9.16)
+Requirement already satisfied: tokenizers==0.15.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 19)) (0.15.2)
+Requirement already satisfied: torchdata==0.10.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 20)) (0.10.0)
+Requirement already satisfied: transformers==4.38.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 21)) (4.38.2)
+Requirement already satisfied: vendi-score==0.0.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 22)) (0.0.3)
+Requirement already satisfied: wandb==0.13.5 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from -r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (0.13.5)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (3.32.2)
+Requirement already satisfied: numpy>=1.17 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (1.26.4)
+Requirement already satisfied: pyarrow>=12.0.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (25.0.0)
+Requirement already satisfied: pyarrow-hotfix in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (0.7)
+Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (0.3.8)
+Requirement already satisfied: requests>=2.19.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (2.34.2)
+Requirement already satisfied: tqdm>=4.62.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (4.70.0)
+Requirement already satisfied: xxhash in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (3.8.1)
+Requirement already satisfied: multiprocess in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (0.70.16)
+Requirement already satisfied: aiohttp in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (3.14.3)
+Requirement already satisfied: packaging in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (24.2)
+Requirement already satisfied: pyyaml>=5.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (6.0.3)
+Requirement already satisfied: nvidia-ml-py>=11.450.129 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (13.610.43)
+Requirement already satisfied: psutil>=5.6.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (7.2.2)
+Requirement already satisfied: blessed>=1.17.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (1.47.0)
+Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from huggingface-hub==0.21.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 7)) (4.16.0)
+Requirement already satisfied: antlr4-python3-runtime==4.9.* in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from hydra-core==1.3.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 8)) (4.9.3)
+Requirement already satisfied: submitit>=1.3.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from hydra-submitit-launcher==1.2.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 9)) (1.5.4)
+Requirement already satisfied: lightning-utilities<2.0,>=0.8.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (0.15.3)
+Requirement already satisfied: torch<4.0,>=1.13.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (2.5.1)
+Requirement already satisfied: torchmetrics<3.0,>=0.7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (1.9.0)
+Requirement already satisfied: pytorch-lightning in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (2.6.5)
+Requirement already satisfied: scikit-learn>=0.22.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 12)) (1.7.2)
+Requirement already satisfied: faiss-cpu>=1.7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 12)) (1.14.3)
+Requirement already satisfied: python-dateutil>=2.8.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 14)) (2.9.0.post0)
+Requirement already satisfied: pytz>=2020.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 14)) (2026.3.post1)
+Requirement already satisfied: tzdata>=2022.7 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from pandas==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 14)) (2026.3)
+Requirement already satisfied: absl-py>=0.4 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (2.5.0)
+Requirement already satisfied: grpcio>=1.48.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (1.83.0)
+Requirement already satisfied: google-auth<3,>=1.6.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (2.56.2)
+Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (1.4.0)
+Requirement already satisfied: markdown>=2.6.8 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (3.10.3)
+Requirement already satisfied: protobuf!=4.24.0,>=3.19.6 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (4.25.9)
+Requirement already satisfied: six>1.9 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (1.17.0)
+Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (0.7.2)
+Requirement already satisfied: werkzeug>=1.0.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (3.1.8)
+Requirement already satisfied: torchvision in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 18)) (0.20.1)
+Requirement already satisfied: urllib3>=1.25 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torchdata==0.10.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 20)) (2.7.0)
+Requirement already satisfied: regex!=2019.12.17 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from transformers==4.38.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 21)) (2026.7.19)
+Requirement already satisfied: scipy>=1.3.2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from vendi-score==0.0.3->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 22)) (1.15.3)
+Requirement already satisfied: Click!=8.0.0,>=7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (8.4.2)
+Requirement already satisfied: GitPython>=1.0.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (3.1.57)
+Requirement already satisfied: promise<3,>=2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (2.3)
+Requirement already satisfied: shortuuid>=0.5.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (1.0.13)
+Requirement already satisfied: sentry-sdk>=1.0.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (2.66.1)
+Requirement already satisfied: docker-pycreds>=0.4.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (0.4.0)
+Requirement already satisfied: pathtools in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (0.1.2)
+Requirement already satisfied: setproctitle in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (1.3.7)
+Requirement already satisfied: wcwidth>=0.8.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from blessed>=1.17.1->gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (0.8.2)
+Requirement already satisfied: jinxed<3,>=2.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from blessed>=1.17.1->gpustat==1.1.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 6)) (2.1.0)
+Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (2.7.1)
+Requirement already satisfied: aiosignal>=1.4.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (1.4.0)
+Requirement already satisfied: async-timeout<6.0,>=4.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (5.0.1)
+Requirement already satisfied: attrs>=17.3.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (26.1.0)
+Requirement already satisfied: frozenlist>=1.1.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (1.8.0)
+Requirement already satisfied: multidict<7.0,>=4.5 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (6.7.1)
+Requirement already satisfied: propcache>=0.2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (0.5.2)
+Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from aiohttp->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (1.24.5)
+Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (4.0.12)
+Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (0.4.2)
+Requirement already satisfied: cryptography>=38.0.3 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (50.0.0)
+Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from google-auth-oauthlib<2,>=0.5->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (2.0.0)
+Requirement already satisfied: charset_normalizer<4,>=2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (3.4.9)
+Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (3.18)
+Requirement already satisfied: certifi>=2023.5.7 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from requests>=2.19.0->datasets==2.18.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 1)) (2026.7.22)
+Requirement already satisfied: joblib>=1.2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from scikit-learn>=0.22.1->mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 12)) (1.5.3)
+Requirement already satisfied: threadpoolctl>=3.1.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from scikit-learn>=0.22.1->mauve-text==0.3.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 12)) (3.6.0)
+Requirement already satisfied: cloudpickle>=1.2.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from submitit>=1.3.3->hydra-submitit-launcher==1.2.0->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 9)) (3.1.2)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (3.4)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (3.1.6)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from sympy==1.13.1->torch<4.0,>=1.13.0->lightning==2.2.1->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 10)) (1.3.0)
+Requirement already satisfied: markupsafe>=2.1.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (3.0.3)
+Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torchvision->timm==0.9.16->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 18)) (9.4.0)
+Requirement already satisfied: cffi>=2.0.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from cryptography>=38.0.3->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (2.1.0)
+Requirement already satisfied: smmap<6,>=3.0.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.13.5->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 23)) (5.0.3)
+Requirement already satisfied: pyasn1<0.7.0,>=0.6.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (0.6.4)
+Requirement already satisfied: oauthlib>=3.0.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (3.3.1)
+Requirement already satisfied: pycparser in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from cffi>=2.0.0->cryptography>=38.0.3->google-auth<3,>=1.6.3->tensorboard==2.15.2->-r /home/ruizzhao/diffusion_baseline/envs/condaenv.7rffn3bk.requirements.txt (line 17)) (3.0)
+
+done
+#
+# To activate this environment, use
+#
+#     $ conda activate dlb-di4c
+#
+# To deactivate an active environment, use
+#
+#     $ conda deactivate
+
+2.5.1 12.4
+
+Requirement already satisfied: flash-attn==2.7.4.post1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (2.7.4.post1)
+Requirement already satisfied: torch in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from flash-attn==2.7.4.post1) (2.5.1)
+Requirement already satisfied: einops in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from flash-attn==2.7.4.post1) (0.7.0)
+Requirement already satisfied: filelock in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.32.2)
+Requirement already satisfied: typing-extensions>=4.8.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (4.16.0)
+Requirement already satisfied: networkx in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.4)
+Requirement already satisfied: jinja2 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (3.1.6)
+Requirement already satisfied: fsspec in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (2024.2.0)
+Requirement already satisfied: sympy==1.13.1 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from torch->flash-attn==2.7.4.post1) (1.13.1)
+Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from sympy==1.13.1->torch->flash-attn==2.7.4.post1) (1.3.0)
+Requirement already satisfied: MarkupSafe>=2.0 in /opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages (from jinja2->torch->flash-attn==2.7.4.post1) (3.0.3)
+

nohup.out

@@ -0,0 +1,6 @@
+Traceback (most recent call last):
+  File "/home/ruizzhao/diffusion_baseline/scripts/run_elf.py", line 303, in <module>
+    main()
+  File "/home/ruizzhao/diffusion_baseline/scripts/run_elf.py", line 67, in main
+    import torch
+ModuleNotFoundError: No module named 'torch'

pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "diffusion-language-baselines"
+version = "0.1.0"
+description = "Reproducible diffusion language model baseline experiments"
+requires-python = ">=3.11"
+dependencies = [
+    "pydantic>=2.0",
+    "PyYAML>=6.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "jsonschema>=4.0",
+    "pytest>=8.0",
+]
+data = [
+    "datasets==3.6.0",
+    "huggingface_hub>=0.33,<1",
+    "pyarrow>=20,<22",
+    "transformers>=4.53,<5",
+]
+checkpoints = [
+    "gdown>=6,<7",
+    "huggingface_hub>=0.33,<1",
+]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]
+filterwarnings = [
+    "ignore:co_lnotab is deprecated, use co_lines instead.:DeprecationWarning:datasets.utils._dill",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/dlb"]

results/_archive/rerun_di4c_gpu4_20260825_110721/checkpoints/reference_reproduction/duo_di4c/lm1b/logs_di4c_train.txt

@@ -0,0 +1,2 @@
+2026-08-25 10:57:51.062 | INFO     | sdtt.main:di4c_train:77 - Starting training
+2026-08-25 10:57:51.063 | INFO     | sdtt.main:di4c_train:104 - Training starting from scratch (no checkpoint to reload)

results/_archive/rerun_di4c_gpu4_20260825_110721/checkpoints/reference_reproduction/duo_di4c/lm1b/provenance.json

@@ -0,0 +1,121 @@
+{
+  "command": [
+    "/opt/anaconda3/envs/dlb-di4c/bin/python",
+    "-u",
+    "/home/ruizzhao/diffusion_baseline/upstreams/di4c/../../adapters/train_di4c.py",
+    "mode=train",
+    "data.train=lm1b",
+    "data.valid=lm1b",
+    "tokenizer.name=/home/ruizzhao/diffusion_baseline/data/raw/huggingface/hub/models--bert-base-uncased/snapshots/86b5e0934494bd15c9632b12f734a8a67f723594",
+    "data_preprocess.data_cache=/home/ruizzhao/diffusion_baseline/data/training_cache/di4c/lm1b",
+    "data_preprocess.seq_len=128",
+    "data_preprocess.group_text=true",
+    "data_preprocess.remove_text=true",
+    "data_preprocess.add_bos=false",
+    "data_preprocess.add_eos=true",
+    "data_preprocess.legacy_start_end_bos=true",
+    "model=dit-orig-small",
+    "model.length=128",
+    "parameterization=multi-round-sdtt",
+    "parameterization.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b/inputs/teacher_adapted.ckpt",
+    "parameterization.start_from_hf=false",
+    "parameterization.grow_dt_every=10000",
+    "parameterization.orig_num_sampling_steps=1024",
+    "loader.global_batch_size=2",
+    "loader.eval_global_batch_size=2",
+    "loader.batch_size=1",
+    "loader.eval_batch_size=1",
+    "trainer.devices=1",
+    "trainer.num_nodes=1",
+    "trainer.accumulate_grad_batches=2",
+    "trainer.max_steps=20000",
+    "trainer.precision=bf16-mixed",
+    "optim.lr=3e-05",
+    "lr_scheduler.num_warmup_steps=2500",
+    "callbacks.checkpoint_every_n_steps.every_n_train_steps=10000",
+    "checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b",
+    "checkpointing.resume_from_ckpt=true",
+    "checkpointing.resume_ckpt_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b/checkpoints/last.ckpt",
+    "hydra.run.dir=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b",
+    "hydra.job.chdir=true",
+    "wandb=null",
+    "+seed=42",
+    "is_di4c=true",
+    "is_teacher_di4c=false",
+    "T=1024",
+    "latent_bsize=16",
+    "round=7",
+    "+dlb_student_init=teacher"
+  ],
+  "composed_config_sha256": "624ca67b5af09ff63ecd60efa7ff08d70ed8391b276f7ad6d3709607b4d43aa8",
+  "cwd": "/home/ruizzhao/diffusion_baseline/upstreams/di4c",
+  "data": {
+    "aliases": {
+      "lm1b_test_bs128_wrapped.dat": "/home/ruizzhao/diffusion_baseline/data/processed/lm1b-bert-128/validation",
+      "lm1b_train_bs128_wrapped.dat": "/home/ruizzhao/diffusion_baseline/data/processed/lm1b-bert-128/train"
+    },
+    "cache": "/home/ruizzhao/diffusion_baseline/data/training_cache/di4c/lm1b",
+    "format": "hf_dataset_with_attention_mask_v1",
+    "mode": "materialized_bridge",
+    "processed_manifest_sha256": "a8465f3fba8c5dc529fc0751039c8ffa32a7710f480208779b55545cb141017c",
+    "processed_path": "/home/ruizzhao/diffusion_baseline/data/processed/lm1b-bert-128"
+  },
+  "devices": 1,
+  "effective_global_batch_size": 2,
+  "gradient_accumulation": 2,
+  "identity_sha256": "0e6b980c0ad2928f4fe7d0a431da99b5658749371a96429bd86e57bd5993ce49",
+  "nodes": 1,
+  "per_device_batch_size": 1,
+  "prepared_at": "2026-08-25T02:57:48.129909Z",
+  "recipe": {
+    "algorithm": "di4c",
+    "checkpoint_every": 10000,
+    "dataset": "lm1b",
+    "default_per_device_batch_size": 1,
+    "distillation_loss": "dimensional_correlations",
+    "entrypoint": "../../adapters/train_di4c.py",
+    "evidence": "paper_appendix_d",
+    "global_batch_size": 2,
+    "learning_rate": 3e-05,
+    "learning_rate_override": "optim.lr",
+    "max_steps": 20000,
+    "max_steps_override": "trainer.max_steps",
+    "model": "duo_di4c",
+    "optimizer": "adam",
+    "output_override": "checkpointing.save_dir",
+    "precision": "bf16",
+    "release_evidence": null,
+    "release_script_max_steps": null,
+    "rounds": null,
+    "sampling_step": 20000,
+    "sequence_length": 128,
+    "source": "di4c",
+    "source_commit": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+    "source_path": "upstreams/di4c",
+    "steps_per_round": null,
+    "teacher_adapter": "uniform_to_absorbing",
+    "teacher_family": "uniform_duo",
+    "warmup_steps": 2500
+  },
+  "schema_version": 1,
+  "seed": 42,
+  "source_commit": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+  "teacher_adaptation": {
+    "hidden_size": 768,
+    "output_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b/inputs/teacher_adapted.ckpt",
+    "output_sha256": "43b276c5749a412dcd5f715f80a43b566e1a500b760a652a486d9b1d3a170f0d",
+    "schema_version": 1,
+    "source_mask_index": null,
+    "source_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/flm_baselines/lm1b/lm1b_Duo.ckpt",
+    "source_sha256": "b11ee9a4cc533435adb9e1149cff74aec0872a9df2a459b4ab58f1867f6cddfb",
+    "source_vocab_size": 30522,
+    "state_keys_sha256": "8212d507b6931a59375f8127f0ccb9e51c24da74fe05e974648d92c5612fb803",
+    "target_mask_index": 30522,
+    "target_vocab_size": 30523,
+    "teacher_family": "uniform_duo",
+    "transformation": "uniform_to_absorbing",
+    "transformation_sha256": "656341f23524d65a7207e53a85fe06088ad7deaa15365d2ee91ac5fb6eac2cd0"
+  },
+  "teacher_sha256": "b11ee9a4cc533435adb9e1149cff74aec0872a9df2a459b4ab58f1867f6cddfb",
+  "teacher_transformation_sha256": "656341f23524d65a7207e53a85fe06088ad7deaa15365d2ee91ac5fb6eac2cd0"
+}

results/_archive/rerun_di4c_gpu4_20260825_110721/checkpoints/reference_reproduction/mdlm_di4c/lm1b/.hydra/hydra.yaml

@@ -0,0 +1,203 @@
+hydra:
+  run:
+    dir: /home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b
+  sweep:
+    dir: multirun/${now:%Y-%m-%d}/${now:%H-%M-%S}
+    subdir: ${hydra.job.num}
+  launcher:
+    _target_: hydra._internal.core_plugins.basic_launcher.BasicLauncher
+  sweeper:
+    _target_: hydra._internal.core_plugins.basic_sweeper.BasicSweeper
+    max_batch_size: null
+    params: null
+  help:
+    app_name: ${hydra.job.name}
+    header: '${hydra.help.app_name} is powered by Hydra.
+
+      '
+    footer: 'Powered by Hydra (https://hydra.cc)
+
+      Use --hydra-help to view Hydra specific help
+
+      '
+    template: '${hydra.help.header}
+
+      == Configuration groups ==
+
+      Compose your configuration from those groups (group=option)
+
+
+      $APP_CONFIG_GROUPS
+
+
+      == Config ==
+
+      Override anything in the config (foo.bar=value)
+
+
+      $CONFIG
+
+
+      ${hydra.help.footer}
+
+      '
+  hydra_help:
+    template: 'Hydra (${hydra.runtime.version})
+
+      See https://hydra.cc for more info.
+
+
+      == Flags ==
+
+      $FLAGS_HELP
+
+
+      == Configuration groups ==
+
+      Compose your configuration from those groups (For example, append hydra/job_logging=disabled
+      to command line)
+
+
+      $HYDRA_CONFIG_GROUPS
+
+
+      Use ''--cfg hydra'' to Show the Hydra config.
+
+      '
+    hydra_help: ???
+  hydra_logging:
+    version: 1
+    formatters:
+      simple:
+        format: '[%(asctime)s][HYDRA] %(message)s'
+    handlers:
+      console:
+        class: logging.StreamHandler
+        formatter: simple
+        stream: ext://sys.stdout
+    root:
+      level: INFO
+      handlers:
+      - console
+    loggers:
+      logging_example:
+        level: DEBUG
+    disable_existing_loggers: false
+  job_logging:
+    version: 1
+    formatters:
+      simple:
+        format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'
+    handlers:
+      console:
+        class: logging.StreamHandler
+        formatter: simple
+        stream: ext://sys.stdout
+      file:
+        class: logging.FileHandler
+        formatter: simple
+        filename: ${hydra.runtime.output_dir}/${hydra.job.name}.log
+    root:
+      level: INFO
+      handlers:
+      - console
+      - file
+    disable_existing_loggers: false
+  env: {}
+  mode: RUN
+  searchpath: []
+  callbacks: {}
+  output_subdir: .hydra
+  overrides:
+    hydra:
+    - hydra.run.dir=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b
+    - hydra.job.chdir=true
+    - hydra.mode=RUN
+    task:
+    - mode=train
+    - data.train=lm1b
+    - data.valid=lm1b
+    - tokenizer.name=/home/ruizzhao/diffusion_baseline/data/raw/huggingface/hub/models--bert-base-uncased/snapshots/86b5e0934494bd15c9632b12f734a8a67f723594
+    - data_preprocess.data_cache=/home/ruizzhao/diffusion_baseline/data/training_cache/di4c/lm1b
+    - data_preprocess.seq_len=128
+    - data_preprocess.group_text=true
+    - data_preprocess.remove_text=true
+    - data_preprocess.add_bos=false
+    - data_preprocess.add_eos=true
+    - data_preprocess.legacy_start_end_bos=true
+    - model=dit-orig-small
+    - model.length=128
+    - parameterization=multi-round-sdtt
+    - parameterization.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b/inputs/teacher_adapted.ckpt
+    - parameterization.start_from_hf=false
+    - parameterization.grow_dt_every=10000
+    - parameterization.orig_num_sampling_steps=1024
+    - loader.global_batch_size=2
+    - loader.eval_global_batch_size=2
+    - loader.batch_size=1
+    - loader.eval_batch_size=1
+    - trainer.devices=1
+    - trainer.num_nodes=1
+    - trainer.accumulate_grad_batches=2
+    - trainer.max_steps=20000
+    - trainer.precision=bf16-mixed
+    - optim.lr=3e-05
+    - lr_scheduler.num_warmup_steps=2500
+    - callbacks.checkpoint_every_n_steps.every_n_train_steps=10000
+    - checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b
+    - checkpointing.resume_from_ckpt=true
+    - checkpointing.resume_ckpt_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b/checkpoints/last.ckpt
+    - wandb=null
+    - +seed=42
+    - is_di4c=true
+    - is_teacher_di4c=false
+    - T=1024
+    - latent_bsize=16
+    - round=7
+    - +dlb_student_init=teacher
+  job:
+    name: train_di4c
+    chdir: true
+    override_dirname: +dlb_student_init=teacher,+seed=42,T=1024,callbacks.checkpoint_every_n_steps.every_n_train_steps=10000,checkpointing.resume_ckpt_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b/checkpoints/last.ckpt,checkpointing.resume_from_ckpt=true,checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b,data.train=lm1b,data.valid=lm1b,data_preprocess.add_bos=false,data_preprocess.add_eos=true,data_preprocess.data_cache=/home/ruizzhao/diffusion_baseline/data/training_cache/di4c/lm1b,data_preprocess.group_text=true,data_preprocess.legacy_start_end_bos=true,data_preprocess.remove_text=true,data_preprocess.seq_len=128,is_di4c=true,is_teacher_di4c=false,latent_bsize=16,loader.batch_size=1,loader.eval_batch_size=1,loader.eval_global_batch_size=2,loader.global_batch_size=2,lr_scheduler.num_warmup_steps=2500,mode=train,model.length=128,model=dit-orig-small,optim.lr=3e-05,parameterization.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b/inputs/teacher_adapted.ckpt,parameterization.grow_dt_every=10000,parameterization.orig_num_sampling_steps=1024,parameterization.start_from_hf=false,parameterization=multi-round-sdtt,round=7,tokenizer.name=/home/ruizzhao/diffusion_baseline/data/raw/huggingface/hub/models--bert-base-uncased/snapshots/86b5e0934494bd15c9632b12f734a8a67f723594,trainer.accumulate_grad_batches=2,trainer.devices=1,trainer.max_steps=20000,trainer.num_nodes=1,trainer.precision=bf16-mixed,wandb=null
+    id: ???
+    num: ???
+    config_name: config
+    env_set: {}
+    env_copy: []
+    config:
+      override_dirname:
+        kv_sep: '='
+        item_sep: ','
+        exclude_keys: []
+  runtime:
+    version: 1.3.2
+    version_base: '1.3'
+    cwd: /home/ruizzhao/diffusion_baseline/upstreams/di4c
+    config_sources:
+    - path: hydra.conf
+      schema: pkg
+      provider: hydra
+    - path: /home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/configs
+      schema: file
+      provider: main
+    - path: ''
+      schema: structured
+      provider: schema
+    output_dir: /home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b
+    choices:
+      parameterization: multi-round-sdtt
+      lr_scheduler: constant_warmup
+      noise: loglinear
+      strategy: ddp
+      model: dit-orig-small
+      data: openwebtext-split
+      hydra/env: default
+      hydra/callbacks: null
+      hydra/job_logging: default
+      hydra/hydra_logging: default
+      hydra/hydra_help: default
+      hydra/help: default
+      hydra/sweeper: basic
+      hydra/launcher: basic
+      hydra/output: default
+  verbose: false

results/_archive/rerun_di4c_gpu4_20260825_110721/checkpoints/reference_reproduction/mdlm_di4c/lm1b/inputs/teacher_adapted.provenance.json

@@ -0,0 +1,16 @@
+{
+  "hidden_size": 768,
+  "output_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/mdlm_di4c/lm1b/inputs/teacher_adapted.ckpt",
+  "output_sha256": "56d653c28097b5ad6fa8dc8101214a7369170f0643899a3d3485303aacbe29c8",
+  "schema_version": 1,
+  "source_mask_index": 103,
+  "source_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/flm_baselines/lm1b/lm1b_MDLM_.ckpt",
+  "source_sha256": "d42739e5a99bbd7c0f604dff7fe4c9ab12348b475e30f1d807e70ef32249e514",
+  "source_vocab_size": 30522,
+  "state_keys_sha256": "8212d507b6931a59375f8127f0ccb9e51c24da74fe05e974648d92c5612fb803",
+  "target_mask_index": 30522,
+  "target_vocab_size": 30523,
+  "teacher_family": "masked_mdlm",
+  "transformation": "masked_to_absorbing",
+  "transformation_sha256": "0d4b7caaa7c4c2de75b488026be396440bb4ea6c4dd11dd3edfbfb83a3364711"
+}

results/_archive/rerun_di4c_gpu4_gpu7_20260825_111320/checkpoints/reference_reproduction/duo_di4c/lm1b/inputs/teacher_adapted.provenance.json

@@ -0,0 +1,16 @@
+{
+  "hidden_size": 768,
+  "output_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/duo_di4c/lm1b/inputs/teacher_adapted.ckpt",
+  "output_sha256": "5f34e31d2743d9851c4314f52861e5f2366bfdbea51b156a9bd102fc592f0b23",
+  "schema_version": 1,
+  "source_mask_index": null,
+  "source_path": "/home/ruizzhao/diffusion_baseline/checkpoints/reference_reproduction/flm_baselines/lm1b/lm1b_Duo.ckpt",
+  "source_sha256": "b11ee9a4cc533435adb9e1149cff74aec0872a9df2a459b4ab58f1867f6cddfb",
+  "source_vocab_size": 30522,
+  "state_keys_sha256": "8212d507b6931a59375f8127f0ccb9e51c24da74fe05e974648d92c5612fb803",
+  "target_mask_index": 30522,
+  "target_vocab_size": 30523,
+  "teacher_family": "uniform_duo",
+  "transformation": "uniform_to_absorbing",
+  "transformation_sha256": "656341f23524d65a7207e53a85fe06088ad7deaa15365d2ee91ac5fb6eac2cd0"
+}

results/_archive/rerun_di4c_gpu4_gpu7_20260825_111320/checkpoints/reference_reproduction/duo_di4c/lm1b/logs_di4c_train.txt

@@ -0,0 +1,2 @@
+2026-08-25 11:07:33.667 | INFO     | sdtt.main:di4c_train:77 - Starting training
+2026-08-25 11:07:33.667 | INFO     | sdtt.main:di4c_train:104 - Training starting from scratch (no checkpoint to reload)

results/conditional/matrix/generation_candi_lm1b_1024_only.tsv

@@ -0,0 +1,3 @@
+# schema=dlb-conditional-generation-matrix-v1
+task_id	category	model	dataset	steps	sample_count	seed	environment	adapter	source	provenance	protocol	conditioning_manifest	conditioning_manifest_sha256	sample_dir	metrics_path	timing_path
+candi-lm1b-steps-1024	many	candi	lm1b	1024	2048	42	dlb-candi	candi	candi	reference_reproduction	c64_zs_v1	/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json	0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039	/home/ruizzhao/diffusion_baseline/results/conditional/samples/lm1b/candi/steps_1024	/home/ruizzhao/diffusion_baseline/results/conditional/metrics/lm1b/candi/steps_1024/metrics.json	/home/ruizzhao/diffusion_baseline/results/conditional/timing/lm1b/candi/steps_1024/timing.json

results/conditional/samples/owt/duo_dcd/steps_16/request.json

@@ -0,0 +1,108 @@
+{
+  "command": [
+    "/opt/anaconda3/envs/dlb-duo/bin/python",
+    "-u",
+    "-m",
+    "dlb.adapters.capture",
+    "--upstream-entrypoint=/home/ruizzhao/diffusion_baseline/upstreams/duo/main.py",
+    "--capture-path=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_16/upstream_token_ids.json",
+    "--generation-mode=conditional_prefix",
+    "--conditioning-manifest=/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "--conditioning-manifest-sha256=7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "--conditioning-config-sha256=89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "--prefix-length=64",
+    "--evaluation-continuation-length=64",
+    "--prompt-count=1024",
+    "--diversity-prompt-count=256",
+    "--completions-per-diversity-prompt=5",
+    "--completion-schedule=c0:p0-1023;c1-4:p0-255",
+    "--",
+    "mode=sample_eval",
+    "seed=42",
+    "data=openwebtext-split",
+    "model=small",
+    "model.length=1024",
+    "loader.batch_size=2",
+    "loader.eval_batch_size=8",
+    "sampling.num_sample_batches=256",
+    "sampling.steps=16",
+    "eval.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/official/duo_dcd/owt",
+    "trainer.devices=1",
+    "hydra.run.dir=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_16/upstream_hydra",
+    "checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_16/upstream_hydra",
+    "+wandb.offline=true",
+    "eval.generated_samples_path=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_16/upstream_samples.json",
+    "algo=duo_base",
+    "algo.backbone=hf_dit",
+    "sampling.predictor=ancestral",
+    "sampling.noise_removal=ancestral",
+    "eval.disable_ema=False"
+  ],
+  "identity": {
+    "adapter_identity": "dlb.adapters.duo:v1|dlb.adapters.duo:DuoAdapter|8878182eb10d7d1e3f18219cb62c2153084842fdb0ab532b1a3c7df0e10da3c6",
+    "checkpoint_lock_id": "duo_dcd_owt_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all",
+    "checkpoint_selection": {
+      "path": null,
+      "resource": "duo_dcd_owt_hf",
+      "teacher_family": null
+    },
+    "checkpoint_sha256": "74770bf9f7305ba80f4cd37d5b0ba78e09a887a7480894a4c6567278e9f6148f",
+    "checkpoint_teacher_family": "uniform_duo",
+    "command_sha256": "fe4e8b4e60f38f61c5c765f662e133ae39f11456895da34cd729b9c0e5f9e6fc",
+    "completion_schedule": "c0:p0-1023;c1-4:p0-255",
+    "completions_per_diversity_prompt": 5,
+    "conditioning_config_sha256": "89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "conditioning_manifest": "/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "conditioning_manifest_sha256": "7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "config_sha256": "6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f",
+    "dataset_id": "owt",
+    "device": null,
+    "diversity_prompt_count": 256,
+    "environment": "dlb-duo",
+    "evaluation_continuation_length": 64,
+    "generation_mode": "conditional_prefix",
+    "model_id": "duo_dcd",
+    "prefix_length": 64,
+    "prompt_count": 1024,
+    "results_root": "/home/ruizzhao/diffusion_baseline/results/conditional",
+    "run_id": "duo_dcd-owt-steps-16",
+    "sample_count": 2048,
+    "seed": 42,
+    "source_sha256": "7c9b498f5b717de064d6fad7e2509c866e6cb620",
+    "step_count": 16
+  },
+  "request": {
+    "adapter_identity": "dlb.adapters.duo:v1|dlb.adapters.duo:DuoAdapter|8878182eb10d7d1e3f18219cb62c2153084842fdb0ab532b1a3c7df0e10da3c6",
+    "checkpoint_lock_id": "duo_dcd_owt_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all",
+    "checkpoint_selection": {
+      "path": null,
+      "resource": "duo_dcd_owt_hf",
+      "teacher_family": null
+    },
+    "checkpoint_sha256": "74770bf9f7305ba80f4cd37d5b0ba78e09a887a7480894a4c6567278e9f6148f",
+    "checkpoint_teacher_family": "uniform_duo",
+    "command": null,
+    "completion_schedule": "c0:p0-1023;c1-4:p0-255",
+    "completions_per_diversity_prompt": 5,
+    "conditioning_config_sha256": "89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "conditioning_manifest": "/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "conditioning_manifest_sha256": "7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "config_sha256": "6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f",
+    "dataset_id": "owt",
+    "device": null,
+    "diversity_prompt_count": 256,
+    "environment": "dlb-duo",
+    "evaluation_continuation_length": 64,
+    "generation_mode": "conditional_prefix",
+    "model_id": "duo_dcd",
+    "prefix_length": 64,
+    "prompt_count": 1024,
+    "results_root": "/home/ruizzhao/diffusion_baseline/results/conditional",
+    "run_id": "duo_dcd-owt-steps-16",
+    "sample_count": 2048,
+    "seed": 42,
+    "source_sha256": "7c9b498f5b717de064d6fad7e2509c866e6cb620",
+    "step_count": 16
+  },
+  "started_at": "2026-08-27T03:20:40.709851Z"
+}

results/conditional/samples/owt/duo_dcd/steps_8/upstream_hydra/dlb_pinned_upstream_main.log

@@ -0,0 +1,14 @@
+[2026-08-25 20:14:13,149][dlb_pinned_upstream_main][INFO] - Starting Sample Eval.
+[2026-08-26 01:38:22,002][dlb_pinned_upstream_main][INFO] - Starting Sample Eval.
+[2026-08-26 01:38:33,346][dlb_pinned_upstream_main][INFO] - Generative perplexity: 130.35482788085938
+[2026-08-26 01:38:33,347][dlb_pinned_upstream_main][INFO] - Sample entropy: 5.518235206604004
+[2026-08-26 01:38:33,348][dlb_pinned_upstream_main][INFO] - Samples saved at: /home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_8/upstream_samples.json
+[2026-08-26 13:59:31,218][dlb_pinned_upstream_main][INFO] - Starting Sample Eval.
+[2026-08-27 11:12:05,657][dlb_pinned_upstream_main][INFO] - Starting Sample Eval.
+[2026-08-27 11:20:33,911][dlb_pinned_upstream_main][INFO] - Generative perplexity: 83.10469055175781
+[2026-08-27 11:20:33,912][dlb_pinned_upstream_main][INFO] - Sample entropy: 5.425522327423096
+[2026-08-27 11:20:33,955][dlb_pinned_upstream_main][INFO] - Samples saved at: /home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_8/upstream_samples.json
+[2026-08-27 16:58:20,881][dlb_pinned_upstream_main][INFO] - Starting Sample Eval.
+[2026-08-27 16:58:31,049][dlb_pinned_upstream_main][INFO] - Generative perplexity: 130.35482788085938
+[2026-08-27 16:58:31,049][dlb_pinned_upstream_main][INFO] - Sample entropy: 5.518235206604004
+[2026-08-27 16:58:31,051][dlb_pinned_upstream_main][INFO] - Samples saved at: /home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/duo_dcd/steps_8/upstream_samples.json

results/conditional/samples/owt/mdlm/steps_4/upstream_hydra/.hydra/hydra.yaml

@@ -0,0 +1,178 @@
+hydra:
+  run:
+    dir: /home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm/steps_4/upstream_hydra
+  sweep:
+    dir: multirun/${now:%Y-%m-%d}/${now:%H-%M-%S}
+    subdir: ${hydra.job.num}
+  launcher:
+    _target_: hydra._internal.core_plugins.basic_launcher.BasicLauncher
+  sweeper:
+    _target_: hydra._internal.core_plugins.basic_sweeper.BasicSweeper
+    max_batch_size: null
+    params: null
+  help:
+    app_name: ${hydra.job.name}
+    header: '${hydra.help.app_name} is powered by Hydra.
+
+      '
+    footer: 'Powered by Hydra (https://hydra.cc)
+
+      Use --hydra-help to view Hydra specific help
+
+      '
+    template: '${hydra.help.header}
+
+      == Configuration groups ==
+
+      Compose your configuration from those groups (group=option)
+
+
+      $APP_CONFIG_GROUPS
+
+
+      == Config ==
+
+      Override anything in the config (foo.bar=value)
+
+
+      $CONFIG
+
+
+      ${hydra.help.footer}
+
+      '
+  hydra_help:
+    template: 'Hydra (${hydra.runtime.version})
+
+      See https://hydra.cc for more info.
+
+
+      == Flags ==
+
+      $FLAGS_HELP
+
+
+      == Configuration groups ==
+
+      Compose your configuration from those groups (For example, append hydra/job_logging=disabled
+      to command line)
+
+
+      $HYDRA_CONFIG_GROUPS
+
+
+      Use ''--cfg hydra'' to Show the Hydra config.
+
+      '
+    hydra_help: ???
+  hydra_logging:
+    version: 1
+    formatters:
+      simple:
+        format: '[%(asctime)s][HYDRA] %(message)s'
+    handlers:
+      console:
+        class: logging.StreamHandler
+        formatter: simple
+        stream: ext://sys.stdout
+    root:
+      level: INFO
+      handlers:
+      - console
+    loggers:
+      logging_example:
+        level: DEBUG
+    disable_existing_loggers: false
+  job_logging:
+    version: 1
+    formatters:
+      simple:
+        format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'
+    handlers:
+      console:
+        class: logging.StreamHandler
+        formatter: simple
+        stream: ext://sys.stdout
+      file:
+        class: logging.FileHandler
+        formatter: simple
+        filename: ${hydra.runtime.output_dir}/${hydra.job.name}.log
+    root:
+      level: INFO
+      handlers:
+      - console
+      - file
+    disable_existing_loggers: false
+  env: {}
+  mode: RUN
+  searchpath: []
+  callbacks: {}
+  output_subdir: .hydra
+  overrides:
+    hydra:
+    - hydra.run.dir=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm/steps_4/upstream_hydra
+    - hydra.mode=RUN
+    task:
+    - mode=sample_eval
+    - seed=42
+    - data=openwebtext-split
+    - model=small
+    - model.length=1024
+    - loader.batch_size=2
+    - loader.eval_batch_size=1
+    - sampling.num_sample_batches=1
+    - sampling.steps=4
+    - eval.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/official/mdlm/owt
+    - trainer.devices=1
+    - checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm/steps_4/upstream_hydra
+    - +wandb.offline=true
+    - backbone=hf_dit
+    - parameterization=subs
+    - sampling.predictor=ddpm
+    - sampling.noise_removal=True
+    - eval.disable_ema=False
+  job:
+    name: dlb_pinned_upstream_main
+    chdir: true
+    override_dirname: +wandb.offline=true,backbone=hf_dit,checkpointing.save_dir=/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm/steps_4/upstream_hydra,data=openwebtext-split,eval.checkpoint_path=/home/ruizzhao/diffusion_baseline/checkpoints/official/mdlm/owt,eval.disable_ema=False,loader.batch_size=2,loader.eval_batch_size=1,mode=sample_eval,model.length=1024,model=small,parameterization=subs,sampling.noise_removal=True,sampling.num_sample_batches=1,sampling.predictor=ddpm,sampling.steps=4,seed=42,trainer.devices=1
+    id: ???
+    num: ???
+    config_name: config
+    env_set: {}
+    env_copy: []
+    config:
+      override_dirname:
+        kv_sep: '='
+        item_sep: ','
+        exclude_keys: []
+  runtime:
+    version: 1.3.2
+    version_base: '1.3'
+    cwd: /home/ruizzhao/diffusion_baseline
+    config_sources:
+    - path: hydra.conf
+      schema: pkg
+      provider: hydra
+    - path: /home/ruizzhao/diffusion_baseline/upstreams/mdlm/configs
+      schema: file
+      provider: main
+    - path: ''
+      schema: structured
+      provider: schema
+    output_dir: /home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm/steps_4/upstream_hydra
+    choices:
+      lr_scheduler: constant_warmup
+      noise: loglinear
+      strategy: ddp
+      model: small
+      data: openwebtext-split
+      hydra/env: default
+      hydra/callbacks: null
+      hydra/job_logging: default
+      hydra/hydra_logging: default
+      hydra/hydra_help: default
+      hydra/help: default
+      hydra/sweeper: basic
+      hydra/launcher: basic
+      hydra/output: default
+  verbose: false

results/conditional/samples/owt/mdlm_di4c/steps_2/request.json

@@ -0,0 +1,141 @@
+{
+  "command": [
+    "/opt/anaconda3/envs/dlb-di4c/bin/python",
+    "-B",
+    "-u",
+    "/home/ruizzhao/diffusion_baseline/adapters/sample_di4c.py",
+    "--upstream-root",
+    "/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt",
+    "--checkpoint",
+    "/home/ruizzhao/diffusion_baseline/checkpoints/official/mdlm_di4c/owt/sdtt7-di4c2.ckpt",
+    "--config",
+    "/home/ruizzhao/diffusion_baseline/configs/sampling/di4c_mdlm_owt.yaml",
+    "--checkpoint-sha256",
+    "4af61572dc82fe012ce395a2e4e4be0585ea09076597ba1f7241ca196b97ecb0",
+    "--config-sha256",
+    "cf627f89b8dede9b25ac8e10407c3b7de203a76c4ba84100723287e571dcec34",
+    "--data-config",
+    "/home/ruizzhao/diffusion_baseline/artifacts/data.yaml",
+    "--downloads-manifest",
+    "/home/ruizzhao/diffusion_baseline/data/manifests/downloads.json",
+    "--tokenizer-snapshot",
+    "/home/ruizzhao/diffusion_baseline/data/raw/huggingface/hub/models--gpt2/snapshots/607a30d783dfa663caf39e06633721c8d4cfcd7e",
+    "--output",
+    "/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm_di4c/steps_2/upstream_token_ids.json",
+    "--sample-count",
+    "2048",
+    "--batch-size",
+    "16",
+    "--num-steps",
+    "2",
+    "--seq-len",
+    "1024",
+    "--seed",
+    "42",
+    "--sampler",
+    "ancestral",
+    "--teacher-family",
+    "masked_mdlm",
+    "--dataset",
+    "owt",
+    "--generation-mode",
+    "conditional_prefix",
+    "--conditioning-manifest",
+    "/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "--conditioning-manifest-sha256",
+    "7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "--conditioning-config-sha256",
+    "89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "--prefix-length",
+    "64",
+    "--evaluation-continuation-length",
+    "64",
+    "--prompt-count",
+    "1024",
+    "--diversity-prompt-count",
+    "256",
+    "--completions-per-diversity-prompt",
+    "5",
+    "--completion-schedule",
+    "c0:p0-1023;c1-4:p0-255",
+    "--offline",
+    "true",
+    "--allow-missing-embedded-config",
+    "true"
+  ],
+  "identity": {
+    "adapter_identity": "dlb.adapters.di4c:v3|dlb.adapters.di4c:Di4CAdapter|a417a282fb2e7548e0965c4417a9e9b1e906dc03c502ed5a0c03a6d0d3eceed9",
+    "checkpoint_lock_id": "di4c_mdlm_owt_zenodo:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all",
+    "checkpoint_selection": {
+      "path": null,
+      "resource": "di4c_mdlm_owt_zenodo",
+      "sampling_config": "configs/sampling/di4c_mdlm_owt.yaml",
+      "sampling_config_sha256": "cf627f89b8dede9b25ac8e10407c3b7de203a76c4ba84100723287e571dcec34",
+      "sampling_config_source": "project",
+      "sampling_config_source_commit": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+      "teacher_family": null
+    },
+    "checkpoint_sha256": "45ec6edfeb0d1c3ed0b43bacf5d53d4f83bf4806af9c742c08cf49e8d79b7b55",
+    "checkpoint_teacher_family": "masked_mdlm",
+    "command_sha256": "e7ea7fa99d1dbab8447d14fddb8740f59100335dc9c51645b75e4efa1d9ca837",
+    "completion_schedule": "c0:p0-1023;c1-4:p0-255",
+    "completions_per_diversity_prompt": 5,
+    "conditioning_config_sha256": "89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "conditioning_manifest": "/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "conditioning_manifest_sha256": "7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "config_sha256": "6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f",
+    "dataset_id": "owt",
+    "device": null,
+    "diversity_prompt_count": 256,
+    "environment": "dlb-di4c",
+    "evaluation_continuation_length": 64,
+    "generation_mode": "conditional_prefix",
+    "model_id": "mdlm_di4c",
+    "prefix_length": 64,
+    "prompt_count": 1024,
+    "results_root": "/home/ruizzhao/diffusion_baseline/results/conditional",
+    "run_id": "mdlm_di4c-owt-steps-2",
+    "sample_count": 2048,
+    "seed": 42,
+    "source_sha256": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+    "step_count": 2
+  },
+  "request": {
+    "adapter_identity": "dlb.adapters.di4c:v3|dlb.adapters.di4c:Di4CAdapter|a417a282fb2e7548e0965c4417a9e9b1e906dc03c502ed5a0c03a6d0d3eceed9",
+    "checkpoint_lock_id": "di4c_mdlm_owt_zenodo:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all",
+    "checkpoint_selection": {
+      "path": null,
+      "resource": "di4c_mdlm_owt_zenodo",
+      "sampling_config": "configs/sampling/di4c_mdlm_owt.yaml",
+      "sampling_config_sha256": "cf627f89b8dede9b25ac8e10407c3b7de203a76c4ba84100723287e571dcec34",
+      "sampling_config_source": "project",
+      "sampling_config_source_commit": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+      "teacher_family": null
+    },
+    "checkpoint_sha256": "45ec6edfeb0d1c3ed0b43bacf5d53d4f83bf4806af9c742c08cf49e8d79b7b55",
+    "checkpoint_teacher_family": "masked_mdlm",
+    "command": null,
+    "completion_schedule": "c0:p0-1023;c1-4:p0-255",
+    "completions_per_diversity_prompt": 5,
+    "conditioning_config_sha256": "89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114",
+    "conditioning_manifest": "/home/ruizzhao/diffusion_baseline/data/manifests/conditional-owt-c64.json",
+    "conditioning_manifest_sha256": "7718eb8d4d3e75ec7628d2f15f3c73c84f6470af0303ad4017da5e0ddae5bd32",
+    "config_sha256": "6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f",
+    "dataset_id": "owt",
+    "device": null,
+    "diversity_prompt_count": 256,
+    "environment": "dlb-di4c",
+    "evaluation_continuation_length": 64,
+    "generation_mode": "conditional_prefix",
+    "model_id": "mdlm_di4c",
+    "prefix_length": 64,
+    "prompt_count": 1024,
+    "results_root": "/home/ruizzhao/diffusion_baseline/results/conditional",
+    "run_id": "mdlm_di4c-owt-steps-2",
+    "sample_count": 2048,
+    "seed": 42,
+    "source_sha256": "ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3",
+    "step_count": 2
+  },
+  "started_at": "2026-08-25T14:04:14.561800Z"
+}

results/conditional/samples/owt/mdlm_sdtt/steps_16/conversion_metadata.json

@@ -0,0 +1,28 @@
+{
+  "checkpoint_lock_id": "sdtt_owt_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all",
+  "checkpoint_selection": {
+    "path": null,
+    "resource": "sdtt_owt_hf",
+    "sampling_config": "config.json",
+    "sampling_config_sha256": null,
+    "sampling_config_source": "resource",
+    "sampling_config_source_commit": null,
+    "teacher_family": null
+  },
+  "checkpoint_sha256": "0bb184e1ea44fe6fa8f6d5bf58f53494ceec96df20e44dd97cf1b46d03e4d508",
+  "conditioning_implementation": "native_projection",
+  "exclude_from_latency": true,
+  "format": "dlb-conditional-token-capture-v1",
+  "generated_samples": 2048,
+  "generation_seconds_sentinel": 0.0,
+  "generation_seconds_source": "unavailable_excluded_sentinel",
+  "paper_conditional_training_reproduced": false,
+  "requested_samples": 2048,
+  "teacher_family": "masked_mdlm",
+  "token_ids_source": "upstream_conditional_capture",
+  "tokenizer": "gpt2",
+  "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e",
+  "trim_policy": "none_exact_conditional_schedule",
+  "trimmed_samples": 0,
+  "upstream_output": "/home/ruizzhao/diffusion_baseline/results/conditional/samples/owt/mdlm_sdtt/steps_16/upstream_token_ids.json"
+}

results/conditional/timing/lm1b/candi/steps_4/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.candi:v1|dlb.adapters.candi:CANDIAdapter|e63560f999a54ea08143cab77049eaa8bbe2965321470716a2fe3a94f2b7ed19","attempt_id":"ce46f243d6c341f189ad1415096cc811","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_CANDI.ckpt","checkpoint_selection":{"path":"lm1b_CANDI.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"hybrid_candi"},"checkpoint_sha256":"2b301a6875a6ba9efa52a2a66773a5bcc49d5397f7c861a94495252e647df185","checkpoint_teacher_family":"hybrid_candi","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-candi","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"candi","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"candi:pinned_internal_bf16_autocast_with_fp32_sensitive_ops","precision_policy_binding":{"adapter_identity":"dlb.adapters.candi:v1|dlb.adapters.candi:CANDIAdapter|e63560f999a54ea08143cab77049eaa8bbe2965321470716a2fe3a94f2b7ed19","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_CANDI.ckpt","checkpoint_repository":null,"checkpoint_resource":"flm_lm1b_reproductions","checkpoint_revision":null,"checkpoint_selection":{"path":"lm1b_CANDI.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"hybrid_candi"},"checkpoint_sha256":"2b301a6875a6ba9efa52a2a66773a5bcc49d5397f7c861a94495252e647df185","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"candi","prefix_length":64,"prompt_count":1024,"source_commit":"cd57ae9eec98d6ac71cd52bdc50eeec8dfd70f91","upstream":"candi"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"bb7d05734503f57d02187a8a94d91394b5a3b394b45a6c3556ed51e612591713","symbol":"_capture_teacher.<locals>.capture.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/algo.py","sha256":"131ffaeb45418710294a70e1258afdaaa213e8a51bd19ef6227eda8d7fd036e9","symbol":"CANDI"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"ContDIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"DDiTFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/models/dit_cont.py","sha256":"3ee29045168f0ea4c0faecc35c053cdd2b872b7ac11eb16def52c6dae084552a","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/candi/trainer_base.py","sha256":"953ab0e52957472772150b32a342fe826dcc1a57ad3a4c48654de4985f0c18a4","symbol":"LogLinear"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"Softplus"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-candi/lib/python3.11/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"}],"sample_count":2048,"seed":42,"source_commit":"cd57ae9eec98d6ac71cd52bdc50eeec8dfd70f91","steps":4,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.025582681642845273,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.025654377415776253,0.025654647033661604,0.025681417901068926,0.02571211475878954,0.02566605480387807,0.025676862336695194,0.02552227396517992,0.025557107757776976,0.025547352619469166,0.025507410988211632,0.025472288019955158,0.02552182273939252,0.025514251086860895,0.029027701821178198,0.025568754877895117,0.025534483138471842,0.0255577489733696,0.02552997600287199,0.02551616495475173,0.028856168035417795,0.02559660840779543,0.02556377835571766,0.025508252903819084,0.02548919292166829,0.02543579088523984,0.02876045135781169,0.026635911781340837,0.027046071831136942,0.027176701929420233,0.02721211686730385,0.03141493396833539,0.02754831174388528],"repeats":32,"seconds_per_sample":0.026302096943254583,"standard_deviation_convention":"population","standard_deviation_seconds":0.0013984036014305402,"warmups":5}}

results/conditional/timing/lm1b/flm/steps_2/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.flm:v2|dlb.adapters.flm:FLMAdapter|c3a6a4573200fecf38a47fc6e7bc78254ca362fff3b96079dc2efa9f8e27aff0","attempt_id":"7181d39abf7c46448f7c6c6daf250549","checkpoint_lock_id":"recipe:flm_lm1b_official_ckpt:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/official/flm_ckpt/lm1b","recipe":"flm_lm1b_official_ckpt","sampling_checkpoint":"lm1b_flm.ckpt","sampling_config":null,"source":"flm","source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","teacher_adapter":null,"teacher_family":"continuous_flm"},"checkpoint_sha256":"99a97513471dc7232d4a0fc6b50d47301d97a6198e0cf13b77b3fe70c12c3665","checkpoint_teacher_family":"continuous_flm","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-flm","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"flm","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed-static-author-policy","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"flm:checkpoint_config_and_loaded_runtime_code_bound","precision_policy_binding":{"adapter_identity":"dlb.adapters.flm:v2|dlb.adapters.flm:FLMAdapter|c3a6a4573200fecf38a47fc6e7bc78254ca362fff3b96079dc2efa9f8e27aff0","checkpoint_lock_id":"recipe:flm_lm1b_official_ckpt:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/official/flm_ckpt/lm1b","recipe":"flm_lm1b_official_ckpt","sampling_checkpoint":"lm1b_flm.ckpt","sampling_config":null,"source":"flm","source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","teacher_adapter":null,"teacher_family":"continuous_flm"},"checkpoint_sha256":"99a97513471dc7232d4a0fc6b50d47301d97a6198e0cf13b77b3fe70c12c3665","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"flm","prefix_length":64,"prompt_count":1024,"source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","upstream":"flm"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"92c69119e85195e099a41668e8f75dd5a908c4e341fb220f714853db555dca0d","symbol":"_capture_teacher.<locals>.capture.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/algo.py","sha256":"65206adc72013bb9b09b68fd847239b3193dd65fd59669bbdee42e061583d76a","symbol":"FLM"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DDiTFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/trainer_base.py","sha256":"4a3cfa69d511023553290aa3ce975b9ced9cb73743f6a06e141de4ebfeec3d36","symbol":"LogLinear"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"Softplus"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"}],"sample_count":2048,"seed":42,"source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","steps":2,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.01166545576415956,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.011584140826016665,0.011667358689010143,0.01167448004707694,0.01161995716392994,0.011673438362777233,0.011663552839308977,0.011603540740907192,0.011659516952931881,0.0116311926394701,0.01228940300643444,0.011637083254754543,0.012476850766688585,0.014373901300132275,0.012706473004072905,0.0115157556720078,0.01156038511544466,0.011537951417267323,0.011575218290090561,0.011547075118869543,0.011675522197037935,0.011591914109885693,0.01162299094721675,0.011569839902222157,0.011668589897453785,0.01156229805201292,0.017901872750371695,0.018812119960784912,0.018786410801112652,0.018782244063913822,0.0188258308917284,0.01874570781365037,0.01887272298336029],"repeats":32,"seconds_per_sample":0.013325479361810721,"standard_deviation_convention":"population","standard_deviation_seconds":0.0028835147626228214,"warmups":5}}

results/conditional/timing/lm1b/fmlm/steps_1/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.flm:v2|dlb.adapters.flm:FLMAdapter|c3a6a4573200fecf38a47fc6e7bc78254ca362fff3b96079dc2efa9f8e27aff0","attempt_id":"bc2f2f295c8f4637a0fff9d4d599a571","checkpoint_lock_id":"recipe:fmlm_lm1b_official_ckpt:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/official/fmlm_ckpt/lm1b","recipe":"fmlm_lm1b_official_ckpt","sampling_checkpoint":"lm1b_fmlm.ckpt","sampling_config":null,"source":"flm","source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","teacher_adapter":null,"teacher_family":"continuous_flm"},"checkpoint_sha256":"cb096021fda9344c88b184d9b3ac8a0a3bc45e30be7bcc917e024971a9d67395","checkpoint_teacher_family":"continuous_flm","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-flm","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"fmlm","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed-static-author-policy","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"flm:checkpoint_config_and_loaded_runtime_code_bound","precision_policy_binding":{"adapter_identity":"dlb.adapters.flm:v2|dlb.adapters.flm:FLMAdapter|c3a6a4573200fecf38a47fc6e7bc78254ca362fff3b96079dc2efa9f8e27aff0","checkpoint_lock_id":"recipe:fmlm_lm1b_official_ckpt:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/official/fmlm_ckpt/lm1b","recipe":"fmlm_lm1b_official_ckpt","sampling_checkpoint":"lm1b_fmlm.ckpt","sampling_config":null,"source":"flm","source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","teacher_adapter":null,"teacher_family":"continuous_flm"},"checkpoint_sha256":"cb096021fda9344c88b184d9b3ac8a0a3bc45e30be7bcc917e024971a9d67395","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"fmlm","prefix_length":64,"prompt_count":1024,"source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","upstream":"flm"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"92c69119e85195e099a41668e8f75dd5a908c4e341fb220f714853db555dca0d","symbol":"_capture_teacher.<locals>.capture.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/algo.py","sha256":"65206adc72013bb9b09b68fd847239b3193dd65fd59669bbdee42e061583d76a","symbol":"FMLM"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DDiTFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"DIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/models/dit.py","sha256":"a5deb76947fa3c31d21e264e10591f504be1c987bc54b98553059f556648ba7a","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/flm/trainer_base.py","sha256":"4a3cfa69d511023553290aa3ce975b9ced9cb73743f6a06e141de4ebfeec3d36","symbol":"LogLinear"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"Softplus"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-flm/lib/python3.11/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"}],"sample_count":2048,"seed":42,"source_commit":"a1918d5164e5038e37d0b7a4fb2010ce75b863b3","steps":1,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.005975442007184029,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.006069451104849577,0.00600470183417201,0.006014927756041288,0.00599653972312808,0.006064232904464006,0.006062130443751812,0.006032113917171955,0.0060233608819544315,0.006025373935699463,0.0060278079472482204,0.00599453691393137,0.00597678916528821,0.005972653161734343,0.0059919217601418495,0.005957338958978653,0.0060112918727099895,0.006009228993207216,0.005948906764388084,0.005927623715251684,0.005974094849079847,0.005958920810371637,0.00597716961055994,0.005938299931585789,0.005967684555798769,0.005941184237599373,0.005948115140199661,0.005960994865745306,0.005960964132100344,0.005942847114056349,0.005961124785244465,0.005966432858258486,0.005963008385151625],"repeats":32,"seconds_per_sample":0.005986617907183245,"standard_deviation_convention":"population","standard_deviation_seconds":3.7829216481046054e-05,"warmups":5}}

results/conditional/timing/lm1b/langflow/steps_1/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.langflow:v1|dlb.adapters.langflow:LangFlowAdapter|7d80794cf6a132fc8d4c179d723e8e486c70476888b2ba51dd92cdc3f5284f84","attempt_id":"98a8de4dfca04d9eac690b53a976afe9","checkpoint_lock_id":"langflow_lm1b_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all","checkpoint_selection":{"path":null,"resource":"langflow_lm1b_hf","teacher_family":null},"checkpoint_sha256":"2050f55b63e952941eb190711828176529f2f510031cc0e0fc1cd39f6998d7bb","checkpoint_teacher_family":"continuous_langflow","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-langflow","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"langflow","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"langflow:pinned_internal_bf16_autocast_with_fp32_sensitive_ops","precision_policy_binding":{"adapter_identity":"dlb.adapters.langflow:v1|dlb.adapters.langflow:LangFlowAdapter|7d80794cf6a132fc8d4c179d723e8e486c70476888b2ba51dd92cdc3f5284f84","checkpoint_lock_id":"langflow_lm1b_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all","checkpoint_repository":"Continuous-Rivals-Discrete/langflow-lm1b","checkpoint_resource":"langflow_lm1b_hf","checkpoint_revision":"d5d395d9bdf01f697ba8830110e3e78c63d77cb5","checkpoint_selection":{"path":null,"resource":"langflow_lm1b_hf","teacher_family":null},"checkpoint_sha256":"2050f55b63e952941eb190711828176529f2f510031cc0e0fc1cd39f6998d7bb","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"langflow","prefix_length":64,"prompt_count":1024,"source_commit":"a712b08570c56c787d6ef24f8e906a2dcdf768f5","upstream":"langflow"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"daaf59774f53f83ef7987b1432cd14769226e701310308480bbac28ef5217256","symbol":"_capture_langflow.<locals>.generate.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"DDiTFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"GumbelProposal"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlow"},{"kind":"sampling_closure","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlow.generate_samples"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlowBackbone"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"TimestepEmbedder"},{"kind":"bound_method_owner","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/autograd/grad_mode.py","sha256":"de61684a7f73eb361c0f47c22159a2bfc67df6f78ddfa5f198049b0654f1f4a7","symbol":"no_grad"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"},{"kind":"sampling_closure","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/utils/_contextlib.py","sha256":"cf7aa5b08f44974ba8c1d08cd71ef70ffd13d1c48a4931576eb235306bfa46b5","symbol":"LangFlow.generate_samples"},{"kind":"sampling_closure","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/utils/_contextlib.py","sha256":"cf7aa5b08f44974ba8c1d08cd71ef70ffd13d1c48a4931576eb235306bfa46b5","symbol":"_DecoratorContextManager.clone"}],"sample_count":2048,"seed":42,"source_commit":"a712b08570c56c787d6ef24f8e906a2dcdf768f5","steps":1,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.007625518133863807,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.021140022203326225,0.021185080986469984,0.021204430144280195,0.02120481012389064,0.02120248693972826,0.013259331230074167,0.00759607320651412,0.00796378729864955,0.007610605098307133,0.007598105818033218,0.0076356432400643826,0.0075834039598703384,0.007601501885801554,0.007644838187843561,0.00793003523722291,0.0076252371072769165,0.00758770015090704,0.007597775664180517,0.007621972821652889,0.0076278517954051495,0.007619458716362715,0.0076209912076592445,0.007612959016114473,0.007638107053935528,0.007606209255754948,0.007626851089298725,0.0076391687616705894,0.007590465247631073,0.00760751124471426,0.007625799160450697,0.007594651076942682,0.009424488991498947],"repeats":32,"seconds_per_sample":0.009988354810047895,"standard_deviation_convention":"population","standard_deviation_seconds":0.0049249346067599935,"warmups":5}}

results/conditional/timing/lm1b/langflow/steps_2/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.langflow:v1|dlb.adapters.langflow:LangFlowAdapter|7d80794cf6a132fc8d4c179d723e8e486c70476888b2ba51dd92cdc3f5284f84","attempt_id":"667041a450f84d6781d1564b2a099f57","checkpoint_lock_id":"langflow_lm1b_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all","checkpoint_selection":{"path":null,"resource":"langflow_lm1b_hf","teacher_family":null},"checkpoint_sha256":"2050f55b63e952941eb190711828176529f2f510031cc0e0fc1cd39f6998d7bb","checkpoint_teacher_family":"continuous_langflow","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-langflow","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"langflow","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"langflow:pinned_internal_bf16_autocast_with_fp32_sensitive_ops","precision_policy_binding":{"adapter_identity":"dlb.adapters.langflow:v1|dlb.adapters.langflow:LangFlowAdapter|7d80794cf6a132fc8d4c179d723e8e486c70476888b2ba51dd92cdc3f5284f84","checkpoint_lock_id":"langflow_lm1b_hf:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:all","checkpoint_repository":"Continuous-Rivals-Discrete/langflow-lm1b","checkpoint_resource":"langflow_lm1b_hf","checkpoint_revision":"d5d395d9bdf01f697ba8830110e3e78c63d77cb5","checkpoint_selection":{"path":null,"resource":"langflow_lm1b_hf","teacher_family":null},"checkpoint_sha256":"2050f55b63e952941eb190711828176529f2f510031cc0e0fc1cd39f6998d7bb","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"langflow","prefix_length":64,"prompt_count":1024,"source_commit":"a712b08570c56c787d6ef24f8e906a2dcdf768f5","upstream":"langflow"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"daaf59774f53f83ef7987b1432cd14769226e701310308480bbac28ef5217256","symbol":"_capture_langflow.<locals>.generate.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"DDiTFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"GumbelProposal"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlow"},{"kind":"sampling_closure","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlow.generate_samples"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LangFlowBackbone"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/langflow/langflow/model.py","sha256":"f3d65b468b61bd8ce46fa94764f1248d3f06e79064b6cfcb97c377be9933c8c3","symbol":"TimestepEmbedder"},{"kind":"bound_method_owner","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/autograd/grad_mode.py","sha256":"de61684a7f73eb361c0f47c22159a2bfc67df6f78ddfa5f198049b0654f1f4a7","symbol":"no_grad"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"},{"kind":"sampling_closure","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/utils/_contextlib.py","sha256":"cf7aa5b08f44974ba8c1d08cd71ef70ffd13d1c48a4931576eb235306bfa46b5","symbol":"LangFlow.generate_samples"},{"kind":"sampling_closure","path":"/opt/anaconda3/envs/dlb-langflow/lib/python3.12/site-packages/torch/utils/_contextlib.py","sha256":"cf7aa5b08f44974ba8c1d08cd71ef70ffd13d1c48a4931576eb235306bfa46b5","symbol":"_DecoratorContextManager.clone"}],"sample_count":2048,"seed":42,"source_commit":"a712b08570c56c787d6ef24f8e906a2dcdf768f5","steps":2,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.013044256484135985,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.03309805691242218,0.032926072366535664,0.03313788678497076,0.033139850944280624,0.015680735930800438,0.013000874780118465,0.012740885838866234,0.012779285199940205,0.01307451818138361,0.013031842187047005,0.01303914375603199,0.012773535680025816,0.012979402206838131,0.013004540000110865,0.01270885718986392,0.012760655954480171,0.012742158025503159,0.012989277020096779,0.01275152200832963,0.012770209927111864,0.01269852090626955,0.013078314252197742,0.013003408908843994,0.01304936921223998,0.020285880658775568,0.03321685781702399,0.033245372120290995,0.030609849374741316,0.026979338843375444,0.0331205609254539,0.03309041541069746,0.03322883741930127],"repeats":32,"seconds_per_sample":0.019898001148249023,"standard_deviation_convention":"population","standard_deviation_seconds":0.009172227812054594,"warmups":5}}

results/conditional/timing/lm1b/mdlm/steps_1/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.mdlm:v1|dlb.adapters.mdlm:MDLMAdapter|56a619efd4f7abcf587f94757a334f9d4481f7f76f19d56bc7a45b930aa9ebd3","attempt_id":"69c53507e094456190c5505a81ab4530","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_MDLM_.ckpt","checkpoint_selection":{"path":"lm1b_MDLM_.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"masked_mdlm"},"checkpoint_sha256":"51910b37ddabb1c716654d934526f483abb04837ddf9ca76c29ad41632807f01","checkpoint_teacher_family":"masked_mdlm","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-mdlm","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"mdlm","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"mdlm:pinned_internal_bf16_autocast_with_fp32_sensitive_ops","precision_policy_binding":{"adapter_identity":"dlb.adapters.mdlm:v1|dlb.adapters.mdlm:MDLMAdapter|56a619efd4f7abcf587f94757a334f9d4481f7f76f19d56bc7a45b930aa9ebd3","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_MDLM_.ckpt","checkpoint_repository":null,"checkpoint_resource":"flm_lm1b_reproductions","checkpoint_revision":null,"checkpoint_selection":{"path":"lm1b_MDLM_.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"masked_mdlm"},"checkpoint_sha256":"51910b37ddabb1c716654d934526f483abb04837ddf9ca76c29ad41632807f01","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"mdlm","prefix_length":64,"prompt_count":1024,"source_commit":"c112c526d193436838c98d81455ee51f90309470","upstream":"mdlm"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"92c69119e85195e099a41668e8f75dd5a908c4e341fb220f714853db555dca0d","symbol":"_capture_teacher.<locals>.capture.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"BPD"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"Diffusion"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"NLL"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"Perplexity"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DDitFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/noise_schedule.py","sha256":"da16320cebbc727c03e8e45297593505fb41a597c7a5ef29eacd7043207a763c","symbol":"LogLinearNoise"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"Softplus"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torchmetrics/collections.py","sha256":"b305613023a66917d5e79639e92ad9c3510dccc043ea5417616fc48612f3db88","symbol":"MetricCollection"}],"sample_count":2048,"seed":42,"source_commit":"c112c526d193436838c98d81455ee51f90309470","steps":1,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.019038962898775935,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.019001980777829885,0.019035592675209045,0.019064967054873705,0.0190742420963943,0.019311956129968166,0.019342373125255108,0.01927006198093295,0.019451941829174757,0.019147424027323723,0.019077126868069172,0.01904738089069724,0.019045136868953705,0.019100111909210682,0.019278876017779112,0.019304526038467884,0.019225834868848324,0.01898276200518012,0.018881106283515692,0.018943871837109327,0.018935158383101225,0.018977513071149588,0.018916981294751167,0.018914907705038786,0.019022813066840172,0.018870199099183083,0.018935618922114372,0.018918583169579506,0.018955409061163664,0.018937732093036175,0.019042333122342825,0.01906721107661724,0.01892730686813593],"repeats":32,"seconds_per_sample":0.019062782506807707,"standard_deviation_convention":"population","standard_deviation_seconds":0.00015022024421004648,"warmups":5}}

results/conditional/timing/lm1b/mdlm/steps_4/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.mdlm:v1|dlb.adapters.mdlm:MDLMAdapter|56a619efd4f7abcf587f94757a334f9d4481f7f76f19d56bc7a45b930aa9ebd3","attempt_id":"188a0c818f644c7d9f229897d9f03fa9","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_MDLM_.ckpt","checkpoint_selection":{"path":"lm1b_MDLM_.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"masked_mdlm"},"checkpoint_sha256":"51910b37ddabb1c716654d934526f483abb04837ddf9ca76c29ad41632807f01","checkpoint_teacher_family":"masked_mdlm","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-mdlm","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"mdlm","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"mdlm:pinned_internal_bf16_autocast_with_fp32_sensitive_ops","precision_policy_binding":{"adapter_identity":"dlb.adapters.mdlm:v1|dlb.adapters.mdlm:MDLMAdapter|56a619efd4f7abcf587f94757a334f9d4481f7f76f19d56bc7a45b930aa9ebd3","checkpoint_lock_id":"flm_lm1b_reproductions:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152:lm1b_MDLM_.ckpt","checkpoint_repository":null,"checkpoint_resource":"flm_lm1b_reproductions","checkpoint_revision":null,"checkpoint_selection":{"path":"lm1b_MDLM_.ckpt","resource":"flm_lm1b_reproductions","teacher_family":"masked_mdlm"},"checkpoint_sha256":"51910b37ddabb1c716654d934526f483abb04837ddf9ca76c29ad41632807f01","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"mdlm","prefix_length":64,"prompt_count":1024,"source_commit":"c112c526d193436838c98d81455ee51f90309470","upstream":"mdlm"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/capture.py","sha256":"92c69119e85195e099a41668e8f75dd5a908c4e341fb220f714853db555dca0d","symbol":"_capture_teacher.<locals>.capture.<locals>.<lambda>"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"BPD"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"Diffusion"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"NLL"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/diffusion.py","sha256":"f8b4f1fdea2ffc960c9852583bee58865aace44baf958f83d5f9b8e352752121","symbol":"Perplexity"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DDitFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"DIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/models/dit.py","sha256":"58751fa6fe0ee555debc86b91995754bf34e818c076efbe0f0a102bee6899fa4","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/mdlm/noise_schedule.py","sha256":"da16320cebbc727c03e8e45297593505fb41a597c7a5ef29eacd7043207a763c","symbol":"LogLinearNoise"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"Softplus"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-mdlm/lib/python3.9/site-packages/torchmetrics/collections.py","sha256":"b305613023a66917d5e79639e92ad9c3510dccc043ea5417616fc48612f3db88","symbol":"MetricCollection"}],"sample_count":2048,"seed":42,"source_commit":"c112c526d193436838c98d81455ee51f90309470","steps":4,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.03478888561949134,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.03513758536428213,0.03471161238849163,0.034986973740160465,0.035053555853664875,0.03470578184351325,0.03503089118748903,0.03486615885049105,0.03510821983218193,0.03523005731403828,0.03515283903107047,0.035110152792185545,0.03508374281227589,0.03500880813226104,0.03775930730625987,0.029046451207250357,0.029012639075517654,0.029064638074487448,0.029064428061246872,0.028911993838846684,0.028944604098796844,0.028966827783733606,0.029003043193370104,0.028995792847126722,0.02888000523671508,0.02898863237351179,0.0288879182189703,0.02904839301481843,0.03288176376372576,0.03508053673431277,0.03506977064535022,0.03497004834935069,0.03501617908477783],"repeats":32,"seconds_per_sample":0.03258685475157108,"standard_deviation_convention":"population","standard_deviation_seconds":0.0030431992789923096,"warmups":5}}

results/conditional/timing/lm1b/mdlm_di4c/steps_16/timing.json

@@ -0,0 +1 @@
+{"metadata":{"adapter_identity":"dlb.adapters.di4c:v3|dlb.adapters.di4c:Di4CAdapter|e1636b46628a6015944553de2d5d8a1b829bb6bf3f9c6b789d829cb6db222ec9","attempt_id":"fc0c87d0397e451fa563e460ff00141d","checkpoint_lock_id":"recipe:mdlm_di4c_lm1b:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/reference_reproduction/mdlm_di4c/lm1b","recipe":"mdlm_di4c_lm1b","sampling_checkpoint":"student_checkpoints/20000.ckpt","sampling_config":"config.yaml","source":"di4c","source_commit":"ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3","teacher_adapter":"masked_to_absorbing","teacher_family":"masked_mdlm"},"checkpoint_sha256":"569faa9af7f866644c622bf3360a246d2ed3f3ddd7572999bdaa7c8a8d3d7f44","checkpoint_teacher_family":"masked_mdlm","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","cuda_runtime_version":"12.4","dataset":"lm1b","diversity_prompt_count":256,"environment":"dlb-di4c","evaluation_continuation_length":64,"generation_mode":"conditional_prefix","gpu_compute_capability":"9.0","gpu_index":0,"gpu_name":"NVIDIA H200 NVL","model":"mdlm_di4c","nvidia_driver_version":"575.57.08","parameter_precision":"fp32","precision":"bf16-mixed","precision_evidence":"static_policy_bound_to_checkpoint_and_runtime_code_not_runtime_autocast_observation","precision_policy":"di4c:pinned_bf16_mixed_config_and_internal_autocast","precision_policy_binding":{"adapter_identity":"dlb.adapters.di4c:v3|dlb.adapters.di4c:Di4CAdapter|e1636b46628a6015944553de2d5d8a1b829bb6bf3f9c6b789d829cb6db222ec9","checkpoint_lock_id":"recipe:mdlm_di4c_lm1b:ce8ff6a327d9f1470fef9a7ac66813ac94d4cc50268bc2998627498dc6f3a152","checkpoint_selection":{"output":"checkpoints/reference_reproduction/mdlm_di4c/lm1b","recipe":"mdlm_di4c_lm1b","sampling_checkpoint":"student_checkpoints/20000.ckpt","sampling_config":"config.yaml","source":"di4c","source_commit":"ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3","teacher_adapter":"masked_to_absorbing","teacher_family":"masked_mdlm"},"checkpoint_sha256":"569faa9af7f866644c622bf3360a246d2ed3f3ddd7572999bdaa7c8a8d3d7f44","completion_schedule":"c0:p0-1023;c1-4:p0-255","completions_per_diversity_prompt":5,"conditioning_config_sha256":"89d70134c4fb1ce21c18e46cfb9753f6cdcd460b879d6e60e99b2c7b56ae5114","conditioning_manifest":"/home/ruizzhao/diffusion_baseline/data/manifests/conditional-lm1b-c64.json","conditioning_manifest_sha256":"0e5c5bf5b8a1445b348a65045f680ac31713dbfa1e690bedb73e0d1e7472a039","dataset":"lm1b","diversity_prompt_count":256,"evaluation_continuation_length":64,"experiment_config_sha256":"6ce963030186c79cab2bc15670737a77ebfad08a0608bc87ee5c54e489ce678f","generation_mode":"conditional_prefix","model":"mdlm_di4c","prefix_length":64,"prompt_count":1024,"source_commit":"ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3","upstream":"di4c"},"prefix_length":64,"prompt_count":1024,"pytorch_compiled_cuda_toolkit":"12.4","requested_precision":"author","runtime_code_binding":[{"kind":"sampling_callable","path":"/home/ruizzhao/diffusion_baseline/adapters/_distilled_runtime.py","sha256":"39268b31d066526a2f0e9bb13bb531f98aaedb1aed46b4f4f0fd722cf02a5683","symbol":"benchmark_model.<locals>.<lambda>"},{"kind":"sampling_closure","path":"/home/ruizzhao/diffusion_baseline/src/dlb/adapters/conditional_runtime.py","sha256":"eef6d1a1062787f949be4656b7a1cc2e81718b42affd9facce4159e2c02c89b8","symbol":"token_project_fn.<locals>.project"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/core/diffusion/noise_schedule.py","sha256":"b2c06e11a4947b306db1c19eac181b99bbc6b36520526ebb36639696c95dcb21","symbol":"LogLinearNoise"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/core/distill/multi_round_sdtt.py","sha256":"75c61fbd1d3e1b87e10b25d67c0143ba0995753f579e0282bc1e3559c145704c","symbol":"MultiRoundSDTT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"DDiTBlock"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"DDitFinalLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"DIT"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"EmbeddingLayer"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"LayerNorm"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"Rotary"},{"kind":"loaded_module","path":"/home/ruizzhao/diffusion_baseline/upstreams/di4c/sdtt/src/sdtt/models/dit_orig.py","sha256":"c2e7ced4ccbb92bcf9e6fdc0fc2b41130447d8d15e5d696e7f3a927177578912","symbol":"TimestepEmbedder"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"GELU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/activation.py","sha256":"1e748e0df0d70491fe82f0178f32126ee6ee138a53de7fc76463af10c75f5477","symbol":"SiLU"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"ModuleList"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/container.py","sha256":"33af8d1fe4fb542cbba5542c5079b1bd8ef8a6493abbef329411f71b2071d24f","symbol":"Sequential"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/dropout.py","sha256":"1fc9974c3cf2078dd4bc257f73afb6a6714adfec2e9d099d47cf923e766d3490","symbol":"Dropout"},{"kind":"loaded_module","path":"/opt/anaconda3/envs/dlb-di4c/lib/python3.10/site-packages/torch/nn/modules/linear.py","sha256":"d7f5cadf773deb5ab4bcc9f57414e8239b2859c70d2aa6f141576de1ec918823","symbol":"Linear"}],"sample_count":2048,"seed":42,"source_commit":"ac61ff9fe8e85120f9e1d2a8c5a332f8b8353dd3","steps":16,"synchronization_policy":"torch.cuda.synchronize_before_start_and_after_generate"},"schema":"dlb-generation-timing-v1","timing":{"batch_size":1,"exclusions":["model_and_checkpoint_loading","first_compilation","token_decoding","metrics","file_io"],"median_seconds":0.11406264989636838,"mode":"primary_latency","num_timed_samples":32,"raw_durations_seconds":[0.11690304288640618,0.11475256411358714,0.11472673388198018,0.11449317587539554,0.1145416391082108,0.11451897490769625,0.11415167059749365,0.11425567092373967,0.11376026086509228,0.1139446240849793,0.1141616371460259,0.11403871886432171,0.11407560482621193,0.11404055217280984,0.11426084907725453,0.11404920415952802,0.11427654279395938,0.11404969496652484,0.11399703472852707,0.1139108119532466,0.11377245979383588,0.11380113381892443,0.11475700093433261,0.11429071519523859,0.11390608502551913,0.11386723490431905,0.11421745223924518,0.11399389989674091,0.11393415834754705,0.11386060481891036,0.1140807131305337,0.11396091897040606],"repeats":32,"seconds_per_sample":0.11422973078151699,"standard_deviation_convention":"population","standard_deviation_seconds":0.0005543853719092198,"warmups":5}}