Blame for data/problems/bleu-1-score.json

{
  "slug": "bleu-1-score",
  "title": "手撕机器翻译 BLEU-1 评测分值 (BLEU-1 Score)",
  "difficulty": "Medium",
  "category": "ML",
  "tags": [
    "自然语言处理",
    "BLEU",
    "文本评估"
  ],
  "description_md": "### 题目描述\n给定候选生成 Token ID 列表 $cand \\in \\mathbb{Z}^C$ 与参考 Token ID 列表 $ref \\in \\mathbb{Z}^R$，计算截断 1-gram 精度 $\\displaystyle p_1 = \\frac{\\sum_{w} \\min(c_{cand}(w), c_{ref}(w))}{C}$，并结合简短惩罚因子 $\\text{BP} = \\min(1, \\exp(1 - R/C))$，输出 $\\text{BLEU-1} = \\text{BP} \\times p_1$。\n\n### 函数签名\n```python\ndef bleu1_score(\n    cand_tokens: List[int],\n    ref_tokens: List[int],\n) -> float:\n```\n\n### 示例\n```python\n# 输入\ncand_tokens = [1, 2, 3, 4]\nref_tokens = [1, 2, 3, 4]\n\n# 输出 (float)\n1.0\n```\n\n### 参数规格与约束\n- **`cand_tokens`**：`List[int]`\n- **`ref_tokens`**：`List[int]`\n- **返回值**：`float`，输出规格 `float` (示例值: `1.0`)\n- **精度容差**：`atol=0.0001, rtol=0.0001`",
  "entry_point": "bleu1_score",
  "starter_code": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\nfrom typing import Optional, List, Tuple\n\ndef bleu1_score(\n    cand_tokens: List[int],\n    ref_tokens: List[int],\n) -> float:\n    \"\"\"\n    手撕机器翻译 BLEU-1 评测分值 (BLEU-1 Score)\n    \"\"\"\n    # 请在此编写代码\n    pass\n",
  "reference_code": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\nfrom typing import Optional, List, Tuple\n\ndef bleu1_score(cand_tokens: List[int], ref_tokens: List[int]) -> float:\n    if len(cand_tokens) == 0:\n        return 0.0\n    from collections import Counter\n    cand_counts = Counter(cand_tokens)\n    ref_counts = Counter(ref_tokens)\n    clipped_hits = sum(min(count, ref_counts.get(w, 0)) for w, count in cand_counts.items())\n    p1 = clipped_hits / len(cand_tokens)\n    bp = 1.0 if len(cand_tokens) >= len(ref_tokens) else math.exp(1.0 - len(ref_tokens) / len(cand_tokens))\n    return float(bp * p1)\n",
  "forbidden_modules": [
    "os",
    "sys",
    "subprocess",
    "shutil",
    "socket",
    "requests"
  ],
  "forbidden_calls": [],
  "time_limit": 3.0,
  "memory_limit_mb": 512,
  "atol": 0.0001,
  "rtol": 0.0001,
  "test_code": "import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\n\ncases = [\n    ('完全匹配 BLEU-1=1.0', [1, 2, 3, 4], [1, 2, 3, 4]),\n    ('过短惩罚 BLEU-1', [1, 2], [1, 2, 3, 4]),\n    ('多维批次与参数泛化验证', [1, 2, 3, 4, 4], [1, 2, 3, 4, 4]),\n    ('边界极值与稳定性验证', [2, 1], [4, 3, 2, 1]),\n]\nfor idx, (name, *args) in enumerate(cases, 1):\n    t0 = time.time()\n    try:\n        exp = ref_obj(*args)\n        act = user_obj(*args)\n        match, err, max_diff = compare_outputs(act, exp, atol=atol, rtol=rtol)\n        passed, msg = match, err\n    except Exception as e:\n        passed, msg = False, f\"Runtime Exception: {str(e)}\"\n        act, exp, max_diff = None, None, None\n    cases_results.append({\n        \"case_id\": idx, \"case_name\": name, \"passed\": passed,\n        \"time_ms\": round((time.time() - t0) * 1000, 2), \"message\": msg,\n        \"user_output_summary\": summarize_obj(act) if act is not None else None,\n        \"expected_summary\": summarize_obj(exp) if exp is not None else None,\n        \"max_diff\": max_diff,\n    })\n",
  "is_featured": false,
  "editorial_md": "#### 💡 核心算法原理与数学推导\n**手撕机器翻译 BLEU-1 评测分值 (BLEU-1 Score)**（`bleu1_score`）聚焦于 **ML** 中的关键计算逻辑：\n\n给定候选生成 Token ID 列表 $cand \\in \\mathbb{Z}^C$ 与参考 Token ID 列表 $ref \\in \\mathbb{Z}^R$，计算截断 1-gram 精度 $\\displaystyle p_1 = \\frac{\\sum_{w} \\min(c_{cand}(w), c_{ref}(w))}{C}$，并结合简短惩罚因子 $\\text{BP} = \\min(1, \\exp(1 - R/C))$，输出 $\\text{BLEU-1} = \\text{BP} \\times p_1$。\n\n在实现中，函数接收参数 `cand_tokens` (`List[int]`)、`ref_tokens` (`List[int]`)，目标是计算出类型与形状严格符合 `float` 契约的结果。\n\n#### 📐 输入输出张量变换与步骤拆解\n- **输入基准规格**：`cand_tokens` 规格为 `List` (长度 `4`)，`ref_tokens` 规格为 `List` (长度 `4`)。\n- **期望输出规格**：`float` (示例值: `1.0`)。\n\n**具体推导与计算步骤如下：**\n1. 条件分支判断 `if len(cand_tokens) == 0`：处理边界条件、掩码过滤或特殊退化情形。\n2. 计算中间量 `cand_counts`：通过表达式 `Counter(cand_tokens)` 完成张量或数值变换。\n3. 计算中间量 `ref_counts`：通过表达式 `Counter(ref_tokens)` 完成张量或数值变换。\n4. 计算中间量 `clipped_hits`：通过表达式 `sum(min(count, ref_counts.get(w, 0)) for w, count in cand_counts.items())` 完成张量或数值变换。\n5. 计算中间量 `p1`：通过表达式 `clipped_hits / len(cand_tokens)` 完成张量或数值变换。\n6. 计算中间量 `bp`：通过表达式 `1.0 if len(cand_tokens) >= len(ref_tokens) else math.exp(1.0 - len(ref_tokens) / len(cand_tokens))` 完成张量或数值变换。\n7. 返回最终结果：输出 `float(bp * p1)`，严格对齐函数签名要求的返回类型与维度。\n\n#### 📊 复杂度分析\n- **时间复杂度**：$O(1)$，仅涉及标量代数运算或闭式公式求值，执行耗时与模型规模参数数值大小无关。\n- **空间复杂度**：$O(1)$，仅使用常数级标量局部变量，无额外张量显存分配。\n\n#### ⚠️ 关键实现细节与易错点\n- **除零保护与数值平滑 (`eps`)**：在执行分母归一化、标准差缩放或对数运算时，必须按公式位置准确加上 `eps`（例如在平方根内 `sqrt(var + eps)` 还是分母外 `+ eps`），防止全零或常量输入触发 `NaN` / `Inf`。\n- **指数与对数域数值稳定性**：直接对大幅值 logits 计算 `exp` 易导致浮点数上溢（Overflow），对接近 0 的概率直接计算 `log` 易导致下溢；应优先使用 `F.log_softmax`、`F.logsigmoid` 或减去最大值的稳定化写法。\n- **纯向量化算子优先**：尽量使用 PyTorch 原生向量化张量算子替代低效的 Python 逐元素循环，以兼顾代码简洁性与执行性能。\n\n#### 💻 官方标准参考实现\n```python\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport math\nfrom typing import Optional, List, Tuple\n\ndef bleu1_score(cand_tokens: List[int], ref_tokens: List[int]) -> float:\n    if len(cand_tokens) == 0:\n        return 0.0\n    from collections import Counter\n    cand_counts = Counter(cand_tokens)\n    ref_counts = Counter(ref_tokens)\n    clipped_hits = sum(min(count, ref_counts.get(w, 0)) for w, count in cand_counts.items())\n    p1 = clipped_hits / len(cand_tokens)\n    bp = 1.0 if len(cand_tokens) >= len(ref_tokens) else math.exp(1.0 - len(ref_tokens) / len(cand_tokens))\n    return float(bp * p1)\n```",
  "difficulty_breakdown": {
    "concept": 3,
    "implementation": 3,
    "math_derivation": 3,
    "boundary_complexity": 3
  },
  "test_cases": [
    {
      "name": "完全匹配 BLEU-1=1.0",
      "input_desc": "真实参数契约用例 (cand_tokens, ref_tokens)",
      "params": [
        {
          "name": "cand_tokens",
          "type": "List[int]",
          "value": "[1, 2, 3, 4]"
        },
        {
          "name": "ref_tokens",
          "type": "List[int]",
          "value": "[1, 2, 3, 4]"
        }
      ],
      "raw_inputs": {
        "cand_tokens": "[1, 2, 3, 4]",
        "ref_tokens": "[1, 2, 3, 4]"
      }
    },
    {
      "name": "过短惩罚 BLEU-1",
      "input_desc": "真实参数契约用例 (cand_tokens, ref_tokens)",
      "params": [
        {
          "name": "cand_tokens",
          "type": "List[int]",
          "value": "[1, 2]"
        },
        {
          "name": "ref_tokens",
          "type": "List[int]",
          "value": "[1, 2, 3, 4]"
        }
      ],
      "raw_inputs": {
        "cand_tokens": "[1, 2]",
        "ref_tokens": "[1, 2, 3, 4]"
      }
    },
    {
      "name": "多维批次与参数泛化验证",
      "input_desc": "真实参数契约用例 (cand_tokens, ref_tokens)",
      "params": [
        {
          "name": "cand_tokens",
          "type": "List[int]",
          "value": "[1, 2, 3, 4, 4]"
        },
        {
          "name": "ref_tokens",
          "type": "List[int]",
          "value": "[1, 2, 3, 4, 4]"
        }
      ],
      "raw_inputs": {
        "cand_tokens": "[1, 2, 3, 4, 4]",
        "ref_tokens": "[1, 2, 3, 4, 4]"
      }
    }
  ]
}