Spaces:
Running on Zero
Running on Zero
File size: 8,571 Bytes
5f7afef | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | {
"slug": "binary-cross-entropy-loss",
"title": "手撕二元交叉熵损失函数 (Binary Cross Entropy Loss)",
"difficulty": "Easy",
"category": "ML",
"tags": [
"常见手撕",
"BCE",
"逻辑回归",
"损失函数",
"基础算法"
],
"description_md": "### 题目描述\n二元交叉熵 (Binary Cross Entropy) 用于评估二分类模型的输出概率与真实标签之间的距离：\n$$\\text{BCE}(p, y) = -\\frac{1}{N} \\sum_{i=1}^N \\Big( y_i \\log(p_i + \\epsilon) + (1 - y_i) \\log(1 - p_i + \\epsilon) \\Big)$$\n\n### 函数签名\n```python\ndef bce_loss(\n probs: torch.Tensor,\n targets: torch.Tensor,\n eps: float = 1e-12,\n) -> torch.Tensor:\n```\n\n### 示例\n```python\n# 输入\nprobs = [0.9, 0.1, 0.8, 0.2]\ntargets = [1.0, 0.0, 1.0, 0.0]\neps = 1e-12\n\n# 输出 (Tensor, shape=[], dtype=float32)\n0.1643\n```\n\n### 参数规格与约束\n- **`probs`**：`torch.Tensor`，测试规格 `shape=[4], dtype=float32`\n- **`targets`**：`torch.Tensor`，测试规格 `shape=[4], dtype=float32`\n- **`eps`**：`float`（默认 `1e-12`）\n- **返回值**：`torch.Tensor`，输出规格 `shape=[], dtype=float32`\n- **精度容差**：`atol=0.0001, rtol=0.0001`",
"entry_point": "bce_loss",
"starter_code": "import torch\n\ndef bce_loss(\n probs: torch.Tensor,\n targets: torch.Tensor,\n eps: float = 1e-12,\n) -> torch.Tensor:\n \"\"\"\n 计算二元交叉熵损失。\n \"\"\"\n # 请在此编写代码\n pass\n",
"reference_code": "import torch\n\ndef bce_loss(probs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-12) -> torch.Tensor:\n p = torch.clamp(probs, eps, 1.0 - eps)\n loss = -(targets * torch.log(p) + (1.0 - targets) * torch.log(1.0 - p))\n return loss.mean()\n",
"forbidden_modules": [
"os",
"sys",
"subprocess",
"shutil",
"socket",
"requests"
],
"forbidden_calls": [],
"time_limit": 3.0,
"memory_limit_mb": 512,
"atol": 0.0001,
"rtol": 0.0001,
"test_code": "import torch\ncases = [\n (\"标准二分类测试\", torch.tensor([0.9, 0.1, 0.8, 0.2]), torch.tensor([1.0, 0.0, 1.0, 0.0])),\n (\"极端边界概率防 log(0) 测试\", torch.tensor([0.9999, 0.0001]), torch.tensor([1.0, 0.0])),\n (\"高不确定性均匀概率测试\", torch.tensor([0.5, 0.5, 0.5, 0.5]), torch.tensor([1.0, 0.0, 1.0, 1.0])),\n (\"高损耗反向预测测试\", torch.tensor([0.15, 0.85, 0.25, 0.75]), torch.tensor([1.0, 0.0, 1.0, 0.0])),\n]\nfor idx, (name, p, y) in enumerate(cases, 1):\n t0 = time.time()\n try:\n exp = ref_obj(p, y)\n act = user_obj(p, y)\n match, err, max_diff = compare_outputs(act, exp, atol=atol, rtol=rtol)\n passed = match\n msg = err\n except Exception as e:\n passed = False\n msg = f\"Runtime Exception: {str(e)}\"\n act, exp, max_diff = None, None, None\n cases_results.append({\n \"case_id\": idx, \"case_name\": name, \"passed\": passed,\n \"time_ms\": round((time.time() - t0) * 1000, 2), \"message\": msg,\n \"user_output_summary\": summarize_obj(act) if act is not None else None,\n \"expected_summary\": summarize_obj(exp) if exp is not None else None,\n \"max_diff\": max_diff,\n })\n",
"is_featured": true,
"editorial_md": "#### 💡 核心算法原理与数学推导\n**手撕二元交叉熵损失函数 (Binary Cross Entropy Loss)**（`bce_loss`）聚焦于 **ML** 中的关键计算逻辑：\n\n二元交叉熵 (Binary Cross Entropy) 用于评估二分类模型的输出概率与真实标签之间的距离：\n$$\\text{BCE}(p, y) = -\\frac{1}{N} \\sum_{i=1}^N \\Big( y_i \\log(p_i + \\epsilon) + (1 - y_i) \\log(1 - p_i + \\epsilon) \\Big)$$\n\n在实现中，函数接收参数 `probs` (`torch.Tensor`)、`targets` (`torch.Tensor`)、`eps` (`float`)，目标是计算出类型与形状严格符合 `torch.Tensor` 契约的结果。\n\n#### 📐 输入输出张量变换与步骤拆解\n- **输入基准规格**：`probs` 规格为 `torch.Tensor(shape=[4], dtype=float32)`，`targets` 规格为 `torch.Tensor(shape=[4], dtype=float32)`。\n- **期望输出规格**：`torch.Tensor(shape=[], dtype=float32)`。\n\n**具体推导与计算步骤如下：**\n1. 计算中间量 `p`：通过表达式 `torch.clamp(probs, eps, 1.0 - eps)` 完成张量或数值变换。\n2. 计算中间量 `loss`：通过表达式 `-(targets * torch.log(p) + (1.0 - targets) * torch.log(1.0 - p))` 完成张量或数值变换。\n3. 返回最终结果：输出 `loss.mean()`，严格对齐函数签名要求的返回类型与维度。\n\n#### 📊 复杂度分析\n- **时间复杂度**：$O(N)$（线性于输入张量总元素个数 $\\prod d_i$），所有运算均为向量化逐元素算子或沿指定维度的归约计算。\n- **空间复杂度**：$O(N)$，需要与输入或输出张量同规模的显存空间保存计算结果。\n\n#### ⚠️ 关键实现细节与易错点\n- **除零保护与数值平滑 (`eps`)**：在执行分母归一化、标准差缩放或对数运算时，必须按公式位置准确加上 `eps`（例如在平方根内 `sqrt(var + eps)` 还是分母外 `+ eps`），防止全零或常量输入触发 `NaN` / `Inf`。\n- **归约维度 (`dim`) 与广播对齐 (`keepdim`)**：调用 `sum`、`mean`、`max`、`norm` 或 `softmax` 时需明确指定目標维度（通常为最后一维 `dim=-1` 或特征维 `dim=0`），并在需要与原张量逐元素相减/相除时开启 `keepdim=True`。\n- **指数与对数域数值稳定性**：直接对大幅值 logits 计算 `exp` 易导致浮点数上溢（Overflow），对接近 0 的概率直接计算 `log` 易导致下溢；应优先使用 `F.log_softmax`、`F.logsigmoid` 或减去最大值的稳定化写法。\n\n#### 💻 官方标准参考实现\n```python\nimport torch\n\ndef bce_loss(probs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-12) -> torch.Tensor:\n p = torch.clamp(probs, eps, 1.0 - eps)\n loss = -(targets * torch.log(p) + (1.0 - targets) * torch.log(1.0 - p))\n return loss.mean()\n```",
"difficulty_breakdown": {
"concept": 2,
"implementation": 2,
"math_derivation": 2,
"boundary_complexity": 2
},
"test_cases": [
{
"name": "标准二分类测试",
"input_desc": "真实参数契约用例 (probs, targets, eps)",
"params": [
{
"name": "probs",
"type": "torch.Tensor",
"value": "torch.tensor([0.9, 0.1, 0.8, 0.2], dtype=torch.float32)"
},
{
"name": "targets",
"type": "torch.Tensor",
"value": "torch.tensor([1.0, 0.0, 1.0, 0.0], dtype=torch.float32)"
},
{
"name": "eps",
"type": "float",
"value": "1e-12"
}
],
"raw_inputs": {
"probs": "torch.tensor([0.9, 0.1, 0.8, 0.2], dtype=torch.float32)",
"targets": "torch.tensor([1.0, 0.0, 1.0, 0.0], dtype=torch.float32)",
"eps": "1e-12"
}
},
{
"name": "极端边界概率防 log(0) 测试",
"input_desc": "真实参数契约用例 (probs, targets, eps)",
"params": [
{
"name": "probs",
"type": "torch.Tensor",
"value": "torch.tensor([0.9999, 0.0001], dtype=torch.float32)"
},
{
"name": "targets",
"type": "torch.Tensor",
"value": "torch.tensor([1.0, 0.0], dtype=torch.float32)"
},
{
"name": "eps",
"type": "float",
"value": "1e-12"
}
],
"raw_inputs": {
"probs": "torch.tensor([0.9999, 0.0001], dtype=torch.float32)",
"targets": "torch.tensor([1.0, 0.0], dtype=torch.float32)",
"eps": "1e-12"
}
},
{
"name": "高不确定性均匀概率测试",
"input_desc": "真实参数契约用例 (probs, targets, eps)",
"params": [
{
"name": "probs",
"type": "torch.Tensor",
"value": "torch.tensor([0.5, 0.5, 0.5, 0.5], dtype=torch.float32)"
},
{
"name": "targets",
"type": "torch.Tensor",
"value": "torch.tensor([1.0, 0.0, 1.0, 1.0], dtype=torch.float32)"
},
{
"name": "eps",
"type": "float",
"value": "1e-12"
}
],
"raw_inputs": {
"probs": "torch.tensor([0.5, 0.5, 0.5, 0.5], dtype=torch.float32)",
"targets": "torch.tensor([1.0, 0.0, 1.0, 1.0], dtype=torch.float32)",
"eps": "1e-12"
}
}
]
} |