laaaarrywang commited on
Commit
476e404
·
verified ·
1 Parent(s): dd17dc4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -30,7 +30,7 @@ The paper reports experiments at **GPT-2** scale on **LM1B** and **OpenWebText**
30
 
31
  ## Method Summary
32
 
33
- For a clean token $x$, SCDD uses the following marginal forward distribution:
34
 
35
  $$
36
  q(z_t \mid x) = \operatorname{Cat}\left(z_t;, \gamma_t\left(\rho_t x + (1-\rho_t)u\right) + (1-\gamma_t)m\right).
@@ -38,10 +38,10 @@ $$
38
 
39
  where:
40
 
41
- * `m` is the `[MASK]` token.
42
- * $`u`$ is the uniform distribution over non-`[MASK]` tokens.
43
- * $`\gamma_t`$ is the marginal probability that $z_t$ is not `[MASK]`.
44
- * $`\rho_t`$ is the marginal probability that $z_t$ retains the clean token among the non-`[MASK]` mass.
45
 
46
 
47
 
 
30
 
31
  ## Method Summary
32
 
33
+ For a clean token \\(x\\), SCDD uses the following marginal forward distribution:
34
 
35
  $$
36
  q(z_t \mid x) = \operatorname{Cat}\left(z_t;, \gamma_t\left(\rho_t x + (1-\rho_t)u\right) + (1-\gamma_t)m\right).
 
38
 
39
  where:
40
 
41
+ * \\(m\\) is the `[MASK]` token.
42
+ * \\(u\\) is the uniform distribution over non-`[MASK]` tokens.
43
+ * \\(\gamma_t\\) is the marginal probability that \\(z_t\\) is not `[MASK]`.
44
+ * \\(\rho_t\\) is the marginal probability that \\(z_t\\) retains the clean token among the non-`[MASK]` mass.
45
 
46
 
47