genderize cora + ultra: open weights (CC BY-NC 4.0), inference code (MIT), model card
Browse files- COMMERCIAL_USE.md +21 -0
- LICENSE +408 -0
- LICENSE-CODE +21 -0
- README.md +216 -0
- genderize_cora.calibration.json +1 -0
- genderize_cora.config.json +1 -0
- genderize_cora.pt +3 -0
- genderize_infer.py +146 -0
- genderize_ultra.calibration.json +1 -0
- genderize_ultra.config.json +1 -0
- genderize_ultra.pt +3 -0
- maps.json +1 -0
COMMERCIAL_USE.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Licensing of the genderize weights
|
| 2 |
+
|
| 3 |
+
**Weights** (`genderize_cora.pt`, `genderize_ultra.pt`, their `.config.json`, `.calibration.json` and `maps.json`)
|
| 4 |
+
are released under **Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)** — see `LICENSE`.
|
| 5 |
+
|
| 6 |
+
In plain words:
|
| 7 |
+
|
| 8 |
+
- You may **download, run, study, modify and redistribute** the weights and derivatives **for non-commercial
|
| 9 |
+
purposes**, provided you give attribution (*dbtool.it, genderize cora/ultra, 2026*) and indicate changes.
|
| 10 |
+
- Research, teaching, personal projects, non-profit and public-sector use are non-commercial uses.
|
| 11 |
+
- **Commercial use is not granted by this licence.** Commercial use means, without limitation: offering the
|
| 12 |
+
model or a derivative as a paid or ad-supported service or API, embedding it in a product that is sold,
|
| 13 |
+
or using it to deliver paid services to third parties.
|
| 14 |
+
- **The only commercial channel is dbtool.it**: the hosted API at https://api.dbtool.it and the `dbtool Local`
|
| 15 |
+
on-premise licence, both offered by the rights holder. A separate written commercial licence can be
|
| 16 |
+
requested from the rights holder, Domenico Gigante (d.gigante@tech-time.it).
|
| 17 |
+
|
| 18 |
+
**Inference code** (`genderize_infer.py`) is released under the **MIT License** (see `LICENSE-CODE`) so that it
|
| 19 |
+
can be freely reused; it carries no model knowledge.
|
| 20 |
+
|
| 21 |
+
No training data, training scripts or dictionary tables are part of this release.
|
LICENSE
ADDED
|
@@ -0,0 +1,408 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Attribution-NonCommercial 4.0 International
|
| 2 |
+
|
| 3 |
+
=======================================================================
|
| 4 |
+
|
| 5 |
+
Creative Commons Corporation ("Creative Commons") is not a law firm and
|
| 6 |
+
does not provide legal services or legal advice. Distribution of
|
| 7 |
+
Creative Commons public licenses does not create a lawyer-client or
|
| 8 |
+
other relationship. Creative Commons makes its licenses and related
|
| 9 |
+
information available on an "as-is" basis. Creative Commons gives no
|
| 10 |
+
warranties regarding its licenses, any material licensed under their
|
| 11 |
+
terms and conditions, or any related information. Creative Commons
|
| 12 |
+
disclaims all liability for damages resulting from their use to the
|
| 13 |
+
fullest extent possible.
|
| 14 |
+
|
| 15 |
+
Using Creative Commons Public Licenses
|
| 16 |
+
|
| 17 |
+
Creative Commons public licenses provide a standard set of terms and
|
| 18 |
+
conditions that creators and other rights holders may use to share
|
| 19 |
+
original works of authorship and other material subject to copyright
|
| 20 |
+
and certain other rights specified in the public license below. The
|
| 21 |
+
following considerations are for informational purposes only, are not
|
| 22 |
+
exhaustive, and do not form part of our licenses.
|
| 23 |
+
|
| 24 |
+
Considerations for licensors: Our public licenses are
|
| 25 |
+
intended for use by those authorized to give the public
|
| 26 |
+
permission to use material in ways otherwise restricted by
|
| 27 |
+
copyright and certain other rights. Our licenses are
|
| 28 |
+
irrevocable. Licensors should read and understand the terms
|
| 29 |
+
and conditions of the license they choose before applying it.
|
| 30 |
+
Licensors should also secure all rights necessary before
|
| 31 |
+
applying our licenses so that the public can reuse the
|
| 32 |
+
material as expected. Licensors should clearly mark any
|
| 33 |
+
material not subject to the license. This includes other CC-
|
| 34 |
+
licensed material, or material used under an exception or
|
| 35 |
+
limitation to copyright. More considerations for licensors:
|
| 36 |
+
wiki.creativecommons.org/Considerations_for_licensors
|
| 37 |
+
|
| 38 |
+
Considerations for the public: By using one of our public
|
| 39 |
+
licenses, a licensor grants the public permission to use the
|
| 40 |
+
licensed material under specified terms and conditions. If
|
| 41 |
+
the licensor's permission is not necessary for any reason--for
|
| 42 |
+
example, because of any applicable exception or limitation to
|
| 43 |
+
copyright--then that use is not regulated by the license. Our
|
| 44 |
+
licenses grant only permissions under copyright and certain
|
| 45 |
+
other rights that a licensor has authority to grant. Use of
|
| 46 |
+
the licensed material may still be restricted for other
|
| 47 |
+
reasons, including because others have copyright or other
|
| 48 |
+
rights in the material. A licensor may make special requests,
|
| 49 |
+
such as asking that all changes be marked or described.
|
| 50 |
+
Although not required by our licenses, you are encouraged to
|
| 51 |
+
respect those requests where reasonable. More considerations
|
| 52 |
+
for the public:
|
| 53 |
+
wiki.creativecommons.org/Considerations_for_licensees
|
| 54 |
+
|
| 55 |
+
=======================================================================
|
| 56 |
+
|
| 57 |
+
Creative Commons Attribution-NonCommercial 4.0 International Public
|
| 58 |
+
License
|
| 59 |
+
|
| 60 |
+
By exercising the Licensed Rights (defined below), You accept and agree
|
| 61 |
+
to be bound by the terms and conditions of this Creative Commons
|
| 62 |
+
Attribution-NonCommercial 4.0 International Public License ("Public
|
| 63 |
+
License"). To the extent this Public License may be interpreted as a
|
| 64 |
+
contract, You are granted the Licensed Rights in consideration of Your
|
| 65 |
+
acceptance of these terms and conditions, and the Licensor grants You
|
| 66 |
+
such rights in consideration of benefits the Licensor receives from
|
| 67 |
+
making the Licensed Material available under these terms and
|
| 68 |
+
conditions.
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
Section 1 -- Definitions.
|
| 72 |
+
|
| 73 |
+
a. Adapted Material means material subject to Copyright and Similar
|
| 74 |
+
Rights that is derived from or based upon the Licensed Material
|
| 75 |
+
and in which the Licensed Material is translated, altered,
|
| 76 |
+
arranged, transformed, or otherwise modified in a manner requiring
|
| 77 |
+
permission under the Copyright and Similar Rights held by the
|
| 78 |
+
Licensor. For purposes of this Public License, where the Licensed
|
| 79 |
+
Material is a musical work, performance, or sound recording,
|
| 80 |
+
Adapted Material is always produced where the Licensed Material is
|
| 81 |
+
synched in timed relation with a moving image.
|
| 82 |
+
|
| 83 |
+
b. Adapter's License means the license You apply to Your Copyright
|
| 84 |
+
and Similar Rights in Your contributions to Adapted Material in
|
| 85 |
+
accordance with the terms and conditions of this Public License.
|
| 86 |
+
|
| 87 |
+
c. Copyright and Similar Rights means copyright and/or similar rights
|
| 88 |
+
closely related to copyright including, without limitation,
|
| 89 |
+
performance, broadcast, sound recording, and Sui Generis Database
|
| 90 |
+
Rights, without regard to how the rights are labeled or
|
| 91 |
+
categorized. For purposes of this Public License, the rights
|
| 92 |
+
specified in Section 2(b)(1)-(2) are not Copyright and Similar
|
| 93 |
+
Rights.
|
| 94 |
+
d. Effective Technological Measures means those measures that, in the
|
| 95 |
+
absence of proper authority, may not be circumvented under laws
|
| 96 |
+
fulfilling obligations under Article 11 of the WIPO Copyright
|
| 97 |
+
Treaty adopted on December 20, 1996, and/or similar international
|
| 98 |
+
agreements.
|
| 99 |
+
|
| 100 |
+
e. Exceptions and Limitations means fair use, fair dealing, and/or
|
| 101 |
+
any other exception or limitation to Copyright and Similar Rights
|
| 102 |
+
that applies to Your use of the Licensed Material.
|
| 103 |
+
|
| 104 |
+
f. Licensed Material means the artistic or literary work, database,
|
| 105 |
+
or other material to which the Licensor applied this Public
|
| 106 |
+
License.
|
| 107 |
+
|
| 108 |
+
g. Licensed Rights means the rights granted to You subject to the
|
| 109 |
+
terms and conditions of this Public License, which are limited to
|
| 110 |
+
all Copyright and Similar Rights that apply to Your use of the
|
| 111 |
+
Licensed Material and that the Licensor has authority to license.
|
| 112 |
+
|
| 113 |
+
h. Licensor means the individual(s) or entity(ies) granting rights
|
| 114 |
+
under this Public License.
|
| 115 |
+
|
| 116 |
+
i. NonCommercial means not primarily intended for or directed towards
|
| 117 |
+
commercial advantage or monetary compensation. For purposes of
|
| 118 |
+
this Public License, the exchange of the Licensed Material for
|
| 119 |
+
other material subject to Copyright and Similar Rights by digital
|
| 120 |
+
file-sharing or similar means is NonCommercial provided there is
|
| 121 |
+
no payment of monetary compensation in connection with the
|
| 122 |
+
exchange.
|
| 123 |
+
|
| 124 |
+
j. Share means to provide material to the public by any means or
|
| 125 |
+
process that requires permission under the Licensed Rights, such
|
| 126 |
+
as reproduction, public display, public performance, distribution,
|
| 127 |
+
dissemination, communication, or importation, and to make material
|
| 128 |
+
available to the public including in ways that members of the
|
| 129 |
+
public may access the material from a place and at a time
|
| 130 |
+
individually chosen by them.
|
| 131 |
+
|
| 132 |
+
k. Sui Generis Database Rights means rights other than copyright
|
| 133 |
+
resulting from Directive 96/9/EC of the European Parliament and of
|
| 134 |
+
the Council of 11 March 1996 on the legal protection of databases,
|
| 135 |
+
as amended and/or succeeded, as well as other essentially
|
| 136 |
+
equivalent rights anywhere in the world.
|
| 137 |
+
|
| 138 |
+
l. You means the individual or entity exercising the Licensed Rights
|
| 139 |
+
under this Public License. Your has a corresponding meaning.
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
Section 2 -- Scope.
|
| 143 |
+
|
| 144 |
+
a. License grant.
|
| 145 |
+
|
| 146 |
+
1. Subject to the terms and conditions of this Public License,
|
| 147 |
+
the Licensor hereby grants You a worldwide, royalty-free,
|
| 148 |
+
non-sublicensable, non-exclusive, irrevocable license to
|
| 149 |
+
exercise the Licensed Rights in the Licensed Material to:
|
| 150 |
+
|
| 151 |
+
a. reproduce and Share the Licensed Material, in whole or
|
| 152 |
+
in part, for NonCommercial purposes only; and
|
| 153 |
+
|
| 154 |
+
b. produce, reproduce, and Share Adapted Material for
|
| 155 |
+
NonCommercial purposes only.
|
| 156 |
+
|
| 157 |
+
2. Exceptions and Limitations. For the avoidance of doubt, where
|
| 158 |
+
Exceptions and Limitations apply to Your use, this Public
|
| 159 |
+
License does not apply, and You do not need to comply with
|
| 160 |
+
its terms and conditions.
|
| 161 |
+
|
| 162 |
+
3. Term. The term of this Public License is specified in Section
|
| 163 |
+
6(a).
|
| 164 |
+
|
| 165 |
+
4. Media and formats; technical modifications allowed. The
|
| 166 |
+
Licensor authorizes You to exercise the Licensed Rights in
|
| 167 |
+
all media and formats whether now known or hereafter created,
|
| 168 |
+
and to make technical modifications necessary to do so. The
|
| 169 |
+
Licensor waives and/or agrees not to assert any right or
|
| 170 |
+
authority to forbid You from making technical modifications
|
| 171 |
+
necessary to exercise the Licensed Rights, including
|
| 172 |
+
technical modifications necessary to circumvent Effective
|
| 173 |
+
Technological Measures. For purposes of this Public License,
|
| 174 |
+
simply making modifications authorized by this Section 2(a)
|
| 175 |
+
(4) never produces Adapted Material.
|
| 176 |
+
|
| 177 |
+
5. Downstream recipients.
|
| 178 |
+
|
| 179 |
+
a. Offer from the Licensor -- Licensed Material. Every
|
| 180 |
+
recipient of the Licensed Material automatically
|
| 181 |
+
receives an offer from the Licensor to exercise the
|
| 182 |
+
Licensed Rights under the terms and conditions of this
|
| 183 |
+
Public License.
|
| 184 |
+
|
| 185 |
+
b. No downstream restrictions. You may not offer or impose
|
| 186 |
+
any additional or different terms or conditions on, or
|
| 187 |
+
apply any Effective Technological Measures to, the
|
| 188 |
+
Licensed Material if doing so restricts exercise of the
|
| 189 |
+
Licensed Rights by any recipient of the Licensed
|
| 190 |
+
Material.
|
| 191 |
+
|
| 192 |
+
6. No endorsement. Nothing in this Public License constitutes or
|
| 193 |
+
may be construed as permission to assert or imply that You
|
| 194 |
+
are, or that Your use of the Licensed Material is, connected
|
| 195 |
+
with, or sponsored, endorsed, or granted official status by,
|
| 196 |
+
the Licensor or others designated to receive attribution as
|
| 197 |
+
provided in Section 3(a)(1)(A)(i).
|
| 198 |
+
|
| 199 |
+
b. Other rights.
|
| 200 |
+
|
| 201 |
+
1. Moral rights, such as the right of integrity, are not
|
| 202 |
+
licensed under this Public License, nor are publicity,
|
| 203 |
+
privacy, and/or other similar personality rights; however, to
|
| 204 |
+
the extent possible, the Licensor waives and/or agrees not to
|
| 205 |
+
assert any such rights held by the Licensor to the limited
|
| 206 |
+
extent necessary to allow You to exercise the Licensed
|
| 207 |
+
Rights, but not otherwise.
|
| 208 |
+
|
| 209 |
+
2. Patent and trademark rights are not licensed under this
|
| 210 |
+
Public License.
|
| 211 |
+
|
| 212 |
+
3. To the extent possible, the Licensor waives any right to
|
| 213 |
+
collect royalties from You for the exercise of the Licensed
|
| 214 |
+
Rights, whether directly or through a collecting society
|
| 215 |
+
under any voluntary or waivable statutory or compulsory
|
| 216 |
+
licensing scheme. In all other cases the Licensor expressly
|
| 217 |
+
reserves any right to collect such royalties, including when
|
| 218 |
+
the Licensed Material is used other than for NonCommercial
|
| 219 |
+
purposes.
|
| 220 |
+
|
| 221 |
+
|
| 222 |
+
Section 3 -- License Conditions.
|
| 223 |
+
|
| 224 |
+
Your exercise of the Licensed Rights is expressly made subject to the
|
| 225 |
+
following conditions.
|
| 226 |
+
|
| 227 |
+
a. Attribution.
|
| 228 |
+
|
| 229 |
+
1. If You Share the Licensed Material (including in modified
|
| 230 |
+
form), You must:
|
| 231 |
+
|
| 232 |
+
a. retain the following if it is supplied by the Licensor
|
| 233 |
+
with the Licensed Material:
|
| 234 |
+
|
| 235 |
+
i. identification of the creator(s) of the Licensed
|
| 236 |
+
Material and any others designated to receive
|
| 237 |
+
attribution, in any reasonable manner requested by
|
| 238 |
+
the Licensor (including by pseudonym if
|
| 239 |
+
designated);
|
| 240 |
+
|
| 241 |
+
ii. a copyright notice;
|
| 242 |
+
|
| 243 |
+
iii. a notice that refers to this Public License;
|
| 244 |
+
|
| 245 |
+
iv. a notice that refers to the disclaimer of
|
| 246 |
+
warranties;
|
| 247 |
+
|
| 248 |
+
v. a URI or hyperlink to the Licensed Material to the
|
| 249 |
+
extent reasonably practicable;
|
| 250 |
+
|
| 251 |
+
b. indicate if You modified the Licensed Material and
|
| 252 |
+
retain an indication of any previous modifications; and
|
| 253 |
+
|
| 254 |
+
c. indicate the Licensed Material is licensed under this
|
| 255 |
+
Public License, and include the text of, or the URI or
|
| 256 |
+
hyperlink to, this Public License.
|
| 257 |
+
|
| 258 |
+
2. You may satisfy the conditions in Section 3(a)(1) in any
|
| 259 |
+
reasonable manner based on the medium, means, and context in
|
| 260 |
+
which You Share the Licensed Material. For example, it may be
|
| 261 |
+
reasonable to satisfy the conditions by providing a URI or
|
| 262 |
+
hyperlink to a resource that includes the required
|
| 263 |
+
information.
|
| 264 |
+
|
| 265 |
+
3. If requested by the Licensor, You must remove any of the
|
| 266 |
+
information required by Section 3(a)(1)(A) to the extent
|
| 267 |
+
reasonably practicable.
|
| 268 |
+
|
| 269 |
+
4. If You Share Adapted Material You produce, the Adapter's
|
| 270 |
+
License You apply must not prevent recipients of the Adapted
|
| 271 |
+
Material from complying with this Public License.
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
Section 4 -- Sui Generis Database Rights.
|
| 275 |
+
|
| 276 |
+
Where the Licensed Rights include Sui Generis Database Rights that
|
| 277 |
+
apply to Your use of the Licensed Material:
|
| 278 |
+
|
| 279 |
+
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
|
| 280 |
+
to extract, reuse, reproduce, and Share all or a substantial
|
| 281 |
+
portion of the contents of the database for NonCommercial purposes
|
| 282 |
+
only;
|
| 283 |
+
|
| 284 |
+
b. if You include all or a substantial portion of the database
|
| 285 |
+
contents in a database in which You have Sui Generis Database
|
| 286 |
+
Rights, then the database in which You have Sui Generis Database
|
| 287 |
+
Rights (but not its individual contents) is Adapted Material; and
|
| 288 |
+
|
| 289 |
+
c. You must comply with the conditions in Section 3(a) if You Share
|
| 290 |
+
all or a substantial portion of the contents of the database.
|
| 291 |
+
|
| 292 |
+
For the avoidance of doubt, this Section 4 supplements and does not
|
| 293 |
+
replace Your obligations under this Public License where the Licensed
|
| 294 |
+
Rights include other Copyright and Similar Rights.
|
| 295 |
+
|
| 296 |
+
|
| 297 |
+
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
|
| 298 |
+
|
| 299 |
+
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
|
| 300 |
+
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
|
| 301 |
+
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
|
| 302 |
+
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
|
| 303 |
+
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
|
| 304 |
+
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
|
| 305 |
+
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
|
| 306 |
+
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
|
| 307 |
+
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
|
| 308 |
+
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
|
| 309 |
+
|
| 310 |
+
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
|
| 311 |
+
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
|
| 312 |
+
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
|
| 313 |
+
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
|
| 314 |
+
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
|
| 315 |
+
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
|
| 316 |
+
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
|
| 317 |
+
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
|
| 318 |
+
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
|
| 319 |
+
|
| 320 |
+
c. The disclaimer of warranties and limitation of liability provided
|
| 321 |
+
above shall be interpreted in a manner that, to the extent
|
| 322 |
+
possible, most closely approximates an absolute disclaimer and
|
| 323 |
+
waiver of all liability.
|
| 324 |
+
|
| 325 |
+
|
| 326 |
+
Section 6 -- Term and Termination.
|
| 327 |
+
|
| 328 |
+
a. This Public License applies for the term of the Copyright and
|
| 329 |
+
Similar Rights licensed here. However, if You fail to comply with
|
| 330 |
+
this Public License, then Your rights under this Public License
|
| 331 |
+
terminate automatically.
|
| 332 |
+
|
| 333 |
+
b. Where Your right to use the Licensed Material has terminated under
|
| 334 |
+
Section 6(a), it reinstates:
|
| 335 |
+
|
| 336 |
+
1. automatically as of the date the violation is cured, provided
|
| 337 |
+
it is cured within 30 days of Your discovery of the
|
| 338 |
+
violation; or
|
| 339 |
+
|
| 340 |
+
2. upon express reinstatement by the Licensor.
|
| 341 |
+
|
| 342 |
+
For the avoidance of doubt, this Section 6(b) does not affect any
|
| 343 |
+
right the Licensor may have to seek remedies for Your violations
|
| 344 |
+
of this Public License.
|
| 345 |
+
|
| 346 |
+
c. For the avoidance of doubt, the Licensor may also offer the
|
| 347 |
+
Licensed Material under separate terms or conditions or stop
|
| 348 |
+
distributing the Licensed Material at any time; however, doing so
|
| 349 |
+
will not terminate this Public License.
|
| 350 |
+
|
| 351 |
+
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
|
| 352 |
+
License.
|
| 353 |
+
|
| 354 |
+
|
| 355 |
+
Section 7 -- Other Terms and Conditions.
|
| 356 |
+
|
| 357 |
+
a. The Licensor shall not be bound by any additional or different
|
| 358 |
+
terms or conditions communicated by You unless expressly agreed.
|
| 359 |
+
|
| 360 |
+
b. Any arrangements, understandings, or agreements regarding the
|
| 361 |
+
Licensed Material not stated herein are separate from and
|
| 362 |
+
independent of the terms and conditions of this Public License.
|
| 363 |
+
|
| 364 |
+
|
| 365 |
+
Section 8 -- Interpretation.
|
| 366 |
+
|
| 367 |
+
a. For the avoidance of doubt, this Public License does not, and
|
| 368 |
+
shall not be interpreted to, reduce, limit, restrict, or impose
|
| 369 |
+
conditions on any use of the Licensed Material that could lawfully
|
| 370 |
+
be made without permission under this Public License.
|
| 371 |
+
|
| 372 |
+
b. To the extent possible, if any provision of this Public License is
|
| 373 |
+
deemed unenforceable, it shall be automatically reformed to the
|
| 374 |
+
minimum extent necessary to make it enforceable. If the provision
|
| 375 |
+
cannot be reformed, it shall be severed from this Public License
|
| 376 |
+
without affecting the enforceability of the remaining terms and
|
| 377 |
+
conditions.
|
| 378 |
+
|
| 379 |
+
c. No term or condition of this Public License will be waived and no
|
| 380 |
+
failure to comply consented to unless expressly agreed to by the
|
| 381 |
+
Licensor.
|
| 382 |
+
|
| 383 |
+
d. Nothing in this Public License constitutes or may be interpreted
|
| 384 |
+
as a limitation upon, or waiver of, any privileges and immunities
|
| 385 |
+
that apply to the Licensor or You, including from the legal
|
| 386 |
+
processes of any jurisdiction or authority.
|
| 387 |
+
|
| 388 |
+
=======================================================================
|
| 389 |
+
|
| 390 |
+
Creative Commons is not a party to its public
|
| 391 |
+
licenses. Notwithstanding, Creative Commons may elect to apply one of
|
| 392 |
+
its public licenses to material it publishes and in those instances
|
| 393 |
+
will be considered the “Licensor.” The text of the Creative Commons
|
| 394 |
+
public licenses is dedicated to the public domain under the CC0 Public
|
| 395 |
+
Domain Dedication. Except for the limited purpose of indicating that
|
| 396 |
+
material is shared under a Creative Commons public license or as
|
| 397 |
+
otherwise permitted by the Creative Commons policies published at
|
| 398 |
+
creativecommons.org/policies, Creative Commons does not authorize the
|
| 399 |
+
use of the trademark "Creative Commons" or any other trademark or logo
|
| 400 |
+
of Creative Commons without its prior written consent including,
|
| 401 |
+
without limitation, in connection with any unauthorized modifications
|
| 402 |
+
to any of its public licenses or any other arrangements,
|
| 403 |
+
understandings, or agreements concerning use of licensed material. For
|
| 404 |
+
the avoidance of doubt, this paragraph does not form part of the
|
| 405 |
+
public licenses.
|
| 406 |
+
|
| 407 |
+
Creative Commons may be contacted at creativecommons.org.
|
| 408 |
+
|
LICENSE-CODE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Domenico Gigante (dbtool.it)
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
ADDED
|
@@ -0,0 +1,216 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- multilingual
|
| 4 |
+
license: cc-by-nc-4.0
|
| 5 |
+
library_name: pytorch
|
| 6 |
+
tags:
|
| 7 |
+
- text-classification
|
| 8 |
+
- gender-detection
|
| 9 |
+
- nationality
|
| 10 |
+
- name-analysis
|
| 11 |
+
- byte-level
|
| 12 |
+
- cpu
|
| 13 |
+
pipeline_tag: text-classification
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# genderize — cora / ultra
|
| 17 |
+
|
| 18 |
+
Two models that take a personal name and return a **gender** (M/F) and a
|
| 19 |
+
**country** (226 ISO-3166 alpha-2 codes), with probabilities. Byte-level,
|
| 20 |
+
CPU-only, no tokenizer and no vocabulary file: you feed them a string.
|
| 21 |
+
|
| 22 |
+
## Package contents
|
| 23 |
+
|
| 24 |
+
| File | What it is |
|
| 25 |
+
|---|---|
|
| 26 |
+
| `genderize_cora.pt` | weights, cora (3.1 MB) — **CC BY-NC 4.0** |
|
| 27 |
+
| `genderize_ultra.pt` | weights, ultra (12.9 MB) — **CC BY-NC 4.0** |
|
| 28 |
+
| `genderize_cora.config.json` | `{"ch": 160}` — CC BY-NC 4.0 |
|
| 29 |
+
| `genderize_ultra.config.json` | `{"ch": 384}` — CC BY-NC 4.0 |
|
| 30 |
+
| `genderize_cora.calibration.json` | per-head temperature — CC BY-NC 4.0 |
|
| 31 |
+
| `genderize_ultra.calibration.json` | per-head temperature (the file additionally records a calibration-error metric) — CC BY-NC 4.0 |
|
| 32 |
+
| `maps.json` | class ids: gender `M/F`, 226 countries — CC BY-NC 4.0 |
|
| 33 |
+
| `genderize_infer.py` | standalone inference code (needs `torch` and `numpy`) — **MIT** |
|
| 34 |
+
| `LICENSE` | full text of CC BY-NC 4.0 (weights) |
|
| 35 |
+
| `LICENSE-CODE` | full text of the MIT License (inference code) |
|
| 36 |
+
| `COMMERCIAL_USE.md` | what counts as non-commercial use, and how to obtain a commercial licence |
|
| 37 |
+
|
| 38 |
+
## What the models do
|
| 39 |
+
|
| 40 |
+
| Variant | Gender | Country | Size |
|
| 41 |
+
|---|---|---|---|
|
| 42 |
+
| `genderize-cora` | 2 classes (M/F) | 226 classes | ~0.77M parameters, ch=160 |
|
| 43 |
+
| `genderize-ultra` | 2 classes (M/F) | 226 classes | ~3.21M parameters, ch=384 |
|
| 44 |
+
|
| 45 |
+
`ultra` is the larger, more accurate variant; `cora` is the lighter one. Both run
|
| 46 |
+
on CPU: on an Intel i3-6100T (2 cores, 2 threads used) the network alone processes
|
| 47 |
+
about **850 names/s** (`cora`) and **160 names/s** (`ultra`) in batches of 100.
|
| 48 |
+
|
| 49 |
+
## How it works
|
| 50 |
+
|
| 51 |
+
**1. Normalisation.** The input name is normalised before anything else:
|
| 52 |
+
Unicode **NFC**, **lowercase**, **whitespace collapsed** (internal runs and
|
| 53 |
+
leading/trailing spaces become a single space). The normalised form is what the
|
| 54 |
+
weights saw, so this step must not be changed or skipped:
|
| 55 |
+
|
| 56 |
+
" MARÍA GARCÍA " -> "maría garcía"
|
| 57 |
+
|
| 58 |
+
**2. Encoding.** The normalised text is encoded as **UTF-8 and truncated to 48
|
| 59 |
+
bytes** (not 48 characters: an accented or non-Latin character takes more than
|
| 60 |
+
one byte, so the effective character budget is smaller). Each byte becomes an
|
| 61 |
+
integer in 0–255; byte value 0 is the padding value. A name shorter than 48 bytes
|
| 62 |
+
is padded with zeros.
|
| 63 |
+
|
| 64 |
+
**3. Network.** Byte-level, dual-head 1-D convolutional classifier:
|
| 65 |
+
|
| 66 |
+
- one embedding table, 256 entries (one per byte value) of width 64, padding
|
| 67 |
+
index 0;
|
| 68 |
+
- a 1×1 convolution projecting 64 → `ch` channels;
|
| 69 |
+
- four residual 1-D convolution blocks, kernel sizes **3, 5, 7, 3**, each with
|
| 70 |
+
convolution + BatchNorm + GELU around a residual connection, padding to keep
|
| 71 |
+
the length;
|
| 72 |
+
- the masked positions are pooled twice — mean-pooling and max-pooling — and the
|
| 73 |
+
two vectors are concatenated;
|
| 74 |
+
- a shared layer of 512 units (GELU, plus a dropout layer that is inactive at
|
| 75 |
+
inference);
|
| 76 |
+
- two linear heads read out from it: **gender**, 2 classes, and **country**,
|
| 77 |
+
226 classes.
|
| 78 |
+
|
| 79 |
+
`cora` uses `ch = 160`, `ultra` `ch = 384`; those are the only differences
|
| 80 |
+
between the two configs.
|
| 81 |
+
|
| 82 |
+
**4. Probabilities.** Because it is a *convolutional* stack, the model sees all
|
| 83 |
+
byte positions at once: character order matters through the convolution kernels,
|
| 84 |
+
not through a recurrent state. Logits are divided by a **per-head temperature**
|
| 85 |
+
before softmax — `temp_gender` and `temp_country` from the calibration file:
|
| 86 |
+
|
| 87 |
+
| Variant | temp_gender | temp_country |
|
| 88 |
+
|---|---|---|
|
| 89 |
+
| cora | 1.0096479654312134 | 1.0 |
|
| 90 |
+
| ultra | 1.0460342168807983 | 0.939997673034668 |
|
| 91 |
+
|
| 92 |
+
**5. Decision rule.** Gender is reported as `male` when P(M) ≥ 0.5, otherwise
|
| 93 |
+
`female`; the reported probability is the probability of the reported class.
|
| 94 |
+
Countries are returned as the **top-k** codes (default 5, `--top`) sorted by
|
| 95 |
+
probability.
|
| 96 |
+
|
| 97 |
+
**6. Output.** One record per input name:
|
| 98 |
+
|
| 99 |
+
{"name": "...", "gender": "male|female", "probability": 0.0-1.0,
|
| 100 |
+
"countries": [{"code": "IT", "probability": 0.0-1.0}, ...], "model": "cora|ultra"}
|
| 101 |
+
|
| 102 |
+
## Loading and using the weights
|
| 103 |
+
|
| 104 |
+
All files sit flat in this repository. Download them (for example with
|
| 105 |
+
`huggingface_hub.snapshot_download("textpie/genderize")`) and run the reference
|
| 106 |
+
driver from that directory:
|
| 107 |
+
|
| 108 |
+
pip install torch numpy
|
| 109 |
+
python genderize_infer.py --variant ultra --top 3 "Andrea Rossi"
|
| 110 |
+
|
| 111 |
+
In Python:
|
| 112 |
+
|
| 113 |
+
```python
|
| 114 |
+
from genderize_infer import Genderize
|
| 115 |
+
|
| 116 |
+
model = Genderize("ultra") # or Genderize("cora"); pass models_dir=... if the files live elsewhere
|
| 117 |
+
model.predict(["Andrea Rossi"], top=3)
|
| 118 |
+
```
|
| 119 |
+
|
| 120 |
+
Output for the five names published with the examples (`--variant ultra`),
|
| 121 |
+
reproduced locally against these exact weight files:
|
| 122 |
+
|
| 123 |
+
| Input | Gender | P | Country top-3 |
|
| 124 |
+
|---|---|---|---|
|
| 125 |
+
| Andrea Rossi | male | 0.849 | IT 0.84, FR 0.05, US 0.02 |
|
| 126 |
+
| Yuki Tanaka | female | 0.589 | JP 0.98, US 0.01, ID 0.00 |
|
| 127 |
+
| María García | female | 0.995 | ES 0.42, MX 0.13, AR 0.09 |
|
| 128 |
+
| Chen Wei | male | 0.692 | CN 0.56, TW 0.13, SG 0.09 |
|
| 129 |
+
| Fatima Al Sayed | female | 0.996 | AE 0.35, OM 0.19, SA 0.14 |
|
| 130 |
+
|
| 131 |
+
## Benchmarks
|
| 132 |
+
|
| 133 |
+
Source: **dbtool.it/benchmark**. These are **not** new measurements taken for
|
| 134 |
+
this card and no re-measurement was performed here.
|
| 135 |
+
|
| 136 |
+
- **Bench**: 25,000 names never seen in training, scored on the **network alone**.
|
| 137 |
+
- The dictionary layer used by the hosted API does not contribute on unseen
|
| 138 |
+
names, so these figures describe the released weights.
|
| 139 |
+
|
| 140 |
+
| Metric | cora | ultra |
|
| 141 |
+
|---|---|---|
|
| 142 |
+
| Gender accuracy | 97.9 % | 98.2 % |
|
| 143 |
+
| Country top-1 accuracy | 82.6 % | 83.7 % |
|
| 144 |
+
|
| 145 |
+
Per-country figures for the full hosted system are on https://dbtool.it/academic; an
|
| 146 |
+
independent open bench on public WGND 2.0 names (network alone, losses included) is
|
| 147 |
+
on https://dbtool.it/benchmark.
|
| 148 |
+
|
| 149 |
+
## What this release does NOT include
|
| 150 |
+
|
| 151 |
+
- **The dictionary layer of the API.** The hosted API combines the network with
|
| 152 |
+
a proprietary frequency layer (exact-name and per-country M/F frequencies)
|
| 153 |
+
derived from production data. That layer is **not part of this release**:
|
| 154 |
+
you get the network alone. Consequence: **the public API can answer
|
| 155 |
+
differently from the weights you download**, especially on very frequent
|
| 156 |
+
names. Documented example: `Yuki Tanaka` scores *female 0.59* with the network
|
| 157 |
+
alone (the value reproduced above), while the API with its dictionary layer
|
| 158 |
+
answers *male 0.58*.
|
| 159 |
+
- **Everything about how the models were trained**: no recipe, no epochs, no
|
| 160 |
+
optimiser, no data split, no training hyperparameters, no data sources and no
|
| 161 |
+
per-country counts. This release ships a usable model, not the process behind
|
| 162 |
+
it, and no training example.
|
| 163 |
+
|
| 164 |
+
## Training data — nature only
|
| 165 |
+
|
| 166 |
+
The models were trained on **name–gender–country pairs covering 226 countries,
|
| 167 |
+
tens of millions of examples**. That is the whole description this release
|
| 168 |
+
provides: the corpus is **not redistributed**, and its sources, composition and
|
| 169 |
+
per-country sizes are not part of the package.
|
| 170 |
+
|
| 171 |
+
## Limitations
|
| 172 |
+
|
| 173 |
+
- **East Asian names**: accuracy drops to roughly **82–88 %**; Chinese, Korean
|
| 174 |
+
and Japanese names are frequently confused with one another.
|
| 175 |
+
- **Names ambiguous across countries**: many names are plausible in several
|
| 176 |
+
countries, so a top-1 of ~83 % means roughly one name in six gets the wrong
|
| 177 |
+
country. `Andrea` is the classic case: male in Italy, female elsewhere. Treat
|
| 178 |
+
the country score as a prior, and never as a single-country verdict.
|
| 179 |
+
- **48-byte truncation**: only the first 48 UTF-8 bytes reach the model. Long
|
| 180 |
+
names and long compounds are silently cut, and multi-byte characters consume
|
| 181 |
+
more of the budget than plain ASCII.
|
| 182 |
+
- **Empty or near-empty input**: an empty or fully-trimmed name is encoded as 48
|
| 183 |
+
padding bytes and still returns a prediction. Validate your input.
|
| 184 |
+
- **Binary gender**: the gender head has two classes; names that do not fit them
|
| 185 |
+
are forced into the closer one.
|
| 186 |
+
- **Transliteration**: results depend on how the name was romanised upstream;
|
| 187 |
+
different romanisations of the same name can disagree.
|
| 188 |
+
- **Dictionary gap**: the released weights are the network alone, so for very
|
| 189 |
+
frequent names they can differ from the hosted API (see above).
|
| 190 |
+
|
| 191 |
+
## Licence
|
| 192 |
+
|
| 193 |
+
Two licences, one per artefact:
|
| 194 |
+
|
| 195 |
+
- **Weights** — `genderize_cora.pt`, `genderize_ultra.pt`, their `.config.json`
|
| 196 |
+
and `.calibration.json`, and `maps.json` — are released under **Creative
|
| 197 |
+
Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)**, full
|
| 198 |
+
text in `LICENSE`. You may download, run, study, modify and redistribute them
|
| 199 |
+
and their derivatives **for non-commercial purposes**, with attribution
|
| 200 |
+
(*dbtool.it, genderize cora/ultra, 2026*) and an indication of changes.
|
| 201 |
+
- **Inference code** — `genderize_infer.py` — is released under the **MIT License**,
|
| 202 |
+
full text in `LICENSE-CODE`. It can be reused freely; it carries no model
|
| 203 |
+
knowledge.
|
| 204 |
+
|
| 205 |
+
**Commercial use of the weights is not granted by this licence.** The commercial
|
| 206 |
+
channel is **dbtool.it** (the hosted API and the on-premise licence), or a
|
| 207 |
+
separate written licence from the rights holder. What counts as commercial use,
|
| 208 |
+
and how to request a licence, is set out in **`COMMERCIAL_USE.md`**.
|
| 209 |
+
|
| 210 |
+
The training data is **not** part of this release, and neither is the dictionary
|
| 211 |
+
layer of the hosted API (see above).
|
| 212 |
+
|
| 213 |
+
## Contact
|
| 214 |
+
|
| 215 |
+
dbtool — https://dbtool.it — open-weights page: https://dbtool.it/open-models.html
|
| 216 |
+
Domenico Gigante, d.gigante@tech-time.it
|
genderize_cora.calibration.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"temp_gender": 1.0096479654312134, "temp_country": 1.0}
|
genderize_cora.config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"ch": 160}
|
genderize_cora.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c0f466b2221f71910b4c83f18a6d207adf837d4cdac2d37da20a5bfdbf068c30
|
| 3 |
+
size 3101302
|
genderize_infer.py
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""genderize (cora / ultra) — standalone inference for the released weights.
|
| 3 |
+
|
| 4 |
+
What the model is
|
| 5 |
+
-----------------
|
| 6 |
+
A byte-level dual-head convolutional classifier. A personal name is normalised
|
| 7 |
+
(Unicode NFC, lowercase, whitespace collapsed), UTF-8 encoded and truncated to
|
| 8 |
+
48 bytes. Each byte is embedded (256 x 64), projected to `ch` channels and passed
|
| 9 |
+
through four residual 1-D convolution blocks (kernels 3, 5, 7, 3; BatchNorm +
|
| 10 |
+
GELU). Masked mean-pooling and max-pooling over the byte positions are
|
| 11 |
+
concatenated and fed to a shared 512-unit layer, from which two linear heads
|
| 12 |
+
read out: gender (2 classes, M/F) and country (226 ISO-3166 alpha-2 codes).
|
| 13 |
+
`cora` uses ch=160 (~0.77M parameters), `ultra` uses ch=384 (~3.2M).
|
| 14 |
+
Logits are divided by a per-head temperature (calibration.json) before softmax.
|
| 15 |
+
|
| 16 |
+
Files expected next to this script (or pass --models-dir):
|
| 17 |
+
genderize_<variant>.pt state_dict (PyTorch)
|
| 18 |
+
genderize_<variant>.config.json {"ch": 160 | 384}
|
| 19 |
+
genderize_<variant>.calibration.json {"temp_gender": t, "temp_country": t}
|
| 20 |
+
maps.json {"gender": {"M":0,"F":1}, "country": {"AD":0, ...}}
|
| 21 |
+
|
| 22 |
+
Usage
|
| 23 |
+
-----
|
| 24 |
+
python genderize_infer.py --variant ultra "Andrea Rossi" "Yuki Tanaka" "María García"
|
| 25 |
+
python genderize_infer.py --variant cora --top 3 --json "Chen Wei"
|
| 26 |
+
|
| 27 |
+
Requires only torch and numpy. CPU is enough.
|
| 28 |
+
"""
|
| 29 |
+
from __future__ import annotations
|
| 30 |
+
|
| 31 |
+
import argparse
|
| 32 |
+
import json
|
| 33 |
+
import sys
|
| 34 |
+
import unicodedata
|
| 35 |
+
from pathlib import Path
|
| 36 |
+
|
| 37 |
+
import numpy as np
|
| 38 |
+
import torch
|
| 39 |
+
import torch.nn as nn
|
| 40 |
+
import torch.nn.functional as F
|
| 41 |
+
|
| 42 |
+
MAXLEN = 48
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
class _GBlock(nn.Module):
|
| 46 |
+
def __init__(self, ch: int, k: int):
|
| 47 |
+
super().__init__()
|
| 48 |
+
self.conv = nn.Conv1d(ch, ch, k, padding=k // 2)
|
| 49 |
+
self.norm = nn.BatchNorm1d(ch)
|
| 50 |
+
|
| 51 |
+
def forward(self, x):
|
| 52 |
+
return x + F.gelu(self.norm(self.conv(x)))
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
class NameModel(nn.Module):
|
| 56 |
+
"""Byte-level dual-head classifier: gender (2) + country (n_countries)."""
|
| 57 |
+
|
| 58 |
+
def __init__(self, n_countries: int, emb: int = 64, ch: int = 160):
|
| 59 |
+
super().__init__()
|
| 60 |
+
self.emb = nn.Embedding(256, emb, padding_idx=0)
|
| 61 |
+
self.proj = nn.Conv1d(emb, ch, 1)
|
| 62 |
+
self.blocks = nn.Sequential(_GBlock(ch, 3), _GBlock(ch, 5), _GBlock(ch, 7), _GBlock(ch, 3))
|
| 63 |
+
self.shared = nn.Sequential(nn.Linear(ch * 2, 512), nn.GELU(), nn.Dropout(0.15))
|
| 64 |
+
self.head_g = nn.Linear(512, 2)
|
| 65 |
+
self.head_c = nn.Linear(512, n_countries)
|
| 66 |
+
|
| 67 |
+
def forward(self, x):
|
| 68 |
+
mask = (x != 0).float().unsqueeze(1)
|
| 69 |
+
h = self.proj(self.emb(x.long()).transpose(1, 2))
|
| 70 |
+
h = self.blocks(h) * mask
|
| 71 |
+
mean = h.sum(-1) / mask.sum(-1).clamp(min=1)
|
| 72 |
+
mx = h.masked_fill(mask == 0, -1e9).max(-1).values
|
| 73 |
+
z = self.shared(torch.cat([mean, mx], -1))
|
| 74 |
+
return self.head_g(z), self.head_c(z)
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def normalise(name: str) -> str:
|
| 78 |
+
"""NFC + lowercase + collapse whitespace. Must match what the weights saw."""
|
| 79 |
+
return " ".join(unicodedata.normalize("NFC", (name or "").lower()).split())
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def encode(texts: list[str], maxlen: int = MAXLEN) -> torch.Tensor:
|
| 83 |
+
X = np.zeros((len(texts), maxlen), dtype=np.uint8)
|
| 84 |
+
for i, t in enumerate(texts):
|
| 85 |
+
b = t.encode("utf-8")[:maxlen]
|
| 86 |
+
X[i, : len(b)] = np.frombuffer(b, dtype=np.uint8)
|
| 87 |
+
return torch.from_numpy(X)
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
class Genderize:
|
| 91 |
+
def __init__(self, variant: str = "ultra", models_dir: str | Path | None = None):
|
| 92 |
+
md = Path(models_dir) if models_dir else Path(__file__).resolve().parent
|
| 93 |
+
maps = json.loads((md / "maps.json").read_text())
|
| 94 |
+
self.inv_country = {v: k for k, v in maps["country"].items()}
|
| 95 |
+
self.idx_m = maps["gender"].get("M", 0)
|
| 96 |
+
cfg = json.loads((md / f"genderize_{variant}.config.json").read_text())
|
| 97 |
+
cal_path = md / f"genderize_{variant}.calibration.json"
|
| 98 |
+
self.cal = json.loads(cal_path.read_text()) if cal_path.exists() else {}
|
| 99 |
+
self.model = NameModel(len(maps["country"]), ch=int(cfg.get("ch", 160)))
|
| 100 |
+
self.model.load_state_dict(torch.load(md / f"genderize_{variant}.pt", map_location="cpu"))
|
| 101 |
+
self.model.eval()
|
| 102 |
+
self.variant = variant
|
| 103 |
+
|
| 104 |
+
@torch.no_grad()
|
| 105 |
+
def predict(self, names: list[str], top: int = 5) -> list[dict]:
|
| 106 |
+
texts = [normalise(n) for n in names]
|
| 107 |
+
lg, lc = self.model(encode(texts))
|
| 108 |
+
pg = F.softmax(lg / self.cal.get("temp_gender", 1.0), dim=-1)
|
| 109 |
+
pc = F.softmax(lc / self.cal.get("temp_country", 1.0), dim=-1)
|
| 110 |
+
vals, idx = pc.topk(min(top, pc.shape[-1]), dim=-1)
|
| 111 |
+
out = []
|
| 112 |
+
for i, name in enumerate(names):
|
| 113 |
+
pm = float(pg[i, self.idx_m])
|
| 114 |
+
gender = "male" if pm >= 0.5 else "female"
|
| 115 |
+
out.append({
|
| 116 |
+
"name": name,
|
| 117 |
+
"gender": gender,
|
| 118 |
+
"probability": round(pm if gender == "male" else 1.0 - pm, 4),
|
| 119 |
+
"countries": [{"code": self.inv_country.get(int(j), "??"), "probability": round(float(v), 4)}
|
| 120 |
+
for v, j in zip(vals[i].tolist(), idx[i].tolist())],
|
| 121 |
+
"model": self.variant,
|
| 122 |
+
})
|
| 123 |
+
return out
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
def main() -> int:
|
| 127 |
+
ap = argparse.ArgumentParser(description="genderize cora/ultra — gender + nationality from a name")
|
| 128 |
+
ap.add_argument("names", nargs="+")
|
| 129 |
+
ap.add_argument("--variant", choices=["cora", "ultra"], default="ultra")
|
| 130 |
+
ap.add_argument("--models-dir", default=None)
|
| 131 |
+
ap.add_argument("--top", type=int, default=5)
|
| 132 |
+
ap.add_argument("--json", action="store_true", help="print JSON instead of a table")
|
| 133 |
+
a = ap.parse_args()
|
| 134 |
+
g = Genderize(a.variant, a.models_dir)
|
| 135 |
+
rows = g.predict(a.names, top=a.top)
|
| 136 |
+
if a.json:
|
| 137 |
+
print(json.dumps(rows, ensure_ascii=False, indent=1))
|
| 138 |
+
else:
|
| 139 |
+
for r in rows:
|
| 140 |
+
cs = ", ".join(f"{c['code']} {c['probability']:.2f}" for c in r["countries"])
|
| 141 |
+
print(f"{r['name']:<28} {r['gender']:<7} {r['probability']:.3f} {cs}")
|
| 142 |
+
return 0
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
if __name__ == "__main__":
|
| 146 |
+
sys.exit(main())
|
genderize_ultra.calibration.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"temp_gender": 1.0460342168807983, "temp_country": 0.939997673034668, "ece_gender_pre": 0.0012399105666638377, "ece_gender_post": 0.0007778512481309008}
|
genderize_ultra.config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"ch": 384}
|
genderize_ultra.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e0b61d9a547aa1b11acad39af573ad63897578e35aa609c26b3de33156303b83
|
| 3 |
+
size 12868705
|
maps.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"gender": {"M": 0, "F": 1}, "country": {"AD": 0, "AE": 1, "AF": 2, "AG": 3, "AI": 4, "AL": 5, "AM": 6, "AO": 7, "AR": 8, "AT": 9, "AU": 10, "AX": 11, "AZ": 12, "BA": 13, "BB": 14, "BD": 15, "BE": 16, "BF": 17, "BG": 18, "BH": 19, "BI": 20, "BJ": 21, "BM": 22, "BN": 23, "BO": 24, "BR": 25, "BS": 26, "BT": 27, "BW": 28, "BY": 29, "BZ": 30, "CA": 31, "CC": 32, "CD": 33, "CF": 34, "CG": 35, "CH": 36, "CI": 37, "CK": 38, "CL": 39, "CM": 40, "CN": 41, "CO": 42, "CR": 43, "CU": 44, "CV": 45, "CY": 46, "CZ": 47, "DE": 48, "DJ": 49, "DK": 50, "DM": 51, "DO": 52, "DZ": 53, "EC": 54, "EE": 55, "EG": 56, "EH": 57, "ER": 58, "ES": 59, "ET": 60, "FI": 61, "FJ": 62, "FK": 63, "FM": 64, "FR": 65, "GA": 66, "GB": 67, "GD": 68, "GE": 69, "GG": 70, "GH": 71, "GI": 72, "GM": 73, "GN": 74, "GP": 75, "GQ": 76, "GR": 77, "GT": 78, "GW": 79, "GY": 80, "HK": 81, "HN": 82, "HR": 83, "HT": 84, "HU": 85, "ID": 86, "IE": 87, "IL": 88, "IM": 89, "IN": 90, "IO": 91, "IQ": 92, "IR": 93, "IS": 94, "IT": 95, "JE": 96, "JM": 97, "JO": 98, "JP": 99, "KE": 100, "KG": 101, "KH": 102, "KI": 103, "KM": 104, "KN": 105, "KP": 106, "KR": 107, "KW": 108, "KY": 109, "KZ": 110, "LA": 111, "LB": 112, "LC": 113, "LI": 114, "LK": 115, "LR": 116, "LS": 117, "LT": 118, "LU": 119, "LV": 120, "LY": 121, "MA": 122, "MC": 123, "MD": 124, "ME": 125, "MF": 126, "MG": 127, "MH": 128, "MK": 129, "ML": 130, "MM": 131, "MN": 132, "MO": 133, "MQ": 134, "MR": 135, "MS": 136, "MT": 137, "MU": 138, "MV": 139, "MW": 140, "MX": 141, "MY": 142, "MZ": 143, "NA": 144, "NC": 145, "NE": 146, "NF": 147, "NG": 148, "NI": 149, "NL": 150, "NO": 151, "NP": 152, "NR": 153, "NU": 154, "NZ": 155, "OM": 156, "PA": 157, "PE": 158, "PG": 159, "PH": 160, "PK": 161, "PL": 162, "PN": 163, "PR": 164, "PS": 165, "PT": 166, "PW": 167, "PY": 168, "QA": 169, "RO": 170, "RS": 171, "RU": 172, "RW": 173, "SA": 174, "SB": 175, "SC": 176, "SD": 177, "SE": 178, "SG": 179, "SI": 180, "SK": 181, "SL": 182, "SM": 183, "SN": 184, "SO": 185, "SR": 186, "SS": 187, "ST": 188, "SV": 189, "SX": 190, "SY": 191, "SZ": 192, "TC": 193, "TD": 194, "TG": 195, "TH": 196, "TJ": 197, "TK": 198, "TL": 199, "TM": 200, "TN": 201, "TO": 202, "TR": 203, "TT": 204, "TV": 205, "TW": 206, "TZ": 207, "UA": 208, "UG": 209, "US": 210, "UY": 211, "UZ": 212, "VA": 213, "VC": 214, "VE": 215, "VG": 216, "VN": 217, "VU": 218, "WF": 219, "WS": 220, "XK": 221, "YE": 222, "ZA": 223, "ZM": 224, "ZW": 225}}
|