diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000000000000000000000000000000000..7ebc8ecdc9db3827575faabc25650179ff71c2a1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,43 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.axmodel filter=lfs diff=lfs merge=lfs -text +*.wav filter=lfs diff=lfs merge=lfs -textcpp_bench/bin/firered_bench filter=lfs diff=lfs merge=lfs -text +cpp_bench/bin/libfirered_sdk_lib.so filter=lfs diff=lfs merge=lfs -text +cpp/bin/firered_bench filter=lfs diff=lfs merge=lfs -text +cpp/bin/libfirered_sdk_lib.so filter=lfs diff=lfs merge=lfs -text +cpp/bin/firered_openai_client filter=lfs diff=lfs merge=lfs -text +cpp/bin/firered_openai_server filter=lfs diff=lfs merge=lfs -text diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..10f5e703af77d90937a6ab6a306f9e16130cd110 --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +.vscode +__pycache__ +cpp/build +cpp/ax650n_bsp_sdk \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..bbd602bfa1c45d28263081e4068436d558f88112 --- /dev/null +++ b/README.md @@ -0,0 +1,62 @@ +--- +license: apache-2.0 +pipeline_tag: audio-to-audio +tags: +- axmodel +- axera +- FireRedASR-AED +--- + +# FireRedASR-AED + +小红书 ASR AED-L 在 AX650N 上的 NPU 部署（原项目： +[FireRedTeam/FireRedASR](https://github.com/FireRedTeam/FireRedASR)）。 +支持中文、英文，最长 10s 输入，超长音频自动 VAD 切分。 + +## 模型 + +| 文件 | 说明 | +|------|------| +| `axmodel/encoder.axmodel` / `axmodel/decoder_loop.axmodel` | Pulsar2 5.1 原版 | +| `axmodel/encoder_u16_7.0.axmodel` / `axmodel/decoder_loop_u8_dense_7.0.axmodel` | Pulsar2 7.0-lite 重转换（推荐） | +| `fsmn_vad/fsmn_vad_10s_fp32.axmodel` + `fsmn_vad/am.mvn` | FSMN-VAD（FunASR）NPU 语音门控 | + +## VAD 说明（2026-08-24） + +- 超长音频按 10s 块处理；每块用 FSMN-VAD 判断是否含语音， + 检出语音 ≥ 1s 才整块送 ASR，静音/噪声块直接返回空。 +- 适配远场/低信噪比录音：3 分钟立体声录音可输出完整转写， + 同时静音/噪声不产生幻觉。 + +## 指标（AX650N，4 条测试语音） + +- CER：3.37% +- RTF：约 0.3（Python），0.225（C++ SDK） + +## 使用 + +```bash +conda create -n fireredasr python=3.12 +conda activate fireredasr +pip install -r requirements.txt +pip install axengine-0.1.3-py3-none-any.whl # pyaxengine +python openai/openai_server.py \ + --encoder axmodel/encoder_u16_7.0.axmodel \ + --decoder axmodel/decoder_loop_u8_dense_7.0.axmodel \ + --fsmn-vad fsmn_vad/fsmn_vad_10s_fp32.axmodel \ + --fsmn-cmvn fsmn_vad/am.mvn \ + --cmvn axmodel/cmvn.ark --dict axmodel/dict.txt \ + --pe axmodel/pe.npy --max-dur 10 --max-steps 128 +``` + +## OpenAI API + +- C++ 预编译：`cpp/bin/firered_openai_server`、`cpp/bin/firered_openai_client` + （用法见 `cpp/bin/README.md`） +- Python：`openai/openai_server.py`、`openai/openai_client.py`（stdlib，零额外依赖） +- Python ASR 核心：`openai/firered_asr.py` + `openai/fsmn_vad_post.py` + （numpy + pyaxengine + kaldi-native-fbank，无 torch/onnxruntime） +- 端点：`POST /v1/audio/transcriptions`（multipart `file`）、`GET /v1/models` + +7.0 重转换与 C++ SDK 详见 [README_7.0.md](README_7.0.md)； +转换源码见 [GitHub](https://github.com/ml-inory/FireRedASR.axera)。 diff --git a/README_7.0.md b/README_7.0.md new file mode 100644 index 0000000000000000000000000000000000000000..9855829c54ef91a575808e007ed7d1722b1d43b5 --- /dev/null +++ b/README_7.0.md @@ -0,0 +1,28 @@ +# FireRedASR-AED · Pulsar2 7.0-lite 重转换 + C++ SDK + +AX650N 上的 7.0-lite 重转换模型与端到端 C++ SDK（板端 VAD + fbank + ASR， +无 torch/onnxruntime 依赖）。 + +## 模型 + +- `axmodel/encoder_u16_7.0.axmodel`（encoder U16） +- `axmodel/decoder_loop_u8_dense_7.0.axmodel`（decoder U8 密集校准） + +## 指标（AX650N，4 条测试语音） + +| 版本 | CER% | RTF | +|------|------|-----| +| Python 优化版 | 3.37 | 0.318 | +| C++ SDK | 3.37 | **0.225** | + +## OpenAI API + +Python 与 C++ 均提供 OpenAI 兼容 server/client： +- C++ 预编译：`cpp/bin/firered_openai_server` / `firered_openai_client` +- Python：`openai/openai_server.py` / `openai/openai_client.py` + +## 使用 + +C++ SDK 源码、构建与运行说明、Pulsar2 编译配置见 +[GitHub](https://github.com/ml-inory/FireRedASR.axera)（`cpp/`、 +`pulsar2_configs/`、`reports/`）。 diff --git a/axmodel/cmvn.ark b/axmodel/cmvn.ark new file mode 100644 index 0000000000000000000000000000000000000000..e26b4a310132492d05bfdd506ab3f6bbf89d9059 Binary files /dev/null and b/axmodel/cmvn.ark differ diff --git a/axmodel/decoder_loop.axmodel b/axmodel/decoder_loop.axmodel new file mode 100644 index 0000000000000000000000000000000000000000..beffeabbfaf2dad8f0303a56c12c155c941e3c3d --- /dev/null +++ b/axmodel/decoder_loop.axmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2912496e6804027f2dc77c903f6b2f76678603dd616e662b78e3f226bcaa91a +size 416269694 diff --git a/axmodel/decoder_loop_u8_dense_7.0.axmodel b/axmodel/decoder_loop_u8_dense_7.0.axmodel new file mode 100644 index 0000000000000000000000000000000000000000..1862638a419f58dd838314373e927209ad4fef85 --- /dev/null +++ b/axmodel/decoder_loop_u8_dense_7.0.axmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b73988101d1593ad76b4fee43e5d3867a30940bd0a3fc49381a1f57edb99c8f +size 416421664 diff --git a/axmodel/dict.txt b/axmodel/dict.txt new file mode 100644 index 0000000000000000000000000000000000000000..afd1d79290c76a05e9eb653d76984434c18bb371 --- /dev/null +++ b/axmodel/dict.txt @@ -0,0 +1,7832 @@ + 0 + 1 + 2 + 3 + 4 +' 5 +▁ 6 +A 7 +▁A 8 +AB 9 +▁AB 10 +ABLE 11 +▁ABOUT 12 +AC 13 +▁AC 14 +▁ACC 15 +ACE 16 +ACH 17 +ACK 18 +ACT 19 +▁ACT 20 +AD 21 +▁AD 22 +▁ADD 23 +ADE 24 +▁ADV 25 +ADY 26 +▁AF 27 +▁AFTER 28 +AG 29 +▁AG 30 +▁AGAIN 31 +▁AGAINST 32 +AGE 33 +AIL 34 +AIN 35 +AINED 36 +AINT 37 +AIR 38 +▁AIR 39 +AK 40 +AKE 41 +AKEN 42 +AKING 43 +AL 44 +▁AL 45 +ALE 46 +ALK 47 +ALL 48 +▁ALL 49 +ALLY 50 +▁ALMOST 51 +ALS 52 +▁ALSO 53 +▁ALWAYS 54 +AM 55 +▁AM 56 +AME 57 +▁AMONG 58 +AMP 59 +AN 60 +▁AN 61 +ANC 62 +ANCE 63 +AND 64 +▁AND 65 +ANG 66 +ANGE 67 +ANK 68 +▁ANOTHER 69 +ANS 70 +▁ANS 71 +ANT 72 +▁ANY 73 +▁ANYTHING 74 +AP 75 +▁APP 76 +▁APPE 77 +APS 78 +AR 79 +▁AR 80 +ARD 81 +ARE 82 +▁ARE 83 +ARED 84 +ARING 85 +ARK 86 +▁ARM 87 +ARS 88 +ART 89 +ARY 90 +AS 91 +▁AS 92 +ASE 93 +▁ASKED 94 +ASON 95 +ASS 96 +▁ASS 97 +AST 98 +ASTER 99 +AT 100 +▁AT 101 +ATE 102 +ATED 103 +ATELY 104 +ATES 105 +ATH 106 +ATHER 107 +ATING 108 +ATION 109 +ATIONS 110 +▁ATT 111 +AU 112 +AUGHT 113 +AUSE 114 +AUT 115 +AV 116 +AVE 117 +▁AW 118 +▁AWAY 119 +AY 120 +B 121 +▁B 122 +▁BACK 123 +▁BAR 124 +BE 125 +▁BE 126 +▁BEAUT 127 +▁BEC 128 +▁BECAUSE 129 +▁BED 130 +▁BEEN 131 +▁BEFORE 132 +▁BEG 133 +▁BEGAN 134 +▁BEH 135 +▁BEING 136 +▁BEL 137 +▁BELIE 138 +BER 139 +▁BEST 140 +▁BET 141 +▁BETTER 142 +▁BETWEEN 143 +▁BL 144 +▁BLACK 145 +BLE 146 +BO 147 +▁BO 148 +▁BOTH 149 +▁BOY 150 +▁BR 151 +▁BRE 152 +▁BRO 153 +▁BROUGHT 154 +▁BU 155 +▁BUS 156 +▁BUT 157 +▁BY 158 +C 159 +▁C 160 +▁CALL 161 +▁CALLED 162 +▁CAME 163 +▁CAN 164 +▁CAP 165 +▁CAR 166 +▁CARE 167 +CC 168 +CE 169 +CED 170 +CEI 171 +CEPT 172 +▁CERTAIN 173 +CES 174 +CESS 175 +CH 176 +▁CH 177 +▁CHAR 178 +▁CHE 179 +CHED 180 +▁CHILD 181 +CI 182 +CK 183 +CL 184 +▁CL 185 +▁CLE 186 +CO 187 +▁CO 188 +▁COL 189 +▁COM 190 +▁COME 191 +▁COMM 192 +▁COMP 193 +▁CON 194 +▁COND 195 +▁CONF 196 +▁CONS 197 +▁CONT 198 +▁CONTIN 199 +▁COULD 200 +▁COUNT 201 +▁COUNTRY 202 +▁COUR 203 +▁COURSE 204 +▁CR 205 +▁CRIED 206 +CT 207 +CTION 208 +▁CUR 209 +D 210 +▁D 211 +▁DARK 212 +▁DAY 213 +▁DAYS 214 +DD 215 +DDEN 216 +DE 217 +▁DE 218 +▁DEAR 219 +▁DEL 220 +DER 221 +▁DES 222 +▁DID 223 +▁DIF 224 +▁DIS 225 +▁DIST 226 +▁DO 227 +▁DOCT 228 +▁DOES 229 +▁DON 230 +▁DONE 231 +▁DOOR 232 +▁DOWN 233 +▁DR 234 +▁DRA 235 +DU 236 +DY 237 +E 238 +▁E 239 +▁EACH 240 +▁EAR 241 +ECT 242 +ED 243 +▁ED 244 +EEN 245 +▁EIGHT 246 +EL 247 +▁EL 248 +EM 249 +▁EM 250 +EN 251 +▁EN 252 +ENCE 253 +▁END 254 +ENED 255 +▁ENG 256 +▁ENOUGH 257 +ENS 258 +ENSE 259 +ENT 260 +▁ENT 261 +ENTLY 262 +ENTS 263 +ENTY 264 +EP 265 +ER 266 +ERED 267 +ERING 268 +ERS 269 +ERT 270 +ERTAIN 271 +ES 272 +▁ES 273 +ESS 274 +EST 275 +ET 276 +ETER 277 +▁EV 278 +▁EVEN 279 +EVER 280 +▁EVER 281 +▁EVERY 282 +EW 283 +▁EX 284 +▁EXP 285 +▁EY 286 +▁EYES 287 +F 288 +▁F 289 +▁FACE 290 +▁FACT 291 +▁FALL 292 +▁FAM 293 +▁FAR 294 +▁FATHER 295 +FE 296 +▁FE 297 +FECT 298 +▁FEEL 299 +▁FEET 300 +▁FELT 301 +▁FEW 302 +FF 303 +▁FIN 304 +▁FIND 305 +▁FIR 306 +▁FIRST 307 +▁FIVE 308 +▁FL 309 +▁FO 310 +▁FOLLOW 311 +▁FOR 312 +FORE 313 +▁FORE 314 +▁FORM 315 +FORT 316 +▁FOUND 317 +▁FOUR 318 +▁FR 319 +▁FRIEND 320 +▁FROM 321 +FT 322 +FUL 323 +▁FULL 324 +FULLY 325 +G 326 +▁G 327 +▁GAVE 328 +GE 329 +GED 330 +▁GEN 331 +▁GENER 332 +▁GENT 333 +GET 334 +▁GET 335 +GETHER 336 +GG 337 +GH 338 +GHT 339 +▁GI 340 +▁GIRL 341 +▁GIVE 342 +▁GL 343 +▁GO 344 +▁GOD 345 +▁GOING 346 +▁GONE 347 +▁GOOD 348 +▁GOT 349 +▁GR 350 +▁GRA 351 +▁GRE 352 +▁GREAT 353 +▁GU 354 +H 355 +▁H 356 +▁HA 357 +▁HAD 358 +▁HALF 359 +▁HAND 360 +▁HANDS 361 +▁HAPP 362 +HAPS 363 +▁HAR 364 +▁HARD 365 +▁HAS 366 +▁HAVE 367 +▁HAVING 368 +HE 369 +▁HE 370 +▁HEAD 371 +▁HEARD 372 +▁HEART 373 +HED 374 +▁HEL 375 +▁HELP 376 +HER 377 +▁HER 378 +▁HERE 379 +▁HERSELF 380 +HES 381 +▁HIGH 382 +▁HIM 383 +▁HIMSELF 384 +HING 385 +▁HIS 386 +▁HO 387 +▁HOME 388 +▁HOR 389 +▁HOUR 390 +▁HOUSE 391 +▁HOW 392 +▁HOWEVER 393 +▁HU 394 +▁HUM 395 +▁HUND 396 +▁HUNDRED 397 +I 398 +▁I 399 +IA 400 +IAL 401 +IAN 402 +IB 403 +IBLE 404 +IC 405 +ICAL 406 +ICE 407 +ICH 408 +ICK 409 +ICT 410 +ID 411 +IDE 412 +▁IDE 413 +IE 414 +IED 415 +IEND 416 +IER 417 +IES 418 +IET 419 +IF 420 +▁IF 421 +IG 422 +IGH 423 +IGHT 424 +IGN 425 +IL 426 +ILD 427 +ILE 428 +ILL 429 +ILY 430 +IM 431 +▁IM 432 +▁IMP 433 +IN 434 +▁IN 435 +▁INC 436 +INCE 437 +IND 438 +▁IND 439 +▁INDE 440 +▁INDEED 441 +INE 442 +INED 443 +INESS 444 +▁INF 445 +ING 446 +INGS 447 +INK 448 +INS 449 +▁INS 450 +▁INST 451 +INT 452 +▁INT 453 +▁INTE 454 +▁INTER 455 +▁INTEREST 456 +▁INTO 457 +ION 458 +IONS 459 +IOUS 460 +IP 461 +IR 462 +IRD 463 +IRE 464 +IRED 465 +IRL 466 +IS 467 +▁IS 468 +ISE 469 +ISED 470 +ISH 471 +ISHED 472 +ISS 473 +IST 474 +IT 475 +▁IT 476 +ITE 477 +ITED 478 +ITH 479 +ITHER 480 +ITION 481 +ITS 482 +▁ITS 483 +ITT 484 +ITTLE 485 +ITY 486 +IV 487 +IVE 488 +IVER 489 +IX 490 +J 491 +▁J 492 +JECT 493 +▁JO 494 +▁JUST 495 +K 496 +▁K 497 +KE 498 +▁KE 499 +KED 500 +▁KEEP 501 +▁KIND 502 +▁KING 503 +▁KN 504 +▁KNEW 505 +▁KNOW 506 +KS 507 +L 508 +▁L 509 +▁LA 510 +▁LADY 511 +LAND 512 +▁LAND 513 +▁LAR 514 +▁LAST 515 +▁LAT 516 +▁LAW 517 +▁LAY 518 +LD 519 +LE 520 +▁LE 521 +LED 522 +▁LEFT 523 +LES 524 +LESS 525 +▁LESS 526 +▁LET 527 +LF 528 +▁LI 529 +▁LIFE 530 +▁LIGHT 531 +▁LIKE 532 +LING 533 +▁LITTLE 534 +LL 535 +LLOW 536 +▁LO 537 +▁LONG 538 +▁LOO 539 +▁LOOK 540 +▁LOOKED 541 +▁LOOKING 542 +▁LORD 543 +▁LOVE 544 +LT 545 +LY 546 +M 547 +▁M 548 +▁MA 549 +▁MADE 550 +▁MAKE 551 +MAN 552 +▁MAN 553 +▁MANY 554 +▁MAR 555 +▁MAT 556 +▁MATTER 557 +▁MAY 558 +▁ME 559 +▁MEAN 560 +MED 561 +▁MEN 562 +MENT 563 +▁MET 564 +▁MIGHT 565 +▁MIL 566 +▁MIN 567 +▁MIND 568 +▁MIS 569 +▁MISS 570 +▁MISSUS 571 +▁MISTER 572 +▁MO 573 +▁MOM 574 +▁MOMENT 575 +▁MON 576 +▁MORE 577 +▁MORN 578 +▁MORNING 579 +MOST 580 +▁MOST 581 +▁MOTHER 582 +▁MU 583 +▁MUCH 584 +▁MUST 585 +▁MY 586 +▁MYSELF 587 +N 588 +▁N 589 +▁NAME 590 +▁NAT 591 +ND 592 +NDED 593 +NDER 594 +NE 595 +▁NE 596 +▁NEAR 597 +NED 598 +▁NEED 599 +NER 600 +NESS 601 +▁NEVER 602 +▁NEW 603 +▁NEXT 604 +NG 605 +▁NIGHT 606 +NING 607 +▁NO 608 +▁NOR 609 +NOT 610 +▁NOT 611 +▁NOTHING 612 +▁NOW 613 +NT 614 +O 615 +▁O 616 +▁OB 617 +OC 618 +▁OCC 619 +OCK 620 +▁OF 621 +▁OFF 622 +▁OH 623 +OL 624 +▁OLD 625 +OM 626 +OME 627 +ON 628 +▁ON 629 +▁ONCE 630 +OND 631 +ONE 632 +▁ONE 633 +ONEY 634 +ONG 635 +▁ONLY 636 +ONS 637 +OO 638 +OOD 639 +OOK 640 +OOL 641 +OP 642 +▁OP 643 +▁OPEN 644 +OR 645 +▁OR 646 +ORD 647 +ORE 648 +ORM 649 +ORN 650 +ORS 651 +ORT 652 +ORY 653 +OS 654 +OSE 655 +OSED 656 +OSS 657 +OST 658 +OT 659 +OTH 660 +OTHER 661 +▁OTHER 662 +OTT 663 +OU 664 +OUB 665 +OUD 666 +OUGH 667 +OUGHT 668 +OULD 669 +OUND 670 +OUNT 671 +OUR 672 +▁OUR 673 +OUS 674 +OUSE 675 +OUT 676 +▁OUT 677 +▁OVER 678 +OW 679 +OWER 680 +OWN 681 +▁OWN 682 +OWS 683 +P 684 +▁P 685 +▁PAR 686 +▁PART 687 +▁PASS 688 +▁PAT 689 +PE 690 +▁PE 691 +PECT 692 +▁PEOP 693 +▁PEOPLE 694 +PER 695 +▁PER 696 +▁PERHAPS 697 +▁PERS 698 +▁PERSON 699 +PH 700 +▁PH 701 +PL 702 +▁PL 703 +▁PLACE 704 +PLE 705 +▁PLE 706 +▁PLEAS 707 +▁PO 708 +▁POINT 709 +▁POOR 710 +▁POSS 711 +▁POWER 712 +PP 713 +PPED 714 +▁PR 715 +▁PRE 716 +▁PRES 717 +▁PRESENT 718 +▁PRIN 719 +▁PRO 720 +▁PROP 721 +PS 722 +PT 723 +▁PUR 724 +▁PUT 725 +Q 726 +QU 727 +▁QU 728 +▁QUEST 729 +▁QUITE 730 +R 731 +▁R 732 +RA 733 +▁RATHER 734 +RE 735 +▁RE 736 +▁READ 737 +REAM 738 +REAT 739 +RED 740 +REE 741 +▁REF 742 +▁REG 743 +▁REL 744 +▁REM 745 +REN 746 +RENT 747 +▁REP 748 +▁REPL 749 +▁REPLIED 750 +RES 751 +▁RES 752 +RESS 753 +RESSED 754 +REST 755 +▁REST 756 +RET 757 +▁RET 758 +▁RETURN 759 +REW 760 +RI 761 +RIED 762 +▁RIGHT 763 +RO 764 +▁RO 765 +▁ROOM 766 +ROSS 767 +▁ROUND 768 +ROW 769 +RU 770 +▁RUN 771 +RY 772 +S 773 +▁S 774 +▁SA 775 +▁SAID 776 +▁SAME 777 +▁SAT 778 +▁SAW 779 +▁SAY 780 +▁SC 781 +SE 782 +▁SE 783 +▁SEE 784 +▁SEEM 785 +▁SEEMED 786 +▁SEEN 787 +SEL 788 +SELF 789 +SELVES 790 +▁SENT 791 +▁SER 792 +▁SERV 793 +▁SET 794 +▁SH 795 +▁SHALL 796 +▁SHE 797 +▁SHOULD 798 +▁SHOW 799 +▁SIDE 800 +▁SIL 801 +▁SIM 802 +▁SINCE 803 +▁SIR 804 +▁SIX 805 +▁SK 806 +▁SL 807 +▁SLE 808 +▁SM 809 +▁SMALL 810 +▁SN 811 +SO 812 +▁SO 813 +▁SOM 814 +▁SOME 815 +▁SOMET 816 +▁SOMETHING 817 +▁SOON 818 +▁SOU 819 +▁SP 820 +▁SPE 821 +▁SPO 822 +SS 823 +ST 824 +▁ST 825 +▁STAND 826 +▁STE 827 +▁STILL 828 +▁STOOD 829 +▁STR 830 +▁SU 831 +▁SUB 832 +▁SUCH 833 +▁SUDDEN 834 +▁SUN 835 +▁SUPP 836 +▁SUR 837 +▁SURE 838 +▁SW 839 +▁SY 840 +T 841 +▁T 842 +TAIN 843 +▁TAKE 844 +TE 845 +▁TE 846 +▁TELL 847 +TER 848 +▁TER 849 +TERS 850 +TH 851 +▁TH 852 +▁THAN 853 +▁THAT 854 +▁THE 855 +▁THEIR 856 +▁THEM 857 +▁THEN 858 +THER 859 +▁THERE 860 +▁THESE 861 +▁THEY 862 +THING 863 +▁THING 864 +▁THINGS 865 +▁THINK 866 +▁THIS 867 +▁THOSE 868 +▁THOUGH 869 +▁THOUGHT 870 +▁THR 871 +▁THREE 872 +▁THROUGH 873 +▁TILL 874 +▁TIM 875 +▁TIME 876 +TING 877 +▁TO 878 +▁TOGETHER 879 +▁TOLD 880 +▁TOO 881 +▁TOOK 882 +▁TOWARD 883 +▁TR 884 +▁TRA 885 +▁TRE 886 +▁TURN 887 +▁TURNED 888 +▁TW 889 +▁TWO 890 +TY 891 +U 892 +▁U 893 +UAL 894 +UB 895 +UCK 896 +UD 897 +UE 898 +UL 899 +ULAR 900 +ULL 901 +ULLY 902 +ULT 903 +UM 904 +UN 905 +▁UN 906 +▁UNC 907 +▁UNDER 908 +▁UNDERST 909 +UNG 910 +▁UNT 911 +▁UNTIL 912 +UP 913 +▁UP 914 +▁UPON 915 +UR 916 +URE 917 +URED 918 +URES 919 +URN 920 +US 921 +▁US 922 +▁USE 923 +UST 924 +UT 925 +UTE 926 +UTH 927 +V 928 +▁V 929 +VE 930 +VED 931 +VEL 932 +VEN 933 +VENT 934 +VER 935 +VERY 936 +▁VERY 937 +VES 938 +VING 939 +▁VIS 940 +▁VO 941 +▁VOICE 942 +W 943 +▁W 944 +▁WA 945 +▁WANT 946 +▁WAR 947 +WARD 948 +▁WAS 949 +▁WAT 950 +▁WATER 951 +WAY 952 +▁WAY 953 +WAYS 954 +WE 955 +▁WE 956 +WEEN 957 +▁WELL 958 +▁WENT 959 +▁WERE 960 +▁WH 961 +▁WHAT 962 +▁WHE 963 +▁WHEN 964 +▁WHERE 965 +▁WHICH 966 +▁WHILE 967 +▁WHITE 968 +▁WHO 969 +▁WHOLE 970 +▁WHOM 971 +▁WHY 972 +▁WI 973 +▁WIFE 974 +▁WILL 975 +▁WIND 976 +▁WITH 977 +▁WITHOUT 978 +WN 979 +▁WO 980 +▁WOM 981 +▁WOMAN 982 +▁WOR 983 +▁WORD 984 +▁WORDS 985 +▁WORK 986 +▁WORLD 987 +▁WOULD 988 +▁WR 989 +X 990 +XT 991 +Y 992 +▁Y 993 +▁YE 994 +▁YEARS 995 +▁YES 996 +▁YET 997 +YING 998 +▁YOU 999 +▁YOUNG 1000 +▁YOUR 1001 +Z 1002 +ZZ 1003 +一 1004 +丁 1005 +七 1006 +万 1007 +丈 1008 +三 1009 +上 1010 +下 1011 +不 1012 +与 1013 +丐 1014 +丑 1015 +专 1016 +且 1017 +丕 1018 +世 1019 +丘 1020 +丙 1021 +业 1022 +丛 1023 +东 1024 +丝 1025 +丞 1026 +丢 1027 +两 1028 +严 1029 +丧 1030 +丨 1031 +个 1032 +丫 1033 +中 1034 +丰 1035 +串 1036 +临 1037 +丸 1038 +丹 1039 +为 1040 +主 1041 +丽 1042 +举 1043 +乃 1044 +久 1045 +么 1046 +义 1047 +之 1048 +乌 1049 +乍 1050 +乎 1051 +乏 1052 +乐 1053 +乒 1054 +乓 1055 +乔 1056 +乖 1057 +乘 1058 +乙 1059 +乜 1060 +九 1061 +乞 1062 +也 1063 +习 1064 +乡 1065 +书 1066 +乩 1067 +买 1068 +乱 1069 +乳 1070 +乾 1071 +了 1072 +予 1073 +争 1074 +事 1075 +二 1076 +亍 1077 +于 1078 +亏 1079 +云 1080 +互 1081 +亓 1082 +五 1083 +井 1084 +亘 1085 +亚 1086 +些 1087 +亟 1088 +亡 1089 +亢 1090 +交 1091 +亥 1092 +亦 1093 +产 1094 +亨 1095 +亩 1096 +享 1097 +京 1098 +亭 1099 +亮 1100 +亲 1101 +亳 1102 +亵 1103 +亶 1104 +亸 1105 +亹 1106 +人 1107 +亿 1108 +什 1109 +仁 1110 +仂 1111 +仃 1112 +仄 1113 +仅 1114 +仆 1115 +仇 1116 +今 1117 +介 1118 +仍 1119 +从 1120 +仑 1121 +仓 1122 +仔 1123 +仕 1124 +他 1125 +仗 1126 +付 1127 +仙 1128 +仝 1129 +仞 1130 +仟 1131 +仡 1132 +代 1133 +令 1134 +以 1135 +仨 1136 +仪 1137 +们 1138 +仮 1139 +仰 1140 +仲 1141 +仵 1142 +件 1143 +价 1144 +任 1145 +份 1146 +仿 1147 +企 1148 +伃 1149 +伉 1150 +伊 1151 +伋 1152 +伍 1153 +伎 1154 +伏 1155 +伐 1156 +休 1157 +众 1158 +优 1159 +伙 1160 +会 1161 +伛 1162 +伝 1163 +伞 1164 +伟 1165 +传 1166 +伢 1167 +伤 1168 +伥 1169 +伦 1170 +伧 1171 +伪 1172 +伫 1173 +伯 1174 +估 1175 +伴 1176 +伶 1177 +伷 1178 +伸 1179 +伺 1180 +似 1181 +伽 1182 +佃 1183 +但 1184 +佉 1185 +位 1186 +低 1187 +住 1188 +佐 1189 +佑 1190 +体 1191 +何 1192 +佗 1193 +佘 1194 +余 1195 +佚 1196 +佛 1197 +作 1198 +佝 1199 +佞 1200 +佟 1201 +你 1202 +佢 1203 +佣 1204 +佤 1205 +佥 1206 +佩 1207 +佬 1208 +佯 1209 +佰 1210 +佳 1211 +佶 1212 +佺 1213 +佻 1214 +佼 1215 +佾 1216 +使 1217 +侂 1218 +侃 1219 +侄 1220 +侈 1221 +侉 1222 +例 1223 +侍 1224 +侏 1225 +侑 1226 +侔 1227 +侗 1228 +侘 1229 +供 1230 +依 1231 +侠 1232 +侣 1233 +侥 1234 +侦 1235 +侧 1236 +侨 1237 +侩 1238 +侪 1239 +侬 1240 +侮 1241 +侯 1242 +侴 1243 +侵 1244 +便 1245 +促 1246 +俄 1247 +俅 1248 +俊 1249 +俎 1250 +俏 1251 +俐 1252 +俑 1253 +俗 1254 +俘 1255 +俚 1256 +俛 1257 +保 1258 +俞 1259 +俟 1260 +信 1261 +俣 1262 +俤 1263 +俦 1264 +俨 1265 +俩 1266 +俪 1267 +俭 1268 +修 1269 +俯 1270 +俱 1271 +俳 1272 +俵 1273 +俶 1274 +俸 1275 +俺 1276 +俾 1277 +倌 1278 +倍 1279 +倏 1280 +倒 1281 +倔 1282 +倘 1283 +候 1284 +倚 1285 +倜 1286 +借 1287 +倡 1288 +倢 1289 +倦 1290 +倧 1291 +倨 1292 +倩 1293 +倪 1294 +倬 1295 +倭 1296 +债 1297 +倻 1298 +值 1299 +倾 1300 +偁 1301 +偃 1302 +假 1303 +偈 1304 +偌 1305 +偎 1306 +偏 1307 +偓 1308 +偕 1309 +做 1310 +停 1311 +健 1312 +偭 1313 +偰 1314 +偲 1315 +偶 1316 +偷 1317 +偻 1318 +偿 1319 +傀 1320 +傅 1321 +傈 1322 +傉 1323 +傍 1324 +傕 1325 +傣 1326 +傥 1327 +储 1328 +傩 1329 +催 1330 +傲 1331 +傺 1332 +傻 1333 +僆 1334 +像 1335 +僖 1336 +僚 1337 +僢 1338 +僧 1339 +僭 1340 +僮 1341 +僳 1342 +僵 1343 +僶 1344 +僻 1345 +儆 1346 +儋 1347 +儍 1348 +儒 1349 +儡 1350 +儿 1351 +兀 1352 +允 1353 +元 1354 +兄 1355 +充 1356 +兆 1357 +先 1358 +光 1359 +克 1360 +免 1361 +兑 1362 +兔 1363 +兖 1364 +党 1365 +兜 1366 +兢 1367 +入 1368 +全 1369 +八 1370 +公 1371 +六 1372 +兮 1373 +兰 1374 +共 1375 +关 1376 +兴 1377 +兵 1378 +其 1379 +具 1380 +典 1381 +兹 1382 +养 1383 +兼 1384 +兽 1385 +冀 1386 +内 1387 +円 1388 +冇 1389 +冈 1390 +冉 1391 +册 1392 +再 1393 +冏 1394 +冒 1395 +冔 1396 +冕 1397 +冗 1398 +写 1399 +军 1400 +农 1401 +冠 1402 +冢 1403 +冤 1404 +冥 1405 +冨 1406 +冫 1407 +冬 1408 +冯 1409 +冰 1410 +冱 1411 +冲 1412 +决 1413 +况 1414 +冶 1415 +冷 1416 +冻 1417 +冼 1418 +冽 1419 +净 1420 +凄 1421 +准 1422 +凇 1423 +凉 1424 +凋 1425 +凌 1426 +减 1427 +凑 1428 +凖 1429 +凛 1430 +凝 1431 +几 1432 +凡 1433 +凤 1434 +凪 1435 +凫 1436 +凭 1437 +凯 1438 +凰 1439 +凳 1440 +凶 1441 +凸 1442 +凹 1443 +出 1444 +击 1445 +凼 1446 +函 1447 +凿 1448 +刀 1449 +刁 1450 +刃 1451 +分 1452 +切 1453 +刈 1454 +刊 1455 +刍 1456 +刎 1457 +刑 1458 +划 1459 +列 1460 +刘 1461 +则 1462 +刚 1463 +创 1464 +初 1465 +删 1466 +判 1467 +刨 1468 +利 1469 +别 1470 +刮 1471 +到 1472 +刳 1473 +制 1474 +刷 1475 +券 1476 +刹 1477 +刺 1478 +刻 1479 +刼 1480 +刽 1481 +刿 1482 +剀 1483 +剁 1484 +剂 1485 +剃 1486 +削 1487 +剌 1488 +前 1489 +剐 1490 +剑 1491 +剔 1492 +剖 1493 +剜 1494 +剡 1495 +剣 1496 +剥 1497 +剧 1498 +剩 1499 +剪 1500 +副 1501 +割 1502 +剽 1503 +剿 1504 +劈 1505 +力 1506 +劝 1507 +办 1508 +功 1509 +加 1510 +务 1511 +劢 1512 +劣 1513 +动 1514 +助 1515 +努 1516 +劫 1517 +劬 1518 +劭 1519 +励 1520 +劲 1521 +劳 1522 +効 1523 +劻 1524 +劼 1525 +劾 1526 +势 1527 +勃 1528 +勇 1529 +勉 1530 +勋 1531 +勍 1532 +勐 1533 +勒 1534 +勖 1535 +勘 1536 +募 1537 +勤 1538 +勧 1539 +勰 1540 +勷 1541 +勺 1542 +勾 1543 +勿 1544 +匀 1545 +匄 1546 +包 1547 +匆 1548 +匈 1549 +匍 1550 +匏 1551 +匐 1552 +匕 1553 +化 1554 +北 1555 +匙 1556 +匝 1557 +匠 1558 +匡 1559 +匣 1560 +匪 1561 +匮 1562 +匹 1563 +区 1564 +医 1565 +匼 1566 +匾 1567 +匿 1568 +十 1569 +千 1570 +卅 1571 +升 1572 +午 1573 +卉 1574 +半 1575 +华 1576 +协 1577 +卑 1578 +卒 1579 +卓 1580 +单 1581 +卖 1582 +南 1583 +単 1584 +博 1585 +卜 1586 +卞 1587 +卟 1588 +占 1589 +卡 1590 +卢 1591 +卤 1592 +卦 1593 +卧 1594 +卫 1595 +卬 1596 +卮 1597 +卯 1598 +印 1599 +危 1600 +卲 1601 +即 1602 +却 1603 +卵 1604 +卷 1605 +卸 1606 +卺 1607 +卿 1608 +厂 1609 +厄 1610 +厅 1611 +历 1612 +厉 1613 +压 1614 +厌 1615 +厍 1616 +厓 1617 +厔 1618 +厕 1619 +厖 1620 +厘 1621 +厚 1622 +厝 1623 +原 1624 +厢 1625 +厥 1626 +厦 1627 +厨 1628 +厩 1629 +厮 1630 +厶 1631 +去 1632 +县 1633 +叁 1634 +参 1635 +叅 1636 +叆 1637 +又 1638 +叉 1639 +及 1640 +友 1641 +双 1642 +反 1643 +发 1644 +叔 1645 +取 1646 +受 1647 +变 1648 +叙 1649 +叛 1650 +叟 1651 +叠 1652 +口 1653 +古 1654 +句 1655 +另 1656 +叨 1657 +叩 1658 +只 1659 +叫 1660 +召 1661 +叭 1662 +叮 1663 +可 1664 +台 1665 +叱 1666 +史 1667 +右 1668 +叵 1669 +叶 1670 +号 1671 +司 1672 +叹 1673 +叻 1674 +叼 1675 +叽 1676 +吁 1677 +吃 1678 +各 1679 +吆 1680 +吇 1681 +合 1682 +吉 1683 +吊 1684 +吋 1685 +同 1686 +名 1687 +后 1688 +吏 1689 +吐 1690 +向 1691 +吒 1692 +吓 1693 +吔 1694 +吕 1695 +吖 1696 +吗 1697 +君 1698 +吝 1699 +吞 1700 +吟 1701 +吠 1702 +吡 1703 +吥 1704 +否 1705 +吧 1706 +吨 1707 +吩 1708 +含 1709 +听 1710 +吭 1711 +吮 1712 +启 1713 +吱 1714 +吲 1715 +吴 1716 +吵 1717 +吸 1718 +吹 1719 +吻 1720 +吼 1721 +吾 1722 +吿 1723 +呀 1724 +呃 1725 +呆 1726 +呈 1727 +告 1728 +呋 1729 +呐 1730 +呒 1731 +呓 1732 +呔 1733 +呕 1734 +呗 1735 +员 1736 +呛 1737 +呜 1738 +呢 1739 +呣 1740 +呤 1741 +呦 1742 +周 1743 +呱 1744 +呲 1745 +味 1746 +呵 1747 +呶 1748 +呷 1749 +呸 1750 +呻 1751 +呼 1752 +命 1753 +咀 1754 +咁 1755 +咂 1756 +咄 1757 +咆 1758 +咋 1759 +和 1760 +咎 1761 +咏 1762 +咐 1763 +咒 1764 +咔 1765 +咕 1766 +咖 1767 +咘 1768 +咙 1769 +咚 1770 +咛 1771 +咝 1772 +咣 1773 +咤 1774 +咦 1775 +咧 1776 +咨 1777 +咩 1778 +咪 1779 +咫 1780 +咬 1781 +咭 1782 +咯 1783 +咱 1784 +咲 1785 +咳 1786 +咵 1787 +咸 1788 +咻 1789 +咽 1790 +咾 1791 +咿 1792 +哀 1793 +品 1794 +哂 1795 +哄 1796 +哆 1797 +哇 1798 +哈 1799 +哉 1800 +哌 1801 +响 1802 +哎 1803 +哏 1804 +哐 1805 +哑 1806 +哒 1807 +哓 1808 +哔 1809 +哕 1810 +哗 1811 +哙 1812 +哚 1813 +哝 1814 +哞 1815 +哟 1816 +哥 1817 +哦 1818 +哧 1819 +哨 1820 +哩 1821 +哪 1822 +哭 1823 +哮 1824 +哲 1825 +哺 1826 +哼 1827 +哽 1828 +唁 1829 +唆 1830 +唇 1831 +唉 1832 +唏 1833 +唐 1834 +唑 1835 +唔 1836 +唗 1837 +唛 1838 +唠 1839 +唢 1840 +唤 1841 +唧 1842 +唪 1843 +唬 1844 +售 1845 +唯 1846 +唰 1847 +唱 1848 +唳 1849 +唷 1850 +唻 1851 +唾 1852 +啁 1853 +啃 1854 +啄 1855 +商 1856 +啉 1857 +啊 1858 +啐 1859 +啕 1860 +啖 1861 +啜 1862 +啡 1863 +啤 1864 +啥 1865 +啦 1866 +啧 1867 +啪 1868 +啫 1869 +啬 1870 +啭 1871 +啮 1872 +啰 1873 +啵 1874 +啶 1875 +啷 1876 +啸 1877 +啻 1878 +啼 1879 +啾 1880 +喀 1881 +喁 1882 +喂 1883 +喃 1884 +善 1885 +喆 1886 +喇 1887 +喈 1888 +喉 1889 +喊 1890 +喋 1891 +喏 1892 +喑 1893 +喔 1894 +喘 1895 +喙 1896 +喜 1897 +喝 1898 +喟 1899 +喧 1900 +喭 1901 +喯 1902 +喰 1903 +喱 1904 +喳 1905 +喵 1906 +営 1907 +喷 1908 +喹 1909 +喻 1910 +喽 1911 +喾 1912 +嗄 1913 +嗅 1914 +嗉 1915 +嗐 1916 +嗑 1917 +嗒 1918 +嗓 1919 +嗔 1920 +嗖 1921 +嗙 1922 +嗜 1923 +嗝 1924 +嗞 1925 +嗟 1926 +嗡 1927 +嗣 1928 +嗤 1929 +嗥 1930 +嗦 1931 +嗨 1932 +嗪 1933 +嗫 1934 +嗬 1935 +嗯 1936 +嗲 1937 +嗳 1938 +嗷 1939 +嗻 1940 +嗽 1941 +嘀 1942 +嘁 1943 +嘅 1944 +嘈 1945 +嘉 1946 +嘌 1947 +嘎 1948 +嘏 1949 +嘘 1950 +嘚 1951 +嘛 1952 +嘞 1953 +嘟 1954 +嘢 1955 +嘣 1956 +嘤 1957 +嘧 1958 +嘬 1959 +嘭 1960 +嘱 1961 +嘲 1962 +嘴 1963 +嘶 1964 +嘹 1965 +嘻 1966 +嘿 1967 +噃 1968 +噌 1969 +噎 1970 +噔 1971 +噗 1972 +噘 1973 +噙 1974 +噜 1975 +噢 1976 +噤 1977 +器 1978 +噩 1979 +噪 1980 +噫 1981 +噬 1982 +噱 1983 +噶 1984 +噻 1985 +噼 1986 +嚅 1987 +嚎 1988 +嚏 1989 +嚒 1990 +嚓 1991 +嚟 1992 +嚣 1993 +嚭 1994 +嚯 1995 +嚷 1996 +嚼 1997 +囊 1998 +囍 1999 +囔 2000 +囗 2001 +囚 2002 +四 2003 +囝 2004 +回 2005 +囟 2006 +因 2007 +囡 2008 +团 2009 +団 2010 +囤 2011 +囧 2012 +囫 2013 +园 2014 +困 2015 +囱 2016 +図 2017 +围 2018 +囵 2019 +囹 2020 +固 2021 +国 2022 +图 2023 +囿 2024 +圃 2025 +圄 2026 +圆 2027 +圈 2028 +圉 2029 +圜 2030 +土 2031 +圣 2032 +圧 2033 +在 2034 +圩 2035 +圪 2036 +圭 2037 +圯 2038 +地 2039 +圳 2040 +圴 2041 +圹 2042 +场 2043 +圻 2044 +圾 2045 +址 2046 +坂 2047 +均 2048 +坊 2049 +坌 2050 +坍 2051 +坎 2052 +坏 2053 +坐 2054 +坑 2055 +坒 2056 +块 2057 +坚 2058 +坛 2059 +坜 2060 +坝 2061 +坞 2062 +坟 2063 +坠 2064 +坡 2065 +坤 2066 +坦 2067 +坨 2068 +坩 2069 +坪 2070 +坭 2071 +坯 2072 +坳 2073 +坷 2074 +坻 2075 +坼 2076 +垂 2077 +垃 2078 +垄 2079 +垅 2080 +垆 2081 +垉 2082 +型 2083 +垌 2084 +垍 2085 +垒 2086 +垓 2087 +垕 2088 +垚 2089 +垛 2090 +垠 2091 +垡 2092 +垢 2093 +垣 2094 +垤 2095 +垦 2096 +垩 2097 +垫 2098 +垭 2099 +垮 2100 +垱 2101 +垴 2102 +垿 2103 +埂 2104 +埃 2105 +埇 2106 +埈 2107 +埋 2108 +城 2109 +埏 2110 +埔 2111 +埕 2112 +埗 2113 +埙 2114 +埚 2115 +埝 2116 +域 2117 +埠 2118 +埤 2119 +埭 2120 +埵 2121 +埸 2122 +培 2123 +基 2124 +埼 2125 +堀 2126 +堂 2127 +堃 2128 +堆 2129 +堇 2130 +堌 2131 +堍 2132 +堎 2133 +堑 2134 +堕 2135 +堡 2136 +堤 2137 +堨 2138 +堪 2139 +堰 2140 +堵 2141 +堺 2142 +塄 2143 +塌 2144 +塍 2145 +塑 2146 +塔 2147 +塘 2148 +塞 2149 +塡 2150 +塩 2151 +填 2152 +塬 2153 +塭 2154 +塽 2155 +塾 2156 +墀 2157 +墁 2158 +境 2159 +墅 2160 +墉 2161 +墋 2162 +墎 2163 +墒 2164 +墓 2165 +増 2166 +墘 2167 +墙 2168 +增 2169 +墟 2170 +墡 2171 +墨 2172 +墩 2173 +墪 2174 +壁 2175 +壅 2176 +壑 2177 +壕 2178 +壤 2179 +壥 2180 +士 2181 +壬 2182 +壮 2183 +声 2184 +売 2185 +壳 2186 +壴 2187 +壵 2188 +壶 2189 +壸 2190 +壹 2191 +壻 2192 +处 2193 +备 2194 +夌 2195 +复 2196 +夏 2197 +夔 2198 +夕 2199 +外 2200 +夙 2201 +多 2202 +夜 2203 +够 2204 +大 2205 +天 2206 +太 2207 +夫 2208 +夭 2209 +央 2210 +夯 2211 +失 2212 +头 2213 +夷 2214 +夸 2215 +夹 2216 +夺 2217 +奁 2218 +奂 2219 +奄 2220 +奇 2221 +奈 2222 +奉 2223 +奋 2224 +奎 2225 +奏 2226 +契 2227 +奔 2228 +奕 2229 +奖 2230 +套 2231 +奘 2232 +奚 2233 +奝 2234 +奠 2235 +奢 2236 +奣 2237 +奥 2238 +奫 2239 +奭 2240 +女 2241 +奴 2242 +奶 2243 +奸 2244 +她 2245 +好 2246 +妁 2247 +如 2248 +妃 2249 +妄 2250 +妆 2251 +妇 2252 +妈 2253 +妊 2254 +妍 2255 +妒 2256 +妓 2257 +妖 2258 +妗 2259 +妙 2260 +妞 2261 +妡 2262 +妣 2263 +妤 2264 +妥 2265 +妧 2266 +妨 2267 +妩 2268 +妪 2269 +妫 2270 +妮 2271 +妯 2272 +妱 2273 +妲 2274 +妹 2275 +妻 2276 +妾 2277 +姆 2278 +姊 2279 +始 2280 +姌 2281 +姐 2282 +姑 2283 +姒 2284 +姓 2285 +委 2286 +姗 2287 +姘 2288 +姚 2289 +姜 2290 +姝 2291 +姞 2292 +姣 2293 +姥 2294 +姨 2295 +姬 2296 +姮 2297 +姹 2298 +姻 2299 +姿 2300 +威 2301 +娃 2302 +娄 2303 +娅 2304 +娆 2305 +娇 2306 +娈 2307 +娉 2308 +娌 2309 +娑 2310 +娓 2311 +娘 2312 +娜 2313 +娟 2314 +娠 2315 +娣 2316 +娥 2317 +娩 2318 +娱 2319 +娲 2320 +娴 2321 +娵 2322 +娶 2323 +娼 2324 +婀 2325 +婄 2326 +婆 2327 +婉 2328 +婊 2329 +婕 2330 +婖 2331 +婚 2332 +婢 2333 +婧 2334 +婪 2335 +婴 2336 +婵 2337 +婶 2338 +婷 2339 +婺 2340 +婿 2341 +媒 2342 +媖 2343 +媚 2344 +媛 2345 +媞 2346 +媪 2347 +媲 2348 +媳 2349 +媵 2350 +媾 2351 +嫁 2352 +嫂 2353 +嫉 2354 +嫌 2355 +嫒 2356 +嫔 2357 +嫖 2358 +嫘 2359 +嫚 2360 +嫡 2361 +嫣 2362 +嫦 2363 +嫩 2364 +嫫 2365 +嫯 2366 +嫱 2367 +嫲 2368 +嬅 2369 +嬉 2370 +嬖 2371 +嬗 2372 +嬛 2373 +嬜 2374 +嬲 2375 +嬴 2376 +嬷 2377 +孀 2378 +子 2379 +孑 2380 +孓 2381 +孔 2382 +孕 2383 +孖 2384 +字 2385 +存 2386 +孙 2387 +孚 2388 +孛 2389 +孜 2390 +孝 2391 +孟 2392 +孢 2393 +季 2394 +孤 2395 +孥 2396 +学 2397 +孩 2398 +孪 2399 +孬 2400 +孰 2401 +孱 2402 +孳 2403 +孵 2404 +孺 2405 +孽 2406 +宁 2407 +它 2408 +宅 2409 +宇 2410 +守 2411 +安 2412 +宋 2413 +完 2414 +宍 2415 +宏 2416 +宓 2417 +宕 2418 +宗 2419 +官 2420 +宙 2421 +定 2422 +宛 2423 +宜 2424 +宝 2425 +实 2426 +実 2427 +宠 2428 +审 2429 +客 2430 +宣 2431 +室 2432 +宥 2433 +宦 2434 +宪 2435 +宫 2436 +宰 2437 +害 2438 +宴 2439 +宵 2440 +家 2441 +宸 2442 +容 2443 +宽 2444 +宾 2445 +宿 2446 +寀 2447 +寂 2448 +寄 2449 +寅 2450 +密 2451 +寇 2452 +富 2453 +寐 2454 +寒 2455 +寓 2456 +寔 2457 +寘 2458 +寛 2459 +寝 2460 +寞 2461 +察 2462 +寡 2463 +寤 2464 +寥 2465 +寨 2466 +寮 2467 +寰 2468 +寸 2469 +对 2470 +寺 2471 +寻 2472 +导 2473 +寿 2474 +封 2475 +専 2476 +射 2477 +尅 2478 +将 2479 +尉 2480 +尊 2481 +小 2482 +少 2483 +尓 2484 +尔 2485 +尕 2486 +尖 2487 +尘 2488 +尚 2489 +尝 2490 +尟 2491 +尤 2492 +尥 2493 +尧 2494 +尨 2495 +尬 2496 +就 2497 +尴 2498 +尸 2499 +尹 2500 +尺 2501 +尻 2502 +尼 2503 +尽 2504 +尾 2505 +尿 2506 +局 2507 +屁 2508 +层 2509 +居 2510 +屈 2511 +屉 2512 +届 2513 +屋 2514 +屌 2515 +屎 2516 +屏 2517 +屐 2518 +屑 2519 +展 2520 +属 2521 +屠 2522 +屡 2523 +履 2524 +屮 2525 +屯 2526 +山 2527 +屹 2528 +屺 2529 +屿 2530 +岀 2531 +岁 2532 +岂 2533 +岈 2534 +岌 2535 +岍 2536 +岐 2537 +岑 2538 +岔 2539 +岖 2540 +岗 2541 +岘 2542 +岚 2543 +岛 2544 +岢 2545 +岩 2546 +岫 2547 +岬 2548 +岭 2549 +岱 2550 +岳 2551 +岵 2552 +岷 2553 +岸 2554 +岿 2555 +峁 2556 +峄 2557 +峋 2558 +峒 2559 +峙 2560 +峡 2561 +峤 2562 +峥 2563 +峦 2564 +峨 2565 +峩 2566 +峪 2567 +峭 2568 +峯 2569 +峰 2570 +峻 2571 +崀 2572 +崁 2573 +崂 2574 +崃 2575 +崆 2576 +崇 2577 +崎 2578 +崐 2579 +崒 2580 +崔 2581 +崖 2582 +崛 2583 +崞 2584 +崤 2585 +崦 2586 +崧 2587 +崩 2588 +崭 2589 +崮 2590 +崴 2591 +崽 2592 +嵂 2593 +嵇 2594 +嵊 2595 +嵋 2596 +嵌 2597 +嵎 2598 +嵖 2599 +嵘 2600 +嵛 2601 +嵨 2602 +嵩 2603 +嵬 2604 +嵯 2605 +嵴 2606 +嶂 2607 +嶃 2608 +嶋 2609 +嶓 2610 +嶙 2611 +嶝 2612 +嶷 2613 +巅 2614 +巍 2615 +川 2616 +州 2617 +巡 2618 +巢 2619 +工 2620 +左 2621 +巧 2622 +巨 2623 +巩 2624 +巫 2625 +差 2626 +巯 2627 +己 2628 +已 2629 +巳 2630 +巴 2631 +巷 2632 +巻 2633 +巽 2634 +巾 2635 +巿 2636 +帀 2637 +币 2638 +市 2639 +布 2640 +帅 2641 +帆 2642 +师 2643 +希 2644 +帏 2645 +帐 2646 +帔 2647 +帕 2648 +帖 2649 +帘 2650 +帙 2651 +帚 2652 +帛 2653 +帜 2654 +帝 2655 +带 2656 +帧 2657 +席 2658 +帮 2659 +帷 2660 +常 2661 +帻 2662 +帼 2663 +帽 2664 +幂 2665 +幄 2666 +幅 2667 +幌 2668 +幔 2669 +幕 2670 +幛 2671 +幡 2672 +幢 2673 +干 2674 +平 2675 +年 2676 +并 2677 +幸 2678 +幺 2679 +幻 2680 +幼 2681 +幽 2682 +广 2683 +庄 2684 +庆 2685 +庇 2686 +床 2687 +序 2688 +庐 2689 +庑 2690 +库 2691 +应 2692 +底 2693 +庖 2694 +店 2695 +庙 2696 +庚 2697 +府 2698 +庞 2699 +废 2700 +庠 2701 +度 2702 +座 2703 +庭 2704 +庵 2705 +庶 2706 +康 2707 +庸 2708 +庹 2709 +庾 2710 +廆 2711 +廉 2712 +廊 2713 +廋 2714 +廓 2715 +廖 2716 +廛 2717 +廪 2718 +延 2719 +廷 2720 +建 2721 +廾 2722 +廿 2723 +开 2724 +弁 2725 +异 2726 +弃 2727 +弄 2728 +弇 2729 +弈 2730 +弊 2731 +弋 2732 +式 2733 +弐 2734 +弑 2735 +弓 2736 +引 2737 +弗 2738 +弘 2739 +弛 2740 +弟 2741 +张 2742 +弢 2743 +弥 2744 +弦 2745 +弧 2746 +弩 2747 +弭 2748 +弯 2749 +弱 2750 +弶 2751 +弹 2752 +强 2753 +弼 2754 +弾 2755 +彀 2756 +归 2757 +当 2758 +录 2759 +彗 2760 +彘 2761 +彝 2762 +形 2763 +彤 2764 +彦 2765 +彧 2766 +彩 2767 +彪 2768 +彬 2769 +彭 2770 +彰 2771 +影 2772 +彳 2773 +彷 2774 +役 2775 +彻 2776 +彼 2777 +往 2778 +征 2779 +徂 2780 +径 2781 +待 2782 +徇 2783 +很 2784 +徉 2785 +徊 2786 +律 2787 +徐 2788 +徒 2789 +徕 2790 +得 2791 +徘 2792 +徙 2793 +徜 2794 +御 2795 +徨 2796 +循 2797 +徭 2798 +微 2799 +徳 2800 +德 2801 +徽 2802 +心 2803 +忄 2804 +必 2805 +忆 2806 +忌 2807 +忍 2808 +忏 2809 +忐 2810 +忑 2811 +忒 2812 +忖 2813 +志 2814 +忘 2815 +忙 2816 +忝 2817 +忠 2818 +忡 2819 +忤 2820 +忧 2821 +忪 2822 +快 2823 +忱 2824 +忳 2825 +念 2826 +忻 2827 +忽 2828 +忾 2829 +忿 2830 +怀 2831 +态 2832 +怂 2833 +怄 2834 +怅 2835 +怆 2836 +怍 2837 +怎 2838 +怏 2839 +怒 2840 +怔 2841 +怕 2842 +怖 2843 +怙 2844 +怛 2845 +怜 2846 +思 2847 +怠 2848 +怡 2849 +急 2850 +怦 2851 +性 2852 +怨 2853 +怩 2854 +怪 2855 +怫 2856 +怯 2857 +怵 2858 +总 2859 +怼 2860 +怿 2861 +恁 2862 +恂 2863 +恃 2864 +恋 2865 +恍 2866 +恐 2867 +恒 2868 +恕 2869 +恙 2870 +恚 2871 +恢 2872 +恣 2873 +恤 2874 +恨 2875 +恩 2876 +恪 2877 +恫 2878 +恬 2879 +恭 2880 +息 2881 +恰 2882 +恳 2883 +恵 2884 +恶 2885 +恸 2886 +恹 2887 +恺 2888 +恻 2889 +恼 2890 +恽 2891 +恿 2892 +悄 2893 +悆 2894 +悉 2895 +悌 2896 +悍 2897 +悔 2898 +悕 2899 +悖 2900 +悚 2901 +悛 2902 +悝 2903 +悟 2904 +悠 2905 +患 2906 +悦 2907 +您 2908 +悫 2909 +悬 2910 +悭 2911 +悯 2912 +悱 2913 +悲 2914 +悳 2915 +悴 2916 +悸 2917 +悻 2918 +悼 2919 +情 2920 +惆 2921 +惇 2922 +惊 2923 +惋 2924 +惎 2925 +惑 2926 +惕 2927 +惘 2928 +惚 2929 +惜 2930 +惟 2931 +惠 2932 +惢 2933 +惦 2934 +惧 2935 +惨 2936 +惩 2937 +惪 2938 +惫 2939 +惬 2940 +惭 2941 +惮 2942 +惯 2943 +惰 2944 +想 2945 +惴 2946 +惶 2947 +惹 2948 +惺 2949 +愀 2950 +愁 2951 +愃 2952 +愆 2953 +愈 2954 +愉 2955 +愍 2956 +愎 2957 +意 2958 +愔 2959 +愕 2960 +愘 2961 +愚 2962 +感 2963 +愠 2964 +愣 2965 +愤 2966 +愧 2967 +愫 2968 +愿 2969 +慈 2970 +慉 2971 +慌 2972 +慎 2973 +慑 2974 +慒 2975 +慕 2976 +慜 2977 +慝 2978 +慢 2979 +慥 2980 +慧 2981 +慨 2982 +慰 2983 +慵 2984 +慷 2985 +憀 2986 +憋 2987 +憍 2988 +憎 2989 +憔 2990 +憙 2991 +憧 2992 +憨 2993 +憩 2994 +憬 2995 +憷 2996 +憾 2997 +懂 2998 +懈 2999 +懊 3000 +懋 3001 +懑 3002 +懒 3003 +懔 3004 +懦 3005 +懵 3006 +懿 3007 +戆 3008 +戈 3009 +戊 3010 +戌 3011 +戍 3012 +戎 3013 +戏 3014 +成 3015 +我 3016 +戒 3017 +戕 3018 +或 3019 +戗 3020 +战 3021 +戚 3022 +戛 3023 +戟 3024 +戡 3025 +戢 3026 +截 3027 +戬 3028 +戮 3029 +戳 3030 +戴 3031 +户 3032 +戾 3033 +房 3034 +所 3035 +扁 3036 +扃 3037 +扇 3038 +扈 3039 +扉 3040 +手 3041 +才 3042 +扎 3043 +扑 3044 +扒 3045 +打 3046 +扔 3047 +托 3048 +扛 3049 +扣 3050 +扥 3051 +扦 3052 +执 3053 +扩 3054 +扪 3055 +扫 3056 +扬 3057 +扭 3058 +扮 3059 +扯 3060 +扰 3061 +扳 3062 +扶 3063 +批 3064 +扼 3065 +扽 3066 +找 3067 +承 3068 +技 3069 +抄 3070 +抉 3071 +把 3072 +抑 3073 +抒 3074 +抓 3075 +抔 3076 +投 3077 +抖 3078 +抗 3079 +折 3080 +抚 3081 +抛 3082 +抟 3083 +抠 3084 +抡 3085 +抢 3086 +护 3087 +报 3088 +抨 3089 +披 3090 +抬 3091 +抱 3092 +抵 3093 +抹 3094 +抻 3095 +押 3096 +抽 3097 +抿 3098 +拂 3099 +拄 3100 +担 3101 +拆 3102 +拇 3103 +拈 3104 +拉 3105 +拊 3106 +拌 3107 +拍 3108 +拎 3109 +拐 3110 +拒 3111 +拓 3112 +拔 3113 +拖 3114 +拗 3115 +拘 3116 +拙 3117 +拚 3118 +招 3119 +拜 3120 +拟 3121 +拢 3122 +拣 3123 +拥 3124 +拦 3125 +拧 3126 +拨 3127 +择 3128 +括 3129 +拭 3130 +拮 3131 +拯 3132 +拱 3133 +拳 3134 +拴 3135 +拷 3136 +拼 3137 +拽 3138 +拾 3139 +拿 3140 +持 3141 +挂 3142 +指 3143 +挈 3144 +按 3145 +挎 3146 +挑 3147 +挖 3148 +挚 3149 +挛 3150 +挝 3151 +挞 3152 +挟 3153 +挠 3154 +挡 3155 +挢 3156 +挣 3157 +挤 3158 +挥 3159 +挨 3160 +挪 3161 +挫 3162 +振 3163 +挲 3164 +挹 3165 +挺 3166 +挽 3167 +捂 3168 +捅 3169 +捆 3170 +捉 3171 +捋 3172 +捌 3173 +捍 3174 +捎 3175 +捏 3176 +捐 3177 +捕 3178 +捞 3179 +损 3180 +捡 3181 +换 3182 +捣 3183 +捧 3184 +捩 3185 +捭 3186 +据 3187 +捯 3188 +捱 3189 +捴 3190 +捶 3191 +捷 3192 +捺 3193 +捻 3194 +掀 3195 +掂 3196 +掇 3197 +授 3198 +掉 3199 +掌 3200 +掏 3201 +掐 3202 +排 3203 +掖 3204 +掘 3205 +掞 3206 +掠 3207 +探 3208 +掣 3209 +接 3210 +控 3211 +推 3212 +掩 3213 +措 3214 +掫 3215 +掬 3216 +掮 3217 +掰 3218 +掳 3219 +掴 3220 +掷 3221 +掸 3222 +掺 3223 +掼 3224 +掾 3225 +掿 3226 +揄 3227 +揆 3228 +揉 3229 +揍 3230 +描 3231 +提 3232 +插 3233 +揖 3234 +揠 3235 +握 3236 +揣 3237 +揦 3238 +揩 3239 +揪 3240 +揭 3241 +援 3242 +揶 3243 +揷 3244 +揸 3245 +揺 3246 +揽 3247 +搀 3248 +搁 3249 +搂 3250 +搅 3251 +搏 3252 +搐 3253 +搓 3254 +搔 3255 +搜 3256 +搞 3257 +搡 3258 +搢 3259 +搧 3260 +搪 3261 +搬 3262 +搭 3263 +搴 3264 +搵 3265 +携 3266 +搽 3267 +摁 3268 +摄 3269 +摅 3270 +摆 3271 +摇 3272 +摈 3273 +摊 3274 +摒 3275 +摔 3276 +摘 3277 +摛 3278 +摞 3279 +摧 3280 +摩 3281 +摭 3282 +摸 3283 +摹 3284 +摽 3285 +撂 3286 +撃 3287 +撅 3288 +撇 3289 +撑 3290 +撒 3291 +撕 3292 +撘 3293 +撞 3294 +撤 3295 +撩 3296 +撬 3297 +播 3298 +撮 3299 +撰 3300 +撵 3301 +撷 3302 +撸 3303 +撺 3304 +撼 3305 +擀 3306 +擂 3307 +擅 3308 +操 3309 +擎 3310 +擒 3311 +擘 3312 +擞 3313 +擢 3314 +擤 3315 +擦 3316 +攀 3317 +攒 3318 +攘 3319 +攥 3320 +攫 3321 +支 3322 +收 3323 +攸 3324 +改 3325 +攻 3326 +放 3327 +政 3328 +故 3329 +效 3330 +敌 3331 +敎 3332 +敏 3333 +救 3334 +敕 3335 +敖 3336 +教 3337 +敛 3338 +敝 3339 +敞 3340 +敢 3341 +散 3342 +敦 3343 +敬 3344 +数 3345 +敲 3346 +整 3347 +敷 3348 +文 3349 +斋 3350 +斌 3351 +斐 3352 +斑 3353 +斓 3354 +斗 3355 +料 3356 +斛 3357 +斜 3358 +斝 3359 +斟 3360 +斡 3361 +斤 3362 +斥 3363 +斧 3364 +斩 3365 +斫 3366 +断 3367 +斯 3368 +新 3369 +斱 3370 +方 3371 +於 3372 +施 3373 +旁 3374 +旃 3375 +旄 3376 +旅 3377 +旆 3378 +旋 3379 +旌 3380 +旎 3381 +族 3382 +旒 3383 +旖 3384 +旗 3385 +无 3386 +既 3387 +日 3388 +旦 3389 +旧 3390 +旨 3391 +早 3392 +旬 3393 +旭 3394 +旮 3395 +旯 3396 +旰 3397 +旱 3398 +旳 3399 +旴 3400 +时 3401 +旷 3402 +旺 3403 +旻 3404 +旼 3405 +昀 3406 +昂 3407 +昃 3408 +昆 3409 +昉 3410 +昊 3411 +昌 3412 +明 3413 +昏 3414 +易 3415 +昔 3416 +昕 3417 +昙 3418 +昝 3419 +星 3420 +映 3421 +春 3422 +昧 3423 +昨 3424 +昭 3425 +是 3426 +昱 3427 +昴 3428 +昵 3429 +昶 3430 +昺 3431 +昼 3432 +显 3433 +晁 3434 +晃 3435 +晊 3436 +晋 3437 +晌 3438 +晏 3439 +晒 3440 +晓 3441 +晔 3442 +晕 3443 +晖 3444 +晗 3445 +晙 3446 +晚 3447 +晞 3448 +晟 3449 +晤 3450 +晦 3451 +晨 3452 +普 3453 +景 3454 +晰 3455 +晳 3456 +晴 3457 +晶 3458 +晷 3459 +智 3460 +晻 3461 +晾 3462 +暂 3463 +暄 3464 +暅 3465 +暇 3466 +暌 3467 +暎 3468 +暑 3469 +暕 3470 +暖 3471 +暗 3472 +暝 3473 +暦 3474 +暧 3475 +暨 3476 +暮 3477 +暴 3478 +暹 3479 +暼 3480 +暾 3481 +曈 3482 +曙 3483 +曜 3484 +曝 3485 +曦 3486 +曩 3487 +曰 3488 +曲 3489 +曳 3490 +更 3491 +曷 3492 +曹 3493 +曼 3494 +曾 3495 +替 3496 +最 3497 +月 3498 +有 3499 +朊 3500 +朋 3501 +服 3502 +朐 3503 +朔 3504 +朕 3505 +朗 3506 +望 3507 +朝 3508 +期 3509 +朦 3510 +木 3511 +未 3512 +末 3513 +本 3514 +札 3515 +术 3516 +朱 3517 +朴 3518 +朵 3519 +机 3520 +朽 3521 +杀 3522 +杂 3523 +权 3524 +杆 3525 +杈 3526 +杉 3527 +杌 3528 +李 3529 +杏 3530 +材 3531 +村 3532 +杓 3533 +杖 3534 +杙 3535 +杜 3536 +杝 3537 +杞 3538 +束 3539 +杠 3540 +条 3541 +来 3542 +杨 3543 +杪 3544 +杬 3545 +杭 3546 +杯 3547 +杰 3548 +杲 3549 +杳 3550 +杵 3551 +杷 3552 +杻 3553 +杼 3554 +松 3555 +板 3556 +极 3557 +构 3558 +枇 3559 +枉 3560 +枋 3561 +析 3562 +枕 3563 +林 3564 +枚 3565 +果 3566 +枝 3567 +枞 3568 +枢 3569 +枣 3570 +枥 3571 +枧 3572 +枨 3573 +枪 3574 +枫 3575 +枭 3576 +枯 3577 +枰 3578 +枱 3579 +枲 3580 +枳 3581 +架 3582 +枷 3583 +枸 3584 +枹 3585 +柃 3586 +柄 3587 +柈 3588 +柊 3589 +柏 3590 +某 3591 +柑 3592 +柒 3593 +染 3594 +柔 3595 +柘 3596 +柚 3597 +柜 3598 +柝 3599 +柞 3600 +柟 3601 +柠 3602 +柢 3603 +柤 3604 +查 3605 +柩 3606 +柬 3607 +柯 3608 +柰 3609 +柱 3610 +柳 3611 +柴 3612 +柷 3613 +柸 3614 +査 3615 +柽 3616 +柿 3617 +栀 3618 +栃 3619 +栄 3620 +栅 3621 +标 3622 +栈 3623 +栉 3624 +栊 3625 +栋 3626 +栌 3627 +栎 3628 +栏 3629 +树 3630 +栓 3631 +栖 3632 +栗 3633 +校 3634 +栢 3635 +栩 3636 +株 3637 +栱 3638 +栲 3639 +栳 3640 +样 3641 +核 3642 +根 3643 +栻 3644 +格 3645 +栽 3646 +栾 3647 +桀 3648 +桁 3649 +桂 3650 +桃 3651 +桄 3652 +桅 3653 +框 3654 +案 3655 +桉 3656 +桌 3657 +桎 3658 +桐 3659 +桑 3660 +桓 3661 +桔 3662 +桕 3663 +桜 3664 +桝 3665 +桠 3666 +桡 3667 +桢 3668 +档 3669 +桤 3670 +桥 3671 +桦 3672 +桧 3673 +桨 3674 +桩 3675 +桫 3676 +桯 3677 +桴 3678 +桶 3679 +桷 3680 +桼 3681 +梁 3682 +梅 3683 +梆 3684 +梏 3685 +梓 3686 +梗 3687 +梢 3688 +梣 3689 +梦 3690 +梧 3691 +梨 3692 +梭 3693 +梯 3694 +械 3695 +梳 3696 +梵 3697 +梶 3698 +梼 3699 +梾 3700 +检 3701 +棁 3702 +棂 3703 +棉 3704 +棋 3705 +棍 3706 +棒 3707 +棕 3708 +棘 3709 +棚 3710 +棠 3711 +棣 3712 +棨 3713 +森 3714 +棱 3715 +棵 3716 +棹 3717 +棺 3718 +棻 3719 +棼 3720 +椁 3721 +椅 3722 +椋 3723 +植 3724 +椎 3725 +椐 3726 +椒 3727 +検 3728 +椟 3729 +椤 3730 +椭 3731 +椰 3732 +椴 3733 +椵 3734 +椹 3735 +椽 3736 +椿 3737 +楀 3738 +楂 3739 +楔 3740 +楙 3741 +楚 3742 +楝 3743 +楞 3744 +楠 3745 +楢 3746 +楣 3747 +楦 3748 +楫 3749 +楮 3750 +楯 3751 +楷 3752 +楸 3753 +楹 3754 +楼 3755 +概 3756 +榄 3757 +榆 3758 +榈 3759 +榉 3760 +榑 3761 +榔 3762 +榕 3763 +榖 3764 +榘 3765 +榙 3766 +榛 3767 +榜 3768 +榧 3769 +榨 3770 +榫 3771 +榭 3772 +榴 3773 +榷 3774 +榻 3775 +榼 3776 +槁 3777 +槊 3778 +槌 3779 +槎 3780 +槐 3781 +様 3782 +槚 3783 +槛 3784 +槟 3785 +槩 3786 +槭 3787 +槱 3788 +槲 3789 +槻 3790 +槽 3791 +槿 3792 +樉 3793 +樊 3794 +樋 3795 +樗 3796 +樟 3797 +模 3798 +樨 3799 +権 3800 +横 3801 +樯 3802 +樱 3803 +樵 3804 +樽 3805 +樾 3806 +橄 3807 +橇 3808 +橐 3809 +橘 3810 +橙 3811 +橛 3812 +橡 3813 +橥 3814 +橱 3815 +橹 3816 +橼 3817 +檀 3818 +檄 3819 +檎 3820 +檐 3821 +檗 3822 +檦 3823 +檫 3824 +檬 3825 +檵 3826 +櫾 3827 +欉 3828 +欠 3829 +次 3830 +欢 3831 +欣 3832 +欤 3833 +欧 3834 +欲 3835 +欹 3836 +欺 3837 +欻 3838 +款 3839 +歃 3840 +歆 3841 +歇 3842 +歉 3843 +歌 3844 +歔 3845 +歙 3846 +止 3847 +正 3848 +此 3849 +步 3850 +武 3851 +歧 3852 +歪 3853 +歯 3854 +歳 3855 +歴 3856 +歹 3857 +死 3858 +歼 3859 +殁 3860 +殂 3861 +殃 3862 +殄 3863 +殆 3864 +殇 3865 +殉 3866 +殊 3867 +残 3868 +殍 3869 +殒 3870 +殓 3871 +殖 3872 +殚 3873 +殛 3874 +殡 3875 +殴 3876 +段 3877 +殷 3878 +殿 3879 +毁 3880 +毂 3881 +毅 3882 +毋 3883 +母 3884 +每 3885 +毐 3886 +毒 3887 +毓 3888 +比 3889 +毕 3890 +毖 3891 +毗 3892 +毘 3893 +毙 3894 +毛 3895 +毡 3896 +毫 3897 +毯 3898 +毳 3899 +毹 3900 +毽 3901 +氅 3902 +氍 3903 +氏 3904 +氐 3905 +民 3906 +氓 3907 +气 3908 +氖 3909 +気 3910 +氘 3911 +氙 3912 +氚 3913 +氛 3914 +氟 3915 +氡 3916 +氢 3917 +氤 3918 +氦 3919 +氧 3920 +氨 3921 +氩 3922 +氪 3923 +氮 3924 +氯 3925 +氰 3926 +氲 3927 +水 3928 +永 3929 +氽 3930 +汀 3931 +汁 3932 +求 3933 +汆 3934 +汇 3935 +汉 3936 +汊 3937 +汏 3938 +汐 3939 +汕 3940 +汗 3941 +汛 3942 +汜 3943 +汝 3944 +汞 3945 +江 3946 +池 3947 +污 3948 +汤 3949 +汨 3950 +汩 3951 +汪 3952 +汰 3953 +汲 3954 +汴 3955 +汶 3956 +汹 3957 +汽 3958 +汾 3959 +沁 3960 +沂 3961 +沃 3962 +沄 3963 +沅 3964 +沆 3965 +沇 3966 +沈 3967 +沉 3968 +沌 3969 +沏 3970 +沐 3971 +沓 3972 +沔 3973 +沙 3974 +沛 3975 +沟 3976 +没 3977 +沢 3978 +沣 3979 +沤 3980 +沥 3981 +沦 3982 +沧 3983 +沪 3984 +沫 3985 +沭 3986 +沮 3987 +沱 3988 +河 3989 +沸 3990 +油 3991 +治 3992 +沼 3993 +沽 3994 +沾 3995 +沿 3996 +泄 3997 +泅 3998 +泆 3999 +泉 4000 +泊 4001 +泌 4002 +泐 4003 +泓 4004 +泔 4005 +法 4006 +泖 4007 +泗 4008 +泘 4009 +泚 4010 +泛 4011 +泞 4012 +泠 4013 +泡 4014 +波 4015 +泣 4016 +泥 4017 +注 4018 +泪 4019 +泫 4020 +泬 4021 +泮 4022 +泯 4023 +泰 4024 +泱 4025 +泳 4026 +泵 4027 +泷 4028 +泸 4029 +泺 4030 +泻 4031 +泼 4032 +泽 4033 +泾 4034 +洁 4035 +洄 4036 +洇 4037 +洋 4038 +洌 4039 +洎 4040 +洐 4041 +洑 4042 +洒 4043 +洗 4044 +洙 4045 +洛 4046 +洞 4047 +洣 4048 +津 4049 +洧 4050 +洪 4051 +洮 4052 +洱 4053 +洲 4054 +洵 4055 +洸 4056 +洹 4057 +洺 4058 +活 4059 +洼 4060 +洽 4061 +派 4062 +流 4063 +浃 4064 +浅 4065 +浆 4066 +浇 4067 +浈 4068 +浉 4069 +浊 4070 +测 4071 +济 4072 +浏 4073 +浐 4074 +浑 4075 +浒 4076 +浓 4077 +浔 4078 +浗 4079 +浙 4080 +浚 4081 +浛 4082 +浜 4083 +浞 4084 +浠 4085 +浣 4086 +浥 4087 +浦 4088 +浩 4089 +浪 4090 +浮 4091 +浯 4092 +浴 4093 +海 4094 +浸 4095 +涂 4096 +涅 4097 +消 4098 +涉 4099 +涌 4100 +涎 4101 +涑 4102 +涓 4103 +涔 4104 +涕 4105 +涘 4106 +涙 4107 +涛 4108 +涝 4109 +涞 4110 +涟 4111 +涠 4112 +涡 4113 +涣 4114 +涤 4115 +润 4116 +涧 4117 +涨 4118 +涩 4119 +涪 4120 +涬 4121 +涮 4122 +涯 4123 +液 4124 +涵 4125 +涸 4126 +涿 4127 +淀 4128 +淄 4129 +淅 4130 +淆 4131 +淇 4132 +淋 4133 +淌 4134 +淑 4135 +淖 4136 +淘 4137 +淙 4138 +淛 4139 +淝 4140 +淞 4141 +淠 4142 +淡 4143 +淤 4144 +淦 4145 +淫 4146 +淬 4147 +淮 4148 +淯 4149 +深 4150 +淳 4151 +混 4152 +淹 4153 +添 4154 +淼 4155 +渀 4156 +清 4157 +済 4158 +渊 4159 +渌 4160 +渍 4161 +渎 4162 +渐 4163 +渑 4164 +渔 4165 +渗 4166 +渚 4167 +渝 4168 +渠 4169 +渡 4170 +渣 4171 +渤 4172 +渥 4173 +温 4174 +渭 4175 +港 4176 +渲 4177 +渴 4178 +游 4179 +渺 4180 +湃 4181 +湄 4182 +湉 4183 +湍 4184 +湎 4185 +湓 4186 +湔 4187 +湖 4188 +湘 4189 +湛 4190 +湜 4191 +湝 4192 +湟 4193 +湫 4194 +湮 4195 +湳 4196 +湾 4197 +湿 4198 +満 4199 +溃 4200 +溅 4201 +溆 4202 +溉 4203 +溍 4204 +溏 4205 +源 4206 +溘 4207 +溜 4208 +溞 4209 +溟 4210 +溢 4211 +溥 4212 +溧 4213 +溪 4214 +溯 4215 +溱 4216 +溲 4217 +溴 4218 +溶 4219 +溺 4220 +溻 4221 +滁 4222 +滂 4223 +滇 4224 +滋 4225 +滏 4226 +滑 4227 +滓 4228 +滔 4229 +滕 4230 +滘 4231 +滚 4232 +滞 4233 +滟 4234 +满 4235 +滢 4236 +滤 4237 +滥 4238 +滦 4239 +滨 4240 +滩 4241 +滴 4242 +滹 4243 +滺 4244 +漂 4245 +漆 4246 +漉 4247 +漏 4248 +漓 4249 +演 4250 +漕 4251 +漠 4252 +漩 4253 +漪 4254 +漫 4255 +漭 4256 +漯 4257 +漱 4258 +漳 4259 +漼 4260 +漾 4261 +潆 4262 +潇 4263 +潋 4264 +潍 4265 +潘 4266 +潜 4267 +潞 4268 +潢 4269 +潦 4270 +潩 4271 +潭 4272 +潮 4273 +潲 4274 +潴 4275 +潸 4276 +潺 4277 +潼 4278 +澄 4279 +澈 4280 +澍 4281 +澎 4282 +澚 4283 +澜 4284 +澡 4285 +澥 4286 +澧 4287 +澪 4288 +澳 4289 +澴 4290 +澶 4291 +澹 4292 +激 4293 +濂 4294 +濉 4295 +濑 4296 +濒 4297 +濞 4298 +濠 4299 +濡 4300 +濩 4301 +濮 4302 +濯 4303 +瀍 4304 +瀑 4305 +瀚 4306 +瀛 4307 +瀣 4308 +瀬 4309 +瀼 4310 +灌 4311 +灏 4312 +灞 4313 +火 4314 +灭 4315 +灯 4316 +灰 4317 +灵 4318 +灶 4319 +灸 4320 +灼 4321 +灾 4322 +灿 4323 +炀 4324 +炁 4325 +炅 4326 +炆 4327 +炉 4328 +炊 4329 +炎 4330 +炒 4331 +炔 4332 +炕 4333 +炖 4334 +炘 4335 +炙 4336 +炜 4337 +炝 4338 +炟 4339 +炫 4340 +炬 4341 +炭 4342 +炮 4343 +炯 4344 +炳 4345 +炴 4346 +炷 4347 +炸 4348 +点 4349 +炼 4350 +炽 4351 +烀 4352 +烁 4353 +烂 4354 +烃 4355 +烈 4356 +烊 4357 +烔 4358 +烘 4359 +烙 4360 +烛 4361 +烜 4362 +烟 4363 +烤 4364 +烦 4365 +烧 4366 +烨 4367 +烩 4368 +烫 4369 +烬 4370 +热 4371 +烯 4372 +烷 4373 +烹 4374 +烺 4375 +烽 4376 +焉 4377 +焊 4378 +焐 4379 +焓 4380 +焕 4381 +焖 4382 +焗 4383 +焘 4384 +焙 4385 +焚 4386 +焜 4387 +焦 4388 +焮 4389 +焯 4390 +焰 4391 +焱 4392 +然 4393 +煅 4394 +煊 4395 +煌 4396 +煎 4397 +煜 4398 +煞 4399 +煤 4400 +煦 4401 +照 4402 +煨 4403 +煮 4404 +煲 4405 +煳 4406 +煸 4407 +煽 4408 +熄 4409 +熈 4410 +熊 4411 +熏 4412 +熔 4413 +熙 4414 +熟 4415 +熠 4416 +熨 4417 +熬 4418 +熵 4419 +熹 4420 +燃 4421 +燊 4422 +燎 4423 +燏 4424 +燕 4425 +燚 4426 +燠 4427 +燥 4428 +燧 4429 +燮 4430 +燹 4431 +爆 4432 +爪 4433 +爬 4434 +爰 4435 +爱 4436 +爵 4437 +父 4438 +爷 4439 +爸 4440 +爹 4441 +爻 4442 +爽 4443 +爿 4444 +牁 4445 +牂 4446 +片 4447 +版 4448 +牌 4449 +牍 4450 +牒 4451 +牕 4452 +牖 4453 +牙 4454 +牛 4455 +牝 4456 +牟 4457 +牡 4458 +牢 4459 +牦 4460 +牧 4461 +物 4462 +牯 4463 +牲 4464 +牵 4465 +特 4466 +牺 4467 +牻 4468 +牾 4469 +犀 4470 +犁 4471 +犄 4472 +犇 4473 +犊 4474 +犍 4475 +犒 4476 +犟 4477 +犬 4478 +犯 4479 +犲 4480 +状 4481 +犷 4482 +犸 4483 +犹 4484 +狂 4485 +狃 4486 +狄 4487 +狈 4488 +狍 4489 +狎 4490 +狐 4491 +狒 4492 +狗 4493 +狙 4494 +狝 4495 +狞 4496 +狠 4497 +狡 4498 +狩 4499 +独 4500 +狭 4501 +狮 4502 +狯 4503 +狰 4504 +狱 4505 +狲 4506 +狷 4507 +狸 4508 +狼 4509 +猁 4510 +猄 4511 +猇 4512 +猎 4513 +猕 4514 +猖 4515 +猗 4516 +猛 4517 +猜 4518 +猝 4519 +猞 4520 +猢 4521 +猥 4522 +猩 4523 +猪 4524 +猫 4525 +猬 4526 +献 4527 +猱 4528 +猲 4529 +猴 4530 +猷 4531 +猹 4532 +猾 4533 +猿 4534 +獐 4535 +獒 4536 +獗 4537 +獠 4538 +獣 4539 +獭 4540 +獴 4541 +獾 4542 +玄 4543 +率 4544 +玉 4545 +王 4546 +玎 4547 +玑 4548 +玖 4549 +玘 4550 +玚 4551 +玛 4552 +玟 4553 +玠 4554 +玢 4555 +玥 4556 +玦 4557 +玩 4558 +玫 4559 +玭 4560 +玮 4561 +环 4562 +现 4563 +玲 4564 +玳 4565 +玶 4566 +玷 4567 +玹 4568 +玺 4569 +玻 4570 +珀 4571 +珂 4572 +珅 4573 +珈 4574 +珉 4575 +珊 4576 +珍 4577 +珏 4578 +珐 4579 +珑 4580 +珙 4581 +珜 4582 +珞 4583 +珠 4584 +珣 4585 +珥 4586 +珧 4587 +珩 4588 +珪 4589 +班 4590 +珮 4591 +珰 4592 +珲 4593 +珺 4594 +珽 4595 +球 4596 +琅 4597 +理 4598 +琇 4599 +琉 4600 +琊 4601 +琌 4602 +琍 4603 +琎 4604 +琏 4605 +琐 4606 +琚 4607 +琛 4608 +琠 4609 +琢 4610 +琤 4611 +琥 4612 +琦 4613 +琨 4614 +琪 4615 +琬 4616 +琮 4617 +琯 4618 +琰 4619 +琲 4620 +琳 4621 +琴 4622 +琵 4623 +琶 4624 +琼 4625 +瑀 4626 +瑁 4627 +瑄 4628 +瑅 4629 +瑊 4630 +瑕 4631 +瑗 4632 +瑙 4633 +瑚 4634 +瑛 4635 +瑜 4636 +瑞 4637 +瑟 4638 +瑠 4639 +瑨 4640 +瑭 4641 +瑮 4642 +瑰 4643 +瑱 4644 +瑳 4645 +瑴 4646 +瑶 4647 +瑷 4648 +瑾 4649 +璀 4650 +璃 4651 +璆 4652 +璇 4653 +璈 4654 +璋 4655 +璎 4656 +璐 4657 +璘 4658 +璜 4659 +璞 4660 +璟 4661 +璠 4662 +璧 4663 +璨 4664 +璩 4665 +璹 4666 +瓒 4667 +瓘 4668 +瓛 4669 +瓜 4670 +瓠 4671 +瓢 4672 +瓣 4673 +瓤 4674 +瓦 4675 +瓮 4676 +瓯 4677 +瓴 4678 +瓶 4679 +瓷 4680 +瓿 4681 +甄 4682 +甑 4683 +甘 4684 +甙 4685 +甚 4686 +甜 4687 +生 4688 +甥 4689 +用 4690 +甩 4691 +甫 4692 +甬 4693 +甭 4694 +甯 4695 +田 4696 +由 4697 +甲 4698 +申 4699 +电 4700 +男 4701 +甸 4702 +町 4703 +画 4704 +甾 4705 +畅 4706 +畈 4707 +畊 4708 +畋 4709 +界 4710 +畏 4711 +畑 4712 +畔 4713 +留 4714 +畜 4715 +略 4716 +畦 4717 +番 4718 +畯 4719 +畲 4720 +畴 4721 +畸 4722 +畿 4723 +疃 4724 +疆 4725 +疋 4726 +疍 4727 +疎 4728 +疏 4729 +疑 4730 +疔 4731 +疖 4732 +疗 4733 +疙 4734 +疚 4735 +疝 4736 +疟 4737 +疠 4738 +疡 4739 +疣 4740 +疤 4741 +疥 4742 +疫 4743 +疬 4744 +疮 4745 +疯 4746 +疰 4747 +疱 4748 +疲 4749 +疳 4750 +疴 4751 +疵 4752 +疸 4753 +疹 4754 +疼 4755 +疽 4756 +疾 4757 +痂 4758 +病 4759 +症 4760 +痈 4761 +痉 4762 +痊 4763 +痍 4764 +痒 4765 +痔 4766 +痕 4767 +痘 4768 +痛 4769 +痞 4770 +痢 4771 +痣 4772 +痤 4773 +痦 4774 +痧 4775 +痨 4776 +痪 4777 +痫 4778 +痰 4779 +痱 4780 +痴 4781 +痹 4782 +痼 4783 +痿 4784 +瘀 4785 +瘁 4786 +瘅 4787 +瘆 4788 +瘗 4789 +瘘 4790 +瘙 4791 +瘟 4792 +瘠 4793 +瘢 4794 +瘤 4795 +瘥 4796 +瘦 4797 +瘩 4798 +瘪 4799 +瘫 4800 +瘰 4801 +瘳 4802 +瘴 4803 +瘸 4804 +瘼 4805 +瘾 4806 +瘿 4807 +癀 4808 +癌 4809 +癍 4810 +癔 4811 +癖 4812 +癜 4813 +癞 4814 +癣 4815 +癫 4816 +癯 4817 +癸 4818 +発 4819 +登 4820 +白 4821 +百 4822 +皂 4823 +的 4824 +皆 4825 +皇 4826 +皈 4827 +皋 4828 +皌 4829 +皎 4830 +皑 4831 +皓 4832 +皕 4833 +皖 4834 +皙 4835 +皝 4836 +皤 4837 +皮 4838 +皱 4839 +皲 4840 +皴 4841 +皿 4842 +盁 4843 +盂 4844 +盅 4845 +盆 4846 +盈 4847 +益 4848 +盍 4849 +盎 4850 +盏 4851 +盐 4852 +监 4853 +盒 4854 +盔 4855 +盖 4856 +盗 4857 +盘 4858 +盛 4859 +盟 4860 +盥 4861 +盩 4862 +目 4863 +盯 4864 +盱 4865 +盲 4866 +直 4867 +相 4868 +盹 4869 +盼 4870 +盾 4871 +省 4872 +眇 4873 +眈 4874 +眉 4875 +看 4876 +県 4877 +眙 4878 +眚 4879 +眛 4880 +真 4881 +眠 4882 +眦 4883 +眨 4884 +眩 4885 +眬 4886 +眯 4887 +眶 4888 +眷 4889 +眸 4890 +眺 4891 +眼 4892 +着 4893 +睁 4894 +睇 4895 +睐 4896 +睑 4897 +睖 4898 +睚 4899 +睛 4900 +睟 4901 +睡 4902 +睢 4903 +督 4904 +睥 4905 +睦 4906 +睨 4907 +睫 4908 +睬 4909 +睹 4910 +睽 4911 +睾 4912 +睿 4913 +瞀 4914 +瞄 4915 +瞅 4916 +瞋 4917 +瞌 4918 +瞎 4919 +瞑 4920 +瞒 4921 +瞟 4922 +瞠 4923 +瞥 4924 +瞧 4925 +瞩 4926 +瞪 4927 +瞫 4928 +瞬 4929 +瞭 4930 +瞰 4931 +瞳 4932 +瞻 4933 +瞽 4934 +瞿 4935 +矅 4936 +矍 4937 +矐 4938 +矗 4939 +矛 4940 +矜 4941 +矢 4942 +矣 4943 +知 4944 +矩 4945 +矫 4946 +矬 4947 +短 4948 +矮 4949 +矱 4950 +石 4951 +矶 4952 +矸 4953 +矽 4954 +矾 4955 +矿 4956 +砀 4957 +码 4958 +砂 4959 +砌 4960 +砍 4961 +砒 4962 +研 4963 +砖 4964 +砗 4965 +砚 4966 +砜 4967 +砝 4968 +砟 4969 +砣 4970 +砥 4971 +砧 4972 +砩 4973 +砫 4974 +砭 4975 +砰 4976 +破 4977 +砵 4978 +砷 4979 +砸 4980 +砺 4981 +砻 4982 +砼 4983 +砾 4984 +础 4985 +硅 4986 +硌 4987 +硎 4988 +硐 4989 +硒 4990 +硕 4991 +硖 4992 +硚 4993 +硝 4994 +硫 4995 +硬 4996 +确 4997 +硼 4998 +碇 4999 +碉 5000 +碌 5001 +碍 5002 +碎 5003 +碑 5004 +碓 5005 +碗 5006 +碘 5007 +碚 5008 +碛 5009 +碜 5010 +碟 5011 +碣 5012 +碥 5013 +碧 5014 +碰 5015 +碱 5016 +碲 5017 +碳 5018 +碴 5019 +碶 5020 +碾 5021 +磁 5022 +磅 5023 +磊 5024 +磋 5025 +磐 5026 +磔 5027 +磕 5028 +磡 5029 +磨 5030 +磬 5031 +磲 5032 +磴 5033 +磷 5034 +磺 5035 +磻 5036 +磾 5037 +礁 5038 +礴 5039 +示 5040 +礻 5041 +礼 5042 +礽 5043 +社 5044 +祀 5045 +祁 5046 +祆 5047 +祈 5048 +祉 5049 +祊 5050 +祋 5051 +祎 5052 +祏 5053 +祓 5054 +祖 5055 +祗 5056 +祚 5057 +祛 5058 +祜 5059 +祝 5060 +神 5061 +祟 5062 +祠 5063 +祢 5064 +祥 5065 +祧 5066 +票 5067 +祭 5068 +祯 5069 +祷 5070 +祸 5071 +祺 5072 +禀 5073 +禁 5074 +禄 5075 +禅 5076 +福 5077 +禑 5078 +禛 5079 +禧 5080 +禨 5081 +禩 5082 +禫 5083 +禳 5084 +禹 5085 +禺 5086 +离 5087 +禽 5088 +禾 5089 +秀 5090 +私 5091 +秃 5092 +秆 5093 +秉 5094 +秋 5095 +种 5096 +科 5097 +秒 5098 +秘 5099 +租 5100 +秣 5101 +秤 5102 +秦 5103 +秧 5104 +秩 5105 +秫 5106 +秭 5107 +积 5108 +称 5109 +秸 5110 +移 5111 +秽 5112 +稀 5113 +稃 5114 +程 5115 +稍 5116 +税 5117 +稔 5118 +稗 5119 +稙 5120 +稚 5121 +稞 5122 +稠 5123 +稣 5124 +稳 5125 +稷 5126 +稹 5127 +稺 5128 +稻 5129 +稼 5130 +稽 5131 +稿 5132 +穂 5133 +穆 5134 +穑 5135 +穗 5136 +穣 5137 +穰 5138 +穴 5139 +究 5140 +穷 5141 +穸 5142 +穹 5143 +空 5144 +穿 5145 +窀 5146 +突 5147 +窃 5148 +窄 5149 +窈 5150 +窍 5151 +窑 5152 +窒 5153 +窕 5154 +窖 5155 +窗 5156 +窘 5157 +窜 5158 +窝 5159 +窟 5160 +窠 5161 +窣 5162 +窥 5163 +窦 5164 +窨 5165 +窭 5166 +窸 5167 +窾 5168 +窿 5169 +立 5170 +竑 5171 +竖 5172 +站 5173 +竚 5174 +竞 5175 +竟 5176 +章 5177 +竣 5178 +童 5179 +竦 5180 +竭 5181 +端 5182 +竲 5183 +竹 5184 +竺 5185 +竽 5186 +竿 5187 +笃 5188 +笄 5189 +笆 5190 +笈 5191 +笋 5192 +笏 5193 +笑 5194 +笔 5195 +笕 5196 +笙 5197 +笛 5198 +笞 5199 +笠 5200 +笤 5201 +符 5202 +笨 5203 +笪 5204 +笫 5205 +第 5206 +笮 5207 +笳 5208 +笸 5209 +笹 5210 +笺 5211 +笼 5212 +筅 5213 +筇 5214 +等 5215 +筋 5216 +筌 5217 +筏 5218 +筐 5219 +筑 5220 +筒 5221 +答 5222 +策 5223 +筚 5224 +筛 5225 +筝 5226 +筠 5227 +筭 5228 +筮 5229 +筱 5230 +筲 5231 +筵 5232 +筷 5233 +筹 5234 +签 5235 +简 5236 +箅 5237 +箍 5238 +箐 5239 +箓 5240 +箔 5241 +箕 5242 +算 5243 +箜 5244 +管 5245 +箦 5246 +箧 5247 +箨 5248 +箩 5249 +箪 5250 +箫 5251 +箬 5252 +箭 5253 +箱 5254 +箴 5255 +箸 5256 +箾 5257 +篁 5258 +篆 5259 +篇 5260 +篌 5261 +篑 5262 +篓 5263 +篙 5264 +篚 5265 +篝 5266 +篡 5267 +篥 5268 +篦 5269 +篪 5270 +篮 5271 +篱 5272 +篷 5273 +篼 5274 +篾 5275 +簃 5276 +簇 5277 +簋 5278 +簌 5279 +簕 5280 +簟 5281 +簧 5282 +簪 5283 +簸 5284 +簿 5285 +籀 5286 +籁 5287 +籍 5288 +籓 5289 +米 5290 +籴 5291 +类 5292 +籼 5293 +籽 5294 +粉 5295 +粑 5296 +粒 5297 +粕 5298 +粗 5299 +粘 5300 +粝 5301 +粟 5302 +粢 5303 +粤 5304 +粥 5305 +粪 5306 +粮 5307 +粱 5308 +粲 5309 +粳 5310 +粹 5311 +粼 5312 +粽 5313 +精 5314 +粿 5315 +糁 5316 +糅 5317 +糊 5318 +糌 5319 +糍 5320 +糕 5321 +糖 5322 +糗 5323 +糙 5324 +糜 5325 +糟 5326 +糠 5327 +糯 5328 +糸 5329 +系 5330 +紊 5331 +紑 5332 +素 5333 +索 5334 +紧 5335 +紫 5336 +累 5337 +経 5338 +絜 5339 +絮 5340 +絺 5341 +綖 5342 +継 5343 +綦 5344 +緁 5345 +縠 5346 +縡 5347 +縻 5348 +繁 5349 +繄 5350 +繇 5351 +纁 5352 +纂 5353 +纛 5354 +纠 5355 +纡 5356 +红 5357 +纣 5358 +纤 5359 +纥 5360 +约 5361 +级 5362 +纨 5363 +纩 5364 +纪 5365 +纫 5366 +纬 5367 +纭 5368 +纮 5369 +纯 5370 +纰 5371 +纱 5372 +纲 5373 +纳 5374 +纵 5375 +纶 5376 +纷 5377 +纸 5378 +纹 5379 +纺 5380 +纻 5381 +纽 5382 +纾 5383 +线 5384 +绀 5385 +绂 5386 +练 5387 +组 5388 +绅 5389 +细 5390 +织 5391 +终 5392 +绉 5393 +绊 5394 +绋 5395 +绌 5396 +绍 5397 +绎 5398 +经 5399 +绐 5400 +绑 5401 +绒 5402 +结 5403 +绔 5404 +绕 5405 +绘 5406 +给 5407 +绚 5408 +绛 5409 +络 5410 +绝 5411 +绞 5412 +统 5413 +绡 5414 +绢 5415 +绣 5416 +绥 5417 +绦 5418 +继 5419 +绨 5420 +绩 5421 +绪 5422 +绫 5423 +续 5424 +绮 5425 +绯 5426 +绰 5427 +绳 5428 +维 5429 +绵 5430 +绶 5431 +绷 5432 +绸 5433 +绺 5434 +绻 5435 +综 5436 +绽 5437 +绾 5438 +绿 5439 +缀 5440 +缄 5441 +缅 5442 +缆 5443 +缇 5444 +缈 5445 +缉 5446 +缎 5447 +缑 5448 +缒 5449 +缓 5450 +缔 5451 +缕 5452 +编 5453 +缗 5454 +缘 5455 +缙 5456 +缚 5457 +缛 5458 +缜 5459 +缝 5460 +缟 5461 +缠 5462 +缢 5463 +缣 5464 +缤 5465 +缥 5466 +缦 5467 +缨 5468 +缩 5469 +缪 5470 +缫 5471 +缬 5472 +缭 5473 +缮 5474 +缯 5475 +缰 5476 +缱 5477 +缳 5478 +缴 5479 +缵 5480 +缶 5481 +缸 5482 +缺 5483 +罂 5484 +罃 5485 +罄 5486 +罅 5487 +罍 5488 +罐 5489 +网 5490 +罔 5491 +罕 5492 +罗 5493 +罘 5494 +罚 5495 +罡 5496 +罢 5497 +罩 5498 +罪 5499 +置 5500 +署 5501 +罹 5502 +罽 5503 +羁 5504 +羊 5505 +羌 5506 +美 5507 +羔 5508 +羚 5509 +羞 5510 +羟 5511 +羡 5512 +羣 5513 +群 5514 +羧 5515 +羯 5516 +羰 5517 +羱 5518 +羲 5519 +羸 5520 +羹 5521 +羽 5522 +羿 5523 +翁 5524 +翃 5525 +翅 5526 +翊 5527 +翌 5528 +翎 5529 +翔 5530 +翕 5531 +翘 5532 +翙 5533 +翚 5534 +翟 5535 +翠 5536 +翡 5537 +翥 5538 +翦 5539 +翩 5540 +翰 5541 +翱 5542 +翳 5543 +翻 5544 +翼 5545 +耀 5546 +老 5547 +考 5548 +耄 5549 +者 5550 +耆 5551 +耋 5552 +而 5553 +耍 5554 +耐 5555 +耒 5556 +耕 5557 +耗 5558 +耘 5559 +耙 5560 +耜 5561 +耦 5562 +耧 5563 +耨 5564 +耪 5565 +耳 5566 +耶 5567 +耷 5568 +耸 5569 +耻 5570 +耽 5571 +耿 5572 +聂 5573 +聃 5574 +聆 5575 +聊 5576 +聋 5577 +职 5578 +聒 5579 +联 5580 +聘 5581 +聚 5582 +聡 5583 +聩 5584 +聪 5585 +聿 5586 +肃 5587 +肄 5588 +肆 5589 +肇 5590 +肉 5591 +肋 5592 +肌 5593 +肓 5594 +肖 5595 +肘 5596 +肚 5597 +肛 5598 +肝 5599 +肟 5600 +肠 5601 +股 5602 +肢 5603 +肣 5604 +肤 5605 +肥 5606 +肩 5607 +肪 5608 +肮 5609 +肯 5610 +肱 5611 +育 5612 +肴 5613 +肸 5614 +肺 5615 +肼 5616 +肽 5617 +肾 5618 +肿 5619 +胀 5620 +胁 5621 +胃 5622 +胄 5623 +胆 5624 +背 5625 +胍 5626 +胎 5627 +胖 5628 +胗 5629 +胚 5630 +胛 5631 +胜 5632 +胝 5633 +胞 5634 +胡 5635 +胤 5636 +胥 5637 +胧 5638 +胪 5639 +胫 5640 +胭 5641 +胯 5642 +胰 5643 +胱 5644 +胳 5645 +胴 5646 +胶 5647 +胸 5648 +胺 5649 +胼 5650 +能 5651 +脁 5652 +脂 5653 +脆 5654 +脉 5655 +脊 5656 +脍 5657 +脏 5658 +脐 5659 +脑 5660 +脓 5661 +脖 5662 +脘 5663 +脚 5664 +脯 5665 +脰 5666 +脱 5667 +脲 5668 +脸 5669 +脾 5670 +腆 5671 +腈 5672 +腊 5673 +腋 5674 +腌 5675 +腐 5676 +腑 5677 +腓 5678 +腔 5679 +腕 5680 +腘 5681 +腚 5682 +腥 5683 +腧 5684 +腩 5685 +腭 5686 +腮 5687 +腰 5688 +腱 5689 +腴 5690 +腹 5691 +腺 5692 +腻 5693 +腼 5694 +腾 5695 +腿 5696 +膀 5697 +膈 5698 +膊 5699 +膏 5700 +膑 5701 +膘 5702 +膛 5703 +膜 5704 +膝 5705 +膦 5706 +膨 5707 +膳 5708 +膺 5709 +膻 5710 +臀 5711 +臂 5712 +臃 5713 +臆 5714 +臊 5715 +臜 5716 +臣 5717 +臧 5718 +自 5719 +臬 5720 +臭 5721 +至 5722 +致 5723 +臻 5724 +臼 5725 +臾 5726 +舀 5727 +舁 5728 +舂 5729 +舄 5730 +舅 5731 +舆 5732 +舌 5733 +舍 5734 +舐 5735 +舒 5736 +舔 5737 +舛 5738 +舜 5739 +舞 5740 +舟 5741 +舢 5742 +舣 5743 +航 5744 +舫 5745 +般 5746 +舰 5747 +舱 5748 +舵 5749 +舶 5750 +舷 5751 +舸 5752 +船 5753 +舺 5754 +舾 5755 +艄 5756 +艇 5757 +艉 5758 +艋 5759 +艘 5760 +艮 5761 +良 5762 +艰 5763 +色 5764 +艳 5765 +艹 5766 +艺 5767 +艽 5768 +艾 5769 +艿 5770 +节 5771 +芈 5772 +芊 5773 +芋 5774 +芍 5775 +芎 5776 +芑 5777 +芒 5778 +芔 5779 +芗 5780 +芘 5781 +芙 5782 +芜 5783 +芝 5784 +芟 5785 +芡 5786 +芥 5787 +芦 5788 +芨 5789 +芩 5790 +芪 5791 +芬 5792 +芭 5793 +芮 5794 +芯 5795 +芰 5796 +花 5797 +芳 5798 +芷 5799 +芸 5800 +芹 5801 +芽 5802 +芾 5803 +苁 5804 +苄 5805 +苇 5806 +苈 5807 +苋 5808 +苌 5809 +苍 5810 +苎 5811 +苏 5812 +苑 5813 +苒 5814 +苓 5815 +苔 5816 +苕 5817 +苗 5818 +苘 5819 +苛 5820 +苜 5821 +苞 5822 +苟 5823 +苡 5824 +苣 5825 +若 5826 +苦 5827 +苪 5828 +苫 5829 +苯 5830 +英 5831 +苳 5832 +苴 5833 +苷 5834 +苹 5835 +苻 5836 +苾 5837 +茀 5838 +茁 5839 +茂 5840 +范 5841 +茄 5842 +茅 5843 +茆 5844 +茈 5845 +茉 5846 +茌 5847 +茎 5848 +茏 5849 +茑 5850 +茔 5851 +茕 5852 +茗 5853 +茛 5854 +茜 5855 +茝 5856 +茧 5857 +茨 5858 +茫 5859 +茬 5860 +茭 5861 +茯 5862 +茱 5863 +茳 5864 +茴 5865 +茵 5866 +茶 5867 +茸 5868 +茹 5869 +茼 5870 +荀 5871 +荁 5872 +荃 5873 +荆 5874 +荇 5875 +草 5876 +荏 5877 +荐 5878 +荑 5879 +荒 5880 +荔 5881 +荚 5882 +荛 5883 +荜 5884 +荞 5885 +荟 5886 +荠 5887 +荡 5888 +荣 5889 +荤 5890 +荥 5891 +荦 5892 +荧 5893 +荨 5894 +荩 5895 +荪 5896 +荫 5897 +药 5898 +荷 5899 +荸 5900 +荻 5901 +荼 5902 +荽 5903 +莅 5904 +莆 5905 +莉 5906 +莎 5907 +莒 5908 +莓 5909 +莘 5910 +莙 5911 +莜 5912 +莞 5913 +莠 5914 +莦 5915 +莨 5916 +莩 5917 +莪 5918 +莫 5919 +莱 5920 +莲 5921 +莳 5922 +莴 5923 +获 5924 +莸 5925 +莹 5926 +莺 5927 +莼 5928 +莽 5929 +菀 5930 +菁 5931 +菅 5932 +菇 5933 +菈 5934 +菉 5935 +菊 5936 +菌 5937 +菏 5938 +菔 5939 +菖 5940 +菘 5941 +菜 5942 +菝 5943 +菟 5944 +菠 5945 +菡 5946 +菩 5947 +菪 5948 +菰 5949 +菱 5950 +菲 5951 +菽 5952 +萁 5953 +萃 5954 +萄 5955 +萋 5956 +萌 5957 +萍 5958 +萎 5959 +萘 5960 +萜 5961 +萝 5962 +萤 5963 +营 5964 +萦 5965 +萧 5966 +萨 5967 +萩 5968 +萱 5969 +萸 5970 +萼 5971 +落 5972 +葆 5973 +葎 5974 +著 5975 +葚 5976 +葛 5977 +葜 5978 +葡 5979 +董 5980 +葩 5981 +葫 5982 +葬 5983 +葭 5984 +葱 5985 +葳 5986 +葵 5987 +葶 5988 +葺 5989 +蒂 5990 +蒋 5991 +蒌 5992 +蒑 5993 +蒙 5994 +蒜 5995 +蒟 5996 +蒡 5997 +蒨 5998 +蒯 5999 +蒲 6000 +蒴 6001 +蒸 6002 +蒹 6003 +蒺 6004 +蒽 6005 +蒿 6006 +蓁 6007 +蓄 6008 +蓉 6009 +蓊 6010 +蓍 6011 +蓑 6012 +蓓 6013 +蓖 6014 +蓝 6015 +蓟 6016 +蓣 6017 +蓥 6018 +蓦 6019 +蓬 6020 +蓼 6021 +蓿 6022 +蔊 6023 +蔑 6024 +蔓 6025 +蔗 6026 +蔚 6027 +蔡 6028 +蔫 6029 +蔬 6030 +蔵 6031 +蔷 6032 +蔸 6033 +蔺 6034 +蔻 6035 +蔼 6036 +蔽 6037 +蕃 6038 +蕈 6039 +蕉 6040 +蕊 6041 +蕐 6042 +蕖 6043 +蕗 6044 +蕙 6045 +蕞 6046 +蕤 6047 +蕨 6048 +蕲 6049 +蕴 6050 +蕻 6051 +蕾 6052 +薁 6053 +薄 6054 +薅 6055 +薇 6056 +薏 6057 +薖 6058 +薛 6059 +薜 6060 +薨 6061 +薪 6062 +薮 6063 +薯 6064 +薰 6065 +薳 6066 +薷 6067 +薹 6068 +藁 6069 +藉 6070 +藏 6071 +藐 6072 +藓 6073 +藔 6074 +藕 6075 +藜 6076 +藠 6077 +藤 6078 +藨 6079 +藩 6080 +藳 6081 +藻 6082 +藿 6083 +蘅 6084 +蘑 6085 +蘖 6086 +蘡 6087 +蘧 6088 +蘸 6089 +蘼 6090 +虎 6091 +虏 6092 +虐 6093 +虑 6094 +虓 6095 +虔 6096 +虚 6097 +虞 6098 +虢 6099 +虫 6100 +虬 6101 +虱 6102 +虹 6103 +虺 6104 +虻 6105 +虽 6106 +虾 6107 +虿 6108 +蚀 6109 +蚁 6110 +蚂 6111 +蚊 6112 +蚋 6113 +蚌 6114 +蚓 6115 +蚕 6116 +蚜 6117 +蚝 6118 +蚡 6119 +蚣 6120 +蚤 6121 +蚧 6122 +蚨 6123 +蚩 6124 +蚪 6125 +蚬 6126 +蚯 6127 +蚱 6128 +蚴 6129 +蚵 6130 +蚶 6131 +蚺 6132 +蚿 6133 +蛀 6134 +蛄 6135 +蛆 6136 +蛇 6137 +蛉 6138 +蛊 6139 +蛋 6140 +蛎 6141 +蛏 6142 +蛐 6143 +蛔 6144 +蛙 6145 +蛛 6146 +蛞 6147 +蛟 6148 +蛤 6149 +蛭 6150 +蛮 6151 +蛰 6152 +蛱 6153 +蛲 6154 +蛳 6155 +蛸 6156 +蛹 6157 +蛾 6158 +蜀 6159 +蜂 6160 +蜃 6161 +蜇 6162 +蜈 6163 +蜉 6164 +蜊 6165 +蜍 6166 +蜑 6167 +蜒 6168 +蜓 6169 +蜕 6170 +蜗 6171 +蜘 6172 +蜚 6173 +蜛 6174 +蜜 6175 +蜡 6176 +蜢 6177 +蜣 6178 +蜥 6179 +蜮 6180 +蜱 6181 +蜴 6182 +蜷 6183 +蜻 6184 +蜿 6185 +蝇 6186 +蝈 6187 +蝉 6188 +蝌 6189 +蝎 6190 +蝓 6191 +蝗 6192 +蝙 6193 +蝠 6194 +蝣 6195 +蝥 6196 +蝮 6197 +蝰 6198 +蝴 6199 +蝶 6200 +蝼 6201 +蝽 6202 +蝾 6203 +螂 6204 +螃 6205 +螅 6206 +螈 6207 +螋 6208 +融 6209 +螟 6210 +螣 6211 +螨 6212 +螫 6213 +螭 6214 +螯 6215 +螳 6216 +螺 6217 +蟀 6218 +蟆 6219 +蟊 6220 +蟋 6221 +蟏 6222 +蟑 6223 +蟒 6224 +蟠 6225 +蟥 6226 +蟪 6227 +蟳 6228 +蟹 6229 +蟾 6230 +蠃 6231 +蠊 6232 +蠋 6233 +蠓 6234 +蠕 6235 +蠛 6236 +蠡 6237 +蠢 6238 +蠲 6239 +蠳 6240 +蠹 6241 +血 6242 +衄 6243 +衅 6244 +行 6245 +衍 6246 +衔 6247 +街 6248 +衙 6249 +衡 6250 +衢 6251 +衣 6252 +补 6253 +表 6254 +衩 6255 +衫 6256 +衬 6257 +衮 6258 +衰 6259 +衲 6260 +衷 6261 +衽 6262 +衾 6263 +衿 6264 +袁 6265 +袂 6266 +袄 6267 +袅 6268 +袆 6269 +袈 6270 +袋 6271 +袍 6272 +袒 6273 +袓 6274 +袖 6275 +袗 6276 +袜 6277 +袝 6278 +袢 6279 +袤 6280 +被 6281 +袭 6282 +袱 6283 +裁 6284 +裂 6285 +装 6286 +裆 6287 +裎 6288 +裔 6289 +裕 6290 +裘 6291 +裙 6292 +裛 6293 +裟 6294 +裢 6295 +裤 6296 +裨 6297 +裬 6298 +裱 6299 +裳 6300 +裴 6301 +裸 6302 +裹 6303 +裾 6304 +褂 6305 +褊 6306 +褐 6307 +褒 6308 +褓 6309 +褙 6310 +褚 6311 +褛 6312 +褡 6313 +褥 6314 +褦 6315 +褧 6316 +褪 6317 +褭 6318 +褴 6319 +褶 6320 +襀 6321 +襁 6322 +襃 6323 +襄 6324 +襞 6325 +襟 6326 +襦 6327 +襶 6328 +襻 6329 +西 6330 +要 6331 +覃 6332 +覆 6333 +覩 6334 +観 6335 +见 6336 +观 6337 +规 6338 +觅 6339 +视 6340 +觇 6341 +览 6342 +觉 6343 +觊 6344 +觌 6345 +觎 6346 +觐 6347 +觑 6348 +角 6349 +觚 6350 +觜 6351 +觞 6352 +解 6353 +觥 6354 +触 6355 +觧 6356 +觯 6357 +觱 6358 +言 6359 +訚 6360 +訾 6361 +詃 6362 +詥 6363 +詧 6364 +詹 6365 +誉 6366 +誊 6367 +誓 6368 +諲 6369 +謇 6370 +謩 6371 +警 6372 +譬 6373 +讃 6374 +讠 6375 +计 6376 +订 6377 +讣 6378 +认 6379 +讥 6380 +讦 6381 +讧 6382 +讨 6383 +让 6384 +讪 6385 +讫 6386 +训 6387 +议 6388 +讯 6389 +记 6390 +讲 6391 +讳 6392 +讴 6393 +讵 6394 +讶 6395 +讷 6396 +许 6397 +讹 6398 +论 6399 +讼 6400 +讽 6401 +设 6402 +访 6403 +诀 6404 +证 6405 +诂 6406 +诃 6407 +评 6408 +诅 6409 +识 6410 +诈 6411 +诉 6412 +诊 6413 +诋 6414 +诌 6415 +词 6416 +诏 6417 +诐 6418 +译 6419 +诒 6420 +诓 6421 +诔 6422 +试 6423 +诗 6424 +诘 6425 +诙 6426 +诚 6427 +诛 6428 +诜 6429 +话 6430 +诞 6431 +诟 6432 +诠 6433 +诡 6434 +询 6435 +诣 6436 +诤 6437 +该 6438 +详 6439 +诧 6440 +诨 6441 +诩 6442 +诫 6443 +诬 6444 +语 6445 +诮 6446 +误 6447 +诰 6448 +诱 6449 +诲 6450 +诳 6451 +说 6452 +诵 6453 +诶 6454 +请 6455 +诸 6456 +诹 6457 +诺 6458 +读 6459 +诽 6460 +课 6461 +诿 6462 +谀 6463 +谁 6464 +谂 6465 +调 6466 +谄 6467 +谅 6468 +谆 6469 +谈 6470 +谊 6471 +谋 6472 +谌 6473 +谍 6474 +谎 6475 +谏 6476 +谐 6477 +谑 6478 +谒 6479 +谓 6480 +谔 6481 +谕 6482 +谖 6483 +谗 6484 +谘 6485 +谙 6486 +谚 6487 +谛 6488 +谜 6489 +谝 6490 +谟 6491 +谠 6492 +谡 6493 +谢 6494 +谣 6495 +谤 6496 +谥 6497 +谦 6498 +谧 6499 +谨 6500 +谩 6501 +谪 6502 +谬 6503 +谭 6504 +谮 6505 +谯 6506 +谰 6507 +谱 6508 +谲 6509 +谳 6510 +谴 6511 +谵 6512 +谶 6513 +谷 6514 +豁 6515 +豆 6516 +豇 6517 +豉 6518 +豊 6519 +豌 6520 +豚 6521 +豝 6522 +象 6523 +豢 6524 +豨 6525 +豪 6526 +豫 6527 +豳 6528 +豷 6529 +豸 6530 +豹 6531 +豺 6532 +貂 6533 +貅 6534 +貉 6535 +貊 6536 +貌 6537 +貔 6538 +貕 6539 +貘 6540 +贝 6541 +贞 6542 +负 6543 +贡 6544 +财 6545 +责 6546 +贤 6547 +败 6548 +账 6549 +货 6550 +质 6551 +贩 6552 +贪 6553 +贫 6554 +贬 6555 +购 6556 +贮 6557 +贯 6558 +贰 6559 +贱 6560 +贲 6561 +贳 6562 +贴 6563 +贵 6564 +贶 6565 +贷 6566 +贸 6567 +费 6568 +贺 6569 +贻 6570 +贼 6571 +贽 6572 +贾 6573 +贿 6574 +赀 6575 +赁 6576 +赂 6577 +赃 6578 +资 6579 +赅 6580 +赈 6581 +赉 6582 +赊 6583 +赋 6584 +赌 6585 +赍 6586 +赎 6587 +赏 6588 +赐 6589 +赓 6590 +赔 6591 +赖 6592 +赘 6593 +赚 6594 +赛 6595 +赜 6596 +赝 6597 +赞 6598 +赟 6599 +赠 6600 +赡 6601 +赢 6602 +赣 6603 +赤 6604 +赦 6605 +赧 6606 +赫 6607 +赭 6608 +走 6609 +赳 6610 +赴 6611 +赵 6612 +赶 6613 +起 6614 +趁 6615 +趄 6616 +超 6617 +越 6618 +趋 6619 +趔 6620 +趟 6621 +趣 6622 +足 6623 +趴 6624 +趵 6625 +趸 6626 +趺 6627 +趾 6628 +趿 6629 +跃 6630 +跄 6631 +跆 6632 +跋 6633 +跌 6634 +跎 6635 +跑 6636 +跖 6637 +跗 6638 +跚 6639 +跛 6640 +距 6641 +跟 6642 +跣 6643 +跤 6644 +跨 6645 +跪 6646 +跬 6647 +路 6648 +跱 6649 +跳 6650 +践 6651 +跶 6652 +跷 6653 +跸 6654 +跹 6655 +跺 6656 +跻 6657 +跽 6658 +踅 6659 +踉 6660 +踊 6661 +踌 6662 +踏 6663 +踘 6664 +踝 6665 +踞 6666 +踟 6667 +踢 6668 +踩 6669 +踪 6670 +踬 6671 +踮 6672 +踯 6673 +踱 6674 +踵 6675 +踹 6676 +踺 6677 +踽 6678 +蹁 6679 +蹂 6680 +蹄 6681 +蹇 6682 +蹈 6683 +蹉 6684 +蹊 6685 +蹋 6686 +蹍 6687 +蹑 6688 +蹒 6689 +蹙 6690 +蹚 6691 +蹦 6692 +蹩 6693 +蹬 6694 +蹭 6695 +蹰 6696 +蹲 6697 +蹴 6698 +蹶 6699 +蹼 6700 +蹿 6701 +躁 6702 +躅 6703 +躇 6704 +躏 6705 +身 6706 +躬 6707 +躯 6708 +躲 6709 +躺 6710 +転 6711 +輀 6712 +轈 6713 +车 6714 +轧 6715 +轨 6716 +轩 6717 +轫 6718 +转 6719 +轭 6720 +轮 6721 +软 6722 +轰 6723 +轱 6724 +轲 6725 +轳 6726 +轴 6727 +轶 6728 +轸 6729 +轻 6730 +轼 6731 +载 6732 +轿 6733 +辂 6734 +较 6735 +辄 6736 +辅 6737 +辆 6738 +辇 6739 +辈 6740 +辉 6741 +辊 6742 +辋 6743 +辍 6744 +辎 6745 +辐 6746 +辑 6747 +输 6748 +辔 6749 +辕 6750 +辖 6751 +辗 6752 +辘 6753 +辙 6754 +辛 6755 +辜 6756 +辞 6757 +辟 6758 +辣 6759 +辨 6760 +辩 6761 +辫 6762 +辰 6763 +辱 6764 +边 6765 +辺 6766 +辻 6767 +込 6768 +辽 6769 +达 6770 +迁 6771 +迂 6772 +迄 6773 +迅 6774 +过 6775 +迈 6776 +迎 6777 +运 6778 +近 6779 +返 6780 +还 6781 +这 6782 +进 6783 +远 6784 +违 6785 +连 6786 +迟 6787 +迢 6788 +迤 6789 +迥 6790 +迦 6791 +迨 6792 +迩 6793 +迪 6794 +迫 6795 +迭 6796 +述 6797 +迳 6798 +迷 6799 +迸 6800 +迹 6801 +追 6802 +退 6803 +送 6804 +适 6805 +逃 6806 +逄 6807 +逅 6808 +逆 6809 +选 6810 +逊 6811 +逋 6812 +逍 6813 +透 6814 +逐 6815 +逑 6816 +递 6817 +逓 6818 +途 6819 +逖 6820 +逗 6821 +通 6822 +逛 6823 +逝 6824 +逞 6825 +速 6826 +造 6827 +逡 6828 +逢 6829 +逮 6830 +逯 6831 +逵 6832 +逶 6833 +逸 6834 +逻 6835 +逼 6836 +逾 6837 +遁 6838 +遂 6839 +遇 6840 +遍 6841 +遏 6842 +遐 6843 +遑 6844 +遒 6845 +道 6846 +遗 6847 +遘 6848 +遛 6849 +遢 6850 +遣 6851 +遥 6852 +遨 6853 +遭 6854 +遮 6855 +遰 6856 +遴 6857 +遵 6858 +遽 6859 +避 6860 +邀 6861 +邂 6862 +邃 6863 +邅 6864 +邈 6865 +邋 6866 +邑 6867 +邓 6868 +邕 6869 +邗 6870 +邘 6871 +邙 6872 +邛 6873 +邝 6874 +邠 6875 +邡 6876 +邢 6877 +那 6878 +邦 6879 +邨 6880 +邪 6881 +邬 6882 +邮 6883 +邯 6884 +邰 6885 +邱 6886 +邳 6887 +邴 6888 +邵 6889 +邸 6890 +邹 6891 +邺 6892 +邻 6893 +邽 6894 +邾 6895 +郁 6896 +郃 6897 +郄 6898 +郅 6899 +郇 6900 +郈 6901 +郊 6902 +郍 6903 +郎 6904 +郏 6905 +郑 6906 +郓 6907 +郕 6908 +郗 6909 +郛 6910 +郜 6911 +郝 6912 +郡 6913 +郢 6914 +郦 6915 +郧 6916 +部 6917 +郫 6918 +郭 6919 +郯 6920 +郴 6921 +郸 6922 +都 6923 +郾 6924 +郿 6925 +鄂 6926 +鄄 6927 +鄅 6928 +鄋 6929 +鄙 6930 +鄜 6931 +鄞 6932 +鄢 6933 +鄣 6934 +鄩 6935 +鄫 6936 +鄮 6937 +鄯 6938 +鄱 6939 +酂 6940 +酃 6941 +酆 6942 +酇 6943 +酉 6944 +酊 6945 +酋 6946 +酌 6947 +配 6948 +酎 6949 +酐 6950 +酒 6951 +酗 6952 +酚 6953 +酝 6954 +酞 6955 +酢 6956 +酣 6957 +酤 6958 +酥 6959 +酩 6960 +酪 6961 +酬 6962 +酮 6963 +酯 6964 +酰 6965 +酱 6966 +酵 6967 +酶 6968 +酷 6969 +酸 6970 +酹 6971 +酺 6972 +酽 6973 +酿 6974 +醅 6975 +醇 6976 +醉 6977 +醋 6978 +醌 6979 +醍 6980 +醐 6981 +醒 6982 +醚 6983 +醛 6984 +醣 6985 +醥 6986 +醪 6987 +醮 6988 +醲 6989 +醴 6990 +醵 6991 +醺 6992 +釆 6993 +采 6994 +釉 6995 +释 6996 +里 6997 +重 6998 +野 6999 +量 7000 +金 7001 +釜 7002 +鉏 7003 +鉴 7004 +銮 7005 +鋆 7006 +鋐 7007 +鋳 7008 +錾 7009 +鍉 7010 +鍒 7011 +鎉 7012 +鎏 7013 +鏊 7014 +鏖 7015 +鑫 7016 +钇 7017 +针 7018 +钉 7019 +钊 7020 +钌 7021 +钍 7022 +钎 7023 +钏 7024 +钐 7025 +钒 7026 +钓 7027 +钕 7028 +钗 7029 +钙 7030 +钚 7031 +钛 7032 +钜 7033 +钝 7034 +钞 7035 +钟 7036 +钠 7037 +钡 7038 +钢 7039 +钣 7040 +钤 7041 +钥 7042 +钦 7043 +钧 7044 +钨 7045 +钩 7046 +钪 7047 +钫 7048 +钬 7049 +钮 7050 +钯 7051 +钰 7052 +钱 7053 +钲 7054 +钳 7055 +钴 7056 +钵 7057 +钹 7058 +钺 7059 +钻 7060 +钼 7061 +钽 7062 +钾 7063 +钿 7064 +铀 7065 +铁 7066 +铂 7067 +铃 7068 +铄 7069 +铅 7070 +铆 7071 +铉 7072 +铊 7073 +铋 7074 +铌 7075 +铍 7076 +铎 7077 +铐 7078 +铑 7079 +铖 7080 +铙 7081 +铚 7082 +铛 7083 +铜 7084 +铝 7085 +铟 7086 +铠 7087 +铡 7088 +铢 7089 +铣 7090 +铤 7091 +铧 7092 +铨 7093 +铩 7094 +铪 7095 +铫 7096 +铬 7097 +铭 7098 +铮 7099 +铯 7100 +铰 7101 +铱 7102 +铲 7103 +铳 7104 +铵 7105 +银 7106 +铷 7107 +铸 7108 +铺 7109 +铼 7110 +铽 7111 +链 7112 +铿 7113 +销 7114 +锁 7115 +锂 7116 +锃 7117 +锄 7118 +锅 7119 +锆 7120 +锇 7121 +锈 7122 +锉 7123 +锊 7124 +锋 7125 +锌 7126 +锎 7127 +锏 7128 +锐 7129 +锑 7130 +锒 7131 +锕 7132 +锗 7133 +错 7134 +锚 7135 +锜 7136 +锟 7137 +锡 7138 +锢 7139 +锣 7140 +锤 7141 +锥 7142 +锦 7143 +锨 7144 +锫 7145 +锭 7146 +键 7147 +锯 7148 +锰 7149 +锱 7150 +锲 7151 +锳 7152 +锴 7153 +锵 7154 +锶 7155 +锷 7156 +锹 7157 +锺 7158 +锻 7159 +镀 7160 +镁 7161 +镂 7162 +镆 7163 +镇 7164 +镈 7165 +镉 7166 +镊 7167 +镋 7168 +镌 7169 +镍 7170 +镎 7171 +镏 7172 +镐 7173 +镑 7174 +镒 7175 +镓 7176 +镔 7177 +镕 7178 +镖 7179 +镗 7180 +镚 7181 +镛 7182 +镜 7183 +镝 7184 +镞 7185 +镠 7186 +镡 7187 +镢 7188 +镣 7189 +镤 7190 +镥 7191 +镧 7192 +镪 7193 +镫 7194 +镭 7195 +镮 7196 +镯 7197 +镰 7198 +镱 7199 +镲 7200 +镳 7201 +镶 7202 +长 7203 +閟 7204 +闘 7205 +门 7206 +闩 7207 +闪 7208 +闫 7209 +闭 7210 +问 7211 +闯 7212 +闰 7213 +闱 7214 +闲 7215 +闳 7216 +间 7217 +闵 7218 +闷 7219 +闸 7220 +闹 7221 +闺 7222 +闻 7223 +闼 7224 +闽 7225 +闾 7226 +闿 7227 +阀 7228 +阁 7229 +阂 7230 +阃 7231 +阄 7232 +阅 7233 +阆 7234 +阇 7235 +阈 7236 +阉 7237 +阊 7238 +阌 7239 +阍 7240 +阎 7241 +阏 7242 +阐 7243 +阑 7244 +阔 7245 +阕 7246 +阖 7247 +阗 7248 +阙 7249 +阚 7250 +阜 7251 +阝 7252 +队 7253 +阡 7254 +阪 7255 +阮 7256 +阱 7257 +防 7258 +阳 7259 +阴 7260 +阵 7261 +阶 7262 +阻 7263 +阼 7264 +阽 7265 +阿 7266 +陀 7267 +陁 7268 +陂 7269 +附 7270 +际 7271 +陆 7272 +陇 7273 +陈 7274 +陉 7275 +陋 7276 +陌 7277 +降 7278 +限 7279 +陔 7280 +陕 7281 +陖 7282 +陛 7283 +陟 7284 +陡 7285 +院 7286 +除 7287 +陨 7288 +险 7289 +陪 7290 +陬 7291 +陲 7292 +陵 7293 +陶 7294 +陷 7295 +険 7296 +隅 7297 +隆 7298 +隈 7299 +隋 7300 +隍 7301 +随 7302 +隐 7303 +隔 7304 +隗 7305 +隘 7306 +隙 7307 +障 7308 +隧 7309 +隰 7310 +隶 7311 +隹 7312 +隼 7313 +隽 7314 +难 7315 +雀 7316 +雁 7317 +雄 7318 +雅 7319 +集 7320 +雇 7321 +雉 7322 +雌 7323 +雍 7324 +雎 7325 +雏 7326 +雑 7327 +雒 7328 +雔 7329 +雕 7330 +雠 7331 +雨 7332 +雩 7333 +雪 7334 +雫 7335 +雯 7336 +雱 7337 +雳 7338 +零 7339 +雷 7340 +雹 7341 +雾 7342 +需 7343 +霁 7344 +霄 7345 +霆 7346 +震 7347 +霈 7348 +霉 7349 +霊 7350 +霍 7351 +霎 7352 +霏 7353 +霑 7354 +霓 7355 +霖 7356 +霜 7357 +霞 7358 +霪 7359 +霭 7360 +霰 7361 +露 7362 +霸 7363 +霹 7364 +霾 7365 +青 7366 +靓 7367 +靖 7368 +静 7369 +靛 7370 +非 7371 +靠 7372 +靡 7373 +面 7374 +靥 7375 +革 7376 +靰 7377 +靳 7378 +靴 7379 +靶 7380 +靺 7381 +靼 7382 +鞅 7383 +鞆 7384 +鞋 7385 +鞍 7386 +鞑 7387 +鞘 7388 +鞠 7389 +鞣 7390 +鞥 7391 +鞨 7392 +鞭 7393 +鞴 7394 +鞶 7395 +韦 7396 +韧 7397 +韩 7398 +韪 7399 +韫 7400 +韬 7401 +韭 7402 +音 7403 +韵 7404 +韶 7405 +頔 7406 +頠 7407 +顗 7408 +页 7409 +顶 7410 +顷 7411 +项 7412 +顺 7413 +须 7414 +顼 7415 +顽 7416 +顾 7417 +顿 7418 +颀 7419 +颁 7420 +颂 7421 +预 7422 +颅 7423 +领 7424 +颇 7425 +颈 7426 +颉 7427 +颊 7428 +颌 7429 +颍 7430 +颎 7431 +颏 7432 +颐 7433 +频 7434 +颓 7435 +颔 7436 +颕 7437 +颖 7438 +颗 7439 +题 7440 +颙 7441 +颚 7442 +颛 7443 +颜 7444 +额 7445 +颞 7446 +颠 7447 +颢 7448 +颤 7449 +颦 7450 +颧 7451 +风 7452 +飒 7453 +飓 7454 +飕 7455 +飖 7456 +飗 7457 +飘 7458 +飙 7459 +飚 7460 +飞 7461 +食 7462 +飧 7463 +飨 7464 +飮 7465 +餍 7466 +餐 7467 +餮 7468 +饔 7469 +饕 7470 +饥 7471 +饨 7472 +饪 7473 +饬 7474 +饭 7475 +饮 7476 +饯 7477 +饰 7478 +饱 7479 +饲 7480 +饴 7481 +饵 7482 +饶 7483 +饷 7484 +饸 7485 +饹 7486 +饺 7487 +饼 7488 +饽 7489 +饿 7490 +馀 7491 +馁 7492 +馄 7493 +馅 7494 +馆 7495 +馈 7496 +馊 7497 +馋 7498 +馍 7499 +馏 7500 +馐 7501 +馑 7502 +馒 7503 +馓 7504 +馔 7505 +馕 7506 +首 7507 +馗 7508 +香 7509 +馥 7510 +馨 7511 +馯 7512 +馲 7513 +駃 7514 +駆 7515 +験 7516 +騕 7517 +騜 7518 +騠 7519 +騪 7520 +騴 7521 +驩 7522 +马 7523 +驭 7524 +驮 7525 +驯 7526 +驰 7527 +驱 7528 +驳 7529 +驴 7530 +驶 7531 +驷 7532 +驸 7533 +驹 7534 +驺 7535 +驻 7536 +驼 7537 +驽 7538 +驾 7539 +驿 7540 +骁 7541 +骂 7542 +骄 7543 +骅 7544 +骆 7545 +骇 7546 +骈 7547 +骊 7548 +骋 7549 +验 7550 +骎 7551 +骏 7552 +骐 7553 +骑 7554 +骓 7555 +骕 7556 +骖 7557 +骗 7558 +骘 7559 +骚 7560 +骛 7561 +骜 7562 +骝 7563 +骞 7564 +骠 7565 +骡 7566 +骢 7567 +骤 7568 +骥 7569 +骧 7570 +骨 7571 +骰 7572 +骶 7573 +骷 7574 +骸 7575 +骹 7576 +骺 7577 +骼 7578 +髁 7579 +髂 7580 +髅 7581 +髋 7582 +髌 7583 +髎 7584 +髑 7585 +髓 7586 +髗 7587 +高 7588 +髡 7589 +髣 7590 +髦 7591 +髪 7592 +髫 7593 +髭 7594 +髯 7595 +髴 7596 +髻 7597 +髽 7598 +鬃 7599 +鬄 7600 +鬐 7601 +鬓 7602 +鬟 7603 +鬣 7604 +鬲 7605 +鬶 7606 +鬻 7607 +鬼 7608 +魁 7609 +魂 7610 +魃 7611 +魄 7612 +魅 7613 +魇 7614 +魉 7615 +魋 7616 +魍 7617 +魏 7618 +魑 7619 +魔 7620 +魟 7621 +鱼 7622 +鱿 7623 +鲀 7624 +鲁 7625 +鲃 7626 +鲅 7627 +鲆 7628 +鲇 7629 +鲈 7630 +鲋 7631 +鲌 7632 +鲍 7633 +鲎 7634 +鲐 7635 +鲑 7636 +鲔 7637 +鲖 7638 +鲚 7639 +鲛 7640 +鲜 7641 +鲟 7642 +鲠 7643 +鲡 7644 +鲢 7645 +鲣 7646 +鲤 7647 +鲧 7648 +鲨 7649 +鲫 7650 +鲭 7651 +鲱 7652 +鲲 7653 +鲳 7654 +鲴 7655 +鲵 7656 +鲶 7657 +鲷 7658 +鲸 7659 +鲹 7660 +鲻 7661 +鲼 7662 +鲽 7663 +鲾 7664 +鲿 7665 +鳃 7666 +鳄 7667 +鳅 7668 +鳊 7669 +鳌 7670 +鳍 7671 +鳎 7672 +鳏 7673 +鳐 7674 +鳒 7675 +鳔 7676 +鳕 7677 +鳖 7678 +鳗 7679 +鳙 7680 +鳚 7681 +鳜 7682 +鳝 7683 +鳞 7684 +鳟 7685 +鳢 7686 +鳣 7687 +鳯 7688 +鴐 7689 +鴶 7690 +鵴 7691 +鶒 7692 +鷏 7693 +鷩 7694 +鸂 7695 +鸟 7696 +鸠 7697 +鸡 7698 +鸢 7699 +鸣 7700 +鸥 7701 +鸦 7702 +鸨 7703 +鸩 7704 +鸪 7705 +鸫 7706 +鸬 7707 +鸭 7708 +鸮 7709 +鸯 7710 +鸰 7711 +鸱 7712 +鸲 7713 +鸳 7714 +鸵 7715 +鸷 7716 +鸸 7717 +鸻 7718 +鸽 7719 +鸾 7720 +鸿 7721 +鹀 7722 +鹁 7723 +鹂 7724 +鹃 7725 +鹄 7726 +鹅 7727 +鹇 7728 +鹈 7729 +鹉 7730 +鹊 7731 +鹋 7732 +鹌 7733 +鹎 7734 +鹏 7735 +鹑 7736 +鹔 7737 +鹕 7738 +鹗 7739 +鹘 7740 +鹚 7741 +鹛 7742 +鹜 7743 +鹞 7744 +鹟 7745 +鹣 7746 +鹤 7747 +鹦 7748 +鹧 7749 +鹩 7750 +鹪 7751 +鹫 7752 +鹬 7753 +鹭 7754 +鹮 7755 +鹯 7756 +鹰 7757 +鹳 7758 +鹿 7759 +麂 7760 +麃 7761 +麇 7762 +麈 7763 +麋 7764 +麒 7765 +麓 7766 +麝 7767 +麟 7768 +麦 7769 +麸 7770 +麹 7771 +麻 7772 +麾 7773 +麿 7774 +黄 7775 +黍 7776 +黎 7777 +黏 7778 +黑 7779 +黔 7780 +默 7781 +黛 7782 +黜 7783 +黝 7784 +黟 7785 +黠 7786 +黡 7787 +黢 7788 +黥 7789 +黧 7790 +黩 7791 +黯 7792 +黻 7793 +黼 7794 +黾 7795 +鼋 7796 +鼎 7797 +鼐 7798 +鼓 7799 +鼙 7800 +鼠 7801 +鼢 7802 +鼩 7803 +鼬 7804 +鼯 7805 +鼱 7806 +鼷 7807 +鼹 7808 +鼻 7809 +鼾 7810 +齁 7811 +齉 7812 +齐 7813 +齑 7814 +齮 7815 +齿 7816 +龃 7817 +龄 7818 +龅 7819 +龇 7820 +龈 7821 +龉 7822 +龊 7823 +龋 7824 +龌 7825 +龙 7826 +龚 7827 +龛 7828 +龟 7829 +龠 7830 +龢 7831 diff --git a/axmodel/encoder.axmodel b/axmodel/encoder.axmodel new file mode 100644 index 0000000000000000000000000000000000000000..ad700fd8412021de964947b27a3d5856ca2421b0 --- /dev/null +++ b/axmodel/encoder.axmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6cc674ba54cf0e57f3c7dffa3824cd53700e4e7709827893f8708c4958e116c1 +size 851656147 diff --git a/axmodel/encoder_u16_7.0.axmodel b/axmodel/encoder_u16_7.0.axmodel new file mode 100644 index 0000000000000000000000000000000000000000..4ef42d416e4e8a96a0f0d5608e54308e56cdd877 --- /dev/null +++ b/axmodel/encoder_u16_7.0.axmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10c123b97d0db91c446dc3e66560686df9ee365c5a6e3ca972a367a303d863d6 +size 851401753 diff --git a/axmodel/pe.npy b/axmodel/pe.npy new file mode 100644 index 0000000000000000000000000000000000000000..6a2d5ca85dee54f05d6e2af29c9ab49748b02276 --- /dev/null +++ b/axmodel/pe.npy @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d0b03c72e71c8e01e36b390154520cc4bb28cf18aed448d6a0c1b93b8603042 +size 25600128 diff --git a/axmodel/train_bpe1000.model b/axmodel/train_bpe1000.model new file mode 100644 index 0000000000000000000000000000000000000000..ab52c664b93b31764820ef193037a209a2089087 --- /dev/null +++ b/axmodel/train_bpe1000.model @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:473bbc157cb4eade2059b30a3c877a1c29bd50cadbfbed869ae36eeade7fee07 +size 251707 diff --git a/config.json b/config.json new file mode 100644 index 0000000000000000000000000000000000000000..0967ef424bce6791893e9a57bb952f80fd536e93 --- /dev/null +++ b/config.json @@ -0,0 +1 @@ +{} diff --git a/cpp/bin/README.md b/cpp/bin/README.md new file mode 100644 index 0000000000000000000000000000000000000000..25c0e3b3fe79a7e1007d1af08cf646a41b8b57df --- /dev/null +++ b/cpp/bin/README.md @@ -0,0 +1,42 @@ +# FireRedASR-AED C++ 预编译产物（AX650N，aarch64） + +包含： +- `firered_sdk`：端到端 CLI（VAD + fbank + encoder + decoder） +- `libfirered_sdk_lib.so`：SDK 动态库（C API：`firered_create/transcribe/destroy`） +- `firered_bench`：NPU 推理基准工具 +- `firered_openai_server`：OpenAI 兼容 API 服务 +- `firered_openai_client`：OpenAI 兼容客户端（CLI） + +依赖：AX650 BSP 运行时（`libax_engine.so`、`libax_sys.so`、 +`libax_interpreter.so`，通常在 `/soc/lib`），以及模型目录下的 +`encoder.axmodel`、`decoder_loop.axmodel`、`fsmn_vad/fsmn_vad_10s_fp32.axmodel`、 +`fsmn_vad/am.mvn`、`cmvn.ark`、`dict.txt`、`pe.bin` +（由 `pe.npy` 按 float32 原样导出）。 + +## CLI + +```bash +LD_LIBRARY_PATH=/soc/lib:. ./firered_sdk \ + encoder.axmodel decoder_loop.axmodel \ + fsmn_vad/fsmn_vad_10s_fp32.axmodel fsmn_vad/am.mvn \ + input.wav cmvn.ark dict.txt 10 128 pe.bin 1000 +``` + +## OpenAI API + +```bash +LD_LIBRARY_PATH=/soc/lib:. ./firered_openai_server \ + encoder.axmodel decoder_loop.axmodel \ + fsmn_vad/fsmn_vad_10s_fp32.axmodel fsmn_vad/am.mvn \ + cmvn.ark dict.txt pe.bin 10 128 8000 1000 +``` + +```bash +curl -F file=@input.wav -F model=fireredasr-aed \ + http://:8000/v1/audio/transcriptions +curl http://:8000/v1/models +./firered_openai_client http://:8000 input.wav fireredasr-aed +``` + +源码、构建与 Python 版 server/client 见 +[GitHub](https://github.com/ml-inory/FireRedASR.axera)。 diff --git a/cpp/bin/firered_bench b/cpp/bin/firered_bench new file mode 100644 index 0000000000000000000000000000000000000000..40b882524d8190e7f6df19ba8fe24f8b335a382c --- /dev/null +++ b/cpp/bin/firered_bench @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fbc9388d0f7747fce2a999827e13f3acb61383e196a18dbb4fafa315cde9c240 +size 129232 diff --git a/cpp/bin/firered_openai_client b/cpp/bin/firered_openai_client new file mode 100644 index 0000000000000000000000000000000000000000..ffc9048d86137567b285e96bd3b7beaf9c8678c4 --- /dev/null +++ b/cpp/bin/firered_openai_client @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7d501356c027d33cb9f48795e0e52fd8a2e64dfedccb1782e8ca0a98032c82c +size 616256 diff --git a/cpp/bin/firered_openai_server b/cpp/bin/firered_openai_server new file mode 100644 index 0000000000000000000000000000000000000000..f346f8cf972870f25df4d4f1ea4903aab6707110 --- /dev/null +++ b/cpp/bin/firered_openai_server @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1023c91f3c7afc35af04bde4176f80e993d4665578b7941decf104730ac0962 +size 510160 diff --git a/cpp/bin/firered_sdk b/cpp/bin/firered_sdk new file mode 100644 index 0000000000000000000000000000000000000000..514fae3e30c4dbec2bf3e7c25a8af95117efe3c4 Binary files /dev/null and b/cpp/bin/firered_sdk differ diff --git a/cpp/bin/libfirered_sdk_lib.so b/cpp/bin/libfirered_sdk_lib.so new file mode 100644 index 0000000000000000000000000000000000000000..5932776b37cce530a54507007b56c45ae3c59693 --- /dev/null +++ b/cpp/bin/libfirered_sdk_lib.so @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f58c27276e51d14f9e8dbb7af04f372e62f62d4ca07388155c41a19a594747b +size 167144 diff --git a/cpp/bin/sdk_lib.hpp b/cpp/bin/sdk_lib.hpp new file mode 100644 index 0000000000000000000000000000000000000000..d9f0ca8407e2a2e2320ce4c7f60ee1c1ae137d78 --- /dev/null +++ b/cpp/bin/sdk_lib.hpp @@ -0,0 +1,48 @@ +#pragma once + +#include +#include + +// FireRedASR-AED AX650N 端到端 SDK（板端 VAD + fbank + ASR） +class FireredSdk { +public: + FireredSdk() = default; + ~FireredSdk(); + + // 初始化：encoder/decoder/vad axmodel、cmvn、词典、pe、max_dur、max_steps + bool Init(const char* encoder, const char* decoder, const char* vad, + const char* cmvn, const char* dict, const char* pe_path, + int max_dur_s, int max_steps, + const char* fsmn_vad = nullptr, const char* fsmn_cmvn = nullptr, + int vad_min_speech_ms = 1000); + + // 识别 wav，输出文本；可选返回各阶段耗时（ms） + std::string Transcribe(const char* wav_path, + double* vad_ms = nullptr, + double* enc_ms = nullptr, + double* dec_ms = nullptr); + +private: + struct Impl; + Impl* impl_ = nullptr; +}; + +// 默认 VAD 最小语音段（ms） + +// 简易 C API（供其他语言/框架调用） +#ifdef __cplusplus +extern "C" { +#endif +typedef void* FireredHandle; +FireredHandle firered_create(const char* encoder, const char* decoder, const char* vad, + const char* cmvn, const char* dict, const char* pe_path, + int max_dur_s, int max_steps, + const char* fsmn_vad = nullptr, + const char* fsmn_cmvn = nullptr, + int vad_min_speech_ms = 1000); +int firered_transcribe(FireredHandle h, const char* wav, char* out, int out_size, + double* vad_ms, double* enc_ms, double* dec_ms); +void firered_destroy(FireredHandle h); +#ifdef __cplusplus +} +#endif diff --git a/fireredasr/data/asr_feat.py b/fireredasr/data/asr_feat.py new file mode 100644 index 0000000000000000000000000000000000000000..81cf0c26b2047ece1f097c24eed1b3cb54f37279 --- /dev/null +++ b/fireredasr/data/asr_feat.py @@ -0,0 +1,122 @@ +import math +import os + +import kaldiio +import kaldi_native_fbank as knf +import numpy as np +import torch + + +class ASRFeatExtractor: + def __init__(self, kaldi_cmvn_file): + self.cmvn = CMVN(kaldi_cmvn_file) if kaldi_cmvn_file != "" else None + self.fbank = KaldifeatFbank(num_mel_bins=80, frame_length=25, + frame_shift=10, dither=0.0) + + def __call__(self, wav_paths): + feats = [] + durs = [] + for wav_path in wav_paths: + sample_rate, wav_np = kaldiio.load_mat(wav_path) + + dur = wav_np.shape[0] / sample_rate + fbank = self.fbank((sample_rate, wav_np)) + if self.cmvn is not None: + fbank = self.cmvn(fbank) + fbank = torch.from_numpy(fbank).float() + feats.append(fbank) + durs.append(dur) + lengths = torch.tensor([feat.size(0) for feat in feats]).long() + feats_pad = self.pad_feat(feats, 0.0) + return feats_pad, lengths, durs + + def run_chunk(self, wav_np, sample_rate): + feats = [] + + dur = wav_np.shape[0] / sample_rate + fbank = self.fbank((sample_rate, wav_np)) + if self.cmvn is not None: + fbank = self.cmvn(fbank) + fbank = torch.from_numpy(fbank).float() + feats.append(fbank) + + lengths = torch.tensor([feat.size(0) for feat in feats]).long() + feats_pad = self.pad_feat(feats, 0.0) + return feats_pad.numpy(), lengths, dur + + def pad_feat(self, xs, pad_value): + # type: (List[Tensor], int) -> Tensor + n_batch = len(xs) + max_len = max([xs[i].size(0) for i in range(n_batch)]) + pad = torch.ones(n_batch, max_len, *xs[0].size()[1:]).to(xs[0].device).to(xs[0].dtype).fill_(pad_value) + for i in range(n_batch): + pad[i, :xs[i].size(0)] = xs[i] + return pad + + + + +class CMVN: + def __init__(self, kaldi_cmvn_file): + self.dim, self.means, self.inverse_std_variences = \ + self.read_kaldi_cmvn(kaldi_cmvn_file) + + def __call__(self, x, is_train=False): + assert x.shape[-1] == self.dim, "CMVN dim mismatch" + out = x - self.means + out = out * self.inverse_std_variences + return out + + def read_kaldi_cmvn(self, kaldi_cmvn_file): + assert os.path.exists(kaldi_cmvn_file) + stats = kaldiio.load_mat(kaldi_cmvn_file) + assert stats.shape[0] == 2 + dim = stats.shape[-1] - 1 + count = stats[0, dim] + assert count >= 1 + floor = 1e-20 + means = [] + inverse_std_variences = [] + for d in range(dim): + mean = stats[0, d] / count + means.append(mean.item()) + varience = (stats[1, d] / count) - mean*mean + if varience < floor: + varience = floor + istd = 1.0 / math.sqrt(varience) + inverse_std_variences.append(istd) + return dim, np.array(means), np.array(inverse_std_variences) + + + +class KaldifeatFbank: + def __init__(self, num_mel_bins=80, frame_length=25, frame_shift=10, + dither=1.0): + self.dither = dither + opts = knf.FbankOptions() + opts.frame_opts.dither = dither + opts.mel_opts.num_bins = num_mel_bins + opts.frame_opts.snip_edges = True + opts.mel_opts.debug_mel = False + self.opts = opts + + def __call__(self, wav, is_train=False): + if type(wav) is str: + sample_rate, wav_np = kaldiio.load_mat(wav) + elif type(wav) in [tuple, list] and len(wav) == 2: + sample_rate, wav_np = wav + assert len(wav_np.shape) == 1 + + dither = self.dither if is_train else 0.0 + self.opts.frame_opts.dither = dither + fbank = knf.OnlineFbank(self.opts) + + fbank.accept_waveform(sample_rate, wav_np.tolist()) + feat = [] + for i in range(fbank.num_frames_ready): + feat.append(fbank.get_frame(i)) + if len(feat) == 0: + print("Check data, len(feat) == 0", wav, flush=True) + return np.zeros((0, self.opts.mel_opts.num_bins)) + feat = np.vstack(feat) + return feat diff --git a/fireredasr/data/token_dict.py b/fireredasr/data/token_dict.py new file mode 100644 index 0000000000000000000000000000000000000000..4bd439683b7d35adbe66f14f91068fbb18bc8337 --- /dev/null +++ b/fireredasr/data/token_dict.py @@ -0,0 +1,59 @@ +import logging + + +class TokenDict: + def __init__(self, dict_path, unk=""): + assert dict_path != "" + self.id2word, self.word2id = self.read_dict(dict_path) + self.unk = unk + assert unk == "" or unk in self.word2id + self.unkid = self.word2id[unk] if unk else -1 + + def get(self, key, default): + if type(default) == str: + default = self.word2id[default] + return self.word2id.get(key, default) + + def __getitem__(self, key): + if type(key) == str: + if self.unk: + return self.word2id.get(key, self.word2id[self.unk]) + else: + return self.word2id[key] + elif type(key) == int: + return self.id2word[key] + else: + raise TypeError("Key should be str or int") + + def __len__(self): + return len(self.id2word) + + def __contains__(self, query): + if type(query) == str: + return query in self.word2id + elif type(query) == int: + return query in self.id2word + else: + raise TypeError("query should be str or int") + + def read_dict(self, dict_path): + id2word, word2id = [], {} + with open(dict_path, encoding='utf8') as f: + for i, line in enumerate(f): + tokens = line.strip().split() + if len(tokens) >= 2: + word, index = tokens[0], int(tokens[1]) + elif len(tokens) == 1: + word, index = tokens[0], i + else: # empty line or space + logging.info(f"Find empty line or space '{line.strip()}' in {dict_path}:L{i}, set to ' '") + word, index = " ", i + assert len(id2word) == index + assert len(word2id) == index + if word == "": + logging.info(f"NOTE: Find in {dict_path}:L{i} and convert it to ' '") + word = " " + word2id[word] = index + id2word.append(word) + assert len(id2word) == len(word2id) + return id2word, word2id diff --git a/fireredasr/models/fireredasr.py b/fireredasr/models/fireredasr.py new file mode 100644 index 0000000000000000000000000000000000000000..4ec88bb38ac850c78bedadd93267c877a5f91710 --- /dev/null +++ b/fireredasr/models/fireredasr.py @@ -0,0 +1,128 @@ +import os +import time + +import torch + +from fireredasr.data.asr_feat import ASRFeatExtractor +from fireredasr.models.fireredasr_aed import FireRedAsrAed +from fireredasr.tokenizer.aed_tokenizer import ChineseCharEnglishSpmTokenizer + + +class FireRedAsr: + @classmethod + def from_pretrained(cls, asr_type, model_dir): + assert asr_type in ["aed", "llm"] + + cmvn_path = os.path.join(model_dir, "cmvn.ark") + feat_extractor = ASRFeatExtractor(cmvn_path) + + if asr_type == "aed": + model_path = os.path.join(model_dir, "model.pth.tar") + dict_path =os.path.join(model_dir, "dict.txt") + spm_model = os.path.join(model_dir, "train_bpe1000.model") + model = load_fireredasr_aed_model(model_path) + tokenizer = ChineseCharEnglishSpmTokenizer(dict_path, spm_model) + elif asr_type == "llm": + model_path = os.path.join(model_dir, "model.pth.tar") + encoder_path = os.path.join(model_dir, "asr_encoder.pth.tar") + llm_dir = os.path.join(model_dir, "Qwen2-7B-Instruct") + model, tokenizer = load_firered_llm_model_and_tokenizer( + model_path, encoder_path, llm_dir) + model.eval() + return cls(asr_type, feat_extractor, model, tokenizer) + + def __init__(self, asr_type, feat_extractor, model, tokenizer): + self.asr_type = asr_type + self.feat_extractor = feat_extractor + self.model = model + self.tokenizer = tokenizer + + @torch.no_grad() + def transcribe(self, batch_uttid, batch_wav_path, args={}): + feats, lengths, durs = self.feat_extractor(batch_wav_path) + total_dur = sum(durs) + if args.get("use_gpu", False): + feats, lengths = feats.cuda(), lengths.cuda() + self.model.cuda() + else: + self.model.cpu() + + if self.asr_type == "aed": + start_time = time.time() + + hyps = self.model.transcribe( + feats, lengths, + args.get("beam_size", 1), + args.get("nbest", 1), + args.get("decode_max_len", 0), + args.get("softmax_smoothing", 1.0), + args.get("aed_length_penalty", 0.0), + args.get("eos_penalty", 1.0) + ) + + elapsed = time.time() - start_time + rtf= elapsed / total_dur if total_dur > 0 else 0 + + results = [] + for uttid, wav, hyp in zip(batch_uttid, batch_wav_path, hyps): + hyp = hyp[0] # only return 1-best + hyp_ids = [int(id) for id in hyp["yseq"].cpu()] + text = self.tokenizer.detokenize(hyp_ids) + results.append({"uttid": uttid, "text": text, "wav": wav, + "rtf": f"{rtf:.4f}"}) + return results + + elif self.asr_type == "llm": + from fireredasr.tokenizer.llm_tokenizer import LlmTokenizerWrapper + + input_ids, attention_mask, _, _ = \ + LlmTokenizerWrapper.preprocess_texts( + origin_texts=[""]*feats.size(0), tokenizer=self.tokenizer, + max_len=128, decode=True) + if args.get("use_gpu", False): + input_ids = input_ids.cuda() + attention_mask = attention_mask.cuda() + start_time = time.time() + + generated_ids = self.model.transcribe( + feats, lengths, input_ids, attention_mask, + args.get("beam_size", 1), + args.get("decode_max_len", 0), + args.get("decode_min_len", 0), + args.get("repetition_penalty", 1.0), + args.get("llm_length_penalty", 0.0), + args.get("temperature", 1.0) + ) + + elapsed = time.time() - start_time + rtf= elapsed / total_dur if total_dur > 0 else 0 + texts = self.tokenizer.batch_decode(generated_ids, + skip_special_tokens=True) + results = [] + for uttid, wav, text in zip(batch_uttid, batch_wav_path, texts): + results.append({"uttid": uttid, "text": text, "wav": wav, + "rtf": f"{rtf:.4f}"}) + return results + + + +def load_fireredasr_aed_model(model_path): + package = torch.load(model_path, map_location=lambda storage, loc: storage, weights_only=False) + print("model args:", package["args"]) + model = FireRedAsrAed.from_args(package["args"]) + model.load_state_dict(package["model_state_dict"], strict=True) + return model + + +def load_firered_llm_model_and_tokenizer(model_path, encoder_path, llm_dir): + from fireredasr.models.fireredasr_llm import FireRedAsrLlm + from fireredasr.tokenizer.llm_tokenizer import LlmTokenizerWrapper + + package = torch.load(model_path, map_location=lambda storage, loc: storage) + package["args"].encoder_path = encoder_path + package["args"].llm_dir = llm_dir + print("model args:", package["args"]) + model = FireRedAsrLlm.from_args(package["args"]) + model.load_state_dict(package["model_state_dict"], strict=False) + tokenizer = LlmTokenizerWrapper.build_llm_tokenizer(llm_dir) + return model, tokenizer diff --git a/fireredasr/models/fireredasr_aed.py b/fireredasr/models/fireredasr_aed.py new file mode 100644 index 0000000000000000000000000000000000000000..4938c2c3c9271b4b65ef38a1fdce464bddef2c02 --- /dev/null +++ b/fireredasr/models/fireredasr_aed.py @@ -0,0 +1,35 @@ +import torch + +from fireredasr.models.module.conformer_encoder import ConformerEncoder +from fireredasr.models.module.transformer_decoder import TransformerDecoder + + +class FireRedAsrAed(torch.nn.Module): + @classmethod + def from_args(cls, args): + return cls(args) + + def __init__(self, args): + super().__init__() + self.sos_id = args.sos_id + self.eos_id = args.eos_id + + self.encoder = ConformerEncoder( + args.idim, args.n_layers_enc, args.n_head, args.d_model, + args.residual_dropout, args.dropout_rate, + args.kernel_size, args.pe_maxlen) + + self.decoder = TransformerDecoder( + args.sos_id, args.eos_id, args.pad_id, args.odim, + args.n_layers_dec, args.n_head, args.d_model, + args.residual_dropout, args.pe_maxlen) + + def transcribe(self, padded_input, input_lengths, + beam_size=1, nbest=1, decode_max_len=0, + softmax_smoothing=1.0, length_penalty=0.0, eos_penalty=1.0): + enc_outputs, _, enc_mask = self.encoder(padded_input, input_lengths) + nbest_hyps = self.decoder.batch_beam_search( + enc_outputs, enc_mask, + beam_size, nbest, decode_max_len, + softmax_smoothing, length_penalty, eos_penalty) + return nbest_hyps diff --git a/fireredasr/models/fireredasr_llm.py b/fireredasr/models/fireredasr_llm.py new file mode 100644 index 0000000000000000000000000000000000000000..17228655941da86e149b116ff324bfe69586a14f --- /dev/null +++ b/fireredasr/models/fireredasr_llm.py @@ -0,0 +1,272 @@ +import logging +import os +import random +import re + +import torch +import torch.nn as nn +from transformers import AutoModelForCausalLM + +from fireredasr.models.fireredasr_aed import FireRedAsrAed +from fireredasr.models.module.adapter import Adapter +from fireredasr.tokenizer.llm_tokenizer import DEFAULT_SPEECH_TOKEN, IGNORE_TOKEN_ID +from fireredasr.tokenizer.llm_tokenizer import LlmTokenizerWrapper +from fireredasr.utils.param import count_model_parameters + + +class FireRedAsrLlm(nn.Module): + @classmethod + def load_encoder(cls, model_path): + assert os.path.exists(model_path) + package = torch.load(model_path, map_location=lambda storage, loc: storage) + model = FireRedAsrAed.from_args(package["args"]) + if "model_state_dict" in package: + model.load_state_dict(package["model_state_dict"], strict=False) + encoder = model.encoder + encoder_dim = encoder.odim + return encoder, encoder_dim + + @classmethod + def from_args(cls, args): + logging.info(args) + logging.info("Build FireRedAsrLlm") + # Build Speech Encoder + encoder, encoder_dim = cls.load_encoder(args.encoder_path) + count_model_parameters(encoder) + if args.freeze_encoder: + logging.info(f"Frezee encoder") + for name, param in encoder.named_parameters(): + param.requires_grad = False + encoder.eval() + + if args.use_flash_attn: + attn_implementation = "flash_attention_2" + if args.use_fp16: + torch_dtype = torch.float16 + else: + torch_dtype = torch.float32 + else: + attn_implementation = "eager" + if args.use_fp16: + torch_dtype = torch.float16 + else: + torch_dtype = torch.float32 + + # Build LLM + llm = AutoModelForCausalLM.from_pretrained( + args.llm_dir, + attn_implementation=attn_implementation, + torch_dtype=torch_dtype, + ) + count_model_parameters(llm) + + # LLM Freeze or LoRA + llm_dim = llm.config.hidden_size + if args.freeze_llm: + logging.info(f"Frezee LLM") + for name, param in llm.named_parameters(): + param.requires_grad = False + llm.eval() + else: + if args.use_lora: + from peft import LoraConfig, get_peft_model + lora_config = LoraConfig( + r=64, + lora_alpha=16, + target_modules=[ + "q_proj", + "k_proj", + "v_proj", + "o_proj", + "up_proj", + "gate_proj", + "down_proj", + ], + lora_dropout=0.05, + task_type="CAUSAL_LM", + ) + llm = get_peft_model(llm, lora_config) + llm.print_trainable_parameters() + + tokenizer = LlmTokenizerWrapper.build_llm_tokenizer(args.llm_dir) + assert tokenizer.pad_token_id == tokenizer.convert_tokens_to_ids("<|endoftext|>") + llm.config.pad_token_id = tokenizer.pad_token_id + llm.config.bos_token_id = tokenizer.convert_tokens_to_ids("<|im_start|>") + llm.config.eos_token_id = tokenizer.convert_tokens_to_ids("<|im_end|>") + llm.config.default_speech_token_id = tokenizer.convert_tokens_to_ids( + DEFAULT_SPEECH_TOKEN + ) + + # Build projector + encoder_projector = Adapter( + encoder_dim, llm_dim, args.encoder_downsample_rate) + count_model_parameters(encoder_projector) + + return cls(encoder, llm, encoder_projector, + args.freeze_encoder, args.freeze_llm) + + def __init__(self, encoder, llm, encoder_projector, + freeze_encoder, freeze_llm): + super().__init__() + self.encoder = encoder + self.llm = llm + self.encoder_projector = encoder_projector + # args + self.freeze_encoder = freeze_encoder + self.freeze_llm = freeze_llm + self.llm_config = llm.config + + def transcribe(self, padded_feat, feat_lengths, padded_input_ids, attention_mask, + beam_size=1, decode_max_len=0, decode_min_len=0, + repetition_penalty=1.0, llm_length_penalty=1.0, temperature=1.0): + encoder_outs, enc_lengths, enc_mask = self.encoder(padded_feat, feat_lengths) + speech_features, speech_lens = self.encoder_projector(encoder_outs, enc_lengths) + inputs_embeds = self.llm.get_input_embeddings()(padded_input_ids) + + inputs_embeds, attention_mask, _ = \ + self._merge_input_ids_with_speech_features( + speech_features.to(inputs_embeds.dtype), inputs_embeds, padded_input_ids, attention_mask, + speech_lens=speech_lens + ) + + max_new_tokens = speech_features.size(1) if decode_max_len < 1 else decode_max_len + max_new_tokens = max(1, max_new_tokens) + + generated_ids = self.llm.generate( + inputs_embeds=inputs_embeds, + max_new_tokens=max_new_tokens, + num_beams=beam_size, + do_sample=False, + min_length=decode_min_len, + top_p=1.0, + repetition_penalty=repetition_penalty, + length_penalty=llm_length_penalty, + temperature=temperature, + bos_token_id=self.llm.config.bos_token_id, + eos_token_id=self.llm.config.eos_token_id, + pad_token_id=self.llm.config.pad_token_id, + ) + + return generated_ids + + + def _merge_input_ids_with_speech_features( + self, speech_features, inputs_embeds, input_ids, attention_mask, labels=None, + speech_lens=None + ): + """ + Modified from: https://github.com/k2-fsa/icefall/blob/master/egs/speech_llm/ASR_LLM/whisper_llm_zh/model.py + """ + speech_lens = None + num_speechs, speech_len, embed_dim = speech_features.shape + batch_size, sequence_length = input_ids.shape + left_padding = not torch.sum( + input_ids[:, -1] == torch.tensor(self.llm.config.pad_token_id) + ) + # 1. Create a mask to know where special speech tokens are + special_speech_token_mask = input_ids == self.llm.config.default_speech_token_id + num_special_speech_tokens = torch.sum(special_speech_token_mask, dim=-1) + # Compute the maximum embed dimension + max_embed_dim = ( + num_special_speech_tokens.max() * (speech_len - 1) + ) + sequence_length + batch_indices, non_speech_indices = torch.where( + input_ids != self.llm.config.default_speech_token_id + ) + + # 2. Compute the positions where text should be written + # Calculate new positions for text tokens in merged speech-text sequence. + # `special_speech_token_mask` identifies speech tokens. Each speech token will be replaced by `nb_text_tokens_per_speechs - 1` text tokens. + # `torch.cumsum` computes how each speech token shifts subsequent text token positions. + # - 1 to adjust for zero-based indexing, as `cumsum` inherently increases indices by one. + new_token_positions = ( + torch.cumsum((special_speech_token_mask * (speech_len - 1) + 1), -1) - 1 + ) # (N,U) + nb_speech_pad = max_embed_dim - 1 - new_token_positions[:, -1] + if left_padding: + new_token_positions += nb_speech_pad[:, None] # offset for left padding + text_to_overwrite = new_token_positions[batch_indices, non_speech_indices] + + # 3. Create the full embedding, already padded to the maximum position + final_embedding = torch.zeros( + batch_size, + max_embed_dim, + embed_dim, + dtype=inputs_embeds.dtype, + device=inputs_embeds.device, + ) + final_attention_mask = torch.zeros( + batch_size, + max_embed_dim, + dtype=attention_mask.dtype, + device=inputs_embeds.device, + ) + if labels is not None: + final_labels = torch.full( + (batch_size, max_embed_dim), + IGNORE_TOKEN_ID, + dtype=input_ids.dtype, + device=input_ids.device, + ) + # In case the Vision model or the Language model has been offloaded to CPU, we need to manually + # set the corresponding tensors into their correct target device. + target_device = inputs_embeds.device + batch_indices, non_speech_indices, text_to_overwrite = ( + batch_indices.to(target_device), + non_speech_indices.to(target_device), + text_to_overwrite.to(target_device), + ) + attention_mask = attention_mask.to(target_device) + + # 4. Fill the embeddings based on the mask. If we have ["hey" "", "how", "are"] + # we need to index copy on [0, 577, 578, 579] for the text and [1:576] for the speech features + final_embedding[batch_indices, text_to_overwrite] = inputs_embeds[ + batch_indices, non_speech_indices + ] + final_attention_mask[batch_indices, text_to_overwrite] = attention_mask[ + batch_indices, non_speech_indices + ] + if labels is not None: + final_labels[batch_indices, text_to_overwrite] = labels[ + batch_indices, non_speech_indices + ] + + # 5. Fill the embeddings corresponding to the speechs. Anything that is not `text_positions` needs filling (#29835) + speech_to_overwrite = torch.full( + (batch_size, max_embed_dim), + True, + dtype=torch.bool, + device=inputs_embeds.device, + ) + speech_to_overwrite[batch_indices, text_to_overwrite] = False + if speech_lens is not None: + speech_pad_position = speech_to_overwrite.cumsum(-1) <= speech_lens[:, None] + speech_to_overwrite &= speech_to_overwrite.cumsum(-1) - 1 >= nb_speech_pad[ + :, None + ].to(target_device) + + if speech_to_overwrite.sum() != speech_features.shape[:-1].numel(): + raise ValueError( + f"The input provided to the model are wrong. The number of speech tokens is {torch.sum(special_speech_token_mask)} while" + f" the number of speech given to the model is {num_speechs}. This prevents correct indexing and breaks batch generation." + ) + + final_embedding[speech_to_overwrite] = ( + speech_features.contiguous().reshape(-1, embed_dim).to(target_device) + ) + if speech_lens is not None: + speech_to_overwrite &= speech_pad_position + final_attention_mask |= speech_to_overwrite + + # 6. Mask out the embedding at padding positions, as we later use the past_key_value value to determine the non-attended tokens. + batch_indices, pad_indices = torch.where( + input_ids == self.llm.config.pad_token_id + ) + indices_to_mask = new_token_positions[batch_indices, pad_indices] + + final_embedding[batch_indices, indices_to_mask] = 0 + + if labels is None: + final_labels = None + + return final_embedding, final_attention_mask, final_labels #, position_ids diff --git a/fireredasr/models/module/adapter.py b/fireredasr/models/module/adapter.py new file mode 100644 index 0000000000000000000000000000000000000000..952dfd3b040fd0580c09be2cbe0c34eba0692107 --- /dev/null +++ b/fireredasr/models/module/adapter.py @@ -0,0 +1,30 @@ +import torch +import torch.nn as nn + + +class Adapter(nn.Module): + def __init__(self, encoder_dim, llm_dim, downsample_rate=2): + super().__init__() + self.ds = downsample_rate + self.linear1 = nn.Linear(encoder_dim * downsample_rate, llm_dim) + self.relu = nn.ReLU() + self.linear2 = nn.Linear(llm_dim, llm_dim) + + def forward(self, x, x_lens): + batch_size, seq_len, feat_dim = x.size() + num_frames_to_discard = seq_len % self.ds + if num_frames_to_discard > 0: + x = x[:, :-num_frames_to_discard, :] + seq_len = x.size(1) + + x = x.contiguous() + x = x.view( + batch_size, seq_len // self.ds, feat_dim * self.ds + ) + + x = self.linear1(x) + x = self.relu(x) + x = self.linear2(x) + + new_x_lens = torch.clamp(x_lens, max=seq_len) // self.ds + return x, new_x_lens diff --git a/fireredasr/models/module/conformer_encoder.py b/fireredasr/models/module/conformer_encoder.py new file mode 100644 index 0000000000000000000000000000000000000000..6ae1307fc92747c5df68263b44d1078ac2e766ab --- /dev/null +++ b/fireredasr/models/module/conformer_encoder.py @@ -0,0 +1,324 @@ +import torch +import torch.nn as nn +import torch.nn.functional as F + + +class ConformerEncoder(nn.Module): + def __init__(self, idim, n_layers, n_head, d_model, + residual_dropout=0.1, dropout_rate=0.1, kernel_size=33, + pe_maxlen=5000): + super().__init__() + self.odim = d_model + + self.input_preprocessor = Conv2dSubsampling(idim, d_model) + self.positional_encoding = RelPositionalEncoding(d_model) + self.dropout = nn.Dropout(residual_dropout) + + self.layer_stack = nn.ModuleList() + for l in range(n_layers): + block = RelPosEmbConformerBlock(d_model, n_head, + residual_dropout, + dropout_rate, kernel_size) + self.layer_stack.append(block) + + def forward(self, padded_input, input_lengths, pad=True): + if pad and self.input_preprocessor.context - 1 > 0: + # padded_input = F.pad(padded_input, + # (0, 0, 0, self.input_preprocessor.context - 1), 'constant', 0.0) + padded_input = torch.concat([padded_input, torch.zeros(padded_input.shape[0], self.input_preprocessor.context - 1, padded_input.shape[2])], dim=1) + + src_mask = self.padding_position_is_0(padded_input, input_lengths) + + embed_output, input_lengths, src_mask = self.input_preprocessor(padded_input, src_mask) + enc_output = self.dropout(embed_output) + + pos_emb = self.dropout(self.positional_encoding(embed_output)) + + enc_outputs = [] + for enc_layer in self.layer_stack: + enc_output = enc_layer(enc_output, pos_emb, slf_attn_mask=src_mask, + pad_mask=src_mask) + enc_outputs.append(enc_output) + + return enc_output, input_lengths, src_mask + + def padding_position_is_0(self, padded_input, input_lengths): + N, T = padded_input.size()[:2] + mask = torch.ones((N, T)).to(padded_input.device) + for i in range(N): + mask[i, input_lengths[i]:] = 0 + mask = mask.unsqueeze(dim=1) + return mask.to(torch.uint8) + + +class RelPosEmbConformerBlock(nn.Module): + def __init__(self, d_model, n_head, + residual_dropout=0.1, + dropout_rate=0.1, kernel_size=33): + super().__init__() + self.ffn1 = ConformerFeedForward(d_model, dropout_rate) + self.mhsa = RelPosMultiHeadAttention(n_head, d_model, + residual_dropout) + self.conv = ConformerConvolution(d_model, kernel_size, + dropout_rate) + self.ffn2 = ConformerFeedForward(d_model, dropout_rate) + self.layer_norm = nn.LayerNorm(d_model) + + def forward(self, x, pos_emb, slf_attn_mask=None, pad_mask=None): + out = 0.5 * x + 0.5 * self.ffn1(x) + out = self.mhsa(out, out, out, pos_emb, mask=slf_attn_mask)[0] + out = self.conv(out, pad_mask) + out = 0.5 * out + 0.5 * self.ffn2(out) + out = self.layer_norm(out) + return out + + +class Swish(nn.Module): + def forward(self, x): + return x * torch.sigmoid(x) + + +class Conv2dSubsampling(nn.Module): + def __init__(self, idim, d_model, out_channels=32): + super().__init__() + self.conv = nn.Sequential( + nn.Conv2d(1, out_channels, 3, 2), + nn.ReLU(), + nn.Conv2d(out_channels, out_channels, 3, 2), + nn.ReLU(), + ) + subsample_idim = ((idim - 1) // 2 - 1) // 2 + self.out = nn.Linear(out_channels * subsample_idim, d_model) + + self.subsampling = 4 + left_context = right_context = 3 # both exclude currect frame + self.context = left_context + 1 + right_context # 7 + + def forward(self, x, x_mask): + x = x.unsqueeze(1) + x = self.conv(x) + N, C, T, D = x.size() + x = self.out(x.transpose(1, 2).contiguous().view(N, T, C * D)) + mask = x_mask[:, :, :-2:2][:, :, :-2:2] + input_lengths = mask[:, -1, :].sum(dim=-1) + return x, input_lengths, mask + + +class RelPositionalEncoding(torch.nn.Module): + def __init__(self, d_model, max_len=5000): + super().__init__() + pe_positive = torch.zeros(max_len, d_model, requires_grad=False) + pe_negative = torch.zeros(max_len, d_model, requires_grad=False) + position = torch.arange(0, max_len).unsqueeze(1).float() + div_term = torch.exp(torch.arange(0, d_model, 2).float() * + -(torch.log(torch.tensor(10000.0)).item()/d_model)) + pe_positive[:, 0::2] = torch.sin(position * div_term) + pe_positive[:, 1::2] = torch.cos(position * div_term) + pe_negative[:, 0::2] = torch.sin(-1 * position * div_term) + pe_negative[:, 1::2] = torch.cos(-1 * position * div_term) + + pe_positive = torch.flip(pe_positive, [0]).unsqueeze(0) + pe_negative = pe_negative[1:].unsqueeze(0) + pe = torch.cat([pe_positive, pe_negative], dim=1) + self.register_buffer('pe', pe) + + def forward(self, x): + # Tmax = 2 * max_len - 1 + Tmax, T = self.pe.size(1), x.size(1) + pos_emb = self.pe[:, Tmax // 2 - T + 1 : Tmax // 2 + T].clone().detach() + return pos_emb + + +class ConformerFeedForward(nn.Module): + def __init__(self, d_model, dropout_rate=0.1): + super().__init__() + pre_layer_norm = nn.LayerNorm(d_model) + linear_expand = nn.Linear(d_model, d_model*4) + nonlinear = Swish() + dropout_pre = nn.Dropout(dropout_rate) + linear_project = nn.Linear(d_model*4, d_model) + dropout_post = nn.Dropout(dropout_rate) + self.net = nn.Sequential(pre_layer_norm, + linear_expand, + nonlinear, + dropout_pre, + linear_project, + dropout_post) + + def forward(self, x): + residual = x + output = self.net(x) + output = output + residual + return output + + +class ConformerConvolution(nn.Module): + def __init__(self, d_model, kernel_size=33, dropout_rate=0.1): + super().__init__() + assert kernel_size % 2 == 1 + self.pre_layer_norm = nn.LayerNorm(d_model) + self.pointwise_conv1 = nn.Conv1d(d_model, d_model*4, kernel_size=1, bias=False) + self.glu = F.glu + self.padding = (kernel_size - 1) // 2 + self.depthwise_conv = nn.Conv1d(d_model*2, d_model*2, + kernel_size, stride=1, + padding=self.padding, + groups=d_model*2, bias=False) + self.batch_norm = nn.LayerNorm(d_model*2) + self.swish = Swish() + self.pointwise_conv2 = nn.Conv1d(d_model*2, d_model, kernel_size=1, bias=False) + self.dropout = nn.Dropout(dropout_rate) + + def forward(self, x, mask=None): + residual = x + out = self.pre_layer_norm(x) + out = out.transpose(1, 2) + if mask is not None: + out.masked_fill_(mask.ne(1), 0.0) + out = self.pointwise_conv1(out) + out = F.glu(out, dim=1) + out = self.depthwise_conv(out) + + out = out.transpose(1, 2) + out = self.swish(self.batch_norm(out)) + out = out.transpose(1, 2) + + out = self.dropout(self.pointwise_conv2(out)) + if mask is not None: + out.masked_fill_(mask.ne(1), 0.0) + out = out.transpose(1, 2) + return out + residual + + +class EncoderMultiHeadAttention(nn.Module): + def __init__(self, n_head, d_model, + residual_dropout=0.1): + super().__init__() + assert d_model % n_head == 0 + self.n_head = n_head + self.d_k = d_model // n_head + self.d_v = self.d_k + + self.w_qs = nn.Linear(d_model, n_head * self.d_k, bias=False) + self.w_ks = nn.Linear(d_model, n_head * self.d_k, bias=False) + self.w_vs = nn.Linear(d_model, n_head * self.d_v, bias=False) + + self.layer_norm_q = nn.LayerNorm(d_model) + self.layer_norm_k = nn.LayerNorm(d_model) + self.layer_norm_v = nn.LayerNorm(d_model) + + self.attention = ScaledDotProductAttention(temperature=self.d_k ** 0.5) + self.fc = nn.Linear(n_head * self.d_v, d_model, bias=False) + self.dropout = nn.Dropout(residual_dropout) + + def forward(self, q, k, v, mask=None): + sz_b, len_q = q.size(0), q.size(1) + + residual = q + q, k, v = self.forward_qkv(q, k, v) + + output, attn = self.attention(q, k, v, mask=mask) + + output = self.forward_output(output, residual, sz_b, len_q) + return output, attn + + def forward_qkv(self, q, k, v): + d_k, d_v, n_head = self.d_k, self.d_v, self.n_head + sz_b, len_q, len_k, len_v = q.size(0), q.size(1), k.size(1), v.size(1) + + q = self.layer_norm_q(q) + k = self.layer_norm_k(k) + v = self.layer_norm_v(v) + + q = self.w_qs(q).view(sz_b, len_q, n_head, d_k) + k = self.w_ks(k).view(sz_b, len_k, n_head, d_k) + v = self.w_vs(v).view(sz_b, len_v, n_head, d_v) + q = q.transpose(1, 2) + k = k.transpose(1, 2) + v = v.transpose(1, 2) + return q, k, v + + def forward_output(self, output, residual, sz_b, len_q): + output = output.transpose(1, 2).contiguous().view(sz_b, len_q, -1) + fc_out = self.fc(output) + output = self.dropout(fc_out) + output = output + residual + return output + + +class ScaledDotProductAttention(nn.Module): + def __init__(self, temperature): + super().__init__() + self.temperature = temperature + self.dropout = nn.Dropout(0.0) + self.INF = float('inf') + + def forward(self, q, k, v, mask=None): + attn = torch.matmul(q, k.transpose(2, 3)) / self.temperature + output, attn = self.forward_attention(attn, v, mask) + return output, attn + + def forward_attention(self, attn, v, mask=None): + if mask is not None: + mask = mask.unsqueeze(1) + mask = mask.eq(0) + attn = attn.masked_fill(mask, -self.INF) + attn = torch.softmax(attn, dim=-1).masked_fill(mask, 0.0) + else: + attn = torch.softmax(attn, dim=-1) + + d_attn = self.dropout(attn) + output = torch.matmul(d_attn, v) + + return output, attn + + +class RelPosMultiHeadAttention(EncoderMultiHeadAttention): + def __init__(self, n_head, d_model, + residual_dropout=0.1): + super().__init__(n_head, d_model, + residual_dropout) + d_k = d_model // n_head + self.scale = 1.0 / (d_k ** 0.5) + self.linear_pos = nn.Linear(d_model, n_head * d_k, bias=False) + self.pos_bias_u = nn.Parameter(torch.FloatTensor(n_head, d_k)) + self.pos_bias_v = nn.Parameter(torch.FloatTensor(n_head, d_k)) + torch.nn.init.xavier_uniform_(self.pos_bias_u) + torch.nn.init.xavier_uniform_(self.pos_bias_v) + + def _rel_shift(self, x): + N, H, T1, T2 = x.size() + zero_pad = torch.zeros((N, H, T1, 1), device=x.device, dtype=x.dtype) + x_padded = torch.cat([zero_pad, x], dim=-1) + + x_padded = x_padded.view(N, H, T2 + 1, T1) + x = x_padded[:, :, 1:].view_as(x) + x = x[:, :, :, : x.size(-1) // 2 + 1] + return x + + def forward(self, q, k, v, pos_emb, mask=None): + sz_b, len_q = q.size(0), q.size(1) + + residual = q + q, k, v = self.forward_qkv(q, k, v) + + q = q.transpose(1, 2) + n_batch_pos = pos_emb.size(0) + p = self.linear_pos(pos_emb).view(n_batch_pos, -1, self.n_head, self.d_k) + p = p.transpose(1, 2) + + q_with_bias_u = (q + self.pos_bias_u).transpose(1, 2) + q_with_bias_v = (q + self.pos_bias_v).transpose(1, 2) + + matrix_ac = torch.matmul(q_with_bias_u, k.transpose(-2, -1)) + + matrix_bd = torch.matmul(q_with_bias_v, p.transpose(-2, -1)) + matrix_bd = self._rel_shift(matrix_bd) + + attn_scores = matrix_ac + matrix_bd + attn_scores.mul_(self.scale) + + output, attn = self.attention.forward_attention(attn_scores, v, mask=mask) + + output = self.forward_output(output, residual, sz_b, len_q) + return output, attn diff --git a/fireredasr/models/module/transformer_decoder.py b/fireredasr/models/module/transformer_decoder.py new file mode 100644 index 0000000000000000000000000000000000000000..99f4ce5c5d506c47fb07bd1e0200970088e565e6 --- /dev/null +++ b/fireredasr/models/module/transformer_decoder.py @@ -0,0 +1,302 @@ +from typing import List, Optional, Dict + +import torch +import torch.nn as nn +import torch.nn.functional as F +from torch import Tensor + + +class TransformerDecoder(nn.Module): + def __init__( + self, sos_id, eos_id, pad_id, odim, + n_layers, n_head, d_model, + residual_dropout=0.1, pe_maxlen=5000): + super().__init__() + self.INF = 1e10 + # parameters + self.pad_id = pad_id + self.sos_id = sos_id + self.eos_id = eos_id + self.n_layers = n_layers + + # Components + self.tgt_word_emb = nn.Embedding(odim, d_model, padding_idx=self.pad_id) + self.positional_encoding = PositionalEncoding(d_model, max_len=pe_maxlen) + self.dropout = nn.Dropout(residual_dropout) + + self.layer_stack = nn.ModuleList() + for l in range(n_layers): + block = DecoderLayer(d_model, n_head, residual_dropout) + self.layer_stack.append(block) + + self.tgt_word_prj = nn.Linear(d_model, odim, bias=False) + self.layer_norm_out = nn.LayerNorm(d_model) + + self.tgt_word_prj.weight = self.tgt_word_emb.weight + self.scale = (d_model ** 0.5) + + def batch_beam_search(self, encoder_outputs, src_masks, + beam_size=1, nbest=1, decode_max_len=0, + softmax_smoothing=1.0, length_penalty=0.0, eos_penalty=1.0): + B = beam_size + N, Ti, H = encoder_outputs.size() + device = encoder_outputs.device + maxlen = decode_max_len if decode_max_len > 0 else Ti + assert eos_penalty > 0.0 and eos_penalty <= 1.0 + + # Init + encoder_outputs = encoder_outputs.unsqueeze(1).repeat(1, B, 1, 1).view(N*B, Ti, H) + src_mask = src_masks.unsqueeze(1).repeat(1, B, 1, 1).view(N*B, -1, Ti) + ys = torch.ones(N*B, 1).fill_(self.sos_id).long().to(device) + caches: List[Optional[Tensor]] = [] + for _ in range(self.n_layers): + caches.append(None) + scores = torch.tensor([0.0] + [-self.INF]*(B-1)).float().to(device) + scores = scores.repeat(N).view(N*B, 1) + is_finished = torch.zeros_like(scores) + + # Autoregressive Prediction + for t in range(maxlen): + tgt_mask = self.ignored_target_position_is_0(ys, self.pad_id) + + dec_output = self.dropout( + self.tgt_word_emb(ys) * self.scale + + self.positional_encoding(ys)) + + i = 0 + for dec_layer in self.layer_stack: + dec_output = dec_layer.forward( + dec_output, encoder_outputs, + tgt_mask, src_mask, + cache=caches[i]) + caches[i] = dec_output + i += 1 + + dec_output = self.layer_norm_out(dec_output) + + t_logit = self.tgt_word_prj(dec_output[:, -1]) + t_scores = F.log_softmax(t_logit / softmax_smoothing, dim=-1) + + if eos_penalty != 1.0: + t_scores[:, self.eos_id] *= eos_penalty + + t_topB_scores, t_topB_ys = torch.topk(t_scores, k=B, dim=1) + t_topB_scores = self.set_finished_beam_score_to_zero(t_topB_scores, is_finished) + t_topB_ys = self.set_finished_beam_y_to_eos(t_topB_ys, is_finished) + + # Accumulated + scores = scores + t_topB_scores + + # Pruning + scores = scores.view(N, B*B) + scores, topB_score_ids = torch.topk(scores, k=B, dim=1) + scores = scores.view(-1, 1) + + topB_row_number_in_each_B_rows_of_ys = torch.div(topB_score_ids, B).view(N*B) + stride = B * torch.arange(N).view(N, 1).repeat(1, B).view(N*B).to(device) + topB_row_number_in_ys = topB_row_number_in_each_B_rows_of_ys.long() + stride.long() + + # Update ys + ys = ys[topB_row_number_in_ys] + t_ys = torch.gather(t_topB_ys.view(N, B*B), dim=1, index=topB_score_ids).view(N*B, 1) + ys = torch.cat((ys, t_ys), dim=1) + + # Update caches + new_caches: List[Optional[Tensor]] = [] + for cache in caches: + if cache is not None: + new_caches.append(cache[topB_row_number_in_ys]) + caches = new_caches + + # Update finished state + is_finished = t_ys.eq(self.eos_id) + if is_finished.sum().item() == N*B: + break + + # Length penalty (follow GNMT) + scores = scores.view(N, B) + ys = ys.view(N, B, -1) + ys_lengths = self.get_ys_lengths(ys) + if length_penalty > 0.0: + penalty = torch.pow((5+ys_lengths.float())/(5.0+1), length_penalty) + scores /= penalty + nbest_scores, nbest_ids = torch.topk(scores, k=int(nbest), dim=1) + nbest_scores = -1.0 * nbest_scores + index = nbest_ids + B * torch.arange(N).view(N, 1).to(device).long() + nbest_ys = ys.view(N*B, -1)[index.view(-1)] + nbest_ys = nbest_ys.view(N, nbest_ids.size(1), -1) + nbest_ys_lengths = ys_lengths.view(N*B)[index.view(-1)].view(N, -1) + + # result + nbest_hyps: List[List[Dict[str, Tensor]]] = [] + for n in range(N): + n_nbest_hyps: List[Dict[str, Tensor]] = [] + for i, score in enumerate(nbest_scores[n]): + new_hyp = { + "yseq": nbest_ys[n, i, 1:nbest_ys_lengths[n, i]] + } + n_nbest_hyps.append(new_hyp) + nbest_hyps.append(n_nbest_hyps) + return nbest_hyps + + def ignored_target_position_is_0(self, padded_targets, ignore_id): + mask = torch.ne(padded_targets, ignore_id) + mask = mask.unsqueeze(dim=1) + T = padded_targets.size(-1) + upper_tri_0_mask = self.upper_triangular_is_0(T).unsqueeze(0).to(mask.dtype) + upper_tri_0_mask = upper_tri_0_mask.to(mask.dtype).to(mask.device) + return mask.to(torch.uint8) & upper_tri_0_mask.to(torch.uint8) + + def upper_triangular_is_0(self, size): + ones = torch.ones(size, size) + tri_left_ones = torch.tril(ones) + return tri_left_ones.to(torch.uint8) + + def set_finished_beam_score_to_zero(self, scores, is_finished): + NB, B = scores.size() + is_finished = is_finished.float() + mask_score = torch.tensor([0.0] + [-self.INF]*(B-1)).float().to(scores.device) + mask_score = mask_score.view(1, B).repeat(NB, 1) + return scores * (1 - is_finished) + mask_score * is_finished + + def set_finished_beam_y_to_eos(self, ys, is_finished): + is_finished = is_finished.long() + return ys * (1 - is_finished) + self.eos_id * is_finished + + def get_ys_lengths(self, ys): + N, B, Tmax = ys.size() + ys_lengths = torch.sum(torch.ne(ys, self.eos_id), dim=-1) + return ys_lengths.int() + + + +class DecoderLayer(nn.Module): + def __init__(self, d_model, n_head, dropout): + super().__init__() + self.self_attn_norm = nn.LayerNorm(d_model) + self.self_attn = DecoderMultiHeadAttention(d_model, n_head, dropout) + + self.cross_attn_norm = nn.LayerNorm(d_model) + self.cross_attn = DecoderMultiHeadAttention(d_model, n_head, dropout) + + self.mlp_norm = nn.LayerNorm(d_model) + self.mlp = PositionwiseFeedForward(d_model, d_model*4, dropout) + + def forward(self, dec_input, enc_output, self_attn_mask, cross_attn_mask, + cache=None): + x = dec_input + residual = x + x = self.self_attn_norm(x) + if cache is not None: + xq = x[:, -1:, :] + residual = residual[:, -1:, :] + self_attn_mask = self_attn_mask[:, -1:, :] + else: + xq = x + x = self.self_attn(xq, x, x, mask=self_attn_mask) + x = residual + x + + residual = x + x = self.cross_attn_norm(x) + x = self.cross_attn(x, enc_output, enc_output, mask=cross_attn_mask) + x = residual + x + + residual = x + x = self.mlp_norm(x) + x = residual + self.mlp(x) + + if cache is not None: + x = torch.cat([cache, x], dim=1) + + return x + + +class DecoderMultiHeadAttention(nn.Module): + def __init__(self, d_model, n_head, dropout=0.1): + super().__init__() + self.d_model = d_model + self.n_head = n_head + self.d_k = d_model // n_head + + self.w_qs = nn.Linear(d_model, n_head * self.d_k) + self.w_ks = nn.Linear(d_model, n_head * self.d_k, bias=False) + self.w_vs = nn.Linear(d_model, n_head * self.d_k) + + self.attention = DecoderScaledDotProductAttention( + temperature=self.d_k ** 0.5) + self.fc = nn.Linear(n_head * self.d_k, d_model) + self.dropout = nn.Dropout(dropout) + + def forward(self, q, k, v, mask=None): + bs = q.size(0) + + q = self.w_qs(q).view(bs, -1, self.n_head, self.d_k) + k = self.w_ks(k).view(bs, -1, self.n_head, self.d_k) + v = self.w_vs(v).view(bs, -1, self.n_head, self.d_k) + q = q.transpose(1, 2) + k = k.transpose(1, 2) + v = v.transpose(1, 2) + + if mask is not None: + mask = mask.unsqueeze(1) + + output = self.attention(q, k, v, mask=mask) + + output = output.transpose(1, 2).contiguous().view(bs, -1, self.d_model) + output = self.fc(output) + output = self.dropout(output) + + return output + + +class DecoderScaledDotProductAttention(nn.Module): + def __init__(self, temperature): + super().__init__() + self.temperature = temperature + self.INF = float("inf") + + def forward(self, q, k, v, mask=None): + attn = torch.matmul(q, k.transpose(2, 3)) / self.temperature + print(f"q.shape: {q.shape}") + print(f"k.shape: {k.shape}") + print(f"attn.shape: {attn.shape}") + if mask is not None: + mask = mask.eq(0) + attn = attn.masked_fill(mask, -self.INF) + attn = torch.softmax(attn, dim=-1).masked_fill(mask, 0.0) + else: + attn = torch.softmax(attn, dim=-1) + output = torch.matmul(attn, v) + return output + + +class PositionwiseFeedForward(nn.Module): + def __init__(self, d_model, d_ff, dropout=0.1): + super().__init__() + self.w_1 = nn.Linear(d_model, d_ff) + self.act = nn.GELU() + self.w_2 = nn.Linear(d_ff, d_model) + self.dropout = nn.Dropout(dropout) + + def forward(self, x): + output = self.w_2(self.act(self.w_1(x))) + output = self.dropout(output) + return output + + +class PositionalEncoding(nn.Module): + def __init__(self, d_model, max_len=5000): + super().__init__() + assert d_model % 2 == 0 + pe = torch.zeros(max_len, d_model, requires_grad=False) + position = torch.arange(0, max_len).unsqueeze(1).float() + div_term = torch.exp(torch.arange(0, d_model, 2).float() * + -(torch.log(torch.tensor(10000.0)).item()/d_model)) + pe[:, 0::2] = torch.sin(position * div_term) + pe[:, 1::2] = torch.cos(position * div_term) + pe = pe.unsqueeze(0) + self.register_buffer('pe', pe) + + def forward(self, x): + length = x.size(1) + return self.pe[:, :length].clone().detach() diff --git a/fireredasr/speech2text.py b/fireredasr/speech2text.py new file mode 100755 index 0000000000000000000000000000000000000000..21f645b411c6007418b7cc9b4cdffa620a73129a --- /dev/null +++ b/fireredasr/speech2text.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python3 + +import argparse +import glob +import os +import sys + +from fireredasr.models.fireredasr import FireRedAsr + + +parser = argparse.ArgumentParser() +parser.add_argument('--asr_type', type=str, required=True, choices=["aed", "llm"]) +parser.add_argument('--model_dir', type=str, required=True) + +# Input / Output +parser.add_argument("--wav_path", type=str) +parser.add_argument("--wav_paths", type=str, nargs="*") +parser.add_argument("--wav_dir", type=str) +parser.add_argument("--wav_scp", type=str) +parser.add_argument("--output", type=str) + +# Decode Options +parser.add_argument('--use_gpu', type=int, default=1) +parser.add_argument("--batch_size", type=int, default=1) +parser.add_argument("--beam_size", type=int, default=1) +parser.add_argument("--decode_max_len", type=int, default=0) +# FireRedASR-AED +parser.add_argument("--nbest", type=int, default=1) +parser.add_argument("--softmax_smoothing", type=float, default=1.0) +parser.add_argument("--aed_length_penalty", type=float, default=0.0) +parser.add_argument("--eos_penalty", type=float, default=1.0) +# FireRedASR-LLM +parser.add_argument("--decode_min_len", type=int, default=0) +parser.add_argument("--repetition_penalty", type=float, default=1.0) +parser.add_argument("--llm_length_penalty", type=float, default=0.0) +parser.add_argument("--temperature", type=float, default=1.0) + + +def main(args): + wavs = get_wav_info(args) + fout = open(args.output, "w") if args.output else None + + model = FireRedAsr.from_pretrained(args.asr_type, args.model_dir) + + batch_uttid = [] + batch_wav_path = [] + for i, wav in enumerate(wavs): + uttid, wav_path = wav + batch_uttid.append(uttid) + batch_wav_path.append(wav_path) + if len(batch_wav_path) < args.batch_size and i != len(wavs) - 1: + continue + + results = model.transcribe( + batch_uttid, + batch_wav_path, + { + "use_gpu": args.use_gpu, + "beam_size": args.beam_size, + "nbest": args.nbest, + "decode_max_len": args.decode_max_len, + "softmax_smoothing": args.softmax_smoothing, + "aed_length_penalty": args.aed_length_penalty, + "eos_penalty": args.eos_penalty, + "decode_min_len": args.decode_min_len, + "repetition_penalty": args.repetition_penalty, + "llm_length_penalty": args.llm_length_penalty, + "temperature": args.temperature + } + ) + + for result in results: + print(result) + if fout is not None: + fout.write(f"{result['uttid']}\t{result['text']}\n") + + batch_uttid = [] + batch_wav_path = [] + + +def get_wav_info(args): + """ + Returns: + wavs: list of (uttid, wav_path) + """ + base = lambda p: os.path.basename(p).replace(".wav", "") + if args.wav_path: + wavs = [(base(args.wav_path), args.wav_path)] + elif args.wav_paths and len(args.wav_paths) >= 1: + wavs = [(base(p), p) for p in sorted(args.wav_paths)] + elif args.wav_scp: + wavs = [line.strip().split() for line in open(args.wav_scp)] + elif args.wav_dir: + wavs = glob.glob(f"{args.wav_dir}/**/*.wav", recursive=True) + wavs = [(base(p), p) for p in sorted(wavs)] + else: + raise ValueError("Please provide valid wav info") + print(f"#wavs={len(wavs)}") + return wavs + + +if __name__ == "__main__": + args = parser.parse_args() + print(args) + main(args) diff --git a/fireredasr/tokenizer/aed_tokenizer.py b/fireredasr/tokenizer/aed_tokenizer.py new file mode 100644 index 0000000000000000000000000000000000000000..00285b5486aeed1626db72fd7640151d032faafc --- /dev/null +++ b/fireredasr/tokenizer/aed_tokenizer.py @@ -0,0 +1,67 @@ +import logging +import re + +import sentencepiece as spm + +from fireredasr.data.token_dict import TokenDict + + +class ChineseCharEnglishSpmTokenizer: + """ + - One Chinese char is a token. + - Split English word into SPM and one piece is a token. + - Ignore ' ' between Chinese char + - Replace ' ' between English word with "▁" by spm_model + - Need to put SPM piece into dict file + - If not set spm_model, will use English char and + """ + SPM_SPACE = "▁" + + def __init__(self, dict_path, spm_model, unk="", space=""): + self.dict = TokenDict(dict_path, unk=unk) + self.space = space + if spm_model: + self.sp = spm.SentencePieceProcessor() + self.sp.Load(spm_model) + else: + self.sp = None + print("[WRAN] Not set spm_model, will use English char") + print("[WARN] Please check how to deal with ' '(space)") + if self.space not in self.dict: + print("Please add to your dict, or it will be ") + + def tokenize(self, text, replace_punc=True): + #if text == "": + # logging.info(f"empty text") + text = text.upper() + tokens = [] + if replace_punc: + text = re.sub("[，。？！,\.?!]", " ", text) + pattern = re.compile(r'([\u3400-\u4dbf\u4e00-\u9fff])') + parts = pattern.split(text.strip()) + parts = [p for p in parts if len(p.strip()) > 0] + for part in parts: + if pattern.fullmatch(part) is not None: + tokens.append(part) + else: + if self.sp: + for piece in self.sp.EncodeAsPieces(part.strip()): + tokens.append(piece) + else: + for char in part.strip(): + tokens.append(char if char != " " else self.space) + tokens_id = [] + for token in tokens: + tokens_id.append(self.dict.get(token, self.dict.unk)) + return tokens, tokens_id + + def detokenize(self, inputs, join_symbol="", replace_spm_space=True): + """inputs is ids or tokens, do not need self.sp""" + if len(inputs) > 0 and type(inputs[0]) == int: + tokens = [self.dict[id] for id in inputs] + else: + tokens = inputs + s = f"{join_symbol}".join(tokens) + if replace_spm_space: + s = s.replace(self.SPM_SPACE, ' ').strip() + return s diff --git a/fireredasr/tokenizer/llm_tokenizer.py b/fireredasr/tokenizer/llm_tokenizer.py new file mode 100644 index 0000000000000000000000000000000000000000..2471409da6508c5775f551f266fced572009abd9 --- /dev/null +++ b/fireredasr/tokenizer/llm_tokenizer.py @@ -0,0 +1,105 @@ +import re + +import torch +from transformers import AutoTokenizer +from transformers.trainer_pt_utils import LabelSmoother + +DEFAULT_SPEECH_TOKEN = "" +IGNORE_TOKEN_ID = LabelSmoother.ignore_index + + +class LlmTokenizerWrapper: + @classmethod + def build_llm_tokenizer(cls, llm_path, use_flash_attn=False): + tokenizer = AutoTokenizer.from_pretrained(llm_path) + if use_flash_attn: + tokenizer.padding_side = "left" + else: + tokenizer.padding_side = "right" + special_tokens_dict = {"additional_special_tokens": [DEFAULT_SPEECH_TOKEN]} + tokenizer.add_special_tokens(special_tokens_dict) + return tokenizer + + @classmethod + def clean_text(cls, origin_text): + """remove punc, remove space between Chinese and keep space between English""" + # remove punc + text = re.sub("[，。？！,\.!?《》（）\·“”、\\/]", "", origin_text) + # merge space + text = re.sub("\s+", " ", text) + + # remove space between Chinese and keep space between English + pattern = re.compile(r'([\u3400-\u4dbf\u4e00-\u9fff])') # Chinese + parts = pattern.split(text.strip()) + parts = [p for p in parts if len(p.strip()) > 0] + text = "".join(parts) + text = text.strip() + + text = text.lower() + return text + + @classmethod + def preprocess_texts(cls, origin_texts, tokenizer, max_len, decode=False): + messages = [] + clean_texts = [] + for i, origin_text in enumerate(origin_texts): + text = cls.clean_text(origin_text) + clean_texts.append(text) + text = text if not decode else "" + message = [ + {"role": "user", "content": f"{DEFAULT_SPEECH_TOKEN}请转写音频为文字"}, + {"role": "assistant", "content": text}, + ] + messages.append(message) + + texts = [] + if not decode: + TEMPLATE = "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if loop.last %}{{ '<|im_end|>'}}{% else %}{{ '<|im_end|>\n' }}{% endif %}{% endfor %}" + else: + TEMPLATE = "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if loop.last %}{{''}}{% else %}{{ '<|im_end|>\n' }}{% endif %}{% endfor %}" + for i, msg in enumerate(messages): + texts.append( + tokenizer.apply_chat_template( + msg, + tokenize=True, + chat_template=TEMPLATE, + add_generation_prompt=False, + padding="longest", + max_length=max_len, + truncation=True, + ) + ) + + # Padding texts + max_len_texts = max([len(text) for text in texts]) + if tokenizer.padding_side == "right": + texts = [ + text + [tokenizer.pad_token_id] * (max_len_texts - len(text)) + for text in texts + ] + else: + texts = [ + [tokenizer.pad_token_id] * (max_len_texts - len(text)) + text + for text in texts + ] + input_ids = torch.tensor(texts, dtype=torch.int) + + target_ids = input_ids.clone() + target_ids[target_ids == tokenizer.pad_token_id] = IGNORE_TOKEN_ID + + # first get the indices of the tokens + mask_prompt = True + if mask_prompt: + mask_indices = torch.where( + input_ids == tokenizer.convert_tokens_to_ids("assistant") + ) + for i in range(mask_indices[0].size(0)): + row = mask_indices[0][i] + col = mask_indices[1][i] + target_ids[row, : col + 2] = IGNORE_TOKEN_ID + + attention_mask = input_ids.ne(tokenizer.pad_token_id) + + target_ids = target_ids.type(torch.LongTensor) + input_ids = input_ids.type(torch.LongTensor) + return input_ids, attention_mask, target_ids, clean_texts diff --git a/fireredasr/utils/param.py b/fireredasr/utils/param.py new file mode 100644 index 0000000000000000000000000000000000000000..8dfa28d2fbc0c4866f1e1f4c437b3b963cea50b8 --- /dev/null +++ b/fireredasr/utils/param.py @@ -0,0 +1,13 @@ +import logging + +import torch + + +def count_model_parameters(model): + if not isinstance(model, torch.nn.Module): + return 0, 0 + name = f"{model.__class__.__name__} {model.__class__}" + num = sum(p.numel() for p in model.parameters() if p.requires_grad) + size = num * 4.0 / 1024.0 / 1024.0 # float32, MB + logging.info(f"#param of {name} is {num} = {size:.1f} MB (float32)") + return num, size diff --git a/fireredasr/utils/wer.py b/fireredasr/utils/wer.py new file mode 100755 index 0000000000000000000000000000000000000000..30107ca881e092e3ddd23972b93a826f326bf597 --- /dev/null +++ b/fireredasr/utils/wer.py @@ -0,0 +1,303 @@ +#!/usr/bin/env python3 + +import argparse +import re +from collections import OrderedDict + + +parser = argparse.ArgumentParser() +parser.add_argument("--ref", type=str, required=True) +parser.add_argument("--hyp", type=str, required=True) +parser.add_argument("--print_sentence_wer", type=int, default=0) +parser.add_argument("--do_tn", type=int, default=0, help="simple tn by cn2an") +parser.add_argument("--rm_special", type=int, default=0, help="remove <\|.*?\|>") + + +def main(args): + uttid2refs = read_uttid2tokens(args.ref, args.do_tn, args.rm_special) + uttid2hyps = read_uttid2tokens(args.hyp, args.do_tn, args.rm_special) + uttid2wer_info, wer_stat, en_dig_stat = compute_uttid2wer_info( + uttid2refs, uttid2hyps, args.print_sentence_wer) + wer_stat.print() + en_dig_stat.print() + + +def read_uttid2tokens(filename, do_tn=False, rm_special=False): + print(f">>> Read uttid to tokens: {filename}", flush=True) + uttid2tokens = OrderedDict() + uttid2text = read_uttid2text(filename, do_tn, rm_special) + for uttid, text in uttid2text.items(): + tokens = text2tokens(text) + uttid2tokens[uttid] = tokens + return uttid2tokens + + +def read_uttid2text(filename, do_tn=False, rm_special=False): + uttid2text = OrderedDict() + with open(filename, "r", encoding="utf8") as fin: + for i, line in enumerate(fin): + cols = line.split() + if len(cols) == 0: + print("[WARN] empty line, continue", i, flush=True) + continue + assert cols[0] not in uttid2text, f"repeated uttid: {line}" + if len(cols) == 1: + uttid2text[cols[0]] = "" + continue + txt = " ".join(cols[1:]) + if rm_special: + txt = " ".join([t for t in re.split("<\|.*?\|>", txt) if t.strip() != ""]) + if do_tn: + import cn2an + txt = cn2an.transform(txt, "an2cn") + uttid2text[cols[0]] = txt + return uttid2text + + +def text2tokens(text): + PUNCTUATIONS = "，。？！,\.?!＂＃＄％＆＇（）＊＋－／：；＜＝＞＠［＼］＾＿｀｛｜｝～｟｠｢｣､　、〃〈〉《》「」『』【】〔〕〖〗〘〙〚〛〜〝〞〟〰〾〿–—‘’‛“”„‟…‧﹏﹑﹔·｡\":" + "()\[\]{}/;`|=+" + if text == "": + return [] + tokens = [] + + text = re.sub("", "", text) + text = re.sub(r"[%s]+" % PUNCTUATIONS, " ", text) + + pattern = re.compile(r'([\u4e00-\u9fff])') + parts = pattern.split(text.strip().upper()) + parts = [p for p in parts if len(p.strip()) > 0] + for part in parts: + if pattern.fullmatch(part) is not None: + tokens.append(part) + else: + for word in part.strip().split(): + tokens.append(word) + return tokens + + +def compute_uttid2wer_info(refs, hyps, print_sentence_wer=False): + print(f">>> Compute uttid to wer info", flush=True) + + uttid2wer_info = OrderedDict() + wer_stat = WerStats() + en_dig_stat = EnDigStats() + + for uttid, ref in refs.items(): + if uttid not in hyps: + print(f"[WARN] No hyp for {uttid}", flush=True) + continue + hyp = hyps[uttid] + + if len(hyp) - len(ref) >= 8: + print(f"[BidLengthDiff]: {uttid} {len(ref)} {len(hyp)}#{' '.join(ref)}#{' '.join(hyp)}") + #continue + + wer_info = compute_one_wer_info(ref, hyp) + uttid2wer_info[uttid] = wer_info + ns = count_english_ditgit(ref, hyp, wer_info) + wer_stat.add(wer_info) + en_dig_stat.add(*ns) + if print_sentence_wer: + print(f"{uttid} {wer_info}") + + return uttid2wer_info, wer_stat, en_dig_stat + + +COST_SUB = 3 +COST_DEL = 3 +COST_INS = 3 + +ALIGN_CRT = 0 +ALIGN_SUB = 1 +ALIGN_DEL = 2 +ALIGN_INS = 3 +ALIGN_END = 4 + + +def compute_one_wer_info(ref, hyp): + """Impl minimum edit distance and backtrace. + Args: + ref, hyp: List[str] + Returns: + WerInfo + """ + ref_len = len(ref) + hyp_len = len(hyp) + + class _DpPoint: + def __init__(self, cost, align): + self.cost = cost + self.align = align + + dp = [] + for i in range(0, ref_len + 1): + dp.append([]) + for j in range(0, hyp_len + 1): + dp[-1].append(_DpPoint(i * j, ALIGN_CRT)) + + # Initialize + for i in range(1, hyp_len + 1): + dp[0][i].cost = dp[0][i - 1].cost + COST_INS; + dp[0][i].align = ALIGN_INS + for i in range(1, ref_len + 1): + dp[i][0].cost = dp[i - 1][0].cost + COST_DEL + dp[i][0].align = ALIGN_DEL + + # DP + for i in range(1, ref_len + 1): + for j in range(1, hyp_len + 1): + min_cost = 0 + min_align = ALIGN_CRT + if hyp[j - 1] == ref[i - 1]: + min_cost = dp[i - 1][j - 1].cost + min_align = ALIGN_CRT + else: + min_cost = dp[i - 1][j - 1].cost + COST_SUB + min_align = ALIGN_SUB + + del_cost = dp[i - 1][j].cost + COST_DEL + if del_cost < min_cost: + min_cost = del_cost + min_align = ALIGN_DEL + + ins_cost = dp[i][j - 1].cost + COST_INS + if ins_cost < min_cost: + min_cost = ins_cost + min_align = ALIGN_INS + + dp[i][j].cost = min_cost + dp[i][j].align = min_align + + # Backtrace + crt = sub = ins = det = 0 + i = ref_len + j = hyp_len + align = [] + while i > 0 or j > 0: + if dp[i][j].align == ALIGN_CRT: + align.append((i, j, ALIGN_CRT)) + i -= 1 + j -= 1 + crt += 1 + elif dp[i][j].align == ALIGN_SUB: + align.append((i, j, ALIGN_SUB)) + i -= 1 + j -= 1 + sub += 1 + elif dp[i][j].align == ALIGN_DEL: + align.append((i, j, ALIGN_DEL)) + i -= 1 + det += 1 + elif dp[i][j].align == ALIGN_INS: + align.append((i, j, ALIGN_INS)) + j -= 1 + ins += 1 + + err = sub + det + ins + align.reverse() + wer_info = WerInfo(ref_len, err, crt, sub, det, ins, align) + return wer_info + + + +class WerInfo: + def __init__(self, ref, err, crt, sub, dele, ins, ali): + self.r = ref + self.e = err + self.c = crt + self.s = sub + self.d = dele + self.i = ins + self.ali = ali + r = max(self.r, 1) + self.wer = 100.0 * (self.s + self.d + self.i) / r + + def __repr__(self): + s = f"wer {self.wer:.2f} ref {self.r:2d} sub {self.s:2d} del {self.d:2d} ins {self.i:2d}" + return s + + +class WerStats: + def __init__(self): + self.infos = [] + + def add(self, wer_info): + self.infos.append(wer_info) + + def print(self): + r = sum(info.r for info in self.infos) + if r <= 0: + print(f"REF len is {r}, check") + r = 1 + s = sum(info.s for info in self.infos) + d = sum(info.d for info in self.infos) + i = sum(info.i for info in self.infos) + se = 100.0 * s / r + de = 100.0 * d / r + ie = 100.0 * i / r + wer = 100.0 * (s + d + i) / r + sen = max(len(self.infos), 1) + errsen = sum(info.e > 0 for info in self.infos) + ser = 100.0 * errsen / sen + print("-"*80) + print(f"ref{r:6d} sub{s:6d} del{d:6d} ins{i:6d}") + print(f"WER{wer:6.2f} sub{se:6.2f} del{de:6.2f} ins{ie:6.2f}") + print(f"SER{ser:6.2f} = {errsen} / {sen}") + print("-"*80) + + +class EnDigStats: + def __init__(self): + self.n_en_word = 0 + self.n_en_correct = 0 + self.n_dig_word = 0 + self.n_dig_correct = 0 + + def add(self, n_en_word, n_en_correct, n_dig_word, n_dig_correct): + self.n_en_word += n_en_word + self.n_en_correct += n_en_correct + self.n_dig_word += n_dig_word + self.n_dig_correct += n_dig_correct + + def print(self): + print(f"English #word={self.n_en_word}, #correct={self.n_en_correct}\n" + f"Digit #word={self.n_dig_word}, #correct={self.n_dig_correct}") + print("-"*80) + + + +def count_english_ditgit(ref, hyp, wer_info): + patt_en = "[a-zA-Z\.\-\']+" + patt_dig = "[0-9]+" + patt_cjk = re.compile(r'([\u4e00-\u9fff])') + n_en_word = 0 + n_en_correct = 0 + n_dig_word = 0 + n_dig_correct = 0 + ali = wer_info.ali + for i, token in enumerate(ref): + if re.match(patt_en, token): + n_en_word += 1 + for y in ali: + if y[0] == i+1 and y[2] == ALIGN_CRT: + j = y[1] - 1 + n_en_correct += 1 + break + if re.match(patt_dig, token): + n_dig_word += 1 + for y in ali: + if y[0] == i+1 and y[2] == ALIGN_CRT: + j = y[1] - 1 + n_dig_correct += 1 + break + if not re.match(patt_cjk, token) and not re.match(patt_en, token) \ + and not re.match(patt_dig, token): + print("[WiredChar]:", token) + return n_en_word, n_en_correct, n_dig_word, n_dig_correct + + + +if __name__ == "__main__": + args = parser.parse_args() + print(args, flush=True) + main(args) diff --git a/fireredasr_onnx.py b/fireredasr_onnx.py new file mode 100644 index 0000000000000000000000000000000000000000..be5bed30b3edd6608aa3edc942a95b9862a6b2fa --- /dev/null +++ b/fireredasr_onnx.py @@ -0,0 +1,529 @@ +from fireredasr.data.asr_feat import ASRFeatExtractor +from fireredasr.tokenizer.aed_tokenizer import ChineseCharEnglishSpmTokenizer + +import onnxruntime as ort +import torch +import torch.nn.functional as F +import numpy as np +from torch import Tensor +from typing import Tuple, List, Dict +import argparse +import os +import time +import logging + +logger = logging.getLogger() +logger.setLevel(logging.INFO) +logger_stream_hander = logging.StreamHandler() +logger_stream_hander.setLevel("INFO") +logger.addHandler(logger_stream_hander) + + +INF = 1e10 + + +def to_numpy(tensor): + if isinstance(tensor, np.ndarray): + return tensor + if tensor.requires_grad: + return tensor.detach().cpu().numpy() + else: + return tensor.cpu().numpy() + + +def set_finished_beam_score_to_zero(scores, is_finished): + NB, B = scores.size() + is_finished = is_finished.float() + mask_score = torch.tensor([0.0] + [-INF] * (B - 1)).float() + mask_score = mask_score.view(1, B).repeat(NB, 1) + return scores * (1 - is_finished) + mask_score * is_finished + + +def set_finished_beam_y_to_eos(ys, is_finished, eos_id): + is_finished = is_finished.long() + return ys * (1 - is_finished) + eos_id * is_finished + + +class FireRedASROnnxModel: + def __init__( + self, + encoder_path: str, + decoder_path: str, + cmvn_file: str, + dict_file: str, + spm_model_path: str, + providers=["CUDAExecutionProvider"], + decode_max_len=128, + audio_dur=10, + ): + session_opts = ort.SessionOptions() + session_opts.inter_op_num_threads = 1 + session_opts.intra_op_num_threads = 1 + # session_opts.log_severity_level = 1 + self.session_opts = session_opts + + # NOTE: 参考whisper设置的最大的解码长度 + # FireRedASR-AED 模型支持的最长语音为 60s + # ref: https://github.com/FireRedTeam/FireRedASR?tab=readme-ov-file#input-length-limitations + self.decode_max_len = decode_max_len + + self.decoder_hidden_dim = 1280 + self.num_decoder_blocks = 16 + self.blank_id = 0 + self.sos_id = 3 + self.eos_id = 4 + self.pad_id = 2 + + self.feature_extractor = ASRFeatExtractor(cmvn_file) + self.tokenizer = ChineseCharEnglishSpmTokenizer(dict_file, spm_model_path) + self.encoder = None + self.decoder = None + self.audio_dur = audio_dur + + self.init_encoder(encoder_path, providers) + self.init_decoder_main(decoder_path, providers) + self.init_decoder_loop(decoder_path, providers) + self.pe = self.init_pe(decoder_path) + + def init_encoder(self, encoder_path, providers=None): + start_time = time.time() + self.encoder = ort.InferenceSession( + encoder_path, sess_options=self.session_opts, providers=providers + ) + end_time = time.time() + logger.info(f"load encoder cost {end_time - start_time} seconds") + + def init_decoder(self, decoder_path, providers=None): + start_time = time.time() + self.decoder = ort.InferenceSession( + decoder_path, sess_options=self.session_opts, providers=providers + ) + end_time = time.time() + logger.info(f"load decoder cost {end_time - start_time} seconds") + + def init_decoder_main(self, decoder_path, providers=None): + decoder_path = os.path.dirname(decoder_path) + decoder_path = os.path.join(decoder_path, "decoder_main.onnx") + start_time = time.time() + self.decoder_main = ort.InferenceSession( + decoder_path, sess_options=self.session_opts, providers=providers + ) + end_time = time.time() + logger.info(f"load decoder_main cost {end_time - start_time} seconds") + + input_names = [i.name for i in self.decoder_main.get_inputs()] + print(f"decoder_main.input_names: {input_names}") + + def init_decoder_loop(self, decoder_path, providers=None): + decoder_path = os.path.dirname(decoder_path) + decoder_path = os.path.join(decoder_path, "decoder_loop.onnx") + + start_time = time.time() + self.decoder_loop = ort.InferenceSession( + decoder_path, sess_options=self.session_opts, providers=providers + ) + end_time = time.time() + logger.info(f"load decoder_loop cost {end_time - start_time} seconds") + + input_names = [i.name for i in self.decoder_loop.get_inputs()] + print(f"decoder_loop.input_names: {input_names}") + + def init_pe(self, decoder_path): + decoder_path = os.path.dirname(decoder_path) + decoder_path = os.path.join(decoder_path, "pe.npy") + + return np.load(decoder_path) + + def run_encoder( + self, input: np.ndarray, input_length: np.ndarray + ) -> Tuple[Tensor, Tensor, Tensor]: + n_layer_cross_k, n_layer_cross_v, cross_attn_mask = self.encoder.run( + None, + { + self.encoder.get_inputs()[0].name: input, + self.encoder.get_inputs()[1].name: input_length, + }, + ) + return (n_layer_cross_k, n_layer_cross_v, cross_attn_mask) + + def decode_one_token( + self, + tokens: np.ndarray, + n_layer_self_k_cache: np.ndarray, + n_layer_self_v_cache: np.ndarray, + n_layer_cross_k_cache: np.ndarray, + n_layer_cross_v_cache: np.ndarray, + offset: np.ndarray, + self_attn_mask: np.ndarray, + cross_attn_mask: np.ndarray, + ) -> Tuple[Tensor, Tensor, Tensor]: + # print("decode:") + # print(f"tokens.shape: {tokens.shape}") + # print(f"n_layer_self_k_cache.shape: {n_layer_self_k_cache.shape}") + # print(f"n_layer_self_v_cache.shape: {n_layer_self_v_cache.shape}") + # print(f"n_layer_cross_k_cache.shape: {n_layer_cross_k_cache.shape}") + # print(f"n_layer_cross_v_cache.shape: {n_layer_cross_v_cache.shape}") + # print(f"offset.shape: {offset.shape}") + # print(f"self_attn_mask.shape: {self_attn_mask.shape}") + # print(f"cross_attn_mask.shape: {cross_attn_mask.shape}") + # print(f"self_attn_mask: {self_attn_mask}") + + logits, out_n_layer_self_k_cache, out_n_layer_self_v_cache = self.decoder.run( + None, + { + self.decoder.get_inputs()[0].name: tokens, + self.decoder.get_inputs()[1].name: n_layer_self_k_cache, + self.decoder.get_inputs()[2].name: n_layer_self_v_cache, + self.decoder.get_inputs()[3].name: n_layer_cross_k_cache, + self.decoder.get_inputs()[4].name: n_layer_cross_v_cache, + self.decoder.get_inputs()[5].name: offset, + self.decoder.get_inputs()[6].name: self_attn_mask, + self.decoder.get_inputs()[7].name: cross_attn_mask, + }, + ) + return (logits, out_n_layer_self_k_cache, out_n_layer_self_v_cache) + + def decode_main_one_token( + self, + tokens: np.ndarray, + n_layer_self_k_cache: np.ndarray, + n_layer_self_v_cache: np.ndarray, + n_layer_cross_k_cache: np.ndarray, + n_layer_cross_v_cache: np.ndarray, + pe: np.ndarray, + self_attn_mask: np.ndarray, + cross_attn_mask: np.ndarray, + ) -> Tuple[Tensor, Tensor, Tensor]: + # print("decode_main:") + # print(f"tokens.shape: {tokens.shape}") + # print(f"n_layer_self_k_cache.shape: {n_layer_self_k_cache.shape}") + # print(f"n_layer_self_v_cache.shape: {n_layer_self_v_cache.shape}") + # print(f"n_layer_cross_k_cache.shape: {n_layer_cross_k_cache.shape}") + # print(f"n_layer_cross_v_cache.shape: {n_layer_cross_v_cache.shape}") + # print(f"pe.shape: {pe.shape}") + # print(f"self_attn_mask.shape: {self_attn_mask.shape}") + # print(f"cross_attn_mask.shape: {cross_attn_mask.shape}") + + ( + logits, + out_n_layer_self_k_cache, + out_n_layer_self_v_cache, + ) = self.decoder_main.run( + None, + { + self.decoder_main.get_inputs()[0].name: tokens, + # self.decoder_main.get_inputs()[1].name: n_layer_self_k_cache, + self.decoder_main.get_inputs()[1].name: n_layer_cross_k_cache, + self.decoder_main.get_inputs()[2].name: n_layer_cross_v_cache, + # self.decoder_main.get_inputs()[3].name: pe, + # self.decoder_main.get_inputs()[4].name: self_attn_mask, + self.decoder_main.get_inputs()[3].name: cross_attn_mask, + # self.decoder_main.get_inputs()[7].name: cross_attn_mask, + }, + ) + return (logits, out_n_layer_self_k_cache, out_n_layer_self_v_cache) + + def decode_loop_one_token( + self, + tokens: np.ndarray, + n_layer_self_k_cache: np.ndarray, + n_layer_self_v_cache: np.ndarray, + n_layer_cross_k_cache: np.ndarray, + n_layer_cross_v_cache: np.ndarray, + pe: np.ndarray, + self_attn_mask: np.ndarray, + cross_attn_mask: np.ndarray, + ) -> Tuple[Tensor, Tensor, Tensor]: + # print("decode_loop:") + # print(f"tokens.shape: {tokens.shape}") + # print(f"n_layer_self_k_cache.shape: {n_layer_self_k_cache.shape}") + # print(f"n_layer_self_v_cache.shape: {n_layer_self_v_cache.shape}") + # print(f"n_layer_cross_k_cache.shape: {n_layer_cross_k_cache.shape}") + # print(f"n_layer_cross_v_cache.shape: {n_layer_cross_v_cache.shape}") + # print(f"pe.shape: {pe.shape}") + # print(f"self_attn_mask.shape: {self_attn_mask.shape}") + # print(f"cross_attn_mask.shape: {cross_attn_mask.shape}") + + ( + logits, + out_n_layer_self_k_cache, + out_n_layer_self_v_cache, + ) = self.decoder_loop.run( + None, + { + self.decoder_loop.get_inputs()[0].name: tokens, + self.decoder_loop.get_inputs()[1].name: n_layer_self_k_cache, + self.decoder_loop.get_inputs()[2].name: n_layer_self_v_cache, + self.decoder_loop.get_inputs()[3].name: n_layer_cross_k_cache, + self.decoder_loop.get_inputs()[4].name: n_layer_cross_v_cache, + self.decoder_loop.get_inputs()[5].name: pe, + self.decoder_loop.get_inputs()[6].name: self_attn_mask, + self.decoder_loop.get_inputs()[7].name: cross_attn_mask, + }, + ) + return (logits, out_n_layer_self_k_cache, out_n_layer_self_v_cache) + + def run_decoder( + self, n_layer_cross_k, n_layer_cross_v, cross_attn_mask, beam_size, nbest + ): + + num_layer, batch_size, Ti, encoder_out_dim = n_layer_cross_k.shape + encoder_out_length = cross_attn_mask.shape[-1] + + cross_attn_mask = torch.from_numpy(cross_attn_mask).to(torch.float32) + cross_attn_mask = ( + cross_attn_mask.unsqueeze(1) + .repeat(1, beam_size, 1, 1) + .view(beam_size * batch_size, -1, encoder_out_length) + ) + + n_layer_cross_k = torch.from_numpy(n_layer_cross_k) + n_layer_cross_v = torch.from_numpy(n_layer_cross_v) + n_layer_cross_k = ( + n_layer_cross_k.unsqueeze(2) + .repeat(1, 1, beam_size, 1, 1) + .view(num_layer, beam_size * batch_size, Ti, encoder_out_dim) + ) + n_layer_cross_v = ( + n_layer_cross_v.unsqueeze(2) + .repeat(1, 1, beam_size, 1, 1) + .view(num_layer, beam_size * batch_size, Ti, encoder_out_dim) + ) + + prediction_tokens = ( + torch.ones(beam_size * batch_size, 1).fill_(self.sos_id).long() + ) + tokens = prediction_tokens + offset = torch.zeros(1, dtype=torch.int64) + n_layer_self_k_cache, n_layer_self_v_cache = self.get_initialized_self_cache( + batch_size, beam_size + ) + + scores = torch.tensor([0.0] + [-INF] * (beam_size - 1)).float() + scores = scores.repeat(batch_size).view(batch_size * beam_size, 1) + is_finished = torch.zeros_like(scores) + + # self_attn_mask = torch.zeros( + # batch_size * beam_size, + # 1, 1 + # ) + + results = [self.sos_id] + for i in range(self.decode_max_len): + + # ==== ORIGIN ==== + # self_attn_mask = torch.empty( + # batch_size * beam_size, + # prediction_tokens.shape[-1], prediction_tokens.shape[-1] + # ).fill_(-np.inf).triu_(1) + # self_attn_mask = self_attn_mask[:, -1:, :] + # self_attn_mask = to_numpy(self_attn_mask) + + # logits, n_layer_self_k_cache, n_layer_self_v_cache = self.decode_one_token( + # to_numpy(tokens), + # to_numpy(n_layer_self_k_cache), + # to_numpy(n_layer_self_v_cache), + # to_numpy(n_layer_cross_k), + # to_numpy(n_layer_cross_v), + # to_numpy(offset), + # to_numpy(self_attn_mask), + # to_numpy(cross_attn_mask) + # ) + # ==== ORIGIN ==== + + tokens = to_numpy(tokens) + n_layer_self_k_cache = to_numpy(n_layer_self_k_cache) + n_layer_self_v_cache = to_numpy(n_layer_self_v_cache) + n_layer_cross_k = to_numpy(n_layer_cross_k) + n_layer_cross_v = to_numpy(n_layer_cross_v) + cross_attn_mask = to_numpy(cross_attn_mask) + + self_attn_mask = np.zeros( + (batch_size * beam_size, 1, self.decode_max_len), dtype=np.float32 + ) + self_attn_mask[:, :, : self.decode_max_len - offset[0] - 1] = -np.inf + + if i == 0: + ( + logits, + n_layer_self_k_cache, + n_layer_self_v_cache, + ) = self.decode_main_one_token( + to_numpy(tokens), + to_numpy(n_layer_self_k_cache), + to_numpy(n_layer_self_v_cache), + to_numpy(n_layer_cross_k), + to_numpy(n_layer_cross_v), + self.pe[0], + self_attn_mask, + to_numpy(cross_attn_mask), + ) + else: + ( + logits, + n_layer_self_k_cache, + n_layer_self_v_cache, + ) = self.decode_loop_one_token( + to_numpy(tokens), + to_numpy(n_layer_self_k_cache), + to_numpy(n_layer_self_v_cache), + to_numpy(n_layer_cross_k), + to_numpy(n_layer_cross_v), + self.pe[offset], + self_attn_mask, + to_numpy(cross_attn_mask), + ) + + # logits, n_layer_self_k_cache, n_layer_self_v_cache = self.decode_loop_one_token( + # to_numpy(tokens), + # to_numpy(n_layer_self_k_cache), + # to_numpy(n_layer_self_v_cache), + # to_numpy(n_layer_cross_k), + # to_numpy(n_layer_cross_v), + # self.pe[offset], + # self_attn_mask, + # to_numpy(cross_attn_mask) + # ) + + offset += 1 + logits = torch.from_numpy(logits) + + logits = logits.squeeze(1) + t_scores = F.log_softmax(logits, dim=-1) + t_topB_scores, t_topB_ys = torch.topk(t_scores, k=beam_size, dim=1) + t_topB_scores = set_finished_beam_score_to_zero(t_topB_scores, is_finished) + t_topB_ys = set_finished_beam_y_to_eos(t_topB_ys, is_finished, self.eos_id) + + scores = scores + t_topB_scores + + scores = scores.view(batch_size, beam_size * beam_size) + scores, topB_score_ids = torch.topk(scores, k=beam_size, dim=1) + scores = scores.view(-1, 1) + + topB_row_number_in_each_B_rows_of_ys = torch.div( + topB_score_ids, beam_size + ).view(batch_size * beam_size) + stride = beam_size * torch.arange(batch_size).view(batch_size, 1).repeat( + 1, beam_size + ).view(batch_size * beam_size) + topB_row_number_in_ys = ( + topB_row_number_in_each_B_rows_of_ys.long() + stride.long() + ) + + prediction_tokens = prediction_tokens[topB_row_number_in_ys] + t_ys = torch.gather( + t_topB_ys.view(batch_size, beam_size * beam_size), + dim=1, + index=topB_score_ids, + ).view(beam_size * batch_size, 1) + + tokens = t_ys + + prediction_tokens = torch.cat((prediction_tokens, t_ys), dim=1) + + n_layer_self_k_cache = torch.from_numpy(n_layer_self_k_cache) + n_layer_self_v_cache = torch.from_numpy(n_layer_self_v_cache) + + for i, self_k_cache in enumerate(n_layer_self_k_cache): + n_layer_self_k_cache[i] = n_layer_self_k_cache[i][topB_row_number_in_ys] + + for i, self_v_cache in enumerate(n_layer_self_v_cache): + n_layer_self_v_cache[i] = n_layer_self_v_cache[i][topB_row_number_in_ys] + + is_finished = t_ys.eq(self.eos_id) + if is_finished.sum().item() == beam_size * batch_size: + break + + scores = scores.view(batch_size, beam_size) + prediction_valid_token_lengths = torch.sum( + torch.ne(prediction_tokens.view(batch_size, beam_size, -1), self.eos_id), + dim=-1, + ).int() + + nbest_scores, nbest_ids = torch.topk(scores, k=nbest, dim=1) + index = ( + nbest_ids + beam_size * torch.arange(batch_size).view(batch_size, 1).long() + ) + nbest_prediction_tokens = prediction_tokens.view(batch_size * beam_size, -1)[ + index.view(-1) + ] + nbest_prediction_tokens = nbest_prediction_tokens.view( + batch_size, nbest_ids.size(1), -1 + ) + nbest_prediction_valid_token_lengths = prediction_valid_token_lengths.view( + batch_size * beam_size + )[index.view(-1)].view(batch_size, -1) + nbest_hyps: List[List[Dict[str, torch.Tensor]]] = [] + for i in range(batch_size): + i_best_hyps: List[Dict[str, torch.Tensor]] = [] + for j, score in enumerate(nbest_scores[i]): + hyp = { + "token_ids": nbest_prediction_tokens[ + i, j, 1 : nbest_prediction_valid_token_lengths[i, j] + ], + "score": score, + } + i_best_hyps.append(hyp) + nbest_hyps.append(i_best_hyps) + + return nbest_hyps + + def get_initialized_self_cache( + self, batch_size, beam_size + ) -> Tuple[Tensor, Tensor]: + n_layer_self_k_cache = torch.zeros( + self.num_decoder_blocks, + batch_size * beam_size, + self.decode_max_len, + self.decoder_hidden_dim, + ) + n_layer_self_v_cache = torch.zeros( + self.num_decoder_blocks, + batch_size * beam_size, + self.decode_max_len, + self.decoder_hidden_dim, + ) + return n_layer_self_k_cache, n_layer_self_v_cache + + def calc_feat_len(self, audio_dur): + import math + + sample_rate = 16000 + frame_length = 25 * sample_rate / 1000 + frame_shift = 10 * sample_rate / 1000 + length = math.floor((audio_dur * sample_rate - frame_length) / frame_shift) + 1 + return length + + def transcribe( + self, batch_wav_path: List[str], beam_size: int = 1, nbest: int = 1 + ) -> List[Dict]: + feats, lengths, wav_durations = self.feature_extractor(batch_wav_path) + maxlen = self.calc_feat_len(self.audio_dur) + if feats.shape[1] < maxlen: + feats = np.concatenate( + [feats, np.zeros((1, maxlen - feats.shape[1], 80), dtype=np.float32)], + axis=1, + ) + feats = feats[:, :maxlen, :] + lengths = torch.minimum(lengths, torch.tensor(maxlen)) + + feats = to_numpy(feats) + lengths = to_numpy(lengths) + + start_time = time.time() + n_layer_cross_k, n_layer_cross_v, cross_attn_mask = self.run_encoder( + to_numpy(feats), to_numpy(lengths) + ) + nbest_hyps = self.run_decoder( + n_layer_cross_k, n_layer_cross_v, cross_attn_mask, beam_size, nbest + ) + transcribe_durations = time.time() - start_time + results: List[Dict] = [] + for wav, hyp in zip(batch_wav_path, nbest_hyps): + hyp = hyp[0] + hyp_ids = [int(id) for id in hyp["token_ids"].cpu()] + score = hyp["score"].item() + text = self.tokenizer.detokenize(hyp_ids) + results.append({"wav": wav, "text": text, "score": score}) + + return results, wav_durations, transcribe_durations diff --git a/fsmn_vad/am.mvn b/fsmn_vad/am.mvn new file mode 100644 index 0000000000000000000000000000000000000000..77070ed5b1c53322ccbcd1e866e715a8b08a5d4c --- /dev/null +++ b/fsmn_vad/am.mvn @@ -0,0 +1,8 @@ + + 400 400 +[ 0 ] + 400 400 + 0 [ -8.311879 -8.600912 -9.615928 -10.43595 -11.21292 -11.88333 -12.36243 -12.63706 -12.8818 -12.83066 -12.89103 -12.95666 -13.19763 -13.40598 -13.49113 -13.5546 -13.55639 -13.51915 -13.68284 -13.53289 -13.42107 -13.65519 -13.50713 -13.75251 -13.76715 -13.87408 -13.73109 -13.70412 -13.56073 -13.53488 -13.54895 -13.56228 -13.59408 -13.62047 -13.64198 -13.66109 -13.62669 -13.58297 -13.57387 -13.4739 -13.53063 -13.48348 -13.61047 -13.64716 -13.71546 -13.79184 -13.90614 -14.03098 -14.18205 -14.35881 -14.48419 -14.60172 -14.70591 -14.83362 -14.92122 -15.00622 -15.05122 -15.03119 -14.99028 -14.92302 -14.86927 -14.82691 -14.7972 -14.76909 -14.71356 -14.61277 -14.51696 -14.42252 -14.36405 -14.30451 -14.23161 -14.19851 -14.16633 -14.15649 -14.10504 -13.99518 -13.79562 -13.3996 -12.7767 -11.71208 -8.311879 -8.600912 -9.615928 -10.43595 -11.21292 -11.88333 -12.36243 -12.63706 -12.8818 -12.83066 -12.89103 -12.95666 -13.19763 -13.40598 -13.49113 -13.5546 -13.55639 -13.51915 -13.68284 -13.53289 -13.42107 -13.65519 -13.50713 -13.75251 -13.76715 -13.87408 -13.73109 -13.70412 -13.56073 -13.53488 -13.54895 -13.56228 -13.59408 -13.62047 -13.64198 -13.66109 -13.62669 -13.58297 -13.57387 -13.4739 -13.53063 -13.48348 -13.61047 -13.64716 -13.71546 -13.79184 -13.90614 -14.03098 -14.18205 -14.35881 -14.48419 -14.60172 -14.70591 -14.83362 -14.92122 -15.00622 -15.05122 -15.03119 -14.99028 -14.92302 -14.86927 -14.82691 -14.7972 -14.76909 -14.71356 -14.61277 -14.51696 -14.42252 -14.36405 -14.30451 -14.23161 -14.19851 -14.16633 -14.15649 -14.10504 -13.99518 -13.79562 -13.3996 -12.7767 -11.71208 -8.311879 -8.600912 -9.615928 -10.43595 -11.21292 -11.88333 -12.36243 -12.63706 -12.8818 -12.83066 -12.89103 -12.95666 -13.19763 -13.40598 -13.49113 -13.5546 -13.55639 -13.51915 -13.68284 -13.53289 -13.42107 -13.65519 -13.50713 -13.75251 -13.76715 -13.87408 -13.73109 -13.70412 -13.56073 -13.53488 -13.54895 -13.56228 -13.59408 -13.62047 -13.64198 -13.66109 -13.62669 -13.58297 -13.57387 -13.4739 -13.53063 -13.48348 -13.61047 -13.64716 -13.71546 -13.79184 -13.90614 -14.03098 -14.18205 -14.35881 -14.48419 -14.60172 -14.70591 -14.83362 -14.92122 -15.00622 -15.05122 -15.03119 -14.99028 -14.92302 -14.86927 -14.82691 -14.7972 -14.76909 -14.71356 -14.61277 -14.51696 -14.42252 -14.36405 -14.30451 -14.23161 -14.19851 -14.16633 -14.15649 -14.10504 -13.99518 -13.79562 -13.3996 -12.7767 -11.71208 -8.311879 -8.600912 -9.615928 -10.43595 -11.21292 -11.88333 -12.36243 -12.63706 -12.8818 -12.83066 -12.89103 -12.95666 -13.19763 -13.40598 -13.49113 -13.5546 -13.55639 -13.51915 -13.68284 -13.53289 -13.42107 -13.65519 -13.50713 -13.75251 -13.76715 -13.87408 -13.73109 -13.70412 -13.56073 -13.53488 -13.54895 -13.56228 -13.59408 -13.62047 -13.64198 -13.66109 -13.62669 -13.58297 -13.57387 -13.4739 -13.53063 -13.48348 -13.61047 -13.64716 -13.71546 -13.79184 -13.90614 -14.03098 -14.18205 -14.35881 -14.48419 -14.60172 -14.70591 -14.83362 -14.92122 -15.00622 -15.05122 -15.03119 -14.99028 -14.92302 -14.86927 -14.82691 -14.7972 -14.76909 -14.71356 -14.61277 -14.51696 -14.42252 -14.36405 -14.30451 -14.23161 -14.19851 -14.16633 -14.15649 -14.10504 -13.99518 -13.79562 -13.3996 -12.7767 -11.71208 -8.311879 -8.600912 -9.615928 -10.43595 -11.21292 -11.88333 -12.36243 -12.63706 -12.8818 -12.83066 -12.89103 -12.95666 -13.19763 -13.40598 -13.49113 -13.5546 -13.55639 -13.51915 -13.68284 -13.53289 -13.42107 -13.65519 -13.50713 -13.75251 -13.76715 -13.87408 -13.73109 -13.70412 -13.56073 -13.53488 -13.54895 -13.56228 -13.59408 -13.62047 -13.64198 -13.66109 -13.62669 -13.58297 -13.57387 -13.4739 -13.53063 -13.48348 -13.61047 -13.64716 -13.71546 -13.79184 -13.90614 -14.03098 -14.18205 -14.35881 -14.48419 -14.60172 -14.70591 -14.83362 -14.92122 -15.00622 -15.05122 -15.03119 -14.99028 -14.92302 -14.86927 -14.82691 -14.7972 -14.76909 -14.71356 -14.61277 -14.51696 -14.42252 -14.36405 -14.30451 -14.23161 -14.19851 -14.16633 -14.15649 -14.10504 -13.99518 -13.79562 -13.3996 -12.7767 -11.71208 ] + 400 400 + 0 [ 0.155775 0.154484 0.1527379 0.1518718 0.1506028 0.1489256 0.147067 0.1447061 0.1436307 0.1443568 0.1451849 0.1455157 0.1452821 0.1445717 0.1439195 0.1435867 0.1436018 0.1438781 0.1442086 0.1448844 0.1454756 0.145663 0.146268 0.1467386 0.1472724 0.147664 0.1480913 0.1483739 0.1488841 0.1493636 0.1497088 0.1500379 0.1502916 0.1505389 0.1506787 0.1507102 0.1505992 0.1505445 0.1505938 0.1508133 0.1509569 0.1512396 0.1514625 0.1516195 0.1516156 0.1515561 0.1514966 0.1513976 0.1512612 0.151076 0.1510596 0.1510431 0.151077 0.1511168 0.1511917 0.151023 0.1508045 0.1505885 0.1503493 0.1502373 0.1501726 0.1500762 0.1500065 0.1499782 0.150057 0.1502658 0.150469 0.1505335 0.1505505 0.1505328 0.1504275 0.1502438 0.1499674 0.1497118 0.1494661 0.1493102 0.1493681 0.1495501 0.1499738 0.1509654 0.155775 0.154484 0.1527379 0.1518718 0.1506028 0.1489256 0.147067 0.1447061 0.1436307 0.1443568 0.1451849 0.1455157 0.1452821 0.1445717 0.1439195 0.1435867 0.1436018 0.1438781 0.1442086 0.1448844 0.1454756 0.145663 0.146268 0.1467386 0.1472724 0.147664 0.1480913 0.1483739 0.1488841 0.1493636 0.1497088 0.1500379 0.1502916 0.1505389 0.1506787 0.1507102 0.1505992 0.1505445 0.1505938 0.1508133 0.1509569 0.1512396 0.1514625 0.1516195 0.1516156 0.1515561 0.1514966 0.1513976 0.1512612 0.151076 0.1510596 0.1510431 0.151077 0.1511168 0.1511917 0.151023 0.1508045 0.1505885 0.1503493 0.1502373 0.1501726 0.1500762 0.1500065 0.1499782 0.150057 0.1502658 0.150469 0.1505335 0.1505505 0.1505328 0.1504275 0.1502438 0.1499674 0.1497118 0.1494661 0.1493102 0.1493681 0.1495501 0.1499738 0.1509654 0.155775 0.154484 0.1527379 0.1518718 0.1506028 0.1489256 0.147067 0.1447061 0.1436307 0.1443568 0.1451849 0.1455157 0.1452821 0.1445717 0.1439195 0.1435867 0.1436018 0.1438781 0.1442086 0.1448844 0.1454756 0.145663 0.146268 0.1467386 0.1472724 0.147664 0.1480913 0.1483739 0.1488841 0.1493636 0.1497088 0.1500379 0.1502916 0.1505389 0.1506787 0.1507102 0.1505992 0.1505445 0.1505938 0.1508133 0.1509569 0.1512396 0.1514625 0.1516195 0.1516156 0.1515561 0.1514966 0.1513976 0.1512612 0.151076 0.1510596 0.1510431 0.151077 0.1511168 0.1511917 0.151023 0.1508045 0.1505885 0.1503493 0.1502373 0.1501726 0.1500762 0.1500065 0.1499782 0.150057 0.1502658 0.150469 0.1505335 0.1505505 0.1505328 0.1504275 0.1502438 0.1499674 0.1497118 0.1494661 0.1493102 0.1493681 0.1495501 0.1499738 0.1509654 0.155775 0.154484 0.1527379 0.1518718 0.1506028 0.1489256 0.147067 0.1447061 0.1436307 0.1443568 0.1451849 0.1455157 0.1452821 0.1445717 0.1439195 0.1435867 0.1436018 0.1438781 0.1442086 0.1448844 0.1454756 0.145663 0.146268 0.1467386 0.1472724 0.147664 0.1480913 0.1483739 0.1488841 0.1493636 0.1497088 0.1500379 0.1502916 0.1505389 0.1506787 0.1507102 0.1505992 0.1505445 0.1505938 0.1508133 0.1509569 0.1512396 0.1514625 0.1516195 0.1516156 0.1515561 0.1514966 0.1513976 0.1512612 0.151076 0.1510596 0.1510431 0.151077 0.1511168 0.1511917 0.151023 0.1508045 0.1505885 0.1503493 0.1502373 0.1501726 0.1500762 0.1500065 0.1499782 0.150057 0.1502658 0.150469 0.1505335 0.1505505 0.1505328 0.1504275 0.1502438 0.1499674 0.1497118 0.1494661 0.1493102 0.1493681 0.1495501 0.1499738 0.1509654 0.155775 0.154484 0.1527379 0.1518718 0.1506028 0.1489256 0.147067 0.1447061 0.1436307 0.1443568 0.1451849 0.1455157 0.1452821 0.1445717 0.1439195 0.1435867 0.1436018 0.1438781 0.1442086 0.1448844 0.1454756 0.145663 0.146268 0.1467386 0.1472724 0.147664 0.1480913 0.1483739 0.1488841 0.1493636 0.1497088 0.1500379 0.1502916 0.1505389 0.1506787 0.1507102 0.1505992 0.1505445 0.1505938 0.1508133 0.1509569 0.1512396 0.1514625 0.1516195 0.1516156 0.1515561 0.1514966 0.1513976 0.1512612 0.151076 0.1510596 0.1510431 0.151077 0.1511168 0.1511917 0.151023 0.1508045 0.1505885 0.1503493 0.1502373 0.1501726 0.1500762 0.1500065 0.1499782 0.150057 0.1502658 0.150469 0.1505335 0.1505505 0.1505328 0.1504275 0.1502438 0.1499674 0.1497118 0.1494661 0.1493102 0.1493681 0.1495501 0.1499738 0.1509654 ] + \ No newline at end of file diff --git a/fsmn_vad/fsmn_vad_10s_fp32.axmodel b/fsmn_vad/fsmn_vad_10s_fp32.axmodel new file mode 100644 index 0000000000000000000000000000000000000000..b1618dff9bf6ea829f88c253a9f778d6942a56b1 --- /dev/null +++ b/fsmn_vad/fsmn_vad_10s_fp32.axmodel @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:75b77c4d40aa37fc23707970079053ba8d325c1b2c17c50c844d8c72301099d4 +size 2439506 diff --git a/openai/firered_asr.py b/openai/firered_asr.py new file mode 100644 index 0000000000000000000000000000000000000000..9547213dbb6a01010b451a6c0891964caafff164 --- /dev/null +++ b/openai/firered_asr.py @@ -0,0 +1,173 @@ +"""FireRedASR-AED numpy+pyaxengine ASR 核心（板端，无 torch/onnxruntime）。""" +import math +from pathlib import Path + +import kaldiio +import kaldi_native_fbank as knf +import numpy as np + +from fsmn_vad_post import vad_segments + + +def _fbank_cmvn(pcm_int16, means, invstds, max_feat_len): + opts = knf.FbankOptions() + opts.frame_opts.dither = 0.0 + opts.frame_opts.snip_edges = True + opts.mel_opts.num_bins = 80 + fbank = knf.OnlineFbank(opts) + fbank.accept_waveform(16000, pcm_int16.tolist()) + n = fbank.num_frames_ready + feat_len = min(n, max_feat_len) + feats = np.zeros((max_feat_len, 80), dtype=np.float32) + for i in range(feat_len): + frame = np.array(fbank.get_frame(i), dtype=np.float32) + feats[i] = (frame - means) * invstds + return feats[None], feat_len + + +def load_cmvn(path): + stats = kaldiio.load_mat(path) + dim = stats.shape[-1] - 1 + count = stats[0, dim] + means = stats[0, :dim] / count + var = np.maximum(stats[1, :dim] / count - means * means, 1e-20) + return np.asarray(means, dtype=np.float32), np.asarray(1.0 / np.sqrt(var), dtype=np.float32) + + +def load_dict(path): + id2word = {} + for line in open(path, encoding="utf-8"): + p = line.strip().split() + if len(p) >= 2: + id2word[int(p[1])] = p[0] + return id2word + + +def load_fsmn_cmvn(path): + means, vars_ = [], [] + lines = open(path, encoding="utf-8").readlines() + for i, line in enumerate(lines): + items = line.split() + if not items: + continue + if items[0] == "": + j = i + 1 + if j < len(lines) and lines[j].split()[0] == "": + means = [float(x) for x in lines[j].split()[3:-1]] + elif items[0] == "": + j = i + 1 + if j < len(lines) and lines[j].split()[0] == "": + vars_ = [float(x) for x in lines[j].split()[3:-1]] + return np.asarray(means, dtype=np.float64), np.asarray(vars_, dtype=np.float64) + + +def _fsmn_feats(chunk, means, vars_): + """FSMN-VAD 前端：fbank(80) + LFR(5) + CMVN，输出 [1,998,400] float32。""" + opts = knf.FbankOptions() + opts.frame_opts.dither = 0.0 + opts.frame_opts.window_type = "hamming" + opts.frame_opts.frame_shift_ms = 10.0 + opts.frame_opts.frame_length_ms = 25.0 + opts.mel_opts.num_bins = 80 + opts.energy_floor = 0.0 + opts.frame_opts.snip_edges = True + fbank = knf.OnlineFbank(opts) + fbank.accept_waveform(16000, (chunk * 32768).tolist()) + n = fbank.num_frames_ready + feat = np.empty((n, 80), dtype=np.float32) + for i in range(n): + feat[i] = fbank.get_frame(i) + left = np.tile(feat[0], (2, 1)) + feat = np.vstack((left, feat)) + out = np.stack([ + feat[i:i + 5].reshape(-1) for i in range(len(feat) - 4) + ]).astype(np.float32) + out = (out + means) * vars_ + if len(out) < 998: + out = np.pad(out, ((0, 998 - len(out)), (0, 0))) + return np.asarray(out[None], dtype=np.float32) + + +class FireredASR: + def __init__(self, encoder, decoder, fsmn_vad, fsmn_cmvn, + cmvn, dict_path, pe, max_dur=10, max_steps=128, + fsmn_threshold=0.6, vad_min_speech_ms=1000): + import axengine as axe + + self.enc = axe.InferenceSession(encoder) + self.dec = axe.InferenceSession(decoder) + self.fsmn = axe.InferenceSession(fsmn_vad) + self.fsmn_means, self.fsmn_vars = load_fsmn_cmvn(fsmn_cmvn) + self.fsmn_threshold = fsmn_threshold + self.vad_min_speech_ms = vad_min_speech_ms + self.means, self.invstds = load_cmvn(cmvn) + self.id2word = load_dict(dict_path) + self.pe = np.ascontiguousarray(np.load(pe), dtype=np.float32) + self.max_feat_len = math.floor((max_dur * 16000 - 400) / 160) + 1 + self.max_chunk_samples = max_dur * 16000 + self.max_steps = max_steps + self.cache_len = self.dec.get_inputs()[1].shape[2] + self.vocab = self.dec.get_outputs()[0].shape[2] + + def _detok(self, ids): + s = "".join(self.id2word.get(i, "") for i in ids if i != 4) + return s.replace("▁", " ").strip() + + def _transcribe_chunk(self, chunk): + pcm16 = (np.clip(chunk, -1, 1) * 32768).astype(np.int16) + feats, length = _fbank_cmvn(pcm16, self.means, self.invstds, self.max_feat_len) + cross_k, cross_v, cross_mask = self.enc.run( + None, {"encoder_input": feats, + "encoder_input_lengths": np.array([length], dtype=np.int32)}) + n_layers, hidden = 16, 1280 + k_cache = np.zeros((n_layers, 1, self.cache_len, hidden), dtype=np.float32) + v_cache = np.zeros((n_layers, 1, self.cache_len, hidden), dtype=np.float32) + tokens = np.array([[3]], dtype=np.int32) + masks = [] + for off in range(self.cache_len): + m = np.zeros((1, 1, self.cache_len), dtype=np.float32) + m[:, :, : self.cache_len - off - 1] = -np.inf + masks.append(m) + ids = [] + for off in range(self.max_steps): + logits, k_cache, v_cache = self.dec.run( + None, { + "tokens": tokens, + "in_n_layer_self_k_cache": k_cache, + "in_n_layer_self_v_cache": v_cache, + "n_layer_cross_k": cross_k, + "n_layer_cross_v": cross_v, + "pe": self.pe[off], + "self_attn_mask": masks[off], + "cross_attn_mask": cross_mask, + }) + nxt = int(np.argmax(logits[0, 0])) + tokens[0, 0] = nxt + ids.append(nxt) + if nxt == 4: + break + return ids + + def transcribe(self, wav_path): + import soundfile as sf + data, sr = sf.read(wav_path, dtype="float32") + if data.ndim == 2: + data = data.mean(axis=1) # 立体声/多声道混单声道 + if sr != 16000: + x = np.arange(0, len(data), sr / 16000) + data = np.interp(x, np.arange(len(data)), data).astype(np.float32) + + all_tokens = [] + step = self.max_chunk_samples + for start in range(0, len(data), step): + chunk = data[start:min(start + step, len(data))] + if len(chunk) < step: + chunk = np.pad(chunk, (0, step - len(chunk))) + feats = _fsmn_feats(chunk, self.fsmn_means, self.fsmn_vars) + logits = np.asarray(self.fsmn.run(None, {"speech": feats})[0]) + segs = vad_segments(logits, chunk[None], self.fsmn_threshold) + if sum(e - s for s, e in segs) < self.vad_min_speech_ms: + continue # 静音/噪声块不送 ASR + all_tokens.extend(self._transcribe_chunk( + data[start:min(start + step, len(data))])) + return self._detok(all_tokens) if all_tokens else "" diff --git a/openai/fsmn_vad_post.py b/openai/fsmn_vad_post.py new file mode 100644 index 0000000000000000000000000000000000000000..41dac31f5f8673204b90c5bac671b9d258b7260d --- /dev/null +++ b/openai/fsmn_vad_post.py @@ -0,0 +1,828 @@ +# -*- coding:utf-8 -*- +# @FileName :e2e_vad.py +# @Time :2023/4/3 17:02 +# @Author :lovemefan +# @Email :lovemefan@outlook.com +import logging +import math +from enum import Enum +from pathlib import Path +from typing import Any, Dict, List, Tuple + +import numpy as np + + +class VadStateMachine(Enum): + kVadInStateStartPointNotDetected = 1 + kVadInStateInSpeechSegment = 2 + kVadInStateEndPointDetected = 3 + + +class FrameState(Enum): + kFrameStateInvalid = -1 + kFrameStateSpeech = 1 + kFrameStateSil = 0 + + +# final voice/unvoice state per frame +class AudioChangeState(Enum): + kChangeStateSpeech2Speech = 0 + kChangeStateSpeech2Sil = 1 + kChangeStateSil2Sil = 2 + kChangeStateSil2Speech = 3 + kChangeStateNoBegin = 4 + kChangeStateInvalid = 5 + + +class VadDetectMode(Enum): + kVadSingleUtteranceDetectMode = 0 + kVadMutipleUtteranceDetectMode = 1 + + +class VADXOptions: + def __init__( + self, + sample_rate: int = 16000, + detect_mode: int = VadDetectMode.kVadMutipleUtteranceDetectMode.value, + snr_mode: int = 0, + max_end_silence_time: int = 800, + max_start_silence_time: int = 3000, + do_start_point_detection: bool = True, + do_end_point_detection: bool = True, + window_size_ms: int = 200, + sil_to_speech_time_thres: int = 150, + speech_to_sil_time_thres: int = 150, + speech_2_noise_ratio: float = 1.0, + do_extend: int = 1, + lookback_time_start_point: int = 200, + lookahead_time_end_point: int = 100, + max_single_segment_time: int = 60000, + nn_eval_block_size: int = 8, + dcd_block_size: int = 4, + snr_thres: int = -100.0, + noise_frame_num_used_for_snr: int = 100, + decibel_thres: int = -100.0, + speech_noise_thres: float = 0.6, + fe_prior_thres: float = 1e-4, + silence_pdf_num: int = 1, + sil_pdf_ids: List[int] = [0], + speech_noise_thresh_low: float = -0.1, + speech_noise_thresh_high: float = 0.3, + output_frame_probs: bool = False, + frame_in_ms: int = 10, + frame_length_ms: int = 25, + ): + self.sample_rate = sample_rate + self.detect_mode = detect_mode + self.snr_mode = snr_mode + self.max_end_silence_time = max_end_silence_time + self.max_start_silence_time = max_start_silence_time + self.do_start_point_detection = do_start_point_detection + self.do_end_point_detection = do_end_point_detection + self.window_size_ms = window_size_ms + self.sil_to_speech_time_thres = sil_to_speech_time_thres + self.speech_to_sil_time_thres = speech_to_sil_time_thres + self.speech_2_noise_ratio = speech_2_noise_ratio + self.do_extend = do_extend + self.lookback_time_start_point = lookback_time_start_point + self.lookahead_time_end_point = lookahead_time_end_point + self.max_single_segment_time = max_single_segment_time + self.nn_eval_block_size = nn_eval_block_size + self.dcd_block_size = dcd_block_size + self.snr_thres = snr_thres + self.noise_frame_num_used_for_snr = noise_frame_num_used_for_snr + self.decibel_thres = decibel_thres + self.speech_noise_thres = speech_noise_thres + self.fe_prior_thres = fe_prior_thres + self.silence_pdf_num = silence_pdf_num + self.sil_pdf_ids = sil_pdf_ids + self.speech_noise_thresh_low = speech_noise_thresh_low + self.speech_noise_thresh_high = speech_noise_thresh_high + self.output_frame_probs = output_frame_probs + self.frame_in_ms = frame_in_ms + self.frame_length_ms = frame_length_ms + + +class E2EVadSpeechBufWithDoa(object): + def __init__(self): + self.start_ms = 0 + self.end_ms = 0 + self.buffer = [] + self.contain_seg_start_point = False + self.contain_seg_end_point = False + self.doa = 0 + + def reset(self): + self.start_ms = 0 + self.end_ms = 0 + self.buffer = [] + self.contain_seg_start_point = False + self.contain_seg_end_point = False + self.doa = 0 + + +class E2EVadFrameProb(object): + def __init__(self): + self.noise_prob = 0.0 + self.speech_prob = 0.0 + self.score = 0.0 + self.frame_id = 0 + self.frm_state = 0 + + +class WindowDetector(object): + def __init__( + self, + window_size_ms: int, + sil_to_speech_time: int, + speech_to_sil_time: int, + frame_size_ms: int, + ): + self.window_size_ms = window_size_ms + self.sil_to_speech_time = sil_to_speech_time + self.speech_to_sil_time = speech_to_sil_time + self.frame_size_ms = frame_size_ms + + self.win_size_frame = int(window_size_ms / frame_size_ms) + self.win_sum = 0 + self.win_state = [0] * self.win_size_frame # 初始化窗 + + self.cur_win_pos = 0 + self.pre_frame_state = FrameState.kFrameStateSil + self.cur_frame_state = FrameState.kFrameStateSil + self.sil_to_speech_frmcnt_thres = int(sil_to_speech_time / frame_size_ms) + self.speech_to_sil_frmcnt_thres = int(speech_to_sil_time / frame_size_ms) + + self.voice_last_frame_count = 0 + self.noise_last_frame_count = 0 + self.hydre_frame_count = 0 + + def reset(self) -> None: + self.cur_win_pos = 0 + self.win_sum = 0 + self.win_state = [0] * self.win_size_frame + self.pre_frame_state = FrameState.kFrameStateSil + self.cur_frame_state = FrameState.kFrameStateSil + self.voice_last_frame_count = 0 + self.noise_last_frame_count = 0 + self.hydre_frame_count = 0 + + def get_win_size(self) -> int: + return int(self.win_size_frame) + + def detect_one_frame( + self, frameState: FrameState, frame_count: int + ) -> AudioChangeState: + cur_frame_state = FrameState.kFrameStateSil + if frameState == FrameState.kFrameStateSpeech: + cur_frame_state = 1 + elif frameState == FrameState.kFrameStateSil: + cur_frame_state = 0 + else: + return AudioChangeState.kChangeStateInvalid + self.win_sum -= self.win_state[self.cur_win_pos] + self.win_sum += cur_frame_state + self.win_state[self.cur_win_pos] = cur_frame_state + self.cur_win_pos = (self.cur_win_pos + 1) % self.win_size_frame + + if ( + self.pre_frame_state == FrameState.kFrameStateSil + and self.win_sum >= self.sil_to_speech_frmcnt_thres + ): + self.pre_frame_state = FrameState.kFrameStateSpeech + return AudioChangeState.kChangeStateSil2Speech + + if ( + self.pre_frame_state == FrameState.kFrameStateSpeech + and self.win_sum <= self.speech_to_sil_frmcnt_thres + ): + self.pre_frame_state = FrameState.kFrameStateSil + return AudioChangeState.kChangeStateSpeech2Sil + + if self.pre_frame_state == FrameState.kFrameStateSil: + return AudioChangeState.kChangeStateSil2Sil + if self.pre_frame_state == FrameState.kFrameStateSpeech: + return AudioChangeState.kChangeStateSpeech2Speech + return AudioChangeState.kChangeStateInvalid + + def frame_size_ms(self) -> int: + return int(self.frame_size_ms) + + +class E2EVadModel: + def __init__(self, speech_noise_thres: float = 0.6): + super(E2EVadModel, self).__init__() + vad_post_args = { + "sample_rate": 16000, "detect_mode": 1, "snr_mode": 0, + "max_end_silence_time": 800, "max_start_silence_time": 3000, + "do_start_point_detection": True, "do_end_point_detection": True, + "window_size_ms": 200, "sil_to_speech_time_thres": 150, + "speech_to_sil_time_thres": 150, "speech_2_noise_ratio": 1.0, + "do_extend": 1, "lookback_time_start_point": 200, + "lookahead_time_end_point": 100, "max_single_segment_time": 20000, + "nn_eval_block_size": 8, "dcd_block_size": 4, + "snr_thres": -100.0, "noise_frame_num_used_for_snr": 100, + "decibel_thres": -100.0, "speech_noise_thres": speech_noise_thres, + "fe_prior_thres": 1e-4, "silence_pdf_num": 1, + "sil_pdf_ids": [0], "speech_noise_thresh_low": -0.1, + "speech_noise_thresh_high": 0.3, "output_frame_probs": False, + "frame_in_ms": 10, "frame_length_ms": 25, + } + self.vad_opts = VADXOptions(**vad_post_args) + self.windows_detector = WindowDetector( + self.vad_opts.window_size_ms, + self.vad_opts.sil_to_speech_time_thres, + self.vad_opts.speech_to_sil_time_thres, + self.vad_opts.frame_in_ms, + ) + self._logits = None + # init variables + self.is_final = False + self.data_buf_start_frame = 0 + self.frm_cnt = 0 + self.latest_confirmed_speech_frame = 0 + self.lastest_confirmed_silence_frame = -1 + self.continous_silence_frame_count = 0 + self.vad_state_machine = VadStateMachine.kVadInStateStartPointNotDetected + self.confirmed_start_frame = -1 + self.confirmed_end_frame = -1 + self.number_end_time_detected = 0 + self.sil_frame = 0 + self.sil_pdf_ids = self.vad_opts.sil_pdf_ids + self.noise_average_decibel = -100.0 + self.pre_end_silence_detected = False + self.next_seg = True + + self.output_data_buf = [] + self.output_data_buf_offset = 0 + self.frame_probs = [] + self.max_end_sil_frame_cnt_thresh = ( + self.vad_opts.max_end_silence_time - self.vad_opts.speech_to_sil_time_thres + ) + self.speech_noise_thres = self.vad_opts.speech_noise_thres + self.scores = None + self.max_time_out = False + self.decibel = [] + self.data_buf = None + self.data_buf_all = None + self.waveform = None + self.reset_detection() + + def all_reset_detection(self): + self.is_final = False + self.data_buf_start_frame = 0 + self.frm_cnt = 0 + self.latest_confirmed_speech_frame = 0 + self.lastest_confirmed_silence_frame = -1 + self.continous_silence_frame_count = 0 + self.vad_state_machine = VadStateMachine.kVadInStateStartPointNotDetected + self.confirmed_start_frame = -1 + self.confirmed_end_frame = -1 + self.number_end_time_detected = 0 + self.sil_frame = 0 + self.sil_pdf_ids = self.vad_opts.sil_pdf_ids + self.noise_average_decibel = -100.0 + self.pre_end_silence_detected = False + self.next_seg = True + + self.output_data_buf = [] + self.output_data_buf_offset = 0 + self.frame_probs = [] + self.max_end_sil_frame_cnt_thresh = ( + self.vad_opts.max_end_silence_time - self.vad_opts.speech_to_sil_time_thres + ) + self.speech_noise_thres = self.vad_opts.speech_noise_thres + self.scores = None + self.max_time_out = False + self.decibel = [] + self.data_buf = None + self.data_buf_all = None + self.waveform = None + self.reset_detection() + + def reset_detection(self): + self.continous_silence_frame_count = 0 + self.latest_confirmed_speech_frame = 0 + self.lastest_confirmed_silence_frame = -1 + self.confirmed_start_frame = -1 + self.confirmed_end_frame = -1 + self.vad_state_machine = VadStateMachine.kVadInStateStartPointNotDetected + self.windows_detector.reset() + self.sil_frame = 0 + self.frame_probs = [] + + def compute_decibel(self) -> None: + frame_sample_length = int( + self.vad_opts.frame_length_ms * self.vad_opts.sample_rate / 1000 + ) + frame_shift_length = int( + self.vad_opts.frame_in_ms * self.vad_opts.sample_rate / 1000 + ) + if self.data_buf_all is None: + self.data_buf_all = self.waveform[ + 0 + ] # self.data_buf is pointed to self.waveform[0] + self.data_buf = self.data_buf_all + else: + self.data_buf_all = np.concatenate((self.data_buf_all, self.waveform[0])) + for offset in range( + 0, self.waveform.shape[1] - frame_sample_length + 1, frame_shift_length + ): + self.decibel.append( + 10 + * np.log10( + np.square( + self.waveform[0][offset: offset + frame_sample_length] + ).sum() + + 1e-6 + ) + ) + + def compute_scores(self, feats: np.ndarray) -> None: + scores = [self._logits] + self.vad_opts.nn_eval_block_size = scores[0].shape[1] + self.frm_cnt += scores[0].shape[1] # count total frames + if isinstance(feats, list): + # return B * T * D + feats = feats[0] + + assert ( + scores[0].shape[1] == feats.shape[1] + ), "The shape between feats and scores does not match" + + if self.scores is None: + self.scores = scores[0] # the first calculation + else: + self.scores = np.concatenate((self.scores, scores[0]), axis=1) + + + return scores[1:] + + def pop_data_buf_till_frame(self, frame_idx: int) -> None: # need check again + while self.data_buf_start_frame < frame_idx: + if len(self.data_buf) >= int( + self.vad_opts.frame_in_ms * self.vad_opts.sample_rate / 1000 + ): + self.data_buf_start_frame += 1 + self.data_buf = self.data_buf_all[ + self.data_buf_start_frame + * int( + self.vad_opts.frame_in_ms * self.vad_opts.sample_rate / 1000 + ): + ] + + def pop_data_to_output_buf( + self, + start_frm: int, + frm_cnt: int, + first_frm_is_start_point: bool, + last_frm_is_end_point: bool, + end_point_is_sent_end: bool, + ) -> None: + self.pop_data_buf_till_frame(start_frm) + expected_sample_number = int( + frm_cnt * self.vad_opts.sample_rate * self.vad_opts.frame_in_ms / 1000 + ) + if last_frm_is_end_point: + extra_sample = max( + 0, + int( + self.vad_opts.frame_length_ms * self.vad_opts.sample_rate / 1000 + - self.vad_opts.sample_rate * self.vad_opts.frame_in_ms / 1000 + ), + ) + expected_sample_number += int(extra_sample) + if end_point_is_sent_end: + expected_sample_number = max(expected_sample_number, len(self.data_buf)) + if len(self.data_buf) < expected_sample_number: + logging.error("error in calling pop data_buf\n") + + if len(self.output_data_buf) == 0 or first_frm_is_start_point: + self.output_data_buf.append(E2EVadSpeechBufWithDoa()) + self.output_data_buf[-1].reset() + self.output_data_buf[-1].start_ms = start_frm * self.vad_opts.frame_in_ms + self.output_data_buf[-1].end_ms = self.output_data_buf[-1].start_ms + self.output_data_buf[-1].doa = 0 + cur_seg = self.output_data_buf[-1] + if cur_seg.end_ms != start_frm * self.vad_opts.frame_in_ms: + logging.error("warning\n") + out_pos = len(cur_seg.buffer) # cur_seg.buff现在没做任何操作 + data_to_pop = 0 + if end_point_is_sent_end: + data_to_pop = expected_sample_number + else: + data_to_pop = int( + frm_cnt * self.vad_opts.frame_in_ms * self.vad_opts.sample_rate / 1000 + ) + if data_to_pop > len(self.data_buf): + logging.error("VAD data_to_pop is bigger than self.data_buf.size()!!!\n") + data_to_pop = len(self.data_buf) + expected_sample_number = len(self.data_buf) + + cur_seg.doa = 0 + for sample_cpy_out in range(0, data_to_pop): + # cur_seg.buffer[out_pos ++] = data_buf_.back(); + out_pos += 1 + for sample_cpy_out in range(data_to_pop, expected_sample_number): + # cur_seg.buffer[out_pos++] = data_buf_.back() + out_pos += 1 + if cur_seg.end_ms != start_frm * self.vad_opts.frame_in_ms: + logging.error("Something wrong with the VAD algorithm\n") + self.data_buf_start_frame += frm_cnt + cur_seg.end_ms = (start_frm + frm_cnt) * self.vad_opts.frame_in_ms + if first_frm_is_start_point: + cur_seg.contain_seg_start_point = True + if last_frm_is_end_point: + cur_seg.contain_seg_end_point = True + + def on_silence_detected(self, valid_frame: int): + self.lastest_confirmed_silence_frame = valid_frame + if self.vad_state_machine == VadStateMachine.kVadInStateStartPointNotDetected: + self.pop_data_buf_till_frame(valid_frame) + # silence_detected_callback_ + # pass + + def on_voice_detected(self, valid_frame: int) -> None: + self.latest_confirmed_speech_frame = valid_frame + self.pop_data_to_output_buf(valid_frame, 1, False, False, False) + + def on_voice_start(self, start_frame: int, fake_result: bool = False) -> None: + if self.vad_opts.do_start_point_detection: + pass + if self.confirmed_start_frame != -1: + logging.error("not reset vad properly\n") + else: + self.confirmed_start_frame = start_frame + + if ( + not fake_result + and self.vad_state_machine + == VadStateMachine.kVadInStateStartPointNotDetected + ): + self.pop_data_to_output_buf( + self.confirmed_start_frame, 1, True, False, False + ) + + def on_voice_end( + self, end_frame: int, fake_result: bool, is_last_frame: bool + ) -> None: + for t in range(self.latest_confirmed_speech_frame + 1, end_frame): + self.on_voice_detected(t) + if self.vad_opts.do_end_point_detection: + pass + if self.confirmed_end_frame != -1: + logging.error("not reset vad properly\n") + else: + self.confirmed_end_frame = end_frame + if not fake_result: + self.sil_frame = 0 + self.pop_data_to_output_buf( + self.confirmed_end_frame, 1, False, True, is_last_frame + ) + self.number_end_time_detected += 1 + + def maybe_on_voice_end_last_frame( + self, is_final_frame: bool, cur_frm_idx: int + ) -> None: + if is_final_frame: + self.on_voice_end(cur_frm_idx, False, True) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + + def get_latency(self) -> int: + return int(self.latency_frm_num_at_start_point() * self.vad_opts.frame_in_ms) + + def latency_frm_num_at_start_point(self) -> int: + vad_latency = self.windows_detector.get_win_size() + if self.vad_opts.do_extend: + vad_latency += int( + self.vad_opts.lookback_time_start_point / self.vad_opts.frame_in_ms + ) + return vad_latency + + def get_frame_state(self, t: int) -> FrameState: + frame_state = FrameState.kFrameStateInvalid + cur_decibel = self.decibel[t] + cur_snr = cur_decibel - self.noise_average_decibel + # for each frame, calc log posterior probability of each state + if cur_decibel < self.vad_opts.decibel_thres: + frame_state = FrameState.kFrameStateSil + self.detect_one_frame(frame_state, t, False) + return frame_state + + sum_score = 0.0 + noise_prob = 0.0 + assert len(self.sil_pdf_ids) == self.vad_opts.silence_pdf_num + if len(self.sil_pdf_ids) > 0: + assert len(self.scores) == 1 # 只支持batch_size = 1的测试 + sil_pdf_scores = [ + self.scores[0][t][sil_pdf_id] for sil_pdf_id in self.sil_pdf_ids + ] + sum_score = sum(sil_pdf_scores) + noise_prob = math.log(sum_score) * self.vad_opts.speech_2_noise_ratio + total_score = 1.0 + sum_score = total_score - sum_score + speech_prob = math.log(sum_score) + if self.vad_opts.output_frame_probs: + frame_prob = E2EVadFrameProb() + frame_prob.noise_prob = noise_prob + frame_prob.speech_prob = speech_prob + frame_prob.score = sum_score + frame_prob.frame_id = t + self.frame_probs.append(frame_prob) + if math.exp(speech_prob) >= math.exp(noise_prob) + self.speech_noise_thres: + if ( + cur_snr >= self.vad_opts.snr_thres + and cur_decibel >= self.vad_opts.decibel_thres + ): + frame_state = FrameState.kFrameStateSpeech + else: + frame_state = FrameState.kFrameStateSil + else: + frame_state = FrameState.kFrameStateSil + if self.noise_average_decibel < -99.9: + self.noise_average_decibel = cur_decibel + else: + self.noise_average_decibel = ( + cur_decibel + + self.noise_average_decibel + * (self.vad_opts.noise_frame_num_used_for_snr - 1) + ) / self.vad_opts.noise_frame_num_used_for_snr + + return frame_state + + def infer_offline( + self, + feats: np.ndarray, + waveform: np.ndarray, + in_cache: Dict[str, np.ndarray] = dict(), + is_final: bool = False, + ) -> Tuple[List[List[List[int]]], Dict[str, np.ndarray]]: + self.waveform = waveform + self.compute_decibel() + + self.compute_scores(feats) + if not is_final: + self.detect_common_frames() + else: + self.detect_last_frames() + segments = [] + for batch_num in range(0, feats.shape[0]): # only support batch_size = 1 now + segment_batch = [] + if len(self.output_data_buf) > 0: + for i in range(self.output_data_buf_offset, len(self.output_data_buf)): + if ( + not self.output_data_buf[i].contain_seg_start_point + or not self.output_data_buf[i].contain_seg_end_point + ): + continue + segment = [ + self.output_data_buf[i].start_ms, + self.output_data_buf[i].end_ms, + ] + segment_batch.append(segment) + self.output_data_buf_offset += 1 # need update this parameter + if segment_batch: + segments.append(segment_batch) + if is_final: + # reset class variables and clear the dict for the next query + self.all_reset_detection() + return segments, in_cache + + def infer_online( + self, + feats: np.ndarray, + waveform: np.ndarray, + in_cache: list = None, + is_final: bool = False, + max_end_sil: int = 800, + ) -> Tuple[List[List[List[int]]], Dict[str, np.ndarray]]: + feats = [feats] + if in_cache is None: + in_cache = [] + + self.max_end_sil_frame_cnt_thresh = ( + max_end_sil - self.vad_opts.speech_to_sil_time_thres + ) + self.waveform = waveform # compute decibel for each frame + feats.extend(in_cache) + in_cache = self.compute_scores(feats) + self.compute_decibel() + + if is_final: + self.detect_last_frames() + else: + self.detect_common_frames() + + segments = [] + # only support batch_size = 1 now + for batch_num in range(0, feats[0].shape[0]): + if len(self.output_data_buf) > 0: + for i in range(self.output_data_buf_offset, len(self.output_data_buf)): + if not self.output_data_buf[i].contain_seg_start_point: + continue + if not self.next_seg and not self.output_data_buf[i].contain_seg_end_point: + continue + start_ms = self.output_data_buf[i].start_ms + if self.output_data_buf[i].contain_seg_end_point: + end_ms = self.output_data_buf[i].end_ms + self.next_seg = True + self.output_data_buf_offset += 1 + else: + end_ms = -1 + self.next_seg = False + + segment = [start_ms, end_ms] + segment = [self.output_data_buf[i].start_ms, self.output_data_buf[i].end_ms] + segments.append(segment) + if is_final: + # reset class variables and clear the dict for the next query + self.all_reset_detection() + return segments, in_cache + + def detect_common_frames(self) -> int: + if self.vad_state_machine == VadStateMachine.kVadInStateEndPointDetected: + return 0 + for i in range(self.vad_opts.nn_eval_block_size - 1, -1, -1): + frame_state = FrameState.kFrameStateInvalid + frame_state = self.get_frame_state(self.frm_cnt - 1 - i) + self.detect_one_frame(frame_state, self.frm_cnt - 1 - i, False) + + return 0 + + def detect_last_frames(self) -> int: + if self.vad_state_machine == VadStateMachine.kVadInStateEndPointDetected: + return 0 + for i in range(self.vad_opts.nn_eval_block_size - 1, -1, -1): + frame_state = FrameState.kFrameStateInvalid + frame_state = self.get_frame_state(self.frm_cnt - 1 - i) + if i != 0: + self.detect_one_frame(frame_state, self.frm_cnt - 1 - i, False) + else: + self.detect_one_frame(frame_state, self.frm_cnt - 1, True) + + return 0 + + def detect_one_frame( + self, cur_frm_state: FrameState, cur_frm_idx: int, is_final_frame: bool + ) -> None: + tmp_cur_frm_state = FrameState.kFrameStateInvalid + if cur_frm_state == FrameState.kFrameStateSpeech: + if math.fabs(1.0) > float(self.vad_opts.fe_prior_thres): + tmp_cur_frm_state = FrameState.kFrameStateSpeech + else: + tmp_cur_frm_state = FrameState.kFrameStateSil + elif cur_frm_state == FrameState.kFrameStateSil: + tmp_cur_frm_state = FrameState.kFrameStateSil + state_change = self.windows_detector.detect_one_frame( + tmp_cur_frm_state, cur_frm_idx + ) + frm_shift_in_ms = self.vad_opts.frame_in_ms + if AudioChangeState.kChangeStateSil2Speech == state_change: + self.continous_silence_frame_count = 0 + self.pre_end_silence_detected = False + + if ( + self.vad_state_machine + == VadStateMachine.kVadInStateStartPointNotDetected + ): + start_frame = max( + self.data_buf_start_frame, + cur_frm_idx - self.latency_frm_num_at_start_point(), + ) + self.on_voice_start(start_frame) + self.vad_state_machine = VadStateMachine.kVadInStateInSpeechSegment + for t in range(start_frame + 1, cur_frm_idx + 1): + self.on_voice_detected(t) + elif self.vad_state_machine == VadStateMachine.kVadInStateInSpeechSegment: + for t in range(self.latest_confirmed_speech_frame + 1, cur_frm_idx): + self.on_voice_detected(t) + if ( + cur_frm_idx - self.confirmed_start_frame + 1 + > self.vad_opts.max_single_segment_time / frm_shift_in_ms + ): + self.on_voice_end(cur_frm_idx, False, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + elif not is_final_frame: + self.on_voice_detected(cur_frm_idx) + else: + self.maybe_on_voice_end_last_frame(is_final_frame, cur_frm_idx) + else: + pass + elif AudioChangeState.kChangeStateSpeech2Sil == state_change: + self.continous_silence_frame_count = 0 + if ( + self.vad_state_machine + == VadStateMachine.kVadInStateStartPointNotDetected + ): + pass + elif self.vad_state_machine == VadStateMachine.kVadInStateInSpeechSegment: + if ( + cur_frm_idx - self.confirmed_start_frame + 1 + > self.vad_opts.max_single_segment_time / frm_shift_in_ms + ): + self.on_voice_end(cur_frm_idx, False, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + elif not is_final_frame: + self.on_voice_detected(cur_frm_idx) + else: + self.maybe_on_voice_end_last_frame(is_final_frame, cur_frm_idx) + else: + pass + elif AudioChangeState.kChangeStateSpeech2Speech == state_change: + self.continous_silence_frame_count = 0 + if self.vad_state_machine == VadStateMachine.kVadInStateInSpeechSegment: + if ( + cur_frm_idx - self.confirmed_start_frame + 1 + > self.vad_opts.max_single_segment_time / frm_shift_in_ms + ): + self.max_time_out = True + self.on_voice_end(cur_frm_idx, False, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + elif not is_final_frame: + self.on_voice_detected(cur_frm_idx) + else: + self.maybe_on_voice_end_last_frame(is_final_frame, cur_frm_idx) + else: + pass + elif AudioChangeState.kChangeStateSil2Sil == state_change: + self.continous_silence_frame_count += 1 + if ( + self.vad_state_machine + == VadStateMachine.kVadInStateStartPointNotDetected + ): + # silence timeout, return zero length decision + if ( + ( + self.vad_opts.detect_mode + == VadDetectMode.kVadSingleUtteranceDetectMode.value + ) + and ( + self.continous_silence_frame_count * frm_shift_in_ms + > self.vad_opts.max_start_silence_time + ) + ) or (is_final_frame and self.number_end_time_detected == 0): + for t in range( + self.lastest_confirmed_silence_frame + 1, cur_frm_idx + ): + self.on_silence_detected(t) + self.on_voice_start(0, True) + self.on_voice_end(0, True, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + else: + if cur_frm_idx >= self.latency_frm_num_at_start_point(): + self.on_silence_detected( + cur_frm_idx - self.latency_frm_num_at_start_point() + ) + elif self.vad_state_machine == VadStateMachine.kVadInStateInSpeechSegment: + if ( + self.continous_silence_frame_count * frm_shift_in_ms + >= self.max_end_sil_frame_cnt_thresh + ): + lookback_frame = int( + self.max_end_sil_frame_cnt_thresh / frm_shift_in_ms + ) + if self.vad_opts.do_extend: + lookback_frame -= int( + self.vad_opts.lookahead_time_end_point / frm_shift_in_ms + ) + lookback_frame -= 1 + lookback_frame = max(0, lookback_frame) + self.on_voice_end(cur_frm_idx - lookback_frame, False, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + elif ( + cur_frm_idx - self.confirmed_start_frame + 1 + > self.vad_opts.max_single_segment_time / frm_shift_in_ms + ): + self.on_voice_end(cur_frm_idx, False, False) + self.vad_state_machine = VadStateMachine.kVadInStateEndPointDetected + elif self.vad_opts.do_extend and not is_final_frame: + if self.continous_silence_frame_count <= int( + self.vad_opts.lookahead_time_end_point / frm_shift_in_ms + ): + self.on_voice_detected(cur_frm_idx) + else: + self.maybe_on_voice_end_last_frame(is_final_frame, cur_frm_idx) + else: + pass + + if ( + self.vad_state_machine == VadStateMachine.kVadInStateEndPointDetected + and self.vad_opts.detect_mode + == VadDetectMode.kVadMutipleUtteranceDetectMode.value + ): + self.reset_detection() + + +def vad_segments(logits: np.ndarray, waveform: np.ndarray, + speech_noise_thres: float = 0.6) -> list: + """用 FSMN-VAD 的 softmax logits 做 FunASR 同款端点后处理。 + + logits: [1, T, 248]（模型输出，已 Softmax） + waveform: [1, T*160] float32，-1~1 + 返回 [[start_ms, end_ms], ...] + """ + model = E2EVadModel(speech_noise_thres) + model._logits = np.asarray(logits, dtype=np.float32) + feats = np.zeros((1, logits.shape[1], 400), dtype=np.float32) + segs, _ = model.infer_offline(feats, waveform, is_final=True) + return segs[0] if segs else [] diff --git a/openai/openai_client.py b/openai/openai_client.py new file mode 100644 index 0000000000000000000000000000000000000000..78dce83e3235dda072e96e38aef16b4edfdce85e --- /dev/null +++ b/openai/openai_client.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python3 +"""FireRedASR-AED OpenAI 兼容客户端（CLI / 函数）。""" +import argparse +import json +import urllib.request +import uuid + + +def transcribe(server_url, wav_path, model="fireredasr-aed", timeout=120): + boundary = uuid.uuid4().hex + with open(wav_path, "rb") as f: + content = f.read() + parts = [] + parts.append( + f'--{boundary}\r\nContent-Disposition: form-data; name="model"\r\n\r\n{model}\r\n'.encode()) + parts.append( + (f'--{boundary}\r\nContent-Disposition: form-data; name="file"; ' + f'filename="{wav_path.split("/")[-1]}"\r\nContent-Type: audio/wav\r\n\r\n').encode() + + content + b"\r\n") + parts.append(f"--{boundary}--\r\n".encode()) + body = b"".join(parts) + url = server_url.rstrip("/") + "/v1/audio/transcriptions" + req = urllib.request.Request( + url, data=body, + headers={"Content-Type": f"multipart/form-data; boundary={boundary}"}) + with urllib.request.urlopen(req, timeout=timeout) as resp: + return json.loads(resp.read().decode("utf-8")) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("server_url") + ap.add_argument("wav") + ap.add_argument("--model", default="fireredasr-aed") + ap.add_argument("--timeout", type=int, default=120) + args = ap.parse_args() + print(transcribe(args.server_url, args.wav, args.model, args.timeout)["text"]) + + +if __name__ == "__main__": + main() diff --git a/openai/openai_server.py b/openai/openai_server.py new file mode 100644 index 0000000000000000000000000000000000000000..bc8142df8adeee45abe4ce8491420a3887cbf6fb --- /dev/null +++ b/openai/openai_server.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +"""FireRedASR-AED OpenAI 兼容服务（/v1/audio/transcriptions）。""" +import argparse +import json +import re +import tempfile +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +from firered_asr import FireredASR + + +def parse_multipart(content_type, body): + m = re.search(r'boundary=(?:"([^"]+)"|([^;]+))', content_type or "") + boundary = (m.group(1) or m.group(2)).strip() if m else None + if not boundary: + return None + delim = b"--" + boundary.encode() + parts = body.split(delim) + files = {} + for part in parts: + if b"\r\n\r\n" not in part: + continue + header, _, data = part.partition(b"\r\n\r\n") + if data.endswith(b"\r\n"): + data = data[:-2] + name = None + filename = None + for line in header.split(b"\r\n"): + if line.lower().startswith(b"content-disposition:"): + nm = re.search(rb'name="([^"]+)"', line, re.I) + fn = re.search(rb'filename="([^"]*)"', line, re.I) + if nm: + name = nm.group(1).decode() + if fn: + filename = fn.group(1).decode() + if name: + files[name] = {"content": data, "filename": filename} + return files + + +def make_handler(asr): + class Handler(BaseHTTPRequestHandler): + def _send_json(self, obj, code=200): + body = json.dumps(obj, ensure_ascii=False).encode() + self.send_response(code) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def do_GET(self): + if self.path.rstrip("/") == "/v1/models": + self._send_json({"object": "list", "data": [ + {"id": "fireredasr-aed", "object": "model", "owned_by": "axera"}]}) + else: + self._send_json({"error": "not found"}, 404) + + def do_POST(self): + if self.path.rstrip("/") != "/v1/audio/transcriptions": + self._send_json({"error": "not found"}, 404) + return + length = int(self.headers.get("Content-Length", 0)) + body = self.rfile.read(length) + files = parse_multipart(self.headers.get("Content-Type", ""), body) + if not files or "file" not in files: + self._send_json({"error": "missing multipart field 'file'"}, 400) + return + data = files["file"]["content"] + with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f: + f.write(data) + tmp = f.name + try: + text = asr.transcribe(tmp) + self._send_json({"text": text}) + except Exception as e: + self._send_json({"error": str(e)}, 500) + + def log_message(self, *args): + pass + return Handler + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--encoder", required=True) + ap.add_argument("--decoder", required=True) + ap.add_argument("--fsmn-vad", required=True) + ap.add_argument("--fsmn-cmvn", required=True) + ap.add_argument("--cmvn", required=True) + ap.add_argument("--dict", required=True) + ap.add_argument("--pe", required=True) + ap.add_argument("--max-dur", type=int, default=10) + ap.add_argument("--max-steps", type=int, default=128) + ap.add_argument("--host", default="0.0.0.0") + ap.add_argument("--port", type=int, default=8000) + args = ap.parse_args() + + asr = FireredASR(args.encoder, args.decoder, args.fsmn_vad, args.fsmn_cmvn, + args.cmvn, args.dict, args.pe, + args.max_dur, args.max_steps) + print(f"OpenAI API listening on {args.host}:{args.port}") + ThreadingHTTPServer((args.host, args.port), make_handler(asr)).serve_forever() + + +if __name__ == "__main__": + main() diff --git a/reports/ACCELERATION.md b/reports/ACCELERATION.md new file mode 100644 index 0000000000000000000000000000000000000000..864b412c6b40ee92420cb6a0cc7f04648c38de5b --- /dev/null +++ b/reports/ACCELERATION.md @@ -0,0 +1,76 @@ +# FireRedASR-AED AX650N 全链路加速方案 + +基线（AX650N，Pulsar2 7.0-lite，U16 encoder + U8 dense decoder， +贪心解码，Python+numpy+pyaxengine 运行时）： + +| 指标 | 数值 | +|------|------| +| CER（4 条测试语音） | 3.37% | +| 全链路 RTF | 0.318 | +| encoder（10s 输入） | 172ms | +| decoder NPU 单步（ax_run_model） | 23.1ms | +| decoder Python 实际单步 | ~53ms（含 ~30ms Python/pyaxengine 开销） | + +## 瓶颈分解 + +- decoder 是绝对大头：5 段共 81 步，NPU 理论 1.87s，Python 实测 4.34s， + 每步额外开销约 30ms。 +- encoder 其次：每段 ~191ms，且 U8 量化在 7.0 下不可用（cos≈0.11）， + 只能 U16。 +- 编译侧已探明：transformer_opt_level=2、tile、data soft compression、 + S8/MSE/Brecq(超时) 均无收益或掉精度；U16 decoder 有 cache 回写缺陷。 + +## 加速手段（按收益排序） + +### 1. C++ 运行时（最大收益，已落地：RTF 0.318 → 0.211） + +基于 HF 仓库 `EngineWrapper` 实现了 C++ 基准（`cpp/`）： +- decoder cache **零拷贝**（下一轮输入直接指向本轮输出 buffer）， + 每步 20MB memcpy 消除； +- 常量输入（cross_k/v/mask）只设置一次； +- 实测 5 段测试集：decoder 每步 23.56ms（NPU 极限约 23.1ms）， + RTF 0.211（Python 优化版 0.318，提速约 34%），详见 `cpp/RESULTS.md`。 + +剩余：板端 fbank/cmvn、VAD、多段流水。 + +### 2. 缩短 encoder 输入时长 max_dur（10s → 5s） + +已实测：encoder 减到 75.6ms/段，decoder 单步 ~20.8ms， +但 **5s decoder 的 U8 量化精度回退**（cos 0.987 vs 10s 的 0.9998， +贪心解码重复/错字）；U16 cache 回写缺陷依旧、BF16 编译不支持。 +结论：**暂不建议采用**，需 QAT 或等工具链修复后再评估。 + +### 3. 缩短 decode_max_len（128 → 64） + +- cache DDR 读写减半（每步 2×10MB → 2×5MB）； +- 自回归最坏步数减半，内存占用降低； +- 本测试集最长 38 步，64 足够。 + +### 4. 运行时流水线/并行 + +- 长音频多段：VAD/特征/encoder 与上一段 decoder 并行（C++ 多线程， + Python 用 ThreadPool 也行，NPU 串行但 CPU 工作可重叠）； +- 板端 VAD 用 FSMN-VAD axmodel（`fsmn_vad/fsmn_vad_10s_fp32.axmodel`），避免主机回传。 + +### 5. 模型结构侧（中期，需重训/蒸馏） + +- decoder 16 层 → 12 层（或 d_model 1280 → 1024）：decoder 每步约减 + 25-35%，但需用原模型蒸馏/微调保精度； +- QAT：若要把 encoder 降到 INT8，7.0 的 PTQ 已证实失效，需 + QAT（QDQ 导出）才能拿到 172 → 115ms 的收益（encoder 占比小，收益有限）。 + +### 6. 已排除/不推荐 + +- transformer_opt_level=2：单步无提速（23.1ms），logits cos 0.9998→0.979； +- tile + data soft compression：23.5ms，无收益； +- S8/MSE/不同校准方法：更差； +- U16 decoder：cache 回写缺陷，不可用； +- U8 encoder：7.0 PTQ 下 cos≈0.11，不可用。 + +## 建议落地顺序 + +1. 先上 C++ 运行时（收益最大、不动模型）； +2. 再按产品约束决定是否把 max_dur 降到 5s / decode_max_len 降到 64 + （需重新导出+编译，精度风险低）； +3. 长音频加段间流水并行； +4. 若仍不够，再评估 decoder 蒸馏/QAT（周期长）。 diff --git a/reports/REPORT.md b/reports/REPORT.md new file mode 100644 index 0000000000000000000000000000000000000000..7a4581d5c46acc51c0d2075dcbd42dc4e7cfefac --- /dev/null +++ b/reports/REPORT.md @@ -0,0 +1,77 @@ +# FireRedASR-AED AX650N 重转换对比报告（Pulsar2 7.0-lite） + +日期：2026-08-24 + +## 结论 + +用 Pulsar2 7.0-lite 重新转换 FireRedASR-AED（参考 +`ml-inory/FireRedASR.axera`），在 AX650N 上与 HF 上 +`AXERA-TECH/FireRedASR-AED`（实际由 Pulsar2 5.1 编译）对比： + +- **精度持平**：最优 7.0 组合（encoder U16 + decoder U8/密集校准） + CER 3.37%，HF 5.1 参考 CER 3.37%（4 条测试语音，VAD 切 5 段）。 +- **速度基本持平**：全链路 RTF 0.329 vs 0.323；decoder 单步 + 23.3ms vs 23.4ms；encoder 10s 输入 171.8ms vs 157.5ms。 + +## 关键发现 + +1. **HF 仓库本体并未损坏**：官方仓库 `AXERA-TECH/FireRedASR-AED` + 的 LFS sha256 为 `6cc674ba...`，与 ModelScope 下载的完好文件完全一致 + （板端可加载，cross cosine≈0.9997）。最初经 hf-mirror 下载的副本 + 被镜像/CDN 损坏（33.5% 字节为 0、尾部约 50MB 全零，SHA + `35cdb624...`），导致“HF encoder 损坏”的误判。改用 ModelScope 或 + 官方 huggingface.co 下载即可。 +2. **7.0-lite 的 U8 量化对校准数据敏感**：用稀疏校准（22 样本）时 + decoder 单步 logits cosine 仅 0.976，贪心解码重复/乱码（CER 85%）； + 改用密集校准（41 样本，解码步每 2 步采样）后 cosine 升至 0.9998， + 与 HF 5.1（0.9998）持平。 +3. **7.0-lite 的 U16 decoder 存在 cache 回写缺陷**：单步 logits + cosine 0.995，但多步解码时 cache 几乎不更新，持续重复首 token， + 不可用（U16 混合精度同样复现）。 +4. **7.0-lite 的 U8 encoder 不可用**：cross_k/v cosine 仅 0.11/-0.002， + 即使配 HF decoder 也乱码；encoder 应使用 U16。 + +## 量化配置探索矩阵（decoder 单步 logits，同一组 ONNX 输入） + +| 配置 | cosine | 单步耗时 | 备注 | +|------|--------|----------|------| +| HF 5.1（官方仓库配置） | 0.999846 | 23.35ms | 参考 | +| 7.0 U8 + 稀疏校准（仓库配置） | 0.976 | 23.23ms | 重复/乱码 | +| 7.0 U8 + 关键算子 FP32 | 0.976 | 23.65ms | 无改善 | +| 7.0 U8 + 关 SmoothQuant/开 auto-refine | 0.976 | 23.21ms | 无改善 | +| 7.0 U8 + 关量化优化 | 0.976 | 23.23ms | 无改善 | +| 7.0 U8 + NPU 后端精度分析 | 0.976 | 23.18ms | 无改善 | +| 7.0 U8 + MSE 校准 | 0.959 | 23.35ms | 更差 | +| 7.0 S8 | 0.946 | 23.78ms | 更差 | +| 7.0 U8 + 密集校准（推荐） | 0.999806 | 23.27ms | **与 5.1 持平** | +| 7.0 U16（全 U16） | 0.995 | 26.41ms | cache 回写缺陷 | +| 7.0 U16 混合（MatMul U16） | 0.995 | 26.81ms | cache 回写缺陷 | + +## 全链路精度/速度（AX650N，VAD 切分 5 段，贪心解码） + +| 组合 | CER% | avg RTF | encoder/段 | decoder/段 | +|------|------|---------|------------|------------| +| HF 5.1（官方 enc + HF dec） | 3.37 | 0.3234 | 178ms | 897ms | +| 7.0 U16 enc + HF 5.1 dec | 3.37 | 0.3342 | 195ms | 917ms | +| 7.0 U16 enc + 7.0 U8 稀疏 dec | 85.4 | 0.26 | — | — | +| 7.0 U16 enc + 7.0 U8 密集 dec（推荐） | 3.37 | 0.3291 | 195ms | 899ms | +| 7.0 U8 enc + HF 5.1 dec | 182 | 0.4835 | — | — | + +## 推荐交付配置 + +- encoder：`compile/encoder_u16_ref.axmodel`（812MB，U16，cos 0.9997） +- decoder：`compile/decoder_loop_u8_dense.axmodel`（398MB，U8，密集校准， + cos 0.9998） +- 校准数据：`export/calib_data/`（4 条真实语音，decoder 每 2 步采样） +- 板端运行：numpy + pyaxengine，不依赖 torch/onnxruntime + +## 复现 + +- 导出：`origin/FireRedASR.axera/model_convert/to_onnx.py` + （beam_size=1, decode_max_len=128, max_dur=10；导出后 onnxslim + 处理供 7.0 编译，或直接用原始 ONNX） +- 校准：`generate_data.py`（CALIB_EVERY=2, CALIB_MAX_STEPS=128） + → `compile/make_calib_tars.py` +- 编译：`compile/pulsar2_decoder_u8_dense.json` / + `compile/pulsar2_encoder_u16_ref_slim.json`（Pulsar2 7.0-lite， + `~/.cache/magnetar/pulsar2/7.0`） diff --git a/reports/RESULTS.md b/reports/RESULTS.md new file mode 100644 index 0000000000000000000000000000000000000000..5052ca6b8d8b6c0427788a5d99ac7b1ce2790a56 --- /dev/null +++ b/reports/RESULTS.md @@ -0,0 +1,89 @@ +# C++ 运行时基准结果（AX650N，U16 encoder + U8 dense decoder） + +输入：HF 测试集 VAD 切分 5 段（总时长 16.626s），特征由主机按仓库 +kaldiio(int16)+fbank+cmvn 管线预计算，C++ 只负责 NPU 推理与贪心解码。 + +| 片段 | 时长(s) | encoder(ms) | decoder(ms) | 步数 | 每步(ms) | ASR(ms) | +|------|--------|-------------|-------------|------|----------|---------| +| BAC009_c0 | 3.292 | 171.8 | 329.6 | 14 | 23.54 | 598.8 | +| IT0011_c0 | 1.244 | 171.8 | 117.6 | 5 | 23.52 | 385.1 | +| MEET_c0 | 8.408 | 171.8 | 1059.6 | 45 | 23.55 | 1328.0 | +| MEET_c1 | 2.140 | 171.8 | 330.3 | 14 | 23.60 | 598.6 | +| NET_c0 | 1.542 | 171.8 | 330.2 | 14 | 23.58 | 597.6 | +| **合计** | **16.626** | **859** | **2167** | **92** | **23.56** | **3508** | + +**RTF = 3508ms / 16626ms ≈ 0.211**（Python 优化版 0.318，提速约 34%）。 + +关键实现： +- 复用 HF 仓库 `EngineWrapper`（ax_engine C API）； +- decoder cache 采用**零拷贝**：下一轮 `in_n_layer_self_k_cache/v_cache` + 直接指向本轮输出 buffer（`SetInputFromOutput`），去掉每步 20MB memcpy； +- 常量输入（cross_k/v/mask）只设置一次； +- 贪心解码直接用 logits argmax（无需 softmax）。 + +## 端到端 C++ SDK（firered_sdk） + +- 板端 VAD（FSMN-VAD axmodel）+ fbank/cmvn（kaldi-native-fbank 源码内嵌） + + encoder + decoder 贪心解码 + detokenize，全部在板端完成； +- 4 条测试语音文本全部正确： + - BAC009：甚至出现交易几乎停滞的情况 + - IT0011：换一首歌 + - TEST_MEETING：与参考一致（VAD 自动切 2 段） + - TEST_NET：有的时候说不清楚你们知道吗（缺“我”，与 Python 版一致） +- 关键修复：VAD 输入需 [-1,1] 归一化、fbank 输入需 int16 原值； + CMVN count 取 data[cols-1]；detokenize 跳过 。 +- 多段流水：后台线程算下一段特征，与当前段 NPU 解码重叠。 + +## max_dur=5s / decode_max_len=64（进行中） + +## max_dur=5s / decode_max_len=64（已实测，不建议采用） + +- ONNX 已导出（encoder T=498，decoder cache=64/cross=125）、onnxslim、 + 5s 校准与编译全部完成； +- 速度：encoder 75.6ms（10s 为 171.8ms），decoder 单步 ~20.8ms； +- **精度不达标**：5s decoder U8 单步 logits cosine 仅 ~0.987（10s 密集 + 校准为 0.9998），贪心解码重复/错字；尝试过 49/95 样本校准、更多 FP32 + 算子、BF16（编译不支持）、U16（cache 回写缺陷依旧），均无法修复。 +- 结论：5s 方案需 QAT 或等 Pulsar2 修复 U8 量化后重试；当前交付采用 + **10s 模型 + C++ SDK**（RTF 0.225，文本正确）。 + +## 最终落地结果（10s 模型） + +| 指标 | Python 优化版 | C++ SDK（含板端 VAD+fbank+流水） | +|------|--------------|----------------------------------| +| 全链路 RTF | 0.318 | **0.225** | +| decoder 每步 | ~53ms | ~23.5ms | +| 4 条测试文本 | 3.37% CER | 3.37% CER（文本一致） | + +## 远场/低信噪比场景复测（2026-08-24） + +问题：静音/背景噪音送 ASR 会产生英文幻觉，预期无输出。 + +### 诊断 + +- 原版 torch FireRedASR-AED 对 `cut_3min.wav` 按 10s 块识别完全正常 + （0-10s：“有没有时间唉你不要让他整个人他就他就会想你就是你这种玩儿啊我开心”等）， + 证明模型本身没问题； +- 原 VAD 方案对该远场录音几乎全判为静音（全段语音概率 < 0.05， + 仅开头 0-5s 0.72），整段只检出 0.3s，导致只输出“嗯”； +- 结论：问题在 VAD，不在 ASR 模型。 + +### 修复 + +替换为 FSMN-VAD（FunASR 16k 通用）NPU 模型，10s 静态输入 FP32 编译： + +- `fsmn_vad/fsmn_vad_10s_fp32.axmodel` +- 按 10s 块做语音门控：块内检出语音 ≥ 1000ms 才整块送 ASR +- Python/C++ SDK 均支持；无 torch/onnxruntime 依赖 + +### 复测结果 + +| 输入 | 结果 | +|------|------| +| `cut_3min.wav`（16k 立体声 180s） | 完整中文转写（约 14/18 个 10s 块；漏检块与噪声样本声学特征重叠） | +| silence9 / noise_40s / noise_80s / noise_120s / noise_rms100 / noise_rms800 | 空（不送 ASR） | +| 语音切片 seg_00~03、05~07、10~17 | 输出对应文本 | +| 标准 4 条测试语音 | 与参考文本一致 | +| 立体声 vs 单声道 | 输出一致 | + +C++ 端到端耗时：cut_3min VAD 约 2.1s、encoder 2.2s、decoder 13.5s。 diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..c775c21764644fa3497dd323ff5fd990bdace08c --- /dev/null +++ b/requirements.txt @@ -0,0 +1,14 @@ +# AX650N AED inference runtime. +kaldiio>=2.18.0 +kaldi_native_fbank>=1.15 +numpy>=1.26.1 +sentencepiece +setuptools<81 +torch==2.6.0 +torchaudio==2.6.0 +axengine @ https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc2/axengine-0.1.3-py3-none-any.whl + +# Optional tools: +# - fireredasr_onnx.py: onnxruntime +# - fireredasr/speech2text.py LLM path: transformers, peft +# - fireredasr/utils/wer.py --do_tn 1: cn2an diff --git a/test_wer.py b/test_wer.py new file mode 100644 index 0000000000000000000000000000000000000000..c3af1284cc387f2f1e5c19dfe8eb466e9710b5c0 --- /dev/null +++ b/test_wer.py @@ -0,0 +1,344 @@ +import argparse +import os +import logging +import re +from fireredasr_axmodel import FireRedASRAxModel + + +def setup_logging(): + """配置日志系统，同时输出到控制台和文件""" + # 获取脚本所在目录 + script_dir = os.path.dirname(os.path.abspath(__file__)) + log_file = os.path.join(script_dir, "test_wer.log") + + # 配置日志格式 + log_format = "%(asctime)s - %(levelname)s - %(message)s" + date_format = "%Y-%m-%d %H:%M:%S" + + # 创建logger + logger = logging.getLogger() + logger.setLevel(logging.INFO) + + # 清除现有的handler + for handler in logger.handlers[:]: + logger.removeHandler(handler) + + # 创建文件handler + file_handler = logging.FileHandler(log_file, mode="a", encoding="utf-8") + file_handler.setLevel(logging.INFO) + file_formatter = logging.Formatter(log_format, date_format) + file_handler.setFormatter(file_formatter) + + # 创建控制台handler + console_handler = logging.StreamHandler() + console_handler.setLevel(logging.INFO) + console_formatter = logging.Formatter(log_format, date_format) + console_handler.setFormatter(console_formatter) + + # 添加handler到logger + logger.addHandler(file_handler) + logger.addHandler(console_handler) + + return logger + + +class AIShellDataset: + def __init__(self, gt_path: str, voice_dir="wav"): + """ + 初始化数据集 + + Args: + json_path: voice.json文件的路径 + """ + self.gt_path = gt_path + self.dataset_dir = os.path.dirname(gt_path) + self.voice_dir = os.path.join(self.dataset_dir, voice_dir) + + # 检查必要文件和文件夹是否存在 + assert os.path.exists(gt_path), f"gt文件不存在: {gt_path}" + assert os.path.exists(self.voice_dir), f"文件夹不存在: {self.voice_dir}" + + # 加载数据 + self.data = [] + with open(gt_path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + audio_path, gt = line.split(" ") + audio_path = os.path.join(self.voice_dir, audio_path + ".wav") + self.data.append({"audio_path": audio_path, "gt": gt}) + + # 使用logging而不是print + logger = logging.getLogger() + logger.info(f"加载了 {len(self.data)} 条数据") + + def __iter__(self): + """返回迭代器""" + self.index = 0 + return self + + def __next__(self): + """返回下一个数据项""" + if self.index >= len(self.data): + raise StopIteration + + item = self.data[self.index] + audio_path = item["audio_path"] + ground_truth = item["gt"] + + self.index += 1 + return audio_path, ground_truth + + def __len__(self): + """返回数据集大小""" + return len(self.data) + + +class CommonVoiceDataset: + """Common Voice数据集解析器""" + + def __init__(self, tsv_path: str): + """ + 初始化数据集 + + Args: + json_path: voice.json文件的路径 + """ + self.tsv_path = tsv_path + self.dataset_dir = os.path.dirname(tsv_path) + self.voice_dir = os.path.join(self.dataset_dir, "clips") + + # 检查必要文件和文件夹是否存在 + assert os.path.exists(tsv_path), f"{tsv_path}文件不存在: {tsv_path}" + assert os.path.exists(self.voice_dir), f"voice文件夹不存在: {self.voice_dir}" + + # 加载JSON数据 + self.data = [] + with open(tsv_path, "r", encoding="utf-8") as f: + f.readline() + for line in f: + line = line.strip() + splits = line.split("\t") + audio_path = splits[1] + gt = splits[2] + audio_path = os.path.join(self.voice_dir, audio_path) + self.data.append({"audio_path": audio_path, "gt": gt}) + + # 使用logging而不是print + logger = logging.getLogger() + logger.info(f"加载了 {len(self.data)} 条数据") + + def __iter__(self): + """返回迭代器""" + self.index = 0 + return self + + def __next__(self): + """返回下一个数据项""" + if self.index >= len(self.data): + raise StopIteration + + item = self.data[self.index] + audio_path = item["audio_path"] + ground_truth = item["gt"] + + self.index += 1 + return audio_path, ground_truth + + def __len__(self): + """返回数据集大小""" + return len(self.data) + + +def get_args(): + parser = argparse.ArgumentParser(prog="whisper", description="Test WER on dataset") + parser.add_argument( + "--dataset", + "-d", + type=str, + required=True, + choices=["aishell", "common_voice"], + help="Test dataset", + ) + parser.add_argument( + "--gt_path", + "-g", + type=str, + required=True, + help="Test dataset ground truth file", + ) + parser.add_argument( + "--max_num", type=int, default=-1, required=False, help="Maximum test data num" + ) + parser.add_argument( + "--language", + "-l", + type=str, + required=False, + default="zh", + help="Target language, support en, zh, ja, and others. See languages.py for more options.", + ) + parser.add_argument( + "--encoder", + type=str, + default="axmodel/encoder.axmodel", + help="Path to onnx encoder", + ) + parser.add_argument( + "--decoder_loop", + type=str, + default="axmodel/decoder_loop.axmodel", + help="Path to axmodel decoder loop", + ) + parser.add_argument( + "--cmvn", type=str, default="axmodel/cmvn.ark", help="Path to cmvn" + ) + parser.add_argument( + "--dict", type=str, default="axmodel/dict.txt", help="Path to dict" + ) + parser.add_argument( + "--spm_model", + type=str, + default="axmodel/train_bpe1000.model", + help="Path to spm model", + ) + parser.add_argument( + "--wavlist", type=str, default="wavlist.txt", help="File to wav path list" + ) + parser.add_argument( + "--hypo", type=str, default="hypo_axmodel.txt", help="File of hypos" + ) + parser.add_argument("--beam_size", type=int, default=1, help="") + parser.add_argument("--nbest", type=int, default=1, help="") + parser.add_argument("--max_len", type=int, default=128, help="") + return parser.parse_args() + + +def print_args(args): + logger = logging.getLogger() + logger.info(f"dataset: {args.dataset}") + logger.info(f"gt_path: {args.gt_path}") + logger.info(f"max_num: {args.max_num}") + logger.info(f"language: {args.language}") + + +def min_distance(word1: str, word2: str) -> int: + + row = len(word1) + 1 + column = len(word2) + 1 + + cache = [[0] * column for i in range(row)] + + for i in range(row): + for j in range(column): + + if i == 0 and j == 0: + cache[i][j] = 0 + elif i == 0 and j != 0: + cache[i][j] = j + elif j == 0 and i != 0: + cache[i][j] = i + else: + if word1[i - 1] == word2[j - 1]: + cache[i][j] = cache[i - 1][j - 1] + else: + replace = cache[i - 1][j - 1] + 1 + insert = cache[i][j - 1] + 1 + remove = cache[i - 1][j] + 1 + + cache[i][j] = min(replace, insert, remove) + + return cache[row - 1][column - 1] + + +def remove_punctuation(text): + # 定义正则表达式模式，匹配所有标点符号 + # 这个模式包括常见的标点符号和中文标点 + pattern = r"[^\w\s]|_" + + # 使用sub方法将所有匹配的标点符号替换为空字符串 + cleaned_text = re.sub(pattern, "", text) + + return cleaned_text + + +def main(): + # 设置日志系统 + logger = setup_logging() + + args = get_args() + print_args(args) + + dataset_type = args.dataset.lower() + if dataset_type == "aishell": + dataset = AIShellDataset(args.gt_path) + elif dataset_type == "common_voice": + dataset = CommonVoiceDataset(args.gt_path) + else: + raise ValueError(f"Unknown dataset type {dataset_type}") + + max_num = args.max_num + + # Load model + model = FireRedASRAxModel( + args.encoder, + args.decoder_loop, + args.cmvn, + args.dict, + args.spm_model, + decode_max_len=args.max_len, + audio_dur=10, + ) + # model = FireRedASROnnxModel( + # args.encoder, + # args.decoder, + # args.cmvn, + # args.dict, + # args.spm_model, + # decode_max_len=args.max_len, + # audio_dur=10 + # ) + # model = FireRedAsr.from_pretrained("aed", "model_convert/pretrained_models/FireRedASR-AED-L") + + # Iterate over dataset + references = [] + hyp = [] + all_character_error_num = 0 + all_character_num = 0 + wer_file = open("wer.txt", "w") + max_data_num = max_num if max_num > 0 else len(dataset) + for n, (audio_path, reference) in enumerate(dataset): + batch_uttid = [os.path.splitext(os.path.basename(audio_path))[0]] + batch_wav = [audio_path] + results, _, _ = model.transcribe(batch_wav, args.beam_size, args.nbest) + + hypothesis = results["text"] + + hypothesis = remove_punctuation(hypothesis) + reference = remove_punctuation(reference) + + character_error_num = min_distance(reference, hypothesis) + character_num = len(reference) + character_error_rate = character_error_num / character_num * 100 + + all_character_error_num += character_error_num + all_character_num += character_num + + hyp.append(hypothesis) + references.append(reference) + + line_content = f"({n+1}/{max_data_num}) {os.path.basename(audio_path)} gt: {reference} predict: {hypothesis} WER: {character_error_rate}%" + wer_file.write(line_content + "\n") + logger.info(line_content) + + if n + 1 >= max_data_num: + break + + total_character_error_rate = all_character_error_num / all_character_num * 100 + + logger.info(f"Total WER: {total_character_error_rate}%") + wer_file.write(f"Total WER: {total_character_error_rate}%") + wer_file.close() + + +if __name__ == "__main__": + main() diff --git a/wav/BAC009S0764W0121.wav b/wav/BAC009S0764W0121.wav new file mode 100644 index 0000000000000000000000000000000000000000..4e35a2c934fec43649b2cb0770dd85e62c931001 --- /dev/null +++ b/wav/BAC009S0764W0121.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46dbc998c9d1d48111267c40741dd3200f2e5bcf4075f8c4c97f4451160dce50 +size 134570 diff --git a/wav/IT0011W0001.wav b/wav/IT0011W0001.wav new file mode 100644 index 0000000000000000000000000000000000000000..fcc219d8315a67f7d1431992c6e3ed00b3554d0c --- /dev/null +++ b/wav/IT0011W0001.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e89a5721b3ec738753f65354457f50a1f8d478ca59cbc07f37e4e80b24c8250c +size 63822 diff --git a/wav/TEST_MEETING_T0000000001_S00000.wav b/wav/TEST_MEETING_T0000000001_S00000.wav new file mode 100644 index 0000000000000000000000000000000000000000..b4059d9fd2816973837c2ee5c471a933435ac1f1 --- /dev/null +++ b/wav/TEST_MEETING_T0000000001_S00000.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16f43567a94f81123bbdab058699ae34fe118a5abd35d294e558997510c2a16e +size 395852 diff --git a/wav/TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000.wav b/wav/TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000.wav new file mode 100644 index 0000000000000000000000000000000000000000..72ea1266aed571e96d963c3b0519c847a5c13876 --- /dev/null +++ b/wav/TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:babe417a3183a4c0e5306e604dae95e81e80e8fb6f2b538bf2530bd5873d2afc +size 57644 diff --git a/wav/text b/wav/text new file mode 100644 index 0000000000000000000000000000000000000000..bdbe25e68b98d3e9a540bf1799b1463fedba6d19 --- /dev/null +++ b/wav/text @@ -0,0 +1,4 @@ +BAC009S0764W0121 甚至 出现 交易 几乎 停滞 的 情况 +IT0011W0001 换一首歌 +TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000 我有的时候说不清楚你们知道吗 +TEST_MEETING_T0000000001_S00000 好首先说一下刚才这个经理说完的这个销售问题咱再说一下咱们的商场问题首先咱们商场上半年业这个先各部门儿汇报一下就是业绩 diff --git a/wav/wav.scp b/wav/wav.scp new file mode 100644 index 0000000000000000000000000000000000000000..db425c665ad76f4946b15285003bee1194e07b58 --- /dev/null +++ b/wav/wav.scp @@ -0,0 +1,4 @@ +BAC009S0764W0121 wav/BAC009S0764W0121.wav +IT0011W0001 wav/IT0011W0001.wav +TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000 wav/TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000.wav +TEST_MEETING_T0000000001_S00000 wav/TEST_MEETING_T0000000001_S00000.wav diff --git a/wavlist.txt b/wavlist.txt new file mode 100644 index 0000000000000000000000000000000000000000..7b23ba6c228eae5d45b464fa5d8e1cb534ce85b7 --- /dev/null +++ b/wavlist.txt @@ -0,0 +1,4 @@ +wav/TEST_NET_Y0000000000_-KTKHdZ2fb8_S00000.wav +wav/TEST_MEETING_T0000000001_S00000.wav +wav/IT0011W0001.wav +wav/BAC009S0764W0121.wav