AgenticASR-Refiner Q8_0

ๅŽŸๅง‹ๆจกๅž‹ Andrew0425/AgenticASR-Refiner ็š„ Q8_0 GGUF ้‡ๅŒ–็‰ˆๆœฌ๏ผŒ็”จไบŽ ASR ่ฝฌๅ†™ๆ–‡ๆœฌ็บ ้”™๏ผŒไธ็›ดๆŽฅๅค„็†้Ÿณ้ข‘ใ€‚ๆœช่ฟฝๅŠ ่ฎญ็ปƒใ€‚

ๆ–‡ไปถไธŽ้‡ๅŒ–

  • ๆ–‡ไปถ๏ผšAgenticASR-Refiner-Q8_0.gguf๏ผŒ1,153,529,088 ๅญ—่Š‚๏ผˆ็บฆ 1.07 GiB๏ผ‰ใ€‚
  • SHA-256๏ผš0cb29dcf5b84a45b0733cd50ac4ad1b879a4aafaa01f6d190c17fc3d9a10961fใ€‚
  • ไปŽๅŽŸๅง‹ FP32 checkpoint ๅฏผๅ‡บ็š„ F16 GGUF ็›ดๆŽฅ้‡ๅŒ–ๅˆฐ Q8_0๏ผŒๆœชๅฏน INT4 ๆƒ้‡ไบŒๆฌก้‡ๅŒ–ใ€‚
  • 170 ไธช Q8_0 ็Ÿฉ้˜ตใ€49 ไธช F32 ๅผ ้‡๏ผ›่พ“ๅ‡บๅคดไธบ Q8_0ใ€‚ๅ—็ผฉๆ”พๅ…ƒๆ•ฐๆฎไฝฟๆƒ้‡ๅนณๅ‡็บฆ 8.5 bit/ๅ‚ๆ•ฐใ€‚
  • llama.cpp ๅŸบ็ก€็‰ˆๆœฌ๏ผšf849095365d5ac1f560886a4e6574531d559e96fใ€‚ๆœฌๆœบ CANN ่ฟ่กŒๆ—ถๅฆๅซๅ…ผๅฎน่กฅไธ๏ผ›่ฟ™ไธๆ˜ฏๅŽŸ็”Ÿ OM ๆ–‡ไปถใ€‚
  • head_dim=128๏ผŒrope_theta=5000000๏ผ›่ฝฌๆขๆฅๆบๅŠ้…็ฝฎ่ง quantization.jsonใ€‚

ไฝฟ็”จ

ไฝฟ็”จๆ”ฏๆŒ่ฏฅๆจกๅž‹ Llama ๆžถๆž„ๅ’Œ GGUF ็š„ llama.cppใ€‚ๅ•่ทฏ็คบไพ‹๏ผš

llama-server --model AgenticASR-Refiner-Q8_0.gguf \
  --alias AgenticASR-Refiner --ctx-size 512 --parallel 1 \
  --jinja --host 127.0.0.1 --port 8080

่พ“ๅ…ฅไธŽ่พ“ๅ‡บๅ…ฑไบซไธŠไธ‹ๆ–‡ใ€‚512 token ๆ˜ฏๆœฌๆฌก็Ÿญๆ–‡ๆœฌๅทฅไฝœ้…็ฝฎ๏ผŒๅนถ้žๆจกๅž‹่ฎญ็ปƒไธŠไธ‹ๆ–‡ไธŠ้™ใ€‚ๅปบ่ฎฎ็บฆ 128 ๅญ—็š„่พ“ๅ…ฅใ€ๆœ€ๅคš 256 token ่พ“ๅ‡บ๏ผ›ๅญ—ๆ•ฐไธŽ token ๆ•ฐไธๅŒใ€‚

่ฐƒ็”จ Chat Completions ๆ—ถๆ˜พๅผไผ ๅ…ฅไปฅไธ‹ system ๆ็คบ่ฏใ€ๅ•ๆก user ๆ–‡ๆœฌ๏ผŒ่ฎพ็ฝฎ temperature: 0ใ€max_tokens: 256 ๅ’Œ stop: ["<|im_end|>"]๏ผš

ไฝ ๆ˜ฏ ASR ๆ–‡ๆœฌ็บ ้”™ๅŠฉๆ‰‹ใ€‚ไฟ็•™ๅŽŸๆ„๏ผŒๆœ€ๅฐไฟฎๆ”น๏ผšๅŽปๅฃ็™–/้‡ๅค๏ผŒไฟฎ้”™ๅญ—๏ผŒ่กฅๅฟ…่ฆๆ ‡็‚น๏ผŒ่ง„่Œƒๆ•ฐๅญ—ใ€ๆ—ฅๆœŸใ€ๆœฏ่ฏญๅ’Œไปฃ็ ็ฌฆๅท๏ผŒๅค„็†่‡ชๆˆ‘ไฟฎๆญฃใ€‚ไธ่ฆๆ€ป็ป“ใ€ๆ‰ฉๅ†™ๆˆ–่งฃ้‡Šใ€‚้‡่ฆๆ˜“้”™ๅฎžไฝ“ๅœจๆœซๅฐพ่ฟฝๅŠ  <KEY>[่ฏ1ใ€่ฏ2]๏ผ›ๆฒกๆœ‰ๅˆ™ไธๅŠ ใ€‚

ไธŠ่ฟฐ้€š็”จๅ‘ฝไปคไฝฟ็”จๅ†…ๅตŒ่Šๅคฉๆจกๆฟ๏ผ›2026-10-28 ็š„ๆœฌๆœบ้ชŒ่ฏไฝฟ็”จ็ญ‰ไปท็š„ๅ•่ฝฎ็บ ้”™ๆจกๆฟ๏ผŒassistant ๅ‰็ผ€ไธๆ’ๅ…ฅๆ€่€ƒๆ ‡็ญพ๏ผŒๅนถ้€ๆกๆฃ€ๆŸฅ prompt token IDs ไธŽ FP32 ๅŸบ็บฟไธ€่‡ดใ€‚็งปๆคๅˆฐๅ…ถไป–็ป‘ๅฎšๆˆ–่Šๅคฉๆจกๆฟๆ—ถๅบ”ๆฃ€ๆŸฅ็”Ÿๆˆๅ‰็ผ€ใ€‚่พ“ๅ‡บไธญ็š„ <KEY>[...] ไธบๆจกๅž‹็”Ÿๆˆ็š„ๅฎžไฝ“ๅญ—ๆฎตใ€‚

ๅทฒๅฎŒๆˆ็š„ๆœฌๆœบ้ชŒ่ฏ

ๆ˜‡่…พ 910B4-1ใ€CANN 9.1.1๏ผŒๅŒไธ€ llama.cpp CANN ๆž„ๅปบ๏ผŒ่ดชๅฟƒ่งฃ็ ๏ผš

  • 64 ๆก็Ÿญๆ–‡ๆœฌๅ…จ้ƒจๆญฃๅธธๅœๆญข๏ผŒ64/64 ่พ“ๅ‡บๆ–‡ๆœฌไธŽ Transformers CPU FP32 ๅŸบ็บฟ้€ๅญ—็›ธๅŒ๏ผ›ๅŒ็ป„ Q4_0 ไธบ 60/64ใ€‚
  • ๅ•ไปฝๆƒ้‡ใ€8 ไธชๆดปๅŠจๆงฝไฝ๏ผŒๆฏๆงฝ 512 token๏ผ›F16 KV ๅ…ฑ 96 MiBใ€‚
  • 8 ่ทฏๅ„ 128 ๅญ—่พ“ๅ…ฅๅ…จ้ƒจๆญฃๅธธ็ป“ๆŸ๏ผŒ8/8 ไธŽไธฒ่กŒ็ป“ๆžœ็›ธๅŒ๏ผ›SSE ๆ‹ผๆŽฅไธŽๆ™ฎ้€šๅ“ๅบ”ไธ€่‡ดใ€‚
  • NPU ๆจกๅž‹่ฟ›็จ‹ๅฐฑ็ปช็บฆ 1213 MiB๏ผŒๆŽจ็†้‡‡ๆ ทๆœ€้ซ˜ 1267 MiBใ€‚ๅŒๆœŸๆœ‰ๅ…ถไป– NPU ๆœๅŠก๏ผŒๆœชๅš้š”็ฆปๆ€ง่ƒฝๅŸบๅ‡†ๆˆ–้•ฟๆœŸๅŽ‹ๅŠ›ๆต‹่ฏ•ใ€‚

Q8 ไปๆ˜ฏๆœ‰ๆŸ้‡ๅŒ–ใ€‚ๅฐๆ ทๆœฌ่พ“ๅ‡บ็›ธๅŒไธ็ญ‰ไบŽๅ…จ้ขๆ— ๆŸ๏ผŒไนŸไธๆ˜ฏ็บ ้”™ๅ‡†็กฎ็އใ€‚ๅŽŸๅง‹ๆจกๅž‹็š„้”™่ฏฏไปๅฏ่ƒฝไฟ็•™๏ผŒไพ‹ๅฆ‚ๆœฌ็ป„โ€œๅไบŒไธ‡ไธ‰ๅƒๅ…ƒโ€ๅœจ FP32 ไธŽ Q8 ไธญ้ƒฝ่ขซ้”™่ฏฏๅ†™ๆˆ 12,300ๅ…ƒใ€‚ๅฎž้™…้ƒจ็ฝฒๅบ”ไฝฟ็”จไธšๅŠกๆ–‡ๆœฌ่ฏ„ไผฐๆ•ฐๅญ—ใ€ๅœฐๅ€ๅ’Œไธ“ๆœ‰ๅ่ฏใ€‚

ๆฅๆบไธŽ่ฎธๅฏ

ๅŽŸๅง‹ๆจกๅž‹๏ผšhttps://huggingface.co/Andrew0425/AgenticASR-Refiner

้‡ๅŒ–ๆฒฟ็”จไธŠๆธธ Apache-2.0 ่ฎธๅฏ๏ผŒ่ง LICENSEใ€‚

Downloads last month
38
GGUF
Model size
1B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for MigoXV/AgenticASR-Refiner-Q8_0

Quantized
(2)
this model