Instructions to use openmodelai/Luna-1.0-Audio with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openmodelai/Luna-1.0-Audio with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openmodelai/Luna-1.0-Audio")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForTextToWaveform

tokenizer = AutoTokenizer.from_pretrained("openmodelai/Luna-1.0-Audio")
model = AutoModelForTextToWaveform.from_pretrained("openmodelai/Luna-1.0-Audio", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openmodelai/Luna-1.0-Audio with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openmodelai/Luna-1.0-Audio"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openmodelai/Luna-1.0-Audio",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openmodelai/Luna-1.0-Audio

SGLang

How to use openmodelai/Luna-1.0-Audio with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openmodelai/Luna-1.0-Audio" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openmodelai/Luna-1.0-Audio",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openmodelai/Luna-1.0-Audio" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openmodelai/Luna-1.0-Audio",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openmodelai/Luna-1.0-Audio with Docker Model Runner:
```
docker model run hf.co/openmodelai/Luna-1.0-Audio
```

Luna-1.0-Audio / model.json

openmodelai

Upload 44 files

b83f555 verified 4 days ago

Raw History Blame Contribute Delete

13.8 kB

{"layer_sizes": [2, 16, 8, 1], "activation": "relu", "weights": [[[-2.9229746985877476, -0.9489083483653213], [-0.16158631352380343, 1.2192898600068751], [-0.7601004126204445, -0.6642078763040936], [1.3106999270140087, 0.5166327428616727], [-1.0356139003754548, 0.11265968561328775], [-1.1915521475551798, 1.0415187412161717], [-1.357355647323099, 0.8749834227149487], [0.013461498167323144, 0.8255065785629377], [2.447479613399256, -3.298439740445513], [0.5410968247135071, 2.935754109918622], [2.1435014780391604, 1.0605454678238622], [0.6660821951856788, -1.5982619981722646], [-0.338864695146397, 0.24472059791898615], [0.48746155182626855, 1.9227166414138754], [-0.578475104609686, -1.1980053175602252], [-0.7639207877302583, 1.5487853057838392]], [[-0.26263492576648867, -0.38769759795750636, -0.09784189120207663, 0.6072600873483914, -0.22277540031468623, -0.02096990886948959, -0.04373286605637957, 0.04744123852003385, 0.4037679357227058, 1.3465019126489157, 0.6141614664156578, -0.3201810703210546, -0.054737448112136766, 0.08456570738129435, 0.2209970080853104, -0.18059905296900344], [1.0509789169949422, 0.17726208348275388, 0.541359688133055, 0.052986949824929554, 0.5311746558374828, 0.13790599332513848, 0.019662931125788062, -0.2809533351737459, 0.5860708190159231, 0.6970577173216018, 0.7431367873826208, -0.24682118544571077, 0.31260262362853486, 0.38940519194488554, 0.5930126090938652, -0.10447426793847739], [-0.3171658325528823, 0.30882261898643454, 0.13451018776659468, 0.2300879331506235, 0.28377988089287853, -0.1411543018894175, 0.20184347883299084, -0.08552807735138712, 0.3863497285212326, 0.31504543520365647, 0.5619825340622131, 0.3991088373379381, -0.5838485323392086, 0.43580524289646383, -0.3432111786869342, -0.15176161635214266], [-0.4782795750309307, 0.06583233241531243, 0.10145728387863615, -0.4458473114126055, -0.6249321286383542, 0.34100028643846836, 0.0653888929263874, 0.4617110237548514, -0.39825376186543177, -0.3690920043800176, -0.4711949755066471, 0.2485087997477923, 0.594826463218114, 0.04519866464475843, -0.4117258779092394, -0.33670190877986916], [-0.3685040593577551, -0.7144804870772361, -0.5627534788390394, -0.5582075077390022, -0.5326042148412484, -0.6766947687931574, -0.8382499053883352, 0.1991611040957724, -0.6749765494765257, -0.000748691229898037, -1.2190360449506197, -0.42660158307314944, 0.9426732182637811, -0.29896330758259737, -0.7873549130479942, -0.4659544215947308], [0.6973085864525479, 0.14804157144858304, 0.1753734411387232, 0.4076401752946796, 1.0413178179570413, 0.49997084498442274, 0.037811046510364626, 0.15285932709701466, 0.3047140970325129, -0.05405860411939351, 0.7997843296015746, 0.35592549151766445, 0.05404758431634929, -0.23520370150961134, 0.9777704526330285, 0.44817464006237306], [2.090710510957523, 0.6665255241560822, 0.21271505075433916, 0.8808911907127206, 0.05876304772107294, 0.596859739823446, 0.936267829944002, -0.4405522268203582, 1.745541287111721, 1.7277754671685481, 1.695882738117948, 0.4972873255456095, -0.7587514879549355, 0.6638862837195482, 0.18342184737535763, 1.1511679486865911], [0.8612886342240752, 0.2345186754935847, 0.892113349131346, 0.2535429752436331, 0.3101684146688546, -0.07497977318314132, -0.04027238910422956, 0.49116754939820223, 0.6724144756469713, -0.08530013582683577, -0.14474386200531547, 0.22483365727880206, -0.4903794679000144, -0.5406041174689109, 0.13504585911391767, 0.2608491819749118]], [[1.170503679415227, 1.496039455944428, 0.6615722550119721, -1.3170746369951198, -2.9644986578272547, 1.3543363856865984, 3.9758603426467842, 1.132019537268394]]], "biases": [[-0.2036951885232317, -0.1373782744345608, -0.04671364969522937, -0.34424180939613064, -0.10269955981823849, -0.020250900451376407, -0.5298454557918851, 0.627313265757605, -0.3347116797236467, -0.3101322448353842, -0.5284117497378984, -0.2130095087845271, 1.1363915114372112, 0.07969259751883738, -0.18550831001723322, -0.324117086119126], [-0.2940110075158081, -0.6820429827809888, 0.06775658620045802, 0.8858799683049982, 2.018186517345542, -0.34694476948854963, -0.9826923913799661, -0.2649976467008574], [-1.9649560613013168]], "v_w": [[[-5.431563084221851e-05, -0.00017081247671554508], [1.284843013377604e-05, -1.7282457345847454e-05], [-2.0228628733457543e-05, -5.6791287385014825e-05], [4.2086361658848136e-05, 2.5306110286476245e-05], [-8.42617953524179e-06, -9.84756085565707e-05], [1.5264312729582347e-06, -6.082565540888484e-05], [4.666649221516426e-05, -4.9119179917511006e-05], [-1.5452443411460747e-05, 5.288801930848568e-05], [-5.507039398534936e-05, -0.00015479506021475693], [3.865721735346629e-05, -3.999925898253855e-05], [8.142989796742754e-05, 4.9332086473626567e-05], [-2.075131478622269e-05, -4.588696127518474e-05], [-2.8373383581237453e-05, 0.00012001998696304949], [3.53667039777422e-05, -1.7837990967046152e-05], [-4.018864997434942e-05, -8.969404333572945e-05], [4.622364658950773e-05, -3.445105976358674e-05]], [[3.230467411451809e-08, -4.315501363852648e-06, 1.2397880720942463e-06, 1.0046616363925697e-05, -1.2004688169732467e-06, -6.731279681059615e-06, 1.43690042861331e-06, -2.063253200204529e-05, 1.836645148796492e-05, -5.234745466053517e-06, 1.7298133843422713e-05, 6.5491850288267146e-06, -3.074568042982308e-05, -7.260921909615762e-06, 2.8658016786728136e-06, -2.0494712175300314e-06], [3.0217371882176097e-05, -4.499094167845985e-06, 1.6914068244327745e-05, 1.284088727362351e-05, 6.601174283958317e-07, -1.9128247270657923e-05, 3.6235009012944463e-06, -3.919173215579144e-05, 3.3717585707010625e-05, -6.516782935442478e-06, 2.2111793292648035e-05, 1.8987774270184815e-05, -3.868926243430937e-05, -9.60315301086873e-06, 2.5893162185817857e-05, -1.9750088163699594e-06], [4.664736393684129e-10, 6.1784092750652316e-06, 7.002950305414727e-07, 5.703776562555353e-06, 1.0449026588317383e-06, 8.331766840204292e-06, 1.672733440397904e-06, 8.572999233100074e-06, 1.0560803673868215e-05, 1.5748169823302202e-05, 9.827019109003956e-06, 3.7294015798961263e-06, 8.971001149419552e-06, 1.3553221958973683e-05, 1.619364711616656e-06, 8.254777591015658e-06], [-2.6599130095945837e-05, 3.961670322740102e-06, -1.4874978966790483e-05, -1.1309448991301401e-05, -5.776871322578134e-07, 1.6862376629208878e-05, -3.190023498900997e-06, 3.4944450848794194e-05, -2.9095538486202695e-05, 5.732976264662245e-06, -1.9474494954676837e-05, -1.6538867189354214e-05, 3.500177394218134e-05, 8.486555330948674e-06, -2.277803290739466e-05, 1.7405236581627376e-06], [-5.9872303248413846e-05, 8.921824028033381e-06, -3.34825906409967e-05, -2.5454008163241137e-05, -1.3000369909477375e-06, 3.795771805780311e-05, -7.180203638318639e-06, 7.867077658719453e-05, -6.549251249530883e-05, 1.2918273738072339e-05, -4.382958408998822e-05, -3.722804859912457e-05, 7.880221241351748e-05, 1.9116097102533006e-05, -5.127186138477889e-05, 3.919528420695303e-06], [2.7339160674315036e-05, -3.4174028667789367e-06, 1.5290777841782898e-05, 1.1683273734601585e-05, 5.928221504489269e-07, -1.682630535738182e-05, 3.2805026162595694e-06, -3.3632115103472854e-05, 2.993457657818457e-05, -3.9947367316599416e-06, 2.031819155945565e-05, 1.700346288410258e-05, -3.300943159188217e-05, -6.7338996609744064e-06, 2.3416167068913914e-05, -1.4414831870351025e-06], [9.155022420050853e-05, -1.1443770128885572e-05, 4.939698902937191e-05, 3.441831946973196e-05, 4.155475850813255e-06, -4.773774238211363e-05, 9.710941653964322e-06, -9.112882633590634e-05, 9.645862364171382e-05, -1.6390155320602792e-05, 5.934957876905855e-05, 5.303472145780748e-05, -7.37646820186524e-05, -2.350863646327647e-05, 7.220727757724969e-05, -4.677528313054465e-06], [2.6323169379997768e-05, 2.492702329656438e-06, 1.2973094934457796e-05, 3.591274665470242e-06, 3.5417752926383616e-06, -3.2778323195506445e-06, 2.279313009479777e-06, -4.732256933623898e-06, 2.4500495871325534e-05, 2.262538037147962e-06, 5.982674898037585e-06, 1.4249490548339086e-05, 9.828777422769084e-06, 4.989600443452209e-06, 1.9584954861134237e-05, 2.8714973980921636e-06]], [[2.0934067781025377e-05, 6.272893112143091e-05, 2.6371050117807567e-05, -7.900701790748727e-05, -0.0001259987954346718, 5.487536617117717e-05, 0.0001514851844017445, 4.088037951899195e-05]]], "v_b": [[9.565051388534745e-05, -9.213113798229408e-05, 3.684064153604115e-05, 1.1860010866020132e-05, -6.451905819696259e-05, -0.00026581996048383367, -0.00022982602948461782, 4.80656542146782e-05, 0.0002776208030138338, -0.00019594586166516903, 2.35698740686424e-05, 8.126207789066574e-05, 7.93757336859044e-05, -4.518580489189918e-05, 0.00014179223057847923, -0.0002502626564586516], [-2.2471032282466253e-05, -2.6077667769347364e-05, 7.79666173242656e-06, 2.385614299510485e-05, 5.371210012940224e-05, -2.1996328547185755e-05, -4.379994053788172e-05, 1.0731607352179675e-05], [-1.8138388673763487e-05]], "m_w": [[[0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0]], [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]], [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]], "v_w_adam": [[[0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0], [0.0, 0.0]], [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]], [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]], "m_b": [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0]], "v_b_adam": [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0]], "t": 0, "epoch": 100, "loss_history": [0.6309883784269327, 0.37057065925555016, 0.14730078744382613, 0.07341941418824555, 0.051578546051268305, 0.03578886638452809, 0.027912246388942154, 0.022699004308159684, 0.018030506150402057, 0.015244821170857695, 0.012397583647251511, 0.012857876358371035, 0.009517345158819497, 0.010430038699219404, 0.007346017945006792, 0.00904558500148795, 0.008303186208106363, 0.0073255425034430745, 0.0062657096216816865, 0.00585630312024781, 0.005164382464381767, 0.006182560787698531, 0.006808185033126516, 0.004709607826471856, 0.004557871810042085, 0.005977454165309984, 0.005437229733099107, 0.004188985519559077, 0.004310681864782301, 0.004343576623780565, 0.005997192259374933, 0.003183546044597947, 0.0033410070523177166, 0.0028641926379182653, 0.002953126980787447, 0.0027834527275027807, 0.0028276424708640604, 0.003284134451683949, 0.003414336644076806, 0.0025091338773310062, 0.002766003463726169, 0.002349846539647206, 0.0027559839288837495, 0.002479674312772259, 0.003156159325497143, 0.0024019005396046563, 0.001980563646156465, 0.0017186245508982504, 0.0018022006633241323, 0.0019429317213472586, 0.0020444527368089632, 0.0026998181255170777, 0.0020330627913692817, 0.0017988462559336868, 0.0020010295255740436, 0.0017717336431980967, 0.0014694125416086275, 0.0015421683967544375, 0.001296099924774927, 0.0013702599598664558, 0.0013214229036784006, 0.0013380596402457277, 0.0012562453414989802, 0.0012182388378994173, 0.0012146764022857015, 0.001234341669591977, 0.0012201543907429725, 0.0014066310085536532, 0.0011254014296256595, 0.0010946725356226518, 0.001359428325280189, 0.001413992578730653, 0.001076742210300881, 0.0010420490941362695, 0.0010535006065298146, 0.000992766322301542, 0.0009981418678105503, 0.0009997613409239829, 0.0009140171873072962, 0.0009601818872580662, 0.0008912749376519886, 0.0009008028554848211, 0.0009114768257810077, 0.0009207399149118657, 0.0009231479402332136, 0.0009364585911901046, 0.000861918045382861, 0.0008367310804534187, 0.0008089635462740302, 0.000907943007403331, 0.0007851015306966462, 0.0008583336944746952, 0.0008586867465268657, 0.0007974961712823352, 0.0007723398091808622, 0.000779619924819028, 0.0007377923568268368, 0.00073133864064005, 0.0007730543110009348, 0.0007080572935341104], "acc_history": [0.8333333333333334, 0.99, 0.9966666666666667, 0.9966666666666667, 0.99, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.9966666666666667, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]}