Spaces:
Sleeping
Sleeping
AnjaJuana commited on
Commit ·
e49048a
1
Parent(s): 1107887
Auto-update from GitHub
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .ipynb_checkpoints/code_blog_post-checkpoint.ipynb +244 -370
- app.py +23 -15
- blog_post/article.md +104 -151
- code_blog_post.ipynb +103 -419
- space/blog_post/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +802 -0
- space/blog_post/article.md +212 -54
- space/blog_post/img/gradio_pred_trip.png +0 -0
- space/code_blog_post.ipynb +184 -355
- space/main_model.ipynb +1 -8
- space/space/space/blog_post/article.md +82 -59
- space/space/space/code_blog_post.ipynb +3 -6
- space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +105 -55
- space/space/space/space/blog_post/img/gradio_pred_trip.png +0 -0
- space/space/space/space/space/blog_post/article.md +39 -51
- space/space/space/space/space/code_blog_post.ipynb +100 -49
- space/space/space/space/space/space/packing_list_api.ipynb +19 -4
- space/space/space/space/space/space/results/MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7_results.pkl +3 -0
- space/space/space/space/space/space/results/pongjin-roberta_with_kornli_results.pkl +3 -0
- space/space/space/space/space/space/results/sileod-deberta-v3-base-tasksource-nli_results.pkl +3 -0
- space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +294 -64
- space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb +25 -0
- space/space/space/space/space/space/space/blog_post/article.md +186 -55
- space/space/space/space/space/space/space/blog_post/img/.DS_Store +0 -0
- space/space/space/space/space/space/space/blog_post/img/gradio_pred_trip.png +0 -0
- space/space/space/space/space/space/space/code_blog_post.ipynb +297 -65
- space/space/space/space/space/space/space/main_model.ipynb +801 -87
- space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +17 -114
- space/space/space/space/space/space/space/space/space/blog_post/.DS_Store +0 -0
- space/space/space/space/space/space/space/space/space/blog_post/article.md +291 -50
- space/space/space/space/space/space/space/space/space/blog_post/img/gradio_pred_trip.png +0 -0
- space/space/space/space/space/space/space/space/space/code_blog_post.ipynb +18 -115
- space/space/space/space/space/space/space/space/space/space/main_model.ipynb +13 -6
- space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/app-checkpoint.py +4 -5
- space/space/space/space/space/space/space/space/space/space/space/app.py +4 -5
- space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +229 -8
- space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb +418 -22
- space/space/space/space/space/space/space/space/space/space/space/space/space/requirements.txt +0 -3
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb +398 -0
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/packing_list_api-checkpoint.ipynb +541 -89
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/README.md +6 -6
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/blog_post/article.md +129 -5
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/code_blog_post.ipynb +619 -0
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/packing_list_api.ipynb +22 -34
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/requirements.txt +7 -2
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/packing_templates_self_supported_offgrid_expanded_dutch_original.json +696 -0
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/app-checkpoint.py +25 -11
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/labels-checkpoint.txt +93 -0
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb +0 -0
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/app.py +25 -11
- space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/blog_post/article.md +44 -0
.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
@@ -20,7 +20,7 @@
|
|
| 20 |
"import requests\n",
|
| 21 |
"import json\n",
|
| 22 |
"\n",
|
| 23 |
-
"load_dotenv()
|
| 24 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 25 |
"\n",
|
| 26 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
|
@@ -37,7 +37,7 @@
|
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
-
"execution_count":
|
| 41 |
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
"metadata": {},
|
| 43 |
"outputs": [
|
|
@@ -47,27 +47,27 @@
|
|
| 47 |
"text": [
|
| 48 |
"[\n",
|
| 49 |
" {\n",
|
| 50 |
-
" \"label\": \"
|
| 51 |
-
" \"score\": 0.
|
| 52 |
" },\n",
|
| 53 |
" {\n",
|
| 54 |
" \"label\": \"health\",\n",
|
| 55 |
-
" \"score\": 0.
|
| 56 |
" },\n",
|
| 57 |
" {\n",
|
| 58 |
-
" \"label\": \"
|
| 59 |
-
" \"score\": 0.
|
| 60 |
" },\n",
|
| 61 |
" {\n",
|
| 62 |
" \"label\": \"politics\",\n",
|
| 63 |
-
" \"score\": 0.
|
| 64 |
" }\n",
|
| 65 |
"]\n"
|
| 66 |
]
|
| 67 |
}
|
| 68 |
],
|
| 69 |
"source": [
|
| 70 |
-
"input_text = \"I just
|
| 71 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
"print(json.dumps(output, indent=4))"
|
| 73 |
]
|
|
@@ -82,7 +82,7 @@
|
|
| 82 |
},
|
| 83 |
{
|
| 84 |
"cell_type": "code",
|
| 85 |
-
"execution_count":
|
| 86 |
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
|
@@ -205,7 +205,7 @@
|
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
-
"execution_count":
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
@@ -255,27 +255,7 @@
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
-
"execution_count":
|
| 259 |
-
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 260 |
-
"metadata": {},
|
| 261 |
-
"outputs": [
|
| 262 |
-
{
|
| 263 |
-
"name": "stdout",
|
| 264 |
-
"output_type": "stream",
|
| 265 |
-
"text": [
|
| 266 |
-
"beach vacation\n"
|
| 267 |
-
]
|
| 268 |
-
}
|
| 269 |
-
],
|
| 270 |
-
"source": [
|
| 271 |
-
"# the labels are sorted by score. We choose the first one as our best guess for a class label\n",
|
| 272 |
-
"class_label = result[\"labels\"][0]\n",
|
| 273 |
-
"print(class_label)"
|
| 274 |
-
]
|
| 275 |
-
},
|
| 276 |
-
{
|
| 277 |
-
"cell_type": "code",
|
| 278 |
-
"execution_count": 7,
|
| 279 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 280 |
"metadata": {
|
| 281 |
"scrolled": true
|
|
@@ -311,7 +291,7 @@
|
|
| 311 |
"22 skiing 0.001169\n",
|
| 312 |
"23 scuba diving 0.000789\n",
|
| 313 |
"24 ski touring 0.000491\n",
|
| 314 |
-
"[]\n"
|
| 315 |
]
|
| 316 |
}
|
| 317 |
],
|
|
@@ -319,22 +299,49 @@
|
|
| 319 |
"# we do things differently for \"activities\"\n",
|
| 320 |
"cut_off = 0.5\n",
|
| 321 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 322 |
-
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
| 323 |
"\n",
|
| 324 |
"df = pd.DataFrame({\n",
|
| 325 |
" \"Label\": result_activ[\"labels\"],\n",
|
| 326 |
" \"Score\": result_activ[\"scores\"]\n",
|
| 327 |
"})\n",
|
|
|
|
|
|
|
| 328 |
"print(df)\n",
|
| 329 |
"print(classes)"
|
| 330 |
]
|
| 331 |
},
|
| 332 |
{
|
| 333 |
"cell_type": "code",
|
| 334 |
-
"execution_count":
|
| 335 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 336 |
-
"metadata": {
|
| 337 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 338 |
"source": [
|
| 339 |
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 340 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
|
@@ -362,13 +369,16 @@
|
|
| 362 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 363 |
" classes = result[\"labels\"][0]\n",
|
| 364 |
" df.loc[i] = [key, classes]\n",
|
| 365 |
-
" return df"
|
|
|
|
|
|
|
|
|
|
| 366 |
]
|
| 367 |
},
|
| 368 |
{
|
| 369 |
"cell_type": "code",
|
| 370 |
-
"execution_count":
|
| 371 |
-
"id": "
|
| 372 |
"metadata": {},
|
| 373 |
"outputs": [
|
| 374 |
{
|
|
@@ -391,169 +401,151 @@
|
|
| 391 |
"5 accommodation huts with half board\n",
|
| 392 |
"6 transportation no own vehicle\n",
|
| 393 |
"7 special_conditions off-grid / no electricity\n",
|
| 394 |
-
"8 trip_length_days 7+ days\n"
|
| 395 |
-
]
|
| 396 |
-
}
|
| 397 |
-
],
|
| 398 |
-
"source": [
|
| 399 |
-
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 400 |
-
"print(result)"
|
| 401 |
-
]
|
| 402 |
-
},
|
| 403 |
-
{
|
| 404 |
-
"cell_type": "markdown",
|
| 405 |
-
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 406 |
-
"metadata": {},
|
| 407 |
-
"source": [
|
| 408 |
-
"Now use gradio app"
|
| 409 |
-
]
|
| 410 |
-
},
|
| 411 |
-
{
|
| 412 |
-
"cell_type": "code",
|
| 413 |
-
"execution_count": 3,
|
| 414 |
-
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 415 |
-
"metadata": {},
|
| 416 |
-
"outputs": [],
|
| 417 |
-
"source": [
|
| 418 |
-
"# Prerequisites\n",
|
| 419 |
-
"#from transformers import pipeline\n",
|
| 420 |
-
"#import json\n",
|
| 421 |
-
"#import pandas as pd\n"
|
| 422 |
-
]
|
| 423 |
-
},
|
| 424 |
-
{
|
| 425 |
-
"cell_type": "code",
|
| 426 |
-
"execution_count": 11,
|
| 427 |
-
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 428 |
-
"metadata": {},
|
| 429 |
-
"outputs": [
|
| 430 |
-
{
|
| 431 |
-
"name": "stdout",
|
| 432 |
-
"output_type": "stream",
|
| 433 |
-
"text": [
|
| 434 |
-
"Running on local URL: http://127.0.0.1:7861\n",
|
| 435 |
"\n",
|
| 436 |
-
"
|
| 437 |
-
]
|
| 438 |
-
},
|
| 439 |
-
{
|
| 440 |
-
"data": {
|
| 441 |
-
"text/html": [
|
| 442 |
-
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 443 |
-
],
|
| 444 |
-
"text/plain": [
|
| 445 |
-
"<IPython.core.display.HTML object>"
|
| 446 |
-
]
|
| 447 |
-
},
|
| 448 |
-
"metadata": {},
|
| 449 |
-
"output_type": "display_data"
|
| 450 |
-
}
|
| 451 |
-
],
|
| 452 |
-
"source": [
|
| 453 |
-
"import gradio as gr\n",
|
| 454 |
-
"\n",
|
| 455 |
-
"demo = gr.Interface(\n",
|
| 456 |
-
" fn=pred_trip,\n",
|
| 457 |
-
" inputs=[\n",
|
| 458 |
-
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 459 |
-
" gr.Textbox(label=\"Trip description\"),\n",
|
| 460 |
-
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 461 |
-
" ],\n",
|
| 462 |
-
" # outputs=\"dataframe\",\n",
|
| 463 |
-
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 464 |
-
" title=\"Trip classification\",\n",
|
| 465 |
-
" description=\"Enter a text describing your trip\",\n",
|
| 466 |
-
")\n",
|
| 467 |
-
"\n",
|
| 468 |
-
"# Launch the Gradio app\n",
|
| 469 |
-
"if __name__ == \"__main__\":\n",
|
| 470 |
-
" demo.launch()\n"
|
| 471 |
-
]
|
| 472 |
-
},
|
| 473 |
-
{
|
| 474 |
-
"cell_type": "code",
|
| 475 |
-
"execution_count": 46,
|
| 476 |
-
"id": "11006b67-bfd5-42a7-99c4-36c3db3affac",
|
| 477 |
-
"metadata": {},
|
| 478 |
-
"outputs": [
|
| 479 |
-
{
|
| 480 |
-
"name": "stderr",
|
| 481 |
-
"output_type": "stream",
|
| 482 |
-
"text": [
|
| 483 |
-
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 484 |
-
]
|
| 485 |
-
},
|
| 486 |
-
{
|
| 487 |
-
"name": "stdout",
|
| 488 |
-
"output_type": "stream",
|
| 489 |
-
"text": [
|
| 490 |
-
"Processing 9/9"
|
| 491 |
-
]
|
| 492 |
-
}
|
| 493 |
-
],
|
| 494 |
-
"source": [
|
| 495 |
-
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 496 |
-
]
|
| 497 |
-
},
|
| 498 |
-
{
|
| 499 |
-
"cell_type": "markdown",
|
| 500 |
-
"id": "39920553-27d3-4a63-8381-8310566c4874",
|
| 501 |
-
"metadata": {},
|
| 502 |
-
"source": [
|
| 503 |
-
"All code for gradio app file"
|
| 504 |
-
]
|
| 505 |
-
},
|
| 506 |
-
{
|
| 507 |
-
"cell_type": "code",
|
| 508 |
-
"execution_count": 2,
|
| 509 |
-
"id": "4ffc76d5-60c3-4bc8-bd4f-a9636077f01d",
|
| 510 |
-
"metadata": {},
|
| 511 |
-
"outputs": [
|
| 512 |
-
{
|
| 513 |
-
"name": "stdout",
|
| 514 |
-
"output_type": "stream",
|
| 515 |
-
"text": [
|
| 516 |
-
"Running on local URL: http://127.0.0.1:7861\n",
|
| 517 |
"\n",
|
| 518 |
-
"
|
| 519 |
-
|
| 520 |
-
|
| 521 |
-
|
| 522 |
-
|
| 523 |
-
"
|
| 524 |
-
|
| 525 |
-
|
| 526 |
-
"
|
| 527 |
-
|
| 528 |
-
|
| 529 |
-
|
| 530 |
-
|
| 531 |
-
|
| 532 |
-
|
| 533 |
-
|
| 534 |
-
|
| 535 |
-
|
| 536 |
-
|
| 537 |
-
"
|
| 538 |
-
|
| 539 |
-
|
| 540 |
-
|
| 541 |
-
|
| 542 |
-
|
| 543 |
-
|
| 544 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 545 |
]
|
| 546 |
}
|
| 547 |
],
|
| 548 |
"source": [
|
| 549 |
-
"
|
| 550 |
-
"
|
| 551 |
-
"
|
| 552 |
-
"import gradio as gr\n",
|
| 553 |
"\n",
|
| 554 |
-
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 555 |
-
" candidate_labels = json.load(file)\n",
|
| 556 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 557 |
"\n",
|
| 558 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 559 |
" \"\"\"\n",
|
|
@@ -580,201 +572,83 @@
|
|
| 580 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 581 |
" classes = result[\"labels\"][0]\n",
|
| 582 |
" df.loc[i] = [key, classes]\n",
|
| 583 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 584 |
"\n",
|
| 585 |
-
"
|
| 586 |
-
" fn=pred_trip,\n",
|
| 587 |
-
" inputs=[\n",
|
| 588 |
-
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 589 |
-
" gr.Textbox(label=\"Trip description\"),\n",
|
| 590 |
-
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 591 |
-
" ],\n",
|
| 592 |
-
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 593 |
-
" title=\"Trip classification\",\n",
|
| 594 |
-
" description=\"Enter a text describing your trip\",\n",
|
| 595 |
-
")\n",
|
| 596 |
"\n",
|
| 597 |
-
"
|
| 598 |
-
"
|
| 599 |
-
"
|
| 600 |
]
|
| 601 |
},
|
| 602 |
{
|
| 603 |
-
"cell_type": "
|
| 604 |
-
"
|
| 605 |
-
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 606 |
"metadata": {},
|
| 607 |
-
"outputs": [],
|
| 608 |
"source": [
|
| 609 |
-
"
|
| 610 |
]
|
| 611 |
},
|
| 612 |
{
|
| 613 |
"cell_type": "code",
|
| 614 |
-
"execution_count":
|
| 615 |
-
"id": "
|
| 616 |
"metadata": {},
|
| 617 |
"outputs": [
|
| 618 |
{
|
| 619 |
"name": "stdout",
|
| 620 |
"output_type": "stream",
|
| 621 |
"text": [
|
| 622 |
-
"
|
| 623 |
-
"\n",
|
| 624 |
-
"
|
| 625 |
-
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 626 |
-
"warm destination / summer\n",
|
| 627 |
-
"lightweight (but comfortable)\n",
|
| 628 |
-
"casual\n",
|
| 629 |
-
"indoor\n",
|
| 630 |
-
"no own vehicle\n",
|
| 631 |
-
"no special conditions to consider\n",
|
| 632 |
-
"7+ days\n",
|
| 633 |
-
"\n",
|
| 634 |
-
"\n",
|
| 635 |
-
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 636 |
-
"\n",
|
| 637 |
-
"city trip\n",
|
| 638 |
-
"['sightseeing']\n",
|
| 639 |
-
"variable weather / spring / autumn\n",
|
| 640 |
-
"luxury (including evening wear)\n",
|
| 641 |
-
"casual\n",
|
| 642 |
-
"indoor\n",
|
| 643 |
-
"no own vehicle\n",
|
| 644 |
-
"no special conditions to consider\n",
|
| 645 |
-
"3 days\n",
|
| 646 |
-
"\n",
|
| 647 |
-
"\n",
|
| 648 |
-
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 649 |
-
"\n",
|
| 650 |
-
"city trip\n",
|
| 651 |
-
"['relaxing']\n",
|
| 652 |
-
"cold destination / winter\n",
|
| 653 |
-
"lightweight (but comfortable)\n",
|
| 654 |
-
"casual\n",
|
| 655 |
-
"indoor\n",
|
| 656 |
-
"no own vehicle\n",
|
| 657 |
-
"no special conditions to consider\n",
|
| 658 |
-
"7+ days\n",
|
| 659 |
-
"\n",
|
| 660 |
-
"\n",
|
| 661 |
-
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 662 |
-
"\n",
|
| 663 |
-
"cultural exploration\n",
|
| 664 |
-
"['sightseeing', 'hiking', 'rafting']\n",
|
| 665 |
-
"variable weather / spring / autumn\n",
|
| 666 |
-
"lightweight (but comfortable)\n",
|
| 667 |
-
"casual\n",
|
| 668 |
-
"indoor\n",
|
| 669 |
-
"no own vehicle\n",
|
| 670 |
-
"rainy climate\n",
|
| 671 |
-
"7+ days\n",
|
| 672 |
-
"\n",
|
| 673 |
-
"\n",
|
| 674 |
-
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 675 |
-
"\n",
|
| 676 |
-
"nature escape\n",
|
| 677 |
-
"['swimming', 'relaxing', 'hiking']\n",
|
| 678 |
-
"warm destination / summer\n",
|
| 679 |
-
"lightweight (but comfortable)\n",
|
| 680 |
-
"casual\n",
|
| 681 |
-
"indoor\n",
|
| 682 |
-
"no own vehicle\n",
|
| 683 |
-
"no special conditions to consider\n",
|
| 684 |
-
"7+ days\n",
|
| 685 |
-
"\n",
|
| 686 |
-
"\n",
|
| 687 |
-
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 688 |
-
"\n",
|
| 689 |
-
"long-distance hike / thru-hike\n",
|
| 690 |
-
"['going to the beach']\n",
|
| 691 |
-
"tropical / humid\n",
|
| 692 |
-
"minimalist\n",
|
| 693 |
-
"casual\n",
|
| 694 |
-
"huts with half board\n",
|
| 695 |
-
"own vehicle\n",
|
| 696 |
-
"off-grid / no electricity\n",
|
| 697 |
-
"6 days\n",
|
| 698 |
-
"\n",
|
| 699 |
-
"\n",
|
| 700 |
-
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 701 |
-
"\n",
|
| 702 |
-
"beach vacation\n",
|
| 703 |
-
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 704 |
-
"cold destination / winter\n",
|
| 705 |
-
"ultralight\n",
|
| 706 |
-
"casual\n",
|
| 707 |
-
"sleeping in a tent\n",
|
| 708 |
-
"own vehicle\n",
|
| 709 |
-
"off-grid / no electricity\n",
|
| 710 |
-
"6 days\n",
|
| 711 |
-
"\n",
|
| 712 |
-
"\n",
|
| 713 |
-
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 714 |
-
"\n",
|
| 715 |
-
"yoga / wellness retreat\n",
|
| 716 |
-
"['hiking', 'yoga']\n",
|
| 717 |
-
"cold destination / winter\n",
|
| 718 |
-
"lightweight (but comfortable)\n",
|
| 719 |
-
"casual\n",
|
| 720 |
-
"indoor\n",
|
| 721 |
-
"no own vehicle\n",
|
| 722 |
-
"snow and ice\n",
|
| 723 |
-
"7 days\n",
|
| 724 |
-
"\n",
|
| 725 |
-
"\n",
|
| 726 |
-
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 727 |
-
"\n",
|
| 728 |
-
"ski tour / skitour\n",
|
| 729 |
-
"['ski touring', 'photography']\n",
|
| 730 |
-
"cold destination / winter\n",
|
| 731 |
-
"minimalist\n",
|
| 732 |
-
"conservative\n",
|
| 733 |
-
"indoor\n",
|
| 734 |
-
"no own vehicle\n",
|
| 735 |
-
"avalanche-prone terrain\n",
|
| 736 |
-
"7+ days\n",
|
| 737 |
-
"\n",
|
| 738 |
-
"\n",
|
| 739 |
-
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 740 |
-
"\n",
|
| 741 |
-
"camping trip (wild camping)\n",
|
| 742 |
-
"['scuba diving', 'kayaking / canoeing']\n",
|
| 743 |
-
"tropical / humid\n",
|
| 744 |
-
"lightweight (but comfortable)\n",
|
| 745 |
-
"casual\n",
|
| 746 |
-
"sleeping in a tent\n",
|
| 747 |
-
"own vehicle\n",
|
| 748 |
-
"self-supported (bring your own cooking gear)\n",
|
| 749 |
-
"3 days\n",
|
| 750 |
-
"\n",
|
| 751 |
-
"\n"
|
| 752 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 753 |
}
|
| 754 |
],
|
| 755 |
"source": [
|
| 756 |
-
"
|
| 757 |
-
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 758 |
-
" packing_data = json.load(file)\n",
|
| 759 |
-
" # Extract trip descriptions and classification (trip_types)\n",
|
| 760 |
-
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 761 |
-
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 762 |
"\n",
|
| 763 |
-
"
|
| 764 |
-
"
|
| 765 |
-
"
|
| 766 |
-
"
|
| 767 |
-
"
|
| 768 |
-
|
| 769 |
-
|
| 770 |
-
|
| 771 |
-
|
| 772 |
-
|
| 773 |
-
|
| 774 |
-
|
| 775 |
-
|
| 776 |
-
|
| 777 |
-
"
|
| 778 |
]
|
| 779 |
}
|
| 780 |
],
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 1,
|
| 14 |
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
|
|
| 20 |
"import requests\n",
|
| 21 |
"import json\n",
|
| 22 |
"\n",
|
| 23 |
+
"load_dotenv() \n",
|
| 24 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 25 |
"\n",
|
| 26 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
|
|
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
+
"execution_count": 6,
|
| 41 |
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
"metadata": {},
|
| 43 |
"outputs": [
|
|
|
|
| 47 |
"text": [
|
| 48 |
"[\n",
|
| 49 |
" {\n",
|
| 50 |
+
" \"label\": \"sports\",\n",
|
| 51 |
+
" \"score\": 0.9877110719680786\n",
|
| 52 |
" },\n",
|
| 53 |
" {\n",
|
| 54 |
" \"label\": \"health\",\n",
|
| 55 |
+
" \"score\": 0.006601463537663221\n",
|
| 56 |
" },\n",
|
| 57 |
" {\n",
|
| 58 |
+
" \"label\": \"technology\",\n",
|
| 59 |
+
" \"score\": 0.004392746835947037\n",
|
| 60 |
" },\n",
|
| 61 |
" {\n",
|
| 62 |
" \"label\": \"politics\",\n",
|
| 63 |
+
" \"score\": 0.0012947289505973458\n",
|
| 64 |
" }\n",
|
| 65 |
"]\n"
|
| 66 |
]
|
| 67 |
}
|
| 68 |
],
|
| 69 |
"source": [
|
| 70 |
+
"input_text = \"I just started to play tennis, and it's so much fun!\"\n",
|
| 71 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
"print(json.dumps(output, indent=4))"
|
| 73 |
]
|
|
|
|
| 82 |
},
|
| 83 |
{
|
| 84 |
"cell_type": "code",
|
| 85 |
+
"execution_count": 7,
|
| 86 |
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
|
|
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
+
"execution_count": 8,
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
+
"execution_count": 9,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 259 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 260 |
"metadata": {
|
| 261 |
"scrolled": true
|
|
|
|
| 291 |
"22 skiing 0.001169\n",
|
| 292 |
"23 scuba diving 0.000789\n",
|
| 293 |
"24 ski touring 0.000491\n",
|
| 294 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 295 |
]
|
| 296 |
}
|
| 297 |
],
|
|
|
|
| 299 |
"# we do things differently for \"activities\"\n",
|
| 300 |
"cut_off = 0.5\n",
|
| 301 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
|
|
|
| 302 |
"\n",
|
| 303 |
"df = pd.DataFrame({\n",
|
| 304 |
" \"Label\": result_activ[\"labels\"],\n",
|
| 305 |
" \"Score\": result_activ[\"scores\"]\n",
|
| 306 |
"})\n",
|
| 307 |
+
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
| 308 |
+
"\n",
|
| 309 |
"print(df)\n",
|
| 310 |
"print(classes)"
|
| 311 |
]
|
| 312 |
},
|
| 313 |
{
|
| 314 |
"cell_type": "code",
|
| 315 |
+
"execution_count": 10,
|
| 316 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 317 |
+
"metadata": {
|
| 318 |
+
"scrolled": true
|
| 319 |
+
},
|
| 320 |
+
"outputs": [
|
| 321 |
+
{
|
| 322 |
+
"name": "stderr",
|
| 323 |
+
"output_type": "stream",
|
| 324 |
+
"text": [
|
| 325 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 326 |
+
]
|
| 327 |
+
},
|
| 328 |
+
{
|
| 329 |
+
"name": "stdout",
|
| 330 |
+
"output_type": "stream",
|
| 331 |
+
"text": [
|
| 332 |
+
"Processing 9/9 superclass pred_class\n",
|
| 333 |
+
"0 activity_type beach vacation\n",
|
| 334 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 335 |
+
"2 climate_or_season warm destination / summer\n",
|
| 336 |
+
"3 style_or_comfort minimalist\n",
|
| 337 |
+
"4 dress_code casual\n",
|
| 338 |
+
"5 accommodation huts with half board\n",
|
| 339 |
+
"6 transportation no own vehicle\n",
|
| 340 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 341 |
+
"8 trip_length_days 7+ days\n"
|
| 342 |
+
]
|
| 343 |
+
}
|
| 344 |
+
],
|
| 345 |
"source": [
|
| 346 |
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 347 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
|
|
|
| 369 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 370 |
" classes = result[\"labels\"][0]\n",
|
| 371 |
" df.loc[i] = [key, classes]\n",
|
| 372 |
+
" return df\n",
|
| 373 |
+
"\n",
|
| 374 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 375 |
+
"print(result)"
|
| 376 |
]
|
| 377 |
},
|
| 378 |
{
|
| 379 |
"cell_type": "code",
|
| 380 |
+
"execution_count": 11,
|
| 381 |
+
"id": "f8b547d4-1858-49d5-b26c-fd30e210cd58",
|
| 382 |
"metadata": {},
|
| 383 |
"outputs": [
|
| 384 |
{
|
|
|
|
| 401 |
"5 accommodation huts with half board\n",
|
| 402 |
"6 transportation no own vehicle\n",
|
| 403 |
"7 special_conditions off-grid / no electricity\n",
|
| 404 |
+
"8 trip_length_days 7+ days\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 405 |
"\n",
|
| 406 |
+
" 1 set of clothing for every situation\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 407 |
"\n",
|
| 408 |
+
" USB hub (for multiple devices)\n",
|
| 409 |
+
"\n",
|
| 410 |
+
" all‑in‑one soap\n",
|
| 411 |
+
"\n",
|
| 412 |
+
" backpack\n",
|
| 413 |
+
"\n",
|
| 414 |
+
" backup lighting (e.g. small flashlight)\n",
|
| 415 |
+
"\n",
|
| 416 |
+
" beach bag\n",
|
| 417 |
+
"\n",
|
| 418 |
+
" beach chair\n",
|
| 419 |
+
"\n",
|
| 420 |
+
" beach towel\n",
|
| 421 |
+
"\n",
|
| 422 |
+
" blister plasters or tape\n",
|
| 423 |
+
"\n",
|
| 424 |
+
" book or e‑reader\n",
|
| 425 |
+
"\n",
|
| 426 |
+
" cap or hat\n",
|
| 427 |
+
"\n",
|
| 428 |
+
" cash for payments\n",
|
| 429 |
+
"\n",
|
| 430 |
+
" comfortable clothing\n",
|
| 431 |
+
"\n",
|
| 432 |
+
" compact toothbrush\n",
|
| 433 |
+
"\n",
|
| 434 |
+
" cooler\n",
|
| 435 |
+
"\n",
|
| 436 |
+
" daypack\n",
|
| 437 |
+
"\n",
|
| 438 |
+
" earplugs\n",
|
| 439 |
+
"\n",
|
| 440 |
+
" emergency communication (e.g. GPS beacon or satellite messenger)\n",
|
| 441 |
+
"\n",
|
| 442 |
+
" extra charger cables\n",
|
| 443 |
+
"\n",
|
| 444 |
+
" extra clothing layer\n",
|
| 445 |
+
"\n",
|
| 446 |
+
" first aid kit\n",
|
| 447 |
+
"\n",
|
| 448 |
+
" flashlight or headlamp\n",
|
| 449 |
+
"\n",
|
| 450 |
+
" flip flops\n",
|
| 451 |
+
"\n",
|
| 452 |
+
" foldable solar panel (if on longer trips)\n",
|
| 453 |
+
"\n",
|
| 454 |
+
" hat or cap\n",
|
| 455 |
+
"\n",
|
| 456 |
+
" headlamp + extra batteries\n",
|
| 457 |
+
"\n",
|
| 458 |
+
" hiking boots or trail runners\n",
|
| 459 |
+
"\n",
|
| 460 |
+
" hiking poles\n",
|
| 461 |
+
"\n",
|
| 462 |
+
" hiking socks (anti-blister)\n",
|
| 463 |
+
"\n",
|
| 464 |
+
" jeans or comfortable pants\n",
|
| 465 |
+
"\n",
|
| 466 |
+
" light pajamas or sleepwear\n",
|
| 467 |
+
"\n",
|
| 468 |
+
" light shoes\n",
|
| 469 |
+
"\n",
|
| 470 |
+
" light towel\n",
|
| 471 |
+
"\n",
|
| 472 |
+
" lightweight clothing\n",
|
| 473 |
+
"\n",
|
| 474 |
+
" lightweight towel\n",
|
| 475 |
+
"\n",
|
| 476 |
+
" music / headphones\n",
|
| 477 |
+
"\n",
|
| 478 |
+
" navigation (map/compass/GPS)\n",
|
| 479 |
+
"\n",
|
| 480 |
+
" navigation device with offline maps\n",
|
| 481 |
+
"\n",
|
| 482 |
+
" notebook + pen\n",
|
| 483 |
+
"\n",
|
| 484 |
+
" number of meals/snacks matched to duration\n",
|
| 485 |
+
"\n",
|
| 486 |
+
" packaging to keep electronics dry\n",
|
| 487 |
+
"\n",
|
| 488 |
+
" paper map and compass\n",
|
| 489 |
+
"\n",
|
| 490 |
+
" power bank (at least 10,000 mAh)\n",
|
| 491 |
+
"\n",
|
| 492 |
+
" public transport app or ticket\n",
|
| 493 |
+
"\n",
|
| 494 |
+
" rain jacket or poncho\n",
|
| 495 |
+
"\n",
|
| 496 |
+
" rechargeable batteries and charger\n",
|
| 497 |
+
"\n",
|
| 498 |
+
" reservation confirmation\n",
|
| 499 |
+
"\n",
|
| 500 |
+
" seat cushion or beach mat\n",
|
| 501 |
+
"\n",
|
| 502 |
+
" sheet liner (often required)\n",
|
| 503 |
+
"\n",
|
| 504 |
+
" slippers or indoor shoes for inside\n",
|
| 505 |
+
"\n",
|
| 506 |
+
" small backpack\n",
|
| 507 |
+
"\n",
|
| 508 |
+
" small toiletry bag\n",
|
| 509 |
+
"\n",
|
| 510 |
+
" snacks / energy bars\n",
|
| 511 |
+
"\n",
|
| 512 |
+
" snacks for along the way\n",
|
| 513 |
+
"\n",
|
| 514 |
+
" sneakers\n",
|
| 515 |
+
"\n",
|
| 516 |
+
" socks per day\n",
|
| 517 |
+
"\n",
|
| 518 |
+
" solar panel or portable charging system\n",
|
| 519 |
+
"\n",
|
| 520 |
+
" sun hat\n",
|
| 521 |
+
"\n",
|
| 522 |
+
" sunglasses\n",
|
| 523 |
+
"\n",
|
| 524 |
+
" sunscreen\n",
|
| 525 |
+
"\n",
|
| 526 |
+
" sunscreen and sunglasses\n",
|
| 527 |
+
"\n",
|
| 528 |
+
" sweater or hoodie\n",
|
| 529 |
+
"\n",
|
| 530 |
+
" swimwear\n",
|
| 531 |
+
"\n",
|
| 532 |
+
" t-shirts\n",
|
| 533 |
+
"\n",
|
| 534 |
+
" toiletry bag\n",
|
| 535 |
+
"\n",
|
| 536 |
+
" underwear per day\n",
|
| 537 |
+
"\n",
|
| 538 |
+
" water bottle\n",
|
| 539 |
+
"\n",
|
| 540 |
+
" water bottle(s) or hydration bladder\n"
|
| 541 |
]
|
| 542 |
}
|
| 543 |
],
|
| 544 |
"source": [
|
| 545 |
+
"# Load packing item data\n",
|
| 546 |
+
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 547 |
+
" packing_items = json.load(file)\n",
|
|
|
|
| 548 |
"\n",
|
|
|
|
|
|
|
|
|
|
| 549 |
"\n",
|
| 550 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 551 |
" \"\"\"\n",
|
|
|
|
| 572 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 573 |
" classes = result[\"labels\"][0]\n",
|
| 574 |
" df.loc[i] = [key, classes]\n",
|
| 575 |
+
" \n",
|
| 576 |
+
" ## Look up and return list of items to pack based on class predictions\n",
|
| 577 |
+
" # make list from dataframe column\n",
|
| 578 |
+
" all_classes = [elem for x in df[\"pred_class\"] for elem in (x if isinstance(x, list) else [x])]\n",
|
| 579 |
+
" # look up packing items for each class/key\n",
|
| 580 |
+
" list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]\n",
|
| 581 |
+
" # combine lists and remove doubble entries\n",
|
| 582 |
+
" flat_unique = []\n",
|
| 583 |
+
" for sublist in list_of_list_of_items:\n",
|
| 584 |
+
" for item in sublist:\n",
|
| 585 |
+
" if item not in flat_unique:\n",
|
| 586 |
+
" flat_unique.append(item)\n",
|
| 587 |
+
" # sort alphabetically\n",
|
| 588 |
+
" sorted_list = sorted(flat_unique) \n",
|
| 589 |
+
" return df, sorted_list\n",
|
| 590 |
"\n",
|
| 591 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 592 |
"\n",
|
| 593 |
+
"print(result[0])\n",
|
| 594 |
+
"for item in result[1]:\n",
|
| 595 |
+
" print(\"\\n\", item)"
|
| 596 |
]
|
| 597 |
},
|
| 598 |
{
|
| 599 |
+
"cell_type": "markdown",
|
| 600 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
|
|
|
| 601 |
"metadata": {},
|
|
|
|
| 602 |
"source": [
|
| 603 |
+
"Now use gradio app"
|
| 604 |
]
|
| 605 |
},
|
| 606 |
{
|
| 607 |
"cell_type": "code",
|
| 608 |
+
"execution_count": 17,
|
| 609 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 610 |
"metadata": {},
|
| 611 |
"outputs": [
|
| 612 |
{
|
| 613 |
"name": "stdout",
|
| 614 |
"output_type": "stream",
|
| 615 |
"text": [
|
| 616 |
+
"Running on local URL: http://127.0.0.1:7865\n",
|
| 617 |
+
"\n",
|
| 618 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 619 |
]
|
| 620 |
+
},
|
| 621 |
+
{
|
| 622 |
+
"data": {
|
| 623 |
+
"text/html": [
|
| 624 |
+
"<div><iframe src=\"http://127.0.0.1:7865/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 625 |
+
],
|
| 626 |
+
"text/plain": [
|
| 627 |
+
"<IPython.core.display.HTML object>"
|
| 628 |
+
]
|
| 629 |
+
},
|
| 630 |
+
"metadata": {},
|
| 631 |
+
"output_type": "display_data"
|
| 632 |
}
|
| 633 |
],
|
| 634 |
"source": [
|
| 635 |
+
"import gradio as gr\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 636 |
"\n",
|
| 637 |
+
"demo = gr.Interface(\n",
|
| 638 |
+
" fn=pred_trip,\n",
|
| 639 |
+
" inputs=[\n",
|
| 640 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 641 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 642 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 643 |
+
" ],\n",
|
| 644 |
+
" outputs=[gr.Dataframe(label=\"Trip classification\"), gr.Textbox(label=\"Items to pack\")],\n",
|
| 645 |
+
" title=\"Trip classification\",\n",
|
| 646 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 647 |
+
")\n",
|
| 648 |
+
"\n",
|
| 649 |
+
"# Launch the Gradio app\n",
|
| 650 |
+
"if __name__ == \"__main__\":\n",
|
| 651 |
+
" demo.launch()"
|
| 652 |
]
|
| 653 |
}
|
| 654 |
],
|
app.py
CHANGED
|
@@ -1,10 +1,6 @@
|
|
| 1 |
-
# Prerequisites
|
| 2 |
-
import os
|
| 3 |
-
os.environ["OMP_NUM_THREADS"] = "1" # Set 1, 2, or 4 depending on CPU usage
|
| 4 |
-
|
| 5 |
-
from transformers import pipeline
|
| 6 |
import json
|
| 7 |
import pandas as pd
|
|
|
|
| 8 |
import gradio as gr
|
| 9 |
|
| 10 |
# get candidate labels
|
|
@@ -16,12 +12,24 @@ keys_list = list(candidate_labels.keys())
|
|
| 16 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 17 |
packing_items = json.load(file)
|
| 18 |
|
| 19 |
-
|
| 20 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 22 |
-
## Create and fill dataframe with class predictions
|
| 23 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 24 |
for i, key in enumerate(keys_list):
|
|
|
|
| 25 |
if key == 'activities':
|
| 26 |
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 27 |
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
|
@@ -30,7 +38,7 @@ def classify(model_name, trip_descr, cut_off = 0.5):
|
|
| 30 |
result = classifier(trip_descr, candidate_labels[key])
|
| 31 |
classes = result["labels"][0]
|
| 32 |
df.loc[i] = [key, classes]
|
| 33 |
-
|
| 34 |
## Look up and return list of items to pack based on class predictions
|
| 35 |
# make list from dataframe column
|
| 36 |
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
|
@@ -42,19 +50,19 @@ def classify(model_name, trip_descr, cut_off = 0.5):
|
|
| 42 |
for item in sublist:
|
| 43 |
if item not in flat_unique:
|
| 44 |
flat_unique.append(item)
|
| 45 |
-
# sort alphabetically
|
| 46 |
-
sorted_list = sorted(flat_unique)
|
| 47 |
-
return df,
|
|
|
|
| 48 |
|
| 49 |
demo = gr.Interface(
|
| 50 |
-
fn=
|
| 51 |
inputs=[
|
| 52 |
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 53 |
gr.Textbox(label="Trip description"),
|
| 54 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 55 |
],
|
| 56 |
-
|
| 57 |
-
outputs=[gr.Dataframe(label="DataFrame"), gr.Textbox(label="List of words")],
|
| 58 |
title="Trip classification",
|
| 59 |
description="Enter a text describing your trip",
|
| 60 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import json
|
| 2 |
import pandas as pd
|
| 3 |
+
from transformers import pipeline
|
| 4 |
import gradio as gr
|
| 5 |
|
| 6 |
# get candidate labels
|
|
|
|
| 12 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 13 |
packing_items = json.load(file)
|
| 14 |
|
| 15 |
+
|
| 16 |
+
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 17 |
+
"""
|
| 18 |
+
Classifies trip
|
| 19 |
+
|
| 20 |
+
Parameters:
|
| 21 |
+
model_name: name of hugging-face model
|
| 22 |
+
trip_descr: text describing the trip
|
| 23 |
+
cut_off: cut_off for choosing activities
|
| 24 |
+
|
| 25 |
+
Returns:
|
| 26 |
+
pd Dataframe: with class predictions
|
| 27 |
+
"""
|
| 28 |
+
|
| 29 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
|
|
|
| 30 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 31 |
for i, key in enumerate(keys_list):
|
| 32 |
+
print(f"\rProcessing {i + 1}/{len(keys_list)}", end="", flush=True)
|
| 33 |
if key == 'activities':
|
| 34 |
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 35 |
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
|
|
|
| 38 |
result = classifier(trip_descr, candidate_labels[key])
|
| 39 |
classes = result["labels"][0]
|
| 40 |
df.loc[i] = [key, classes]
|
| 41 |
+
|
| 42 |
## Look up and return list of items to pack based on class predictions
|
| 43 |
# make list from dataframe column
|
| 44 |
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
|
|
|
| 50 |
for item in sublist:
|
| 51 |
if item not in flat_unique:
|
| 52 |
flat_unique.append(item)
|
| 53 |
+
# sort alphabetically and add newlines
|
| 54 |
+
sorted_list = "\n".join(sorted(flat_unique))
|
| 55 |
+
return df, sorted_list
|
| 56 |
+
|
| 57 |
|
| 58 |
demo = gr.Interface(
|
| 59 |
+
fn=pred_trip,
|
| 60 |
inputs=[
|
| 61 |
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 62 |
gr.Textbox(label="Trip description"),
|
| 63 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 64 |
],
|
| 65 |
+
outputs=[gr.Dataframe(label="Trip classification"), gr.Textbox(label="Items to pack")],
|
|
|
|
| 66 |
title="Trip classification",
|
| 67 |
description="Enter a text describing your trip",
|
| 68 |
)
|
blog_post/article.md
CHANGED
|
@@ -428,19 +428,18 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 428 |
for item in sublist:
|
| 429 |
if item not in flat_unique:
|
| 430 |
flat_unique.append(item)
|
| 431 |
-
# sort alphabetically
|
| 432 |
-
sorted_list = sorted(flat_unique)
|
| 433 |
return df, sorted_list
|
| 434 |
|
| 435 |
result = pred_trip(model_name, trip_descr, cut_off = 0.5)
|
| 436 |
|
| 437 |
print(result[0])
|
| 438 |
-
|
| 439 |
-
print("\n", item)
|
| 440 |
```
|
| 441 |
|
| 442 |
```text
|
| 443 |
-
|
| 444 |
0 activity_type beach vacation
|
| 445 |
1 activities [going to the beach, relaxing, hiking]
|
| 446 |
2 climate_or_season warm destination / summer
|
|
@@ -450,142 +449,74 @@ for item in result[1]:
|
|
| 450 |
6 transportation no own vehicle
|
| 451 |
7 special_conditions off-grid / no electricity
|
| 452 |
8 trip_length_days 7+ days
|
| 453 |
-
|
| 454 |
-
|
| 455 |
-
|
| 456 |
-
|
| 457 |
-
|
| 458 |
-
|
| 459 |
-
|
| 460 |
-
|
| 461 |
-
|
| 462 |
-
|
| 463 |
-
|
| 464 |
-
|
| 465 |
-
|
| 466 |
-
|
| 467 |
-
|
| 468 |
-
|
| 469 |
-
|
| 470 |
-
|
| 471 |
-
|
| 472 |
-
|
| 473 |
-
|
| 474 |
-
|
| 475 |
-
|
| 476 |
-
|
| 477 |
-
|
| 478 |
-
|
| 479 |
-
|
| 480 |
-
|
| 481 |
-
|
| 482 |
-
|
| 483 |
-
|
| 484 |
-
|
| 485 |
-
|
| 486 |
-
|
| 487 |
-
|
| 488 |
-
|
| 489 |
-
|
| 490 |
-
|
| 491 |
-
|
| 492 |
-
|
| 493 |
-
|
| 494 |
-
|
| 495 |
-
|
| 496 |
-
|
| 497 |
-
|
| 498 |
-
|
| 499 |
-
|
| 500 |
-
|
| 501 |
-
|
| 502 |
-
|
| 503 |
-
|
| 504 |
-
|
| 505 |
-
|
| 506 |
-
|
| 507 |
-
|
| 508 |
-
|
| 509 |
-
|
| 510 |
-
|
| 511 |
-
|
| 512 |
-
|
| 513 |
-
|
| 514 |
-
|
| 515 |
-
|
| 516 |
-
|
| 517 |
-
|
| 518 |
-
|
| 519 |
-
|
| 520 |
-
|
| 521 |
-
|
| 522 |
-
lightweight towel
|
| 523 |
-
|
| 524 |
-
music / headphones
|
| 525 |
-
|
| 526 |
-
navigation (map/compass/GPS)
|
| 527 |
-
|
| 528 |
-
navigation device with offline maps
|
| 529 |
-
|
| 530 |
-
notebook + pen
|
| 531 |
-
|
| 532 |
-
number of meals/snacks matched to duration
|
| 533 |
-
|
| 534 |
-
packaging to keep electronics dry
|
| 535 |
-
|
| 536 |
-
paper map and compass
|
| 537 |
-
|
| 538 |
-
power bank (at least 10,000 mAh)
|
| 539 |
-
|
| 540 |
-
public transport app or ticket
|
| 541 |
-
|
| 542 |
-
rain jacket or poncho
|
| 543 |
-
|
| 544 |
-
rechargeable batteries and charger
|
| 545 |
-
|
| 546 |
-
reservation confirmation
|
| 547 |
-
|
| 548 |
-
seat cushion or beach mat
|
| 549 |
-
|
| 550 |
-
sheet liner (often required)
|
| 551 |
-
|
| 552 |
-
slippers or indoor shoes for inside
|
| 553 |
-
|
| 554 |
-
small backpack
|
| 555 |
-
|
| 556 |
-
small toiletry bag
|
| 557 |
-
|
| 558 |
-
snacks / energy bars
|
| 559 |
-
|
| 560 |
-
snacks for along the way
|
| 561 |
-
|
| 562 |
-
sneakers
|
| 563 |
-
|
| 564 |
-
socks per day
|
| 565 |
-
|
| 566 |
-
solar panel or portable charging system
|
| 567 |
-
|
| 568 |
-
sun hat
|
| 569 |
-
|
| 570 |
-
sunglasses
|
| 571 |
-
|
| 572 |
-
sunscreen
|
| 573 |
-
|
| 574 |
-
sunscreen and sunglasses
|
| 575 |
-
|
| 576 |
-
sweater or hoodie
|
| 577 |
-
|
| 578 |
-
swimwear
|
| 579 |
-
|
| 580 |
-
t-shirts
|
| 581 |
-
|
| 582 |
-
toiletry bag
|
| 583 |
-
|
| 584 |
-
underwear per day
|
| 585 |
-
|
| 586 |
-
water bottle
|
| 587 |
-
|
| 588 |
-
water bottle(s) or hydration bladder
|
| 589 |
```
|
| 590 |
We are ready! But let's make it a bit more user friendly in the next step!
|
| 591 |
|
|
@@ -635,26 +566,32 @@ git clone https://huggingface.co/spaces/<username>/<space_name>
|
|
| 635 |
```
|
| 636 |
As prompted, go to https://huggingface.co/settings/tokens to generate an access token. Click on *+ Create new token*, set the token type to *Write*. Give your token a name and click on *Create Token*. You'll use this token as a password to push to your remote repository.
|
| 637 |
|
| 638 |
-
Next, open the command line, navigate to your project folder
|
| 639 |
|
| 640 |
```bash
|
| 641 |
-
cd path/to/
|
| 642 |
-
git
|
| 643 |
-
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 644 |
```
|
|
|
|
| 645 |
|
| 646 |
The Space automatically runs whatever model code you put in a file called app.py. In your project folder, create that file (e.g. on mac in command line: touch app.py) and open it. Copy all relevant code for your Gradio app into this file and save it.
|
| 647 |
|
| 648 |
```python
|
| 649 |
-
from transformers import pipeline
|
| 650 |
import json
|
| 651 |
import pandas as pd
|
|
|
|
| 652 |
import gradio as gr
|
| 653 |
|
|
|
|
| 654 |
with open("packing_label_structure.json", "r") as file:
|
| 655 |
candidate_labels = json.load(file)
|
| 656 |
keys_list = list(candidate_labels.keys())
|
| 657 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 658 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 659 |
"""
|
| 660 |
Classifies trip
|
|
@@ -671,7 +608,7 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 671 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 672 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 673 |
for i, key in enumerate(keys_list):
|
| 674 |
-
|
| 675 |
if key == 'activities':
|
| 676 |
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 677 |
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
|
@@ -680,7 +617,22 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 680 |
result = classifier(trip_descr, candidate_labels[key])
|
| 681 |
classes = result["labels"][0]
|
| 682 |
df.loc[i] = [key, classes]
|
| 683 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 684 |
|
| 685 |
demo = gr.Interface(
|
| 686 |
fn=pred_trip,
|
|
@@ -712,12 +664,13 @@ pandas
|
|
| 712 |
gradio
|
| 713 |
```
|
| 714 |
|
| 715 |
-
After that, add, commit, and push your changes to the remote repository.
|
| 716 |
|
| 717 |
```bash
|
|
|
|
| 718 |
git add .
|
| 719 |
-
git commit -m "
|
| 720 |
-
git push origin
|
| 721 |
```
|
| 722 |
|
| 723 |
Once the push is complete, go to the URL of your Space and try it out!
|
|
|
|
| 428 |
for item in sublist:
|
| 429 |
if item not in flat_unique:
|
| 430 |
flat_unique.append(item)
|
| 431 |
+
# sort alphabetically and add newlines
|
| 432 |
+
sorted_list = "\n".join(sorted(flat_unique))
|
| 433 |
return df, sorted_list
|
| 434 |
|
| 435 |
result = pred_trip(model_name, trip_descr, cut_off = 0.5)
|
| 436 |
|
| 437 |
print(result[0])
|
| 438 |
+
print(result[1])
|
|
|
|
| 439 |
```
|
| 440 |
|
| 441 |
```text
|
| 442 |
+
superclass pred_class
|
| 443 |
0 activity_type beach vacation
|
| 444 |
1 activities [going to the beach, relaxing, hiking]
|
| 445 |
2 climate_or_season warm destination / summer
|
|
|
|
| 449 |
6 transportation no own vehicle
|
| 450 |
7 special_conditions off-grid / no electricity
|
| 451 |
8 trip_length_days 7+ days
|
| 452 |
+
1 set of clothing for every situation
|
| 453 |
+
USB hub (for multiple devices)
|
| 454 |
+
all‑in‑one soap
|
| 455 |
+
backpack
|
| 456 |
+
backup lighting (e.g. small flashlight)
|
| 457 |
+
beach bag
|
| 458 |
+
beach chair
|
| 459 |
+
beach towel
|
| 460 |
+
blister plasters or tape
|
| 461 |
+
book or e‑reader
|
| 462 |
+
cap or hat
|
| 463 |
+
cash for payments
|
| 464 |
+
comfortable clothing
|
| 465 |
+
compact toothbrush
|
| 466 |
+
cooler
|
| 467 |
+
daypack
|
| 468 |
+
earplugs
|
| 469 |
+
emergency communication (e.g. GPS beacon or satellite messenger)
|
| 470 |
+
extra charger cables
|
| 471 |
+
extra clothing layer
|
| 472 |
+
first aid kit
|
| 473 |
+
flashlight or headlamp
|
| 474 |
+
flip flops
|
| 475 |
+
foldable solar panel (if on longer trips)
|
| 476 |
+
hat or cap
|
| 477 |
+
headlamp + extra batteries
|
| 478 |
+
hiking boots or trail runners
|
| 479 |
+
hiking poles
|
| 480 |
+
hiking socks (anti-blister)
|
| 481 |
+
jeans or comfortable pants
|
| 482 |
+
light pajamas or sleepwear
|
| 483 |
+
light shoes
|
| 484 |
+
light towel
|
| 485 |
+
lightweight clothing
|
| 486 |
+
lightweight towel
|
| 487 |
+
music / headphones
|
| 488 |
+
navigation (map/compass/GPS)
|
| 489 |
+
navigation device with offline maps
|
| 490 |
+
notebook + pen
|
| 491 |
+
number of meals/snacks matched to duration
|
| 492 |
+
packaging to keep electronics dry
|
| 493 |
+
paper map and compass
|
| 494 |
+
power bank (at least 10,000 mAh)
|
| 495 |
+
public transport app or ticket
|
| 496 |
+
rain jacket or poncho
|
| 497 |
+
rechargeable batteries and charger
|
| 498 |
+
reservation confirmation
|
| 499 |
+
seat cushion or beach mat
|
| 500 |
+
sheet liner (often required)
|
| 501 |
+
slippers or indoor shoes for inside
|
| 502 |
+
small backpack
|
| 503 |
+
small toiletry bag
|
| 504 |
+
snacks / energy bars
|
| 505 |
+
snacks for along the way
|
| 506 |
+
sneakers
|
| 507 |
+
socks per day
|
| 508 |
+
solar panel or portable charging system
|
| 509 |
+
sun hat
|
| 510 |
+
sunglasses
|
| 511 |
+
sunscreen
|
| 512 |
+
sunscreen and sunglasses
|
| 513 |
+
sweater or hoodie
|
| 514 |
+
swimwear
|
| 515 |
+
t-shirts
|
| 516 |
+
toiletry bag
|
| 517 |
+
underwear per day
|
| 518 |
+
water bottle
|
| 519 |
+
water bottle(s) or hydration bladder
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 520 |
```
|
| 521 |
We are ready! But let's make it a bit more user friendly in the next step!
|
| 522 |
|
|
|
|
| 566 |
```
|
| 567 |
As prompted, go to https://huggingface.co/settings/tokens to generate an access token. Click on *+ Create new token*, set the token type to *Write*. Give your token a name and click on *Create Token*. You'll use this token as a password to push to your remote repository.
|
| 568 |
|
| 569 |
+
Next, open the command line, navigate to where you want your project folder and clone the remote repository:
|
| 570 |
|
| 571 |
```bash
|
| 572 |
+
cd path/to/where/you/want/the-folder
|
| 573 |
+
git clone https://huggingface.co/spaces/<username>/<space-name>
|
|
|
|
| 574 |
```
|
| 575 |
+
Put your project files into the newly created folder named <space-name>.
|
| 576 |
|
| 577 |
The Space automatically runs whatever model code you put in a file called app.py. In your project folder, create that file (e.g. on mac in command line: touch app.py) and open it. Copy all relevant code for your Gradio app into this file and save it.
|
| 578 |
|
| 579 |
```python
|
|
|
|
| 580 |
import json
|
| 581 |
import pandas as pd
|
| 582 |
+
from transformers import pipeline
|
| 583 |
import gradio as gr
|
| 584 |
|
| 585 |
+
# get candidate labels
|
| 586 |
with open("packing_label_structure.json", "r") as file:
|
| 587 |
candidate_labels = json.load(file)
|
| 588 |
keys_list = list(candidate_labels.keys())
|
| 589 |
|
| 590 |
+
# Load packing item data
|
| 591 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 592 |
+
packing_items = json.load(file)
|
| 593 |
+
|
| 594 |
+
|
| 595 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 596 |
"""
|
| 597 |
Classifies trip
|
|
|
|
| 608 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 609 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 610 |
for i, key in enumerate(keys_list):
|
| 611 |
+
print(f"\rProcessing {i + 1}/{len(keys_list)}", end="", flush=True)
|
| 612 |
if key == 'activities':
|
| 613 |
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 614 |
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
|
|
|
| 617 |
result = classifier(trip_descr, candidate_labels[key])
|
| 618 |
classes = result["labels"][0]
|
| 619 |
df.loc[i] = [key, classes]
|
| 620 |
+
|
| 621 |
+
## Look up and return list of items to pack based on class predictions
|
| 622 |
+
# make list from dataframe column
|
| 623 |
+
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
| 624 |
+
# look up packing items for each class/key
|
| 625 |
+
list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]
|
| 626 |
+
# combine lists and remove doubble entries
|
| 627 |
+
flat_unique = []
|
| 628 |
+
for sublist in list_of_list_of_items:
|
| 629 |
+
for item in sublist:
|
| 630 |
+
if item not in flat_unique:
|
| 631 |
+
flat_unique.append(item)
|
| 632 |
+
# sort alphabetically and add newlines
|
| 633 |
+
sorted_list = "\n".join(sorted(flat_unique))
|
| 634 |
+
return df, sorted_list
|
| 635 |
+
|
| 636 |
|
| 637 |
demo = gr.Interface(
|
| 638 |
fn=pred_trip,
|
|
|
|
| 664 |
gradio
|
| 665 |
```
|
| 666 |
|
| 667 |
+
After that, navigate to your project folder, add, commit, and push your changes to the remote repository.
|
| 668 |
|
| 669 |
```bash
|
| 670 |
+
cd <space-name>
|
| 671 |
git add .
|
| 672 |
+
git commit -m "initial commit"
|
| 673 |
+
git push origin master
|
| 674 |
```
|
| 675 |
|
| 676 |
Once the push is complete, go to the URL of your Space and try it out!
|
code_blog_post.ipynb
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
@@ -37,7 +37,7 @@
|
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
-
"execution_count":
|
| 41 |
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
"metadata": {},
|
| 43 |
"outputs": [
|
|
@@ -82,7 +82,7 @@
|
|
| 82 |
},
|
| 83 |
{
|
| 84 |
"cell_type": "code",
|
| 85 |
-
"execution_count":
|
| 86 |
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
|
@@ -205,7 +205,7 @@
|
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
-
"execution_count":
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
@@ -255,7 +255,7 @@
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
-
"execution_count":
|
| 259 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 260 |
"metadata": {
|
| 261 |
"scrolled": true
|
|
@@ -312,7 +312,7 @@
|
|
| 312 |
},
|
| 313 |
{
|
| 314 |
"cell_type": "code",
|
| 315 |
-
"execution_count":
|
| 316 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 317 |
"metadata": {
|
| 318 |
"scrolled": true
|
|
@@ -377,7 +377,7 @@
|
|
| 377 |
},
|
| 378 |
{
|
| 379 |
"cell_type": "code",
|
| 380 |
-
"execution_count":
|
| 381 |
"id": "f8b547d4-1858-49d5-b26c-fd30e210cd58",
|
| 382 |
"metadata": {},
|
| 383 |
"outputs": [
|
|
@@ -392,152 +392,7 @@
|
|
| 392 |
"name": "stdout",
|
| 393 |
"output_type": "stream",
|
| 394 |
"text": [
|
| 395 |
-
"Processing 9/9
|
| 396 |
-
"0 activity_type beach vacation\n",
|
| 397 |
-
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 398 |
-
"2 climate_or_season warm destination / summer\n",
|
| 399 |
-
"3 style_or_comfort minimalist\n",
|
| 400 |
-
"4 dress_code casual\n",
|
| 401 |
-
"5 accommodation huts with half board\n",
|
| 402 |
-
"6 transportation no own vehicle\n",
|
| 403 |
-
"7 special_conditions off-grid / no electricity\n",
|
| 404 |
-
"8 trip_length_days 7+ days\n",
|
| 405 |
-
"\n",
|
| 406 |
-
" 1 set of clothing for every situation\n",
|
| 407 |
-
"\n",
|
| 408 |
-
" USB hub (for multiple devices)\n",
|
| 409 |
-
"\n",
|
| 410 |
-
" all‑in‑one soap\n",
|
| 411 |
-
"\n",
|
| 412 |
-
" backpack\n",
|
| 413 |
-
"\n",
|
| 414 |
-
" backup lighting (e.g. small flashlight)\n",
|
| 415 |
-
"\n",
|
| 416 |
-
" beach bag\n",
|
| 417 |
-
"\n",
|
| 418 |
-
" beach chair\n",
|
| 419 |
-
"\n",
|
| 420 |
-
" beach towel\n",
|
| 421 |
-
"\n",
|
| 422 |
-
" blister plasters or tape\n",
|
| 423 |
-
"\n",
|
| 424 |
-
" book or e‑reader\n",
|
| 425 |
-
"\n",
|
| 426 |
-
" cap or hat\n",
|
| 427 |
-
"\n",
|
| 428 |
-
" cash for payments\n",
|
| 429 |
-
"\n",
|
| 430 |
-
" comfortable clothing\n",
|
| 431 |
-
"\n",
|
| 432 |
-
" compact toothbrush\n",
|
| 433 |
-
"\n",
|
| 434 |
-
" cooler\n",
|
| 435 |
-
"\n",
|
| 436 |
-
" daypack\n",
|
| 437 |
-
"\n",
|
| 438 |
-
" earplugs\n",
|
| 439 |
-
"\n",
|
| 440 |
-
" emergency communication (e.g. GPS beacon or satellite messenger)\n",
|
| 441 |
-
"\n",
|
| 442 |
-
" extra charger cables\n",
|
| 443 |
-
"\n",
|
| 444 |
-
" extra clothing layer\n",
|
| 445 |
-
"\n",
|
| 446 |
-
" first aid kit\n",
|
| 447 |
-
"\n",
|
| 448 |
-
" flashlight or headlamp\n",
|
| 449 |
-
"\n",
|
| 450 |
-
" flip flops\n",
|
| 451 |
-
"\n",
|
| 452 |
-
" foldable solar panel (if on longer trips)\n",
|
| 453 |
-
"\n",
|
| 454 |
-
" hat or cap\n",
|
| 455 |
-
"\n",
|
| 456 |
-
" headlamp + extra batteries\n",
|
| 457 |
-
"\n",
|
| 458 |
-
" hiking boots or trail runners\n",
|
| 459 |
-
"\n",
|
| 460 |
-
" hiking poles\n",
|
| 461 |
-
"\n",
|
| 462 |
-
" hiking socks (anti-blister)\n",
|
| 463 |
-
"\n",
|
| 464 |
-
" jeans or comfortable pants\n",
|
| 465 |
-
"\n",
|
| 466 |
-
" light pajamas or sleepwear\n",
|
| 467 |
-
"\n",
|
| 468 |
-
" light shoes\n",
|
| 469 |
-
"\n",
|
| 470 |
-
" light towel\n",
|
| 471 |
-
"\n",
|
| 472 |
-
" lightweight clothing\n",
|
| 473 |
-
"\n",
|
| 474 |
-
" lightweight towel\n",
|
| 475 |
-
"\n",
|
| 476 |
-
" music / headphones\n",
|
| 477 |
-
"\n",
|
| 478 |
-
" navigation (map/compass/GPS)\n",
|
| 479 |
-
"\n",
|
| 480 |
-
" navigation device with offline maps\n",
|
| 481 |
-
"\n",
|
| 482 |
-
" notebook + pen\n",
|
| 483 |
-
"\n",
|
| 484 |
-
" number of meals/snacks matched to duration\n",
|
| 485 |
-
"\n",
|
| 486 |
-
" packaging to keep electronics dry\n",
|
| 487 |
-
"\n",
|
| 488 |
-
" paper map and compass\n",
|
| 489 |
-
"\n",
|
| 490 |
-
" power bank (at least 10,000 mAh)\n",
|
| 491 |
-
"\n",
|
| 492 |
-
" public transport app or ticket\n",
|
| 493 |
-
"\n",
|
| 494 |
-
" rain jacket or poncho\n",
|
| 495 |
-
"\n",
|
| 496 |
-
" rechargeable batteries and charger\n",
|
| 497 |
-
"\n",
|
| 498 |
-
" reservation confirmation\n",
|
| 499 |
-
"\n",
|
| 500 |
-
" seat cushion or beach mat\n",
|
| 501 |
-
"\n",
|
| 502 |
-
" sheet liner (often required)\n",
|
| 503 |
-
"\n",
|
| 504 |
-
" slippers or indoor shoes for inside\n",
|
| 505 |
-
"\n",
|
| 506 |
-
" small backpack\n",
|
| 507 |
-
"\n",
|
| 508 |
-
" small toiletry bag\n",
|
| 509 |
-
"\n",
|
| 510 |
-
" snacks / energy bars\n",
|
| 511 |
-
"\n",
|
| 512 |
-
" snacks for along the way\n",
|
| 513 |
-
"\n",
|
| 514 |
-
" sneakers\n",
|
| 515 |
-
"\n",
|
| 516 |
-
" socks per day\n",
|
| 517 |
-
"\n",
|
| 518 |
-
" solar panel or portable charging system\n",
|
| 519 |
-
"\n",
|
| 520 |
-
" sun hat\n",
|
| 521 |
-
"\n",
|
| 522 |
-
" sunglasses\n",
|
| 523 |
-
"\n",
|
| 524 |
-
" sunscreen\n",
|
| 525 |
-
"\n",
|
| 526 |
-
" sunscreen and sunglasses\n",
|
| 527 |
-
"\n",
|
| 528 |
-
" sweater or hoodie\n",
|
| 529 |
-
"\n",
|
| 530 |
-
" swimwear\n",
|
| 531 |
-
"\n",
|
| 532 |
-
" t-shirts\n",
|
| 533 |
-
"\n",
|
| 534 |
-
" toiletry bag\n",
|
| 535 |
-
"\n",
|
| 536 |
-
" underwear per day\n",
|
| 537 |
-
"\n",
|
| 538 |
-
" water bottle\n",
|
| 539 |
-
"\n",
|
| 540 |
-
" water bottle(s) or hydration bladder\n"
|
| 541 |
]
|
| 542 |
}
|
| 543 |
],
|
|
@@ -584,94 +439,128 @@
|
|
| 584 |
" for item in sublist:\n",
|
| 585 |
" if item not in flat_unique:\n",
|
| 586 |
" flat_unique.append(item)\n",
|
| 587 |
-
" # sort alphabetically\n",
|
| 588 |
-
" sorted_list = sorted(flat_unique) \n",
|
| 589 |
" return df, sorted_list\n",
|
| 590 |
"\n",
|
| 591 |
-
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)
|
| 592 |
-
"\n",
|
| 593 |
-
"print(result[0])\n",
|
| 594 |
-
"for item in result[1]:\n",
|
| 595 |
-
" print(\"\\n\", item)"
|
| 596 |
-
]
|
| 597 |
-
},
|
| 598 |
-
{
|
| 599 |
-
"cell_type": "markdown",
|
| 600 |
-
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 601 |
-
"metadata": {},
|
| 602 |
-
"source": [
|
| 603 |
-
"Now use gradio app"
|
| 604 |
]
|
| 605 |
},
|
| 606 |
{
|
| 607 |
"cell_type": "code",
|
| 608 |
-
"execution_count":
|
| 609 |
-
"id": "
|
| 610 |
"metadata": {},
|
| 611 |
"outputs": [
|
| 612 |
{
|
| 613 |
"name": "stdout",
|
| 614 |
"output_type": "stream",
|
| 615 |
"text": [
|
| 616 |
-
"
|
| 617 |
-
"\n",
|
| 618 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 619 |
]
|
| 620 |
-
},
|
| 621 |
-
{
|
| 622 |
-
"data": {
|
| 623 |
-
"text/html": [
|
| 624 |
-
"<div><iframe src=\"http://127.0.0.1:7865/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 625 |
-
],
|
| 626 |
-
"text/plain": [
|
| 627 |
-
"<IPython.core.display.HTML object>"
|
| 628 |
-
]
|
| 629 |
-
},
|
| 630 |
-
"metadata": {},
|
| 631 |
-
"output_type": "display_data"
|
| 632 |
}
|
| 633 |
],
|
| 634 |
"source": [
|
| 635 |
-
"
|
| 636 |
-
"
|
| 637 |
-
"demo = gr.Interface(\n",
|
| 638 |
-
" fn=pred_trip,\n",
|
| 639 |
-
" inputs=[\n",
|
| 640 |
-
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 641 |
-
" gr.Textbox(label=\"Trip description\"),\n",
|
| 642 |
-
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 643 |
-
" ],\n",
|
| 644 |
-
" outputs=[gr.Dataframe(label=\"Trip classification\"), gr.Textbox(label=\"Items to pack\")],\n",
|
| 645 |
-
" title=\"Trip classification\",\n",
|
| 646 |
-
" description=\"Enter a text describing your trip\",\n",
|
| 647 |
-
")\n",
|
| 648 |
-
"\n",
|
| 649 |
-
"# Launch the Gradio app\n",
|
| 650 |
-
"if __name__ == \"__main__\":\n",
|
| 651 |
-
" demo.launch()"
|
| 652 |
]
|
| 653 |
},
|
| 654 |
{
|
| 655 |
"cell_type": "markdown",
|
| 656 |
-
"id": "
|
| 657 |
"metadata": {},
|
| 658 |
"source": [
|
| 659 |
-
"
|
| 660 |
]
|
| 661 |
},
|
| 662 |
{
|
| 663 |
"cell_type": "code",
|
| 664 |
-
"execution_count":
|
| 665 |
-
"id": "
|
| 666 |
-
"metadata": {
|
| 667 |
-
"scrolled": true
|
| 668 |
-
},
|
| 669 |
"outputs": [
|
| 670 |
{
|
| 671 |
"name": "stdout",
|
| 672 |
"output_type": "stream",
|
| 673 |
"text": [
|
| 674 |
-
"Running on local URL: http://127.0.0.1:
|
| 675 |
"\n",
|
| 676 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 677 |
]
|
|
@@ -679,7 +568,7 @@
|
|
| 679 |
{
|
| 680 |
"data": {
|
| 681 |
"text/html": [
|
| 682 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 683 |
],
|
| 684 |
"text/plain": [
|
| 685 |
"<IPython.core.display.HTML object>"
|
|
@@ -690,42 +579,8 @@
|
|
| 690 |
}
|
| 691 |
],
|
| 692 |
"source": [
|
| 693 |
-
"from transformers import pipeline\n",
|
| 694 |
-
"import json\n",
|
| 695 |
-
"import pandas as pd\n",
|
| 696 |
"import gradio as gr\n",
|
| 697 |
"\n",
|
| 698 |
-
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 699 |
-
" candidate_labels = json.load(file)\n",
|
| 700 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 701 |
-
"\n",
|
| 702 |
-
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 703 |
-
" \"\"\"\n",
|
| 704 |
-
" Classifies trip\n",
|
| 705 |
-
" \n",
|
| 706 |
-
" Parameters:\n",
|
| 707 |
-
" model_name: name of hugging-face model\n",
|
| 708 |
-
" trip_descr: text describing the trip\n",
|
| 709 |
-
" cut_off: cut_off for choosing activities\n",
|
| 710 |
-
"\n",
|
| 711 |
-
" Returns:\n",
|
| 712 |
-
" pd Dataframe: with class predictions\n",
|
| 713 |
-
" \"\"\"\n",
|
| 714 |
-
" \n",
|
| 715 |
-
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 716 |
-
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 717 |
-
" for i, key in enumerate(keys_list):\n",
|
| 718 |
-
" # print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 719 |
-
" if key == 'activities':\n",
|
| 720 |
-
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 721 |
-
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 722 |
-
" classes = [result['labels'][i] for i in indices]\n",
|
| 723 |
-
" else:\n",
|
| 724 |
-
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 725 |
-
" classes = result[\"labels\"][0]\n",
|
| 726 |
-
" df.loc[i] = [key, classes]\n",
|
| 727 |
-
" return df\n",
|
| 728 |
-
"\n",
|
| 729 |
"demo = gr.Interface(\n",
|
| 730 |
" fn=pred_trip,\n",
|
| 731 |
" inputs=[\n",
|
|
@@ -740,187 +595,16 @@
|
|
| 740 |
"\n",
|
| 741 |
"# Launch the Gradio app\n",
|
| 742 |
"if __name__ == \"__main__\":\n",
|
| 743 |
-
" demo.launch()
|
| 744 |
-
" "
|
| 745 |
-
]
|
| 746 |
-
},
|
| 747 |
-
{
|
| 748 |
-
"cell_type": "code",
|
| 749 |
-
"execution_count": null,
|
| 750 |
-
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 751 |
-
"metadata": {},
|
| 752 |
-
"outputs": [],
|
| 753 |
-
"source": [
|
| 754 |
-
"Print test data set"
|
| 755 |
-
]
|
| 756 |
-
},
|
| 757 |
-
{
|
| 758 |
-
"cell_type": "code",
|
| 759 |
-
"execution_count": 4,
|
| 760 |
-
"id": "f25e1462-ca48-4853-ae44-dfc096c5011d",
|
| 761 |
-
"metadata": {},
|
| 762 |
-
"outputs": [
|
| 763 |
-
{
|
| 764 |
-
"name": "stdout",
|
| 765 |
-
"output_type": "stream",
|
| 766 |
-
"text": [
|
| 767 |
-
"0 . I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 768 |
-
"\n",
|
| 769 |
-
"beach vacation\n",
|
| 770 |
-
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 771 |
-
"warm destination / summer\n",
|
| 772 |
-
"lightweight (but comfortable)\n",
|
| 773 |
-
"casual\n",
|
| 774 |
-
"indoor\n",
|
| 775 |
-
"no own vehicle\n",
|
| 776 |
-
"no special conditions to consider\n",
|
| 777 |
-
"7+ days\n",
|
| 778 |
-
"\n",
|
| 779 |
-
"\n",
|
| 780 |
-
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 781 |
-
"\n",
|
| 782 |
-
"city trip\n",
|
| 783 |
-
"['sightseeing']\n",
|
| 784 |
-
"variable weather / spring / autumn\n",
|
| 785 |
-
"luxury (including evening wear)\n",
|
| 786 |
-
"casual\n",
|
| 787 |
-
"indoor\n",
|
| 788 |
-
"no own vehicle\n",
|
| 789 |
-
"no special conditions to consider\n",
|
| 790 |
-
"3 days\n",
|
| 791 |
-
"\n",
|
| 792 |
-
"\n",
|
| 793 |
-
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 794 |
-
"\n",
|
| 795 |
-
"city trip\n",
|
| 796 |
-
"['relaxing']\n",
|
| 797 |
-
"cold destination / winter\n",
|
| 798 |
-
"lightweight (but comfortable)\n",
|
| 799 |
-
"casual\n",
|
| 800 |
-
"indoor\n",
|
| 801 |
-
"no own vehicle\n",
|
| 802 |
-
"no special conditions to consider\n",
|
| 803 |
-
"7+ days\n",
|
| 804 |
-
"\n",
|
| 805 |
-
"\n",
|
| 806 |
-
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 807 |
-
"\n",
|
| 808 |
-
"cultural exploration\n",
|
| 809 |
-
"['sightseeing', 'hiking', 'rafting']\n",
|
| 810 |
-
"variable weather / spring / autumn\n",
|
| 811 |
-
"lightweight (but comfortable)\n",
|
| 812 |
-
"casual\n",
|
| 813 |
-
"indoor\n",
|
| 814 |
-
"no own vehicle\n",
|
| 815 |
-
"rainy climate\n",
|
| 816 |
-
"7+ days\n",
|
| 817 |
-
"\n",
|
| 818 |
-
"\n",
|
| 819 |
-
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 820 |
-
"\n",
|
| 821 |
-
"nature escape\n",
|
| 822 |
-
"['swimming', 'relaxing', 'hiking']\n",
|
| 823 |
-
"warm destination / summer\n",
|
| 824 |
-
"lightweight (but comfortable)\n",
|
| 825 |
-
"casual\n",
|
| 826 |
-
"indoor\n",
|
| 827 |
-
"no own vehicle\n",
|
| 828 |
-
"no special conditions to consider\n",
|
| 829 |
-
"7+ days\n",
|
| 830 |
-
"\n",
|
| 831 |
-
"\n",
|
| 832 |
-
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 833 |
-
"\n",
|
| 834 |
-
"long-distance hike / thru-hike\n",
|
| 835 |
-
"['going to the beach']\n",
|
| 836 |
-
"tropical / humid\n",
|
| 837 |
-
"minimalist\n",
|
| 838 |
-
"casual\n",
|
| 839 |
-
"huts with half board\n",
|
| 840 |
-
"own vehicle\n",
|
| 841 |
-
"off-grid / no electricity\n",
|
| 842 |
-
"6 days\n",
|
| 843 |
-
"\n",
|
| 844 |
-
"\n",
|
| 845 |
-
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 846 |
-
"\n",
|
| 847 |
-
"beach vacation\n",
|
| 848 |
-
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 849 |
-
"cold destination / winter\n",
|
| 850 |
-
"ultralight\n",
|
| 851 |
-
"casual\n",
|
| 852 |
-
"sleeping in a tent\n",
|
| 853 |
-
"own vehicle\n",
|
| 854 |
-
"off-grid / no electricity\n",
|
| 855 |
-
"6 days\n",
|
| 856 |
-
"\n",
|
| 857 |
-
"\n",
|
| 858 |
-
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 859 |
-
"\n",
|
| 860 |
-
"yoga / wellness retreat\n",
|
| 861 |
-
"['hiking', 'yoga']\n",
|
| 862 |
-
"cold destination / winter\n",
|
| 863 |
-
"lightweight (but comfortable)\n",
|
| 864 |
-
"casual\n",
|
| 865 |
-
"indoor\n",
|
| 866 |
-
"no own vehicle\n",
|
| 867 |
-
"snow and ice\n",
|
| 868 |
-
"7 days\n",
|
| 869 |
-
"\n",
|
| 870 |
-
"\n",
|
| 871 |
-
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 872 |
-
"\n",
|
| 873 |
-
"ski tour / skitour\n",
|
| 874 |
-
"['ski touring', 'photography']\n",
|
| 875 |
-
"cold destination / winter\n",
|
| 876 |
-
"minimalist\n",
|
| 877 |
-
"conservative\n",
|
| 878 |
-
"indoor\n",
|
| 879 |
-
"no own vehicle\n",
|
| 880 |
-
"avalanche-prone terrain\n",
|
| 881 |
-
"7+ days\n",
|
| 882 |
-
"\n",
|
| 883 |
-
"\n",
|
| 884 |
-
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 885 |
-
"\n",
|
| 886 |
-
"camping trip (wild camping)\n",
|
| 887 |
-
"['scuba diving', 'kayaking / canoeing']\n",
|
| 888 |
-
"tropical / humid\n",
|
| 889 |
-
"lightweight (but comfortable)\n",
|
| 890 |
-
"casual\n",
|
| 891 |
-
"sleeping in a tent\n",
|
| 892 |
-
"own vehicle\n",
|
| 893 |
-
"self-supported (bring your own cooking gear)\n",
|
| 894 |
-
"3 days\n",
|
| 895 |
-
"\n",
|
| 896 |
-
"\n"
|
| 897 |
-
]
|
| 898 |
-
}
|
| 899 |
-
],
|
| 900 |
-
"source": [
|
| 901 |
-
"# Load test data (list of dictionaries)\n",
|
| 902 |
-
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 903 |
-
" packing_data = json.load(file)\n",
|
| 904 |
-
" # Extract trip descriptions and classification (trip_types)\n",
|
| 905 |
-
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 906 |
-
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 907 |
-
"\n",
|
| 908 |
-
"for i, item in enumerate(trip_descriptions):\n",
|
| 909 |
-
" print(i, \".\", item, \"\\n\")\n",
|
| 910 |
-
" for elem in trip_types[i]:\n",
|
| 911 |
-
" print(elem)\n",
|
| 912 |
-
" print(\"\\n\")"
|
| 913 |
]
|
| 914 |
},
|
| 915 |
{
|
| 916 |
"cell_type": "code",
|
| 917 |
"execution_count": null,
|
| 918 |
-
"id": "
|
| 919 |
"metadata": {},
|
| 920 |
"outputs": [],
|
| 921 |
-
"source": [
|
| 922 |
-
"# "
|
| 923 |
-
]
|
| 924 |
}
|
| 925 |
],
|
| 926 |
"metadata": {
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 2,
|
| 14 |
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
|
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
+
"execution_count": 3,
|
| 41 |
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
"metadata": {},
|
| 43 |
"outputs": [
|
|
|
|
| 82 |
},
|
| 83 |
{
|
| 84 |
"cell_type": "code",
|
| 85 |
+
"execution_count": 4,
|
| 86 |
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
|
|
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
+
"execution_count": 5,
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
+
"execution_count": 6,
|
| 259 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 260 |
"metadata": {
|
| 261 |
"scrolled": true
|
|
|
|
| 312 |
},
|
| 313 |
{
|
| 314 |
"cell_type": "code",
|
| 315 |
+
"execution_count": 7,
|
| 316 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 317 |
"metadata": {
|
| 318 |
"scrolled": true
|
|
|
|
| 377 |
},
|
| 378 |
{
|
| 379 |
"cell_type": "code",
|
| 380 |
+
"execution_count": 8,
|
| 381 |
"id": "f8b547d4-1858-49d5-b26c-fd30e210cd58",
|
| 382 |
"metadata": {},
|
| 383 |
"outputs": [
|
|
|
|
| 392 |
"name": "stdout",
|
| 393 |
"output_type": "stream",
|
| 394 |
"text": [
|
| 395 |
+
"Processing 9/9"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 396 |
]
|
| 397 |
}
|
| 398 |
],
|
|
|
|
| 439 |
" for item in sublist:\n",
|
| 440 |
" if item not in flat_unique:\n",
|
| 441 |
" flat_unique.append(item)\n",
|
| 442 |
+
" # sort alphabetically and add newlines\n",
|
| 443 |
+
" sorted_list = \"\\n\".join(sorted(flat_unique)) \n",
|
| 444 |
" return df, sorted_list\n",
|
| 445 |
"\n",
|
| 446 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 447 |
]
|
| 448 |
},
|
| 449 |
{
|
| 450 |
"cell_type": "code",
|
| 451 |
+
"execution_count": 11,
|
| 452 |
+
"id": "162482dd-d2ce-44b7-9a6d-c34a96c1a6ed",
|
| 453 |
"metadata": {},
|
| 454 |
"outputs": [
|
| 455 |
{
|
| 456 |
"name": "stdout",
|
| 457 |
"output_type": "stream",
|
| 458 |
"text": [
|
| 459 |
+
" superclass pred_class\n",
|
| 460 |
+
"0 activity_type beach vacation\n",
|
| 461 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 462 |
+
"2 climate_or_season warm destination / summer\n",
|
| 463 |
+
"3 style_or_comfort minimalist\n",
|
| 464 |
+
"4 dress_code casual\n",
|
| 465 |
+
"5 accommodation huts with half board\n",
|
| 466 |
+
"6 transportation no own vehicle\n",
|
| 467 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 468 |
+
"8 trip_length_days 7+ days\n",
|
| 469 |
+
"1 set of clothing for every situation\n",
|
| 470 |
+
"USB hub (for multiple devices)\n",
|
| 471 |
+
"all‑in‑one soap\n",
|
| 472 |
+
"backpack\n",
|
| 473 |
+
"backup lighting (e.g. small flashlight)\n",
|
| 474 |
+
"beach bag\n",
|
| 475 |
+
"beach chair\n",
|
| 476 |
+
"beach towel\n",
|
| 477 |
+
"blister plasters or tape\n",
|
| 478 |
+
"book or e‑reader\n",
|
| 479 |
+
"cap or hat\n",
|
| 480 |
+
"cash for payments\n",
|
| 481 |
+
"comfortable clothing\n",
|
| 482 |
+
"compact toothbrush\n",
|
| 483 |
+
"cooler\n",
|
| 484 |
+
"daypack\n",
|
| 485 |
+
"earplugs\n",
|
| 486 |
+
"emergency communication (e.g. GPS beacon or satellite messenger)\n",
|
| 487 |
+
"extra charger cables\n",
|
| 488 |
+
"extra clothing layer\n",
|
| 489 |
+
"first aid kit\n",
|
| 490 |
+
"flashlight or headlamp\n",
|
| 491 |
+
"flip flops\n",
|
| 492 |
+
"foldable solar panel (if on longer trips)\n",
|
| 493 |
+
"hat or cap\n",
|
| 494 |
+
"headlamp + extra batteries\n",
|
| 495 |
+
"hiking boots or trail runners\n",
|
| 496 |
+
"hiking poles\n",
|
| 497 |
+
"hiking socks (anti-blister)\n",
|
| 498 |
+
"jeans or comfortable pants\n",
|
| 499 |
+
"light pajamas or sleepwear\n",
|
| 500 |
+
"light shoes\n",
|
| 501 |
+
"light towel\n",
|
| 502 |
+
"lightweight clothing\n",
|
| 503 |
+
"lightweight towel\n",
|
| 504 |
+
"music / headphones\n",
|
| 505 |
+
"navigation (map/compass/GPS)\n",
|
| 506 |
+
"navigation device with offline maps\n",
|
| 507 |
+
"notebook + pen\n",
|
| 508 |
+
"number of meals/snacks matched to duration\n",
|
| 509 |
+
"packaging to keep electronics dry\n",
|
| 510 |
+
"paper map and compass\n",
|
| 511 |
+
"power bank (at least 10,000 mAh)\n",
|
| 512 |
+
"public transport app or ticket\n",
|
| 513 |
+
"rain jacket or poncho\n",
|
| 514 |
+
"rechargeable batteries and charger\n",
|
| 515 |
+
"reservation confirmation\n",
|
| 516 |
+
"seat cushion or beach mat\n",
|
| 517 |
+
"sheet liner (often required)\n",
|
| 518 |
+
"slippers or indoor shoes for inside\n",
|
| 519 |
+
"small backpack\n",
|
| 520 |
+
"small toiletry bag\n",
|
| 521 |
+
"snacks / energy bars\n",
|
| 522 |
+
"snacks for along the way\n",
|
| 523 |
+
"sneakers\n",
|
| 524 |
+
"socks per day\n",
|
| 525 |
+
"solar panel or portable charging system\n",
|
| 526 |
+
"sun hat\n",
|
| 527 |
+
"sunglasses\n",
|
| 528 |
+
"sunscreen\n",
|
| 529 |
+
"sunscreen and sunglasses\n",
|
| 530 |
+
"sweater or hoodie\n",
|
| 531 |
+
"swimwear\n",
|
| 532 |
+
"t-shirts\n",
|
| 533 |
+
"toiletry bag\n",
|
| 534 |
+
"underwear per day\n",
|
| 535 |
+
"water bottle\n",
|
| 536 |
+
"water bottle(s) or hydration bladder\n"
|
| 537 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 538 |
}
|
| 539 |
],
|
| 540 |
"source": [
|
| 541 |
+
"print(result[0])\n",
|
| 542 |
+
"print(result[1])"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 543 |
]
|
| 544 |
},
|
| 545 |
{
|
| 546 |
"cell_type": "markdown",
|
| 547 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 548 |
"metadata": {},
|
| 549 |
"source": [
|
| 550 |
+
"Now use gradio app"
|
| 551 |
]
|
| 552 |
},
|
| 553 |
{
|
| 554 |
"cell_type": "code",
|
| 555 |
+
"execution_count": 9,
|
| 556 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 557 |
+
"metadata": {},
|
|
|
|
|
|
|
| 558 |
"outputs": [
|
| 559 |
{
|
| 560 |
"name": "stdout",
|
| 561 |
"output_type": "stream",
|
| 562 |
"text": [
|
| 563 |
+
"Running on local URL: http://127.0.0.1:7860\n",
|
| 564 |
"\n",
|
| 565 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 566 |
]
|
|
|
|
| 568 |
{
|
| 569 |
"data": {
|
| 570 |
"text/html": [
|
| 571 |
+
"<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 572 |
],
|
| 573 |
"text/plain": [
|
| 574 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 579 |
}
|
| 580 |
],
|
| 581 |
"source": [
|
|
|
|
|
|
|
|
|
|
| 582 |
"import gradio as gr\n",
|
| 583 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 584 |
"demo = gr.Interface(\n",
|
| 585 |
" fn=pred_trip,\n",
|
| 586 |
" inputs=[\n",
|
|
|
|
| 595 |
"\n",
|
| 596 |
"# Launch the Gradio app\n",
|
| 597 |
"if __name__ == \"__main__\":\n",
|
| 598 |
+
" demo.launch()"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 599 |
]
|
| 600 |
},
|
| 601 |
{
|
| 602 |
"cell_type": "code",
|
| 603 |
"execution_count": null,
|
| 604 |
+
"id": "0aa38a02-266c-4628-8f90-0d198d2f9897",
|
| 605 |
"metadata": {},
|
| 606 |
"outputs": [],
|
| 607 |
+
"source": []
|
|
|
|
|
|
|
| 608 |
}
|
| 609 |
],
|
| 610 |
"metadata": {
|
space/blog_post/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
ADDED
|
@@ -0,0 +1,802 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cells": [
|
| 3 |
+
{
|
| 4 |
+
"cell_type": "markdown",
|
| 5 |
+
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
+
"metadata": {},
|
| 7 |
+
"source": [
|
| 8 |
+
"**Hugging Face API**"
|
| 9 |
+
]
|
| 10 |
+
},
|
| 11 |
+
{
|
| 12 |
+
"cell_type": "code",
|
| 13 |
+
"execution_count": 2,
|
| 14 |
+
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
+
"metadata": {},
|
| 16 |
+
"outputs": [],
|
| 17 |
+
"source": [
|
| 18 |
+
"from dotenv import load_dotenv\n",
|
| 19 |
+
"import os\n",
|
| 20 |
+
"import requests\n",
|
| 21 |
+
"import json\n",
|
| 22 |
+
"\n",
|
| 23 |
+
"load_dotenv() \n",
|
| 24 |
+
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 25 |
+
"\n",
|
| 26 |
+
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 27 |
+
"\n",
|
| 28 |
+
"def query(model, input_text):\n",
|
| 29 |
+
" API_URL = f\"https://router.huggingface.co/hf-inference/models/{model}\"\n",
|
| 30 |
+
" payload = {\n",
|
| 31 |
+
" \"inputs\": input_text,\n",
|
| 32 |
+
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
| 33 |
+
" }\n",
|
| 34 |
+
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 35 |
+
" return response.json()"
|
| 36 |
+
]
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"cell_type": "code",
|
| 40 |
+
"execution_count": 3,
|
| 41 |
+
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
+
"metadata": {},
|
| 43 |
+
"outputs": [
|
| 44 |
+
{
|
| 45 |
+
"name": "stdout",
|
| 46 |
+
"output_type": "stream",
|
| 47 |
+
"text": [
|
| 48 |
+
"[\n",
|
| 49 |
+
" {\n",
|
| 50 |
+
" \"label\": \"technology\",\n",
|
| 51 |
+
" \"score\": 0.970917284488678\n",
|
| 52 |
+
" },\n",
|
| 53 |
+
" {\n",
|
| 54 |
+
" \"label\": \"health\",\n",
|
| 55 |
+
" \"score\": 0.014999152161180973\n",
|
| 56 |
+
" },\n",
|
| 57 |
+
" {\n",
|
| 58 |
+
" \"label\": \"sports\",\n",
|
| 59 |
+
" \"score\": 0.008272469975054264\n",
|
| 60 |
+
" },\n",
|
| 61 |
+
" {\n",
|
| 62 |
+
" \"label\": \"politics\",\n",
|
| 63 |
+
" \"score\": 0.005811101291328669\n",
|
| 64 |
+
" }\n",
|
| 65 |
+
"]\n"
|
| 66 |
+
]
|
| 67 |
+
}
|
| 68 |
+
],
|
| 69 |
+
"source": [
|
| 70 |
+
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 71 |
+
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
+
"print(json.dumps(output, indent=4))"
|
| 73 |
+
]
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"cell_type": "markdown",
|
| 77 |
+
"id": "01dc84d7-6d0f-4409-8966-c1adac5fd0c8",
|
| 78 |
+
"metadata": {},
|
| 79 |
+
"source": [
|
| 80 |
+
"**Model implementation**"
|
| 81 |
+
]
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"cell_type": "code",
|
| 85 |
+
"execution_count": 4,
|
| 86 |
+
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
+
"metadata": {},
|
| 88 |
+
"outputs": [
|
| 89 |
+
{
|
| 90 |
+
"name": "stdout",
|
| 91 |
+
"output_type": "stream",
|
| 92 |
+
"text": [
|
| 93 |
+
"\n",
|
| 94 |
+
" activity_type :\n",
|
| 95 |
+
"\t hut trek (summer)\n",
|
| 96 |
+
"\t hut trek (winter)\n",
|
| 97 |
+
"\t camping trip (wild camping)\n",
|
| 98 |
+
"\t camping trip (campground)\n",
|
| 99 |
+
"\t ski tour / skitour\n",
|
| 100 |
+
"\t snowboard / splitboard trip\n",
|
| 101 |
+
"\t long-distance hike / thru-hike\n",
|
| 102 |
+
"\t digital nomad trip\n",
|
| 103 |
+
"\t city trip\n",
|
| 104 |
+
"\t road trip (car/camper)\n",
|
| 105 |
+
"\t festival trip\n",
|
| 106 |
+
"\t yoga / wellness retreat\n",
|
| 107 |
+
"\t micro-adventure / weekend trip\n",
|
| 108 |
+
"\t beach vacation\n",
|
| 109 |
+
"\t cultural exploration\n",
|
| 110 |
+
"\t nature escape\n",
|
| 111 |
+
"\n",
|
| 112 |
+
" activities :\n",
|
| 113 |
+
"\t swimming\n",
|
| 114 |
+
"\t going to the beach\n",
|
| 115 |
+
"\t relaxing\n",
|
| 116 |
+
"\t sightseeing\n",
|
| 117 |
+
"\t biking\n",
|
| 118 |
+
"\t running\n",
|
| 119 |
+
"\t skiing\n",
|
| 120 |
+
"\t cross-country skiing\n",
|
| 121 |
+
"\t ski touring\n",
|
| 122 |
+
"\t hiking\n",
|
| 123 |
+
"\t hut-to-hut hiking\n",
|
| 124 |
+
"\t rock climbing\n",
|
| 125 |
+
"\t ice climbing\n",
|
| 126 |
+
"\t snowshoe hiking\n",
|
| 127 |
+
"\t kayaking / canoeing\n",
|
| 128 |
+
"\t stand-up paddleboarding (SUP)\n",
|
| 129 |
+
"\t snorkeling\n",
|
| 130 |
+
"\t scuba diving\n",
|
| 131 |
+
"\t surfing\n",
|
| 132 |
+
"\t paragliding\n",
|
| 133 |
+
"\t horseback riding\n",
|
| 134 |
+
"\t photography\n",
|
| 135 |
+
"\t fishing\n",
|
| 136 |
+
"\t rafting\n",
|
| 137 |
+
"\t yoga\n",
|
| 138 |
+
"\n",
|
| 139 |
+
" climate_or_season :\n",
|
| 140 |
+
"\t cold destination / winter\n",
|
| 141 |
+
"\t warm destination / summer\n",
|
| 142 |
+
"\t variable weather / spring / autumn\n",
|
| 143 |
+
"\t tropical / humid\n",
|
| 144 |
+
"\t dry / desert-like\n",
|
| 145 |
+
"\t rainy climate\n",
|
| 146 |
+
"\n",
|
| 147 |
+
" style_or_comfort :\n",
|
| 148 |
+
"\t ultralight\n",
|
| 149 |
+
"\t lightweight (but comfortable)\n",
|
| 150 |
+
"\t luxury (including evening wear)\n",
|
| 151 |
+
"\t minimalist\n",
|
| 152 |
+
"\n",
|
| 153 |
+
" dress_code :\n",
|
| 154 |
+
"\t casual\n",
|
| 155 |
+
"\t formal (business trip)\n",
|
| 156 |
+
"\t conservative\n",
|
| 157 |
+
"\n",
|
| 158 |
+
" accommodation :\n",
|
| 159 |
+
"\t indoor\n",
|
| 160 |
+
"\t huts with half board\n",
|
| 161 |
+
"\t sleeping in a tent\n",
|
| 162 |
+
"\t sleeping in a car\n",
|
| 163 |
+
"\n",
|
| 164 |
+
" transportation :\n",
|
| 165 |
+
"\t own vehicle\n",
|
| 166 |
+
"\t no own vehicle\n",
|
| 167 |
+
"\n",
|
| 168 |
+
" special_conditions :\n",
|
| 169 |
+
"\t off-grid / no electricity\n",
|
| 170 |
+
"\t self-supported (bring your own cooking gear)\n",
|
| 171 |
+
"\t travel with children\n",
|
| 172 |
+
"\t pet-friendly\n",
|
| 173 |
+
"\t snow and ice\n",
|
| 174 |
+
"\t high alpine terrain\n",
|
| 175 |
+
"\t snow, ice and avalanche-prone terrain\n",
|
| 176 |
+
"\t no special conditions to consider\n",
|
| 177 |
+
"\n",
|
| 178 |
+
" trip_length_days :\n",
|
| 179 |
+
"\t 1 day\n",
|
| 180 |
+
"\t 2 days\n",
|
| 181 |
+
"\t 3 days\n",
|
| 182 |
+
"\t 4 days\n",
|
| 183 |
+
"\t 5 days\n",
|
| 184 |
+
"\t 6 days\n",
|
| 185 |
+
"\t 7 days\n",
|
| 186 |
+
"\t 7+ days\n"
|
| 187 |
+
]
|
| 188 |
+
}
|
| 189 |
+
],
|
| 190 |
+
"source": [
|
| 191 |
+
"import json\n",
|
| 192 |
+
"import pandas as pd\n",
|
| 193 |
+
"from tabulate import tabulate\n",
|
| 194 |
+
"from transformers import pipeline\n",
|
| 195 |
+
"\n",
|
| 196 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 197 |
+
" candidate_labels = json.load(file)\n",
|
| 198 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 199 |
+
"\n",
|
| 200 |
+
"for key in candidate_labels:\n",
|
| 201 |
+
" print(\"\\n\", key, \":\")\n",
|
| 202 |
+
" for item in candidate_labels[key]:\n",
|
| 203 |
+
" print(\"\\t\", item)"
|
| 204 |
+
]
|
| 205 |
+
},
|
| 206 |
+
{
|
| 207 |
+
"cell_type": "code",
|
| 208 |
+
"execution_count": 5,
|
| 209 |
+
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
+
"metadata": {},
|
| 211 |
+
"outputs": [
|
| 212 |
+
{
|
| 213 |
+
"name": "stderr",
|
| 214 |
+
"output_type": "stream",
|
| 215 |
+
"text": [
|
| 216 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 217 |
+
]
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"name": "stdout",
|
| 221 |
+
"output_type": "stream",
|
| 222 |
+
"text": [
|
| 223 |
+
" Label Score\n",
|
| 224 |
+
"0 beach vacation 0.376311\n",
|
| 225 |
+
"1 micro-adventure / weekend trip 0.350168\n",
|
| 226 |
+
"2 nature escape 0.133974\n",
|
| 227 |
+
"3 digital nomad trip 0.031636\n",
|
| 228 |
+
"4 cultural exploration 0.031271\n",
|
| 229 |
+
"5 yoga / wellness retreat 0.012846\n",
|
| 230 |
+
"6 festival trip 0.012700\n",
|
| 231 |
+
"7 long-distance hike / thru-hike 0.009527\n",
|
| 232 |
+
"8 hut trek (summer) 0.008148\n",
|
| 233 |
+
"9 city trip 0.007793\n",
|
| 234 |
+
"10 road trip (car/camper) 0.006512\n",
|
| 235 |
+
"11 ski tour / skitour 0.005670\n",
|
| 236 |
+
"12 camping trip (campground) 0.004448\n",
|
| 237 |
+
"13 snowboard / splitboard trip 0.004113\n",
|
| 238 |
+
"14 camping trip (wild camping) 0.002714\n",
|
| 239 |
+
"15 hut trek (winter) 0.002170\n"
|
| 240 |
+
]
|
| 241 |
+
}
|
| 242 |
+
],
|
| 243 |
+
"source": [
|
| 244 |
+
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 245 |
+
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 246 |
+
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 247 |
+
"result = classifier(trip_descr, candidate_labels[\"activity_type\"])\n",
|
| 248 |
+
"\n",
|
| 249 |
+
"df = pd.DataFrame({\n",
|
| 250 |
+
" \"Label\": result[\"labels\"],\n",
|
| 251 |
+
" \"Score\": result[\"scores\"]\n",
|
| 252 |
+
"})\n",
|
| 253 |
+
"print(df)"
|
| 254 |
+
]
|
| 255 |
+
},
|
| 256 |
+
{
|
| 257 |
+
"cell_type": "code",
|
| 258 |
+
"execution_count": 6,
|
| 259 |
+
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 260 |
+
"metadata": {},
|
| 261 |
+
"outputs": [
|
| 262 |
+
{
|
| 263 |
+
"name": "stdout",
|
| 264 |
+
"output_type": "stream",
|
| 265 |
+
"text": [
|
| 266 |
+
"beach vacation\n"
|
| 267 |
+
]
|
| 268 |
+
}
|
| 269 |
+
],
|
| 270 |
+
"source": [
|
| 271 |
+
"# the labels are sorted by score. We choose the first one as our best guess for a class label\n",
|
| 272 |
+
"class_label = result[\"labels\"][0]\n",
|
| 273 |
+
"print(class_label)"
|
| 274 |
+
]
|
| 275 |
+
},
|
| 276 |
+
{
|
| 277 |
+
"cell_type": "code",
|
| 278 |
+
"execution_count": 7,
|
| 279 |
+
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 280 |
+
"metadata": {
|
| 281 |
+
"scrolled": true
|
| 282 |
+
},
|
| 283 |
+
"outputs": [
|
| 284 |
+
{
|
| 285 |
+
"name": "stdout",
|
| 286 |
+
"output_type": "stream",
|
| 287 |
+
"text": [
|
| 288 |
+
" Label Score\n",
|
| 289 |
+
"0 going to the beach 0.991486\n",
|
| 290 |
+
"1 relaxing 0.977136\n",
|
| 291 |
+
"2 hiking 0.942628\n",
|
| 292 |
+
"3 swimming 0.219020\n",
|
| 293 |
+
"4 sightseeing 0.175862\n",
|
| 294 |
+
"5 running 0.098545\n",
|
| 295 |
+
"6 hut-to-hut hiking 0.083704\n",
|
| 296 |
+
"7 biking 0.036792\n",
|
| 297 |
+
"8 photography 0.036690\n",
|
| 298 |
+
"9 surfing 0.030993\n",
|
| 299 |
+
"10 stand-up paddleboarding (SUP) 0.025300\n",
|
| 300 |
+
"11 snorkeling 0.021451\n",
|
| 301 |
+
"12 yoga 0.011070\n",
|
| 302 |
+
"13 kayaking / canoeing 0.007511\n",
|
| 303 |
+
"14 rock climbing 0.006307\n",
|
| 304 |
+
"15 fishing 0.003497\n",
|
| 305 |
+
"16 paragliding 0.002656\n",
|
| 306 |
+
"17 rafting 0.001970\n",
|
| 307 |
+
"18 horseback riding 0.001560\n",
|
| 308 |
+
"19 snowshoe hiking 0.001528\n",
|
| 309 |
+
"20 cross-country skiing 0.001502\n",
|
| 310 |
+
"21 ice climbing 0.001434\n",
|
| 311 |
+
"22 skiing 0.001169\n",
|
| 312 |
+
"23 scuba diving 0.000789\n",
|
| 313 |
+
"24 ski touring 0.000491\n",
|
| 314 |
+
"[]\n"
|
| 315 |
+
]
|
| 316 |
+
}
|
| 317 |
+
],
|
| 318 |
+
"source": [
|
| 319 |
+
"# we do things differently for \"activities\"\n",
|
| 320 |
+
"cut_off = 0.5\n",
|
| 321 |
+
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 322 |
+
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
| 323 |
+
"\n",
|
| 324 |
+
"df = pd.DataFrame({\n",
|
| 325 |
+
" \"Label\": result_activ[\"labels\"],\n",
|
| 326 |
+
" \"Score\": result_activ[\"scores\"]\n",
|
| 327 |
+
"})\n",
|
| 328 |
+
"print(df)\n",
|
| 329 |
+
"print(classes)"
|
| 330 |
+
]
|
| 331 |
+
},
|
| 332 |
+
{
|
| 333 |
+
"cell_type": "code",
|
| 334 |
+
"execution_count": 8,
|
| 335 |
+
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 336 |
+
"metadata": {},
|
| 337 |
+
"outputs": [],
|
| 338 |
+
"source": [
|
| 339 |
+
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 340 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 341 |
+
" \"\"\"\n",
|
| 342 |
+
" Classifies trip\n",
|
| 343 |
+
" \n",
|
| 344 |
+
" Parameters:\n",
|
| 345 |
+
" model_name: name of hugging-face model\n",
|
| 346 |
+
" trip_descr: text describing the trip\n",
|
| 347 |
+
" cut_off: cut_off for choosing activities\n",
|
| 348 |
+
"\n",
|
| 349 |
+
" Returns:\n",
|
| 350 |
+
" pd Dataframe: with class predictions\n",
|
| 351 |
+
" \"\"\"\n",
|
| 352 |
+
" \n",
|
| 353 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 354 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 355 |
+
" for i, key in enumerate(keys_list):\n",
|
| 356 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 357 |
+
" if key == 'activities':\n",
|
| 358 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 359 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 360 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 361 |
+
" else:\n",
|
| 362 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 363 |
+
" classes = result[\"labels\"][0]\n",
|
| 364 |
+
" df.loc[i] = [key, classes]\n",
|
| 365 |
+
" return df"
|
| 366 |
+
]
|
| 367 |
+
},
|
| 368 |
+
{
|
| 369 |
+
"cell_type": "code",
|
| 370 |
+
"execution_count": 9,
|
| 371 |
+
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 372 |
+
"metadata": {},
|
| 373 |
+
"outputs": [
|
| 374 |
+
{
|
| 375 |
+
"name": "stderr",
|
| 376 |
+
"output_type": "stream",
|
| 377 |
+
"text": [
|
| 378 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 379 |
+
]
|
| 380 |
+
},
|
| 381 |
+
{
|
| 382 |
+
"name": "stdout",
|
| 383 |
+
"output_type": "stream",
|
| 384 |
+
"text": [
|
| 385 |
+
"Processing 9/9 superclass pred_class\n",
|
| 386 |
+
"0 activity_type beach vacation\n",
|
| 387 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 388 |
+
"2 climate_or_season warm destination / summer\n",
|
| 389 |
+
"3 style_or_comfort minimalist\n",
|
| 390 |
+
"4 dress_code casual\n",
|
| 391 |
+
"5 accommodation huts with half board\n",
|
| 392 |
+
"6 transportation no own vehicle\n",
|
| 393 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 394 |
+
"8 trip_length_days 7+ days\n"
|
| 395 |
+
]
|
| 396 |
+
}
|
| 397 |
+
],
|
| 398 |
+
"source": [
|
| 399 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 400 |
+
"print(result)"
|
| 401 |
+
]
|
| 402 |
+
},
|
| 403 |
+
{
|
| 404 |
+
"cell_type": "markdown",
|
| 405 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 406 |
+
"metadata": {},
|
| 407 |
+
"source": [
|
| 408 |
+
"Now use gradio app"
|
| 409 |
+
]
|
| 410 |
+
},
|
| 411 |
+
{
|
| 412 |
+
"cell_type": "code",
|
| 413 |
+
"execution_count": 3,
|
| 414 |
+
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 415 |
+
"metadata": {},
|
| 416 |
+
"outputs": [],
|
| 417 |
+
"source": [
|
| 418 |
+
"# Prerequisites\n",
|
| 419 |
+
"#from transformers import pipeline\n",
|
| 420 |
+
"#import json\n",
|
| 421 |
+
"#import pandas as pd\n"
|
| 422 |
+
]
|
| 423 |
+
},
|
| 424 |
+
{
|
| 425 |
+
"cell_type": "code",
|
| 426 |
+
"execution_count": 11,
|
| 427 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 428 |
+
"metadata": {},
|
| 429 |
+
"outputs": [
|
| 430 |
+
{
|
| 431 |
+
"name": "stdout",
|
| 432 |
+
"output_type": "stream",
|
| 433 |
+
"text": [
|
| 434 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 435 |
+
"\n",
|
| 436 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 437 |
+
]
|
| 438 |
+
},
|
| 439 |
+
{
|
| 440 |
+
"data": {
|
| 441 |
+
"text/html": [
|
| 442 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 443 |
+
],
|
| 444 |
+
"text/plain": [
|
| 445 |
+
"<IPython.core.display.HTML object>"
|
| 446 |
+
]
|
| 447 |
+
},
|
| 448 |
+
"metadata": {},
|
| 449 |
+
"output_type": "display_data"
|
| 450 |
+
}
|
| 451 |
+
],
|
| 452 |
+
"source": [
|
| 453 |
+
"import gradio as gr\n",
|
| 454 |
+
"\n",
|
| 455 |
+
"demo = gr.Interface(\n",
|
| 456 |
+
" fn=pred_trip,\n",
|
| 457 |
+
" inputs=[\n",
|
| 458 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 459 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 460 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 461 |
+
" ],\n",
|
| 462 |
+
" # outputs=\"dataframe\",\n",
|
| 463 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 464 |
+
" title=\"Trip classification\",\n",
|
| 465 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 466 |
+
")\n",
|
| 467 |
+
"\n",
|
| 468 |
+
"# Launch the Gradio app\n",
|
| 469 |
+
"if __name__ == \"__main__\":\n",
|
| 470 |
+
" demo.launch()\n"
|
| 471 |
+
]
|
| 472 |
+
},
|
| 473 |
+
{
|
| 474 |
+
"cell_type": "code",
|
| 475 |
+
"execution_count": 46,
|
| 476 |
+
"id": "11006b67-bfd5-42a7-99c4-36c3db3affac",
|
| 477 |
+
"metadata": {},
|
| 478 |
+
"outputs": [
|
| 479 |
+
{
|
| 480 |
+
"name": "stderr",
|
| 481 |
+
"output_type": "stream",
|
| 482 |
+
"text": [
|
| 483 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 484 |
+
]
|
| 485 |
+
},
|
| 486 |
+
{
|
| 487 |
+
"name": "stdout",
|
| 488 |
+
"output_type": "stream",
|
| 489 |
+
"text": [
|
| 490 |
+
"Processing 9/9"
|
| 491 |
+
]
|
| 492 |
+
}
|
| 493 |
+
],
|
| 494 |
+
"source": [
|
| 495 |
+
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 496 |
+
]
|
| 497 |
+
},
|
| 498 |
+
{
|
| 499 |
+
"cell_type": "markdown",
|
| 500 |
+
"id": "39920553-27d3-4a63-8381-8310566c4874",
|
| 501 |
+
"metadata": {},
|
| 502 |
+
"source": [
|
| 503 |
+
"All code for gradio app file"
|
| 504 |
+
]
|
| 505 |
+
},
|
| 506 |
+
{
|
| 507 |
+
"cell_type": "code",
|
| 508 |
+
"execution_count": 2,
|
| 509 |
+
"id": "4ffc76d5-60c3-4bc8-bd4f-a9636077f01d",
|
| 510 |
+
"metadata": {},
|
| 511 |
+
"outputs": [
|
| 512 |
+
{
|
| 513 |
+
"name": "stdout",
|
| 514 |
+
"output_type": "stream",
|
| 515 |
+
"text": [
|
| 516 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 517 |
+
"\n",
|
| 518 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 519 |
+
]
|
| 520 |
+
},
|
| 521 |
+
{
|
| 522 |
+
"data": {
|
| 523 |
+
"text/html": [
|
| 524 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 525 |
+
],
|
| 526 |
+
"text/plain": [
|
| 527 |
+
"<IPython.core.display.HTML object>"
|
| 528 |
+
]
|
| 529 |
+
},
|
| 530 |
+
"metadata": {},
|
| 531 |
+
"output_type": "display_data"
|
| 532 |
+
},
|
| 533 |
+
{
|
| 534 |
+
"name": "stderr",
|
| 535 |
+
"output_type": "stream",
|
| 536 |
+
"text": [
|
| 537 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 538 |
+
]
|
| 539 |
+
},
|
| 540 |
+
{
|
| 541 |
+
"name": "stdout",
|
| 542 |
+
"output_type": "stream",
|
| 543 |
+
"text": [
|
| 544 |
+
"Processing 9/9"
|
| 545 |
+
]
|
| 546 |
+
}
|
| 547 |
+
],
|
| 548 |
+
"source": [
|
| 549 |
+
"from transformers import pipeline\n",
|
| 550 |
+
"import json\n",
|
| 551 |
+
"import pandas as pd\n",
|
| 552 |
+
"import gradio as gr\n",
|
| 553 |
+
"\n",
|
| 554 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 555 |
+
" candidate_labels = json.load(file)\n",
|
| 556 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 557 |
+
"\n",
|
| 558 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 559 |
+
" \"\"\"\n",
|
| 560 |
+
" Classifies trip\n",
|
| 561 |
+
" \n",
|
| 562 |
+
" Parameters:\n",
|
| 563 |
+
" model_name: name of hugging-face model\n",
|
| 564 |
+
" trip_descr: text describing the trip\n",
|
| 565 |
+
" cut_off: cut_off for choosing activities\n",
|
| 566 |
+
"\n",
|
| 567 |
+
" Returns:\n",
|
| 568 |
+
" pd Dataframe: with class predictions\n",
|
| 569 |
+
" \"\"\"\n",
|
| 570 |
+
" \n",
|
| 571 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 572 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 573 |
+
" for i, key in enumerate(keys_list):\n",
|
| 574 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 575 |
+
" if key == 'activities':\n",
|
| 576 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 577 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 578 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 579 |
+
" else:\n",
|
| 580 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 581 |
+
" classes = result[\"labels\"][0]\n",
|
| 582 |
+
" df.loc[i] = [key, classes]\n",
|
| 583 |
+
" return df\n",
|
| 584 |
+
"\n",
|
| 585 |
+
"demo = gr.Interface(\n",
|
| 586 |
+
" fn=pred_trip,\n",
|
| 587 |
+
" inputs=[\n",
|
| 588 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 589 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 590 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 591 |
+
" ],\n",
|
| 592 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 593 |
+
" title=\"Trip classification\",\n",
|
| 594 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 595 |
+
")\n",
|
| 596 |
+
"\n",
|
| 597 |
+
"# Launch the Gradio app\n",
|
| 598 |
+
"if __name__ == \"__main__\":\n",
|
| 599 |
+
" demo.launch()\n"
|
| 600 |
+
]
|
| 601 |
+
},
|
| 602 |
+
{
|
| 603 |
+
"cell_type": "code",
|
| 604 |
+
"execution_count": null,
|
| 605 |
+
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 606 |
+
"metadata": {},
|
| 607 |
+
"outputs": [],
|
| 608 |
+
"source": [
|
| 609 |
+
"Print test data set"
|
| 610 |
+
]
|
| 611 |
+
},
|
| 612 |
+
{
|
| 613 |
+
"cell_type": "code",
|
| 614 |
+
"execution_count": 4,
|
| 615 |
+
"id": "f25e1462-ca48-4853-ae44-dfc096c5011d",
|
| 616 |
+
"metadata": {},
|
| 617 |
+
"outputs": [
|
| 618 |
+
{
|
| 619 |
+
"name": "stdout",
|
| 620 |
+
"output_type": "stream",
|
| 621 |
+
"text": [
|
| 622 |
+
"0 . I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 623 |
+
"\n",
|
| 624 |
+
"beach vacation\n",
|
| 625 |
+
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 626 |
+
"warm destination / summer\n",
|
| 627 |
+
"lightweight (but comfortable)\n",
|
| 628 |
+
"casual\n",
|
| 629 |
+
"indoor\n",
|
| 630 |
+
"no own vehicle\n",
|
| 631 |
+
"no special conditions to consider\n",
|
| 632 |
+
"7+ days\n",
|
| 633 |
+
"\n",
|
| 634 |
+
"\n",
|
| 635 |
+
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 636 |
+
"\n",
|
| 637 |
+
"city trip\n",
|
| 638 |
+
"['sightseeing']\n",
|
| 639 |
+
"variable weather / spring / autumn\n",
|
| 640 |
+
"luxury (including evening wear)\n",
|
| 641 |
+
"casual\n",
|
| 642 |
+
"indoor\n",
|
| 643 |
+
"no own vehicle\n",
|
| 644 |
+
"no special conditions to consider\n",
|
| 645 |
+
"3 days\n",
|
| 646 |
+
"\n",
|
| 647 |
+
"\n",
|
| 648 |
+
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 649 |
+
"\n",
|
| 650 |
+
"city trip\n",
|
| 651 |
+
"['relaxing']\n",
|
| 652 |
+
"cold destination / winter\n",
|
| 653 |
+
"lightweight (but comfortable)\n",
|
| 654 |
+
"casual\n",
|
| 655 |
+
"indoor\n",
|
| 656 |
+
"no own vehicle\n",
|
| 657 |
+
"no special conditions to consider\n",
|
| 658 |
+
"7+ days\n",
|
| 659 |
+
"\n",
|
| 660 |
+
"\n",
|
| 661 |
+
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 662 |
+
"\n",
|
| 663 |
+
"cultural exploration\n",
|
| 664 |
+
"['sightseeing', 'hiking', 'rafting']\n",
|
| 665 |
+
"variable weather / spring / autumn\n",
|
| 666 |
+
"lightweight (but comfortable)\n",
|
| 667 |
+
"casual\n",
|
| 668 |
+
"indoor\n",
|
| 669 |
+
"no own vehicle\n",
|
| 670 |
+
"rainy climate\n",
|
| 671 |
+
"7+ days\n",
|
| 672 |
+
"\n",
|
| 673 |
+
"\n",
|
| 674 |
+
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 675 |
+
"\n",
|
| 676 |
+
"nature escape\n",
|
| 677 |
+
"['swimming', 'relaxing', 'hiking']\n",
|
| 678 |
+
"warm destination / summer\n",
|
| 679 |
+
"lightweight (but comfortable)\n",
|
| 680 |
+
"casual\n",
|
| 681 |
+
"indoor\n",
|
| 682 |
+
"no own vehicle\n",
|
| 683 |
+
"no special conditions to consider\n",
|
| 684 |
+
"7+ days\n",
|
| 685 |
+
"\n",
|
| 686 |
+
"\n",
|
| 687 |
+
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 688 |
+
"\n",
|
| 689 |
+
"long-distance hike / thru-hike\n",
|
| 690 |
+
"['going to the beach']\n",
|
| 691 |
+
"tropical / humid\n",
|
| 692 |
+
"minimalist\n",
|
| 693 |
+
"casual\n",
|
| 694 |
+
"huts with half board\n",
|
| 695 |
+
"own vehicle\n",
|
| 696 |
+
"off-grid / no electricity\n",
|
| 697 |
+
"6 days\n",
|
| 698 |
+
"\n",
|
| 699 |
+
"\n",
|
| 700 |
+
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 701 |
+
"\n",
|
| 702 |
+
"beach vacation\n",
|
| 703 |
+
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 704 |
+
"cold destination / winter\n",
|
| 705 |
+
"ultralight\n",
|
| 706 |
+
"casual\n",
|
| 707 |
+
"sleeping in a tent\n",
|
| 708 |
+
"own vehicle\n",
|
| 709 |
+
"off-grid / no electricity\n",
|
| 710 |
+
"6 days\n",
|
| 711 |
+
"\n",
|
| 712 |
+
"\n",
|
| 713 |
+
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 714 |
+
"\n",
|
| 715 |
+
"yoga / wellness retreat\n",
|
| 716 |
+
"['hiking', 'yoga']\n",
|
| 717 |
+
"cold destination / winter\n",
|
| 718 |
+
"lightweight (but comfortable)\n",
|
| 719 |
+
"casual\n",
|
| 720 |
+
"indoor\n",
|
| 721 |
+
"no own vehicle\n",
|
| 722 |
+
"snow and ice\n",
|
| 723 |
+
"7 days\n",
|
| 724 |
+
"\n",
|
| 725 |
+
"\n",
|
| 726 |
+
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 727 |
+
"\n",
|
| 728 |
+
"ski tour / skitour\n",
|
| 729 |
+
"['ski touring', 'photography']\n",
|
| 730 |
+
"cold destination / winter\n",
|
| 731 |
+
"minimalist\n",
|
| 732 |
+
"conservative\n",
|
| 733 |
+
"indoor\n",
|
| 734 |
+
"no own vehicle\n",
|
| 735 |
+
"avalanche-prone terrain\n",
|
| 736 |
+
"7+ days\n",
|
| 737 |
+
"\n",
|
| 738 |
+
"\n",
|
| 739 |
+
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 740 |
+
"\n",
|
| 741 |
+
"camping trip (wild camping)\n",
|
| 742 |
+
"['scuba diving', 'kayaking / canoeing']\n",
|
| 743 |
+
"tropical / humid\n",
|
| 744 |
+
"lightweight (but comfortable)\n",
|
| 745 |
+
"casual\n",
|
| 746 |
+
"sleeping in a tent\n",
|
| 747 |
+
"own vehicle\n",
|
| 748 |
+
"self-supported (bring your own cooking gear)\n",
|
| 749 |
+
"3 days\n",
|
| 750 |
+
"\n",
|
| 751 |
+
"\n"
|
| 752 |
+
]
|
| 753 |
+
}
|
| 754 |
+
],
|
| 755 |
+
"source": [
|
| 756 |
+
"# Load test data (list of dictionaries)\n",
|
| 757 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 758 |
+
" packing_data = json.load(file)\n",
|
| 759 |
+
" # Extract trip descriptions and classification (trip_types)\n",
|
| 760 |
+
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 761 |
+
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 762 |
+
"\n",
|
| 763 |
+
"for i, item in enumerate(trip_descriptions):\n",
|
| 764 |
+
" print(i, \".\", item, \"\\n\")\n",
|
| 765 |
+
" for elem in trip_types[i]:\n",
|
| 766 |
+
" print(elem)\n",
|
| 767 |
+
" print(\"\\n\")"
|
| 768 |
+
]
|
| 769 |
+
},
|
| 770 |
+
{
|
| 771 |
+
"cell_type": "code",
|
| 772 |
+
"execution_count": null,
|
| 773 |
+
"id": "23a306bc-5dac-4569-a727-d140e1b8da7a",
|
| 774 |
+
"metadata": {},
|
| 775 |
+
"outputs": [],
|
| 776 |
+
"source": [
|
| 777 |
+
"# "
|
| 778 |
+
]
|
| 779 |
+
}
|
| 780 |
+
],
|
| 781 |
+
"metadata": {
|
| 782 |
+
"kernelspec": {
|
| 783 |
+
"display_name": "Python (huggingface_env)",
|
| 784 |
+
"language": "python",
|
| 785 |
+
"name": "huggingface_env"
|
| 786 |
+
},
|
| 787 |
+
"language_info": {
|
| 788 |
+
"codemirror_mode": {
|
| 789 |
+
"name": "ipython",
|
| 790 |
+
"version": 3
|
| 791 |
+
},
|
| 792 |
+
"file_extension": ".py",
|
| 793 |
+
"mimetype": "text/x-python",
|
| 794 |
+
"name": "python",
|
| 795 |
+
"nbconvert_exporter": "python",
|
| 796 |
+
"pygments_lexer": "ipython3",
|
| 797 |
+
"version": "3.8.20"
|
| 798 |
+
}
|
| 799 |
+
},
|
| 800 |
+
"nbformat": 4,
|
| 801 |
+
"nbformat_minor": 5
|
| 802 |
+
}
|
space/blog_post/article.md
CHANGED
|
@@ -10,7 +10,7 @@ A cool characteristic of LLM’s, is that they can respond in a very logical and
|
|
| 10 |
|
| 11 |
## The application: Personalized Holiday packing lists
|
| 12 |
|
| 13 |
-
The goal of the project is to develop a framework using LLM’s that automatically generates packing lists based on a textual description of a trip. This description can be really short (“a weekend camping in the mountains”) or much more elaborate (“10 days tour through Iceland in summer with hikes, camping and remote workdays. I will also do strength training, yoga and sleep outside”). The model will for instance try to understand the type, destination and duration of the trip, the season, and potential extra activities or characteristics and will then
|
| 14 |
|
| 15 |
We tested several open source NLI en zero-shot models (such as BART and DeBERTa - more about them later) to see which one performs the best on several different indicators. We have found that the standard models and options are not always directly fittable to any situation, and that when you plan to ‘use an LLM in practice’ for your own application, you might need some extra steps to make the output more reliable, consistent and usable.
|
| 16 |
|
|
@@ -38,7 +38,7 @@ Many powerful LLM’s are available on platforms such as
|
|
| 38 |
|
| 39 |
This makes it possible to use existing models, compare them before using on the platforms and adjust them to your own application without having to train a model from scratch.
|
| 40 |
|
| 41 |
-
For our project we have used the
|
| 42 |
- [Transformers library](https://pypi.org/project/transformers/) in Python to test and combine models, and
|
| 43 |
- [Gradio](https://www.gradio.dev/) to build a simple user interface around it.
|
| 44 |
|
|
@@ -47,14 +47,11 @@ This means we can experiment with input and output in a visual interface, and ob
|
|
| 47 |
## Implementation of packing list model
|
| 48 |
### Set up: Anja
|
| 49 |
|
| 50 |
-
|
| 51 |
-
To be able to use the full functionality offered by Hugging Face (e.g. access to models, spaces, datasets, API access) you can create a free account on their website https://huggingface.co/.
|
| 52 |
-
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 53 |
-
|
| 54 |
|
| 55 |
To develop our model, we use the Anaconda Navigator, which includes the package and environment manager conda, as well as Jupyter Notebook for writing and running Python code. You can download the Anaconda navigator from their website https://www.anaconda.com/products/navigator. (Python is installed automatically)
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
```bash
|
| 60 |
conda create --name hf_env
|
|
@@ -68,21 +65,20 @@ pip install transformers torch numpy tabulate gradio pandas scikit-learn
|
|
| 68 |
conda install jupyter
|
| 69 |
jupyter-notebook
|
| 70 |
```
|
| 71 |
-
Create a new Jupyter Notebook for this project.
|
| 72 |
|
| 73 |
|
| 74 |
### Hugging face API
|
| 75 |
-
Let
|
| 76 |
|
| 77 |
-
To use
|
| 78 |
-
Next, save this
|
| 79 |
|
| 80 |
```text
|
| 81 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 82 |
-
```
|
| 83 |
-
, where you replace YOUR_OWN_ACCESS_TOKEN with your actual access token.
|
| 84 |
|
| 85 |
-
Now it
|
| 86 |
|
| 87 |
```python
|
| 88 |
from dotenv import load_dotenv
|
|
@@ -106,12 +102,12 @@ def query(model, input_text):
|
|
| 106 |
|
| 107 |
```
|
| 108 |
|
| 109 |
-
In this code, we first load the
|
| 110 |
|
| 111 |
-
Trying the query function with the model "facebook/bart-large-mnli"
|
| 112 |
|
| 113 |
```python
|
| 114 |
-
input_text = "I just
|
| 115 |
output = query("facebook/bart-large-mnli", input_text)
|
| 116 |
print(json.dumps(output, indent=4))
|
| 117 |
```
|
|
@@ -119,37 +115,38 @@ print(json.dumps(output, indent=4))
|
|
| 119 |
```json
|
| 120 |
[
|
| 121 |
{
|
| 122 |
-
"label": "
|
| 123 |
-
"score": 0.
|
| 124 |
},
|
| 125 |
{
|
| 126 |
"label": "health",
|
| 127 |
-
"score": 0.
|
| 128 |
},
|
| 129 |
{
|
| 130 |
-
"label": "
|
| 131 |
-
"score": 0.
|
| 132 |
},
|
| 133 |
{
|
| 134 |
"label": "politics",
|
| 135 |
-
"score": 0.
|
| 136 |
}
|
| 137 |
]
|
| 138 |
```
|
| 139 |
The scores represent the probabilities of the text belonging to a particular class label.
|
| 140 |
|
| 141 |
-
This approach worked great! However, using the API
|
|
|
|
| 142 |
|
| 143 |
|
| 144 |
### Predefine outputs/classes: Nikky
|
| 145 |
|
| 146 |
### Model implementation: Anja
|
| 147 |
|
| 148 |
-
Now we load the model locally and work with additional functionality. We import the required libraries and load our class labels from a JSON file. The last code block prints out these
|
| 149 |
|
| 150 |
```python
|
|
|
|
| 151 |
import pandas as pd
|
| 152 |
-
import matplotlib.pyplot as plt
|
| 153 |
from tabulate import tabulate
|
| 154 |
from transformers import pipeline
|
| 155 |
|
|
@@ -259,7 +256,7 @@ trip_length_days :
|
|
| 259 |
7+ days
|
| 260 |
```
|
| 261 |
|
| 262 |
-
Next, we use the pipeline function to load the model *facebook/bart-large-mnli* from Hugging Face.
|
| 263 |
|
| 264 |
```python
|
| 265 |
model_name = "facebook/bart-large-mnli"
|
|
@@ -294,7 +291,7 @@ print(df)
|
|
| 294 |
15 hut trek (winter) 0.002170
|
| 295 |
```
|
| 296 |
|
| 297 |
-
The
|
| 298 |
|
| 299 |
```python
|
| 300 |
cut_off = 0.5
|
|
@@ -339,7 +336,7 @@ print(classes)
|
|
| 339 |
['going to the beach', 'relaxing', 'hiking']
|
| 340 |
```
|
| 341 |
|
| 342 |
-
|
| 343 |
|
| 344 |
```python
|
| 345 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
@@ -374,7 +371,7 @@ print(result)
|
|
| 374 |
```
|
| 375 |
|
| 376 |
```text
|
| 377 |
-
superclass pred_class
|
| 378 |
0 activity_type beach vacation
|
| 379 |
1 activities [going to the beach, relaxing, hiking]
|
| 380 |
2 climate_or_season warm destination / summer
|
|
@@ -386,10 +383,145 @@ print(result)
|
|
| 386 |
8 trip_length_days 7+ days
|
| 387 |
```
|
| 388 |
|
| 389 |
-
And
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 390 |
|
| 391 |
### Gradio App: Anja
|
| 392 |
-
|
| 393 |
|
| 394 |
```python
|
| 395 |
import gradio as gr
|
|
@@ -397,12 +529,11 @@ import gradio as gr
|
|
| 397 |
demo = gr.Interface(
|
| 398 |
fn=pred_trip,
|
| 399 |
inputs=[
|
| 400 |
-
gr.Textbox(label="Model name", value = "
|
| 401 |
gr.Textbox(label="Trip description"),
|
| 402 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 403 |
],
|
| 404 |
-
|
| 405 |
-
outputs=[gr.Dataframe(label="DataFrame")],
|
| 406 |
title="Trip classification",
|
| 407 |
description="Enter a text describing your trip",
|
| 408 |
)
|
|
@@ -416,41 +547,51 @@ if __name__ == "__main__":
|
|
| 416 |
|
| 417 |

|
| 418 |
|
| 419 |
-
The app is now ready to take your trip description and return a list
|
| 420 |
|
| 421 |
|
| 422 |
### Share your model: Anja
|
| 423 |
**Hugging Face Spaces**
|
| 424 |
-
A simple way to share your model with others is to use Hugging Face Spaces, where you can create a free Space that can be expanded later.
|
| 425 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 426 |
|
| 427 |
```bash
|
| 428 |
# When prompted for a password, use an access token with write permissions.
|
| 429 |
# Generate one from your settings: https://huggingface.co/settings/tokens
|
| 430 |
git clone https://huggingface.co/spaces/<username>/<space_name>
|
| 431 |
```
|
| 432 |
-
As prompted, go to https://huggingface.co/settings/tokens to generate an access token. Click on *+ Create new token*, set the token type to *Write*. Give your token a name and click on *Create Token*. You
|
| 433 |
|
| 434 |
-
Next, open the command line, navigate to your project folder
|
| 435 |
|
| 436 |
```bash
|
| 437 |
-
cd path/to/
|
| 438 |
-
git
|
| 439 |
-
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 440 |
```
|
|
|
|
| 441 |
|
| 442 |
-
The Space
|
| 443 |
|
| 444 |
```python
|
| 445 |
-
from transformers import pipeline
|
| 446 |
import json
|
| 447 |
import pandas as pd
|
|
|
|
| 448 |
import gradio as gr
|
| 449 |
|
|
|
|
| 450 |
with open("packing_label_structure.json", "r") as file:
|
| 451 |
candidate_labels = json.load(file)
|
| 452 |
keys_list = list(candidate_labels.keys())
|
| 453 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 454 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 455 |
"""
|
| 456 |
Classifies trip
|
|
@@ -476,16 +617,31 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 476 |
result = classifier(trip_descr, candidate_labels[key])
|
| 477 |
classes = result["labels"][0]
|
| 478 |
df.loc[i] = [key, classes]
|
| 479 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 480 |
|
| 481 |
demo = gr.Interface(
|
| 482 |
fn=pred_trip,
|
| 483 |
inputs=[
|
| 484 |
-
gr.Textbox(label="Model name", value = "
|
| 485 |
gr.Textbox(label="Trip description"),
|
| 486 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 487 |
],
|
| 488 |
-
outputs=[gr.Dataframe(label="
|
| 489 |
title="Trip classification",
|
| 490 |
description="Enter a text describing your trip",
|
| 491 |
)
|
|
@@ -493,6 +649,7 @@ demo = gr.Interface(
|
|
| 493 |
# Launch the Gradio app
|
| 494 |
if __name__ == "__main__":
|
| 495 |
demo.launch()
|
|
|
|
| 496 |
```
|
| 497 |
|
| 498 |
Additionally, in your project folder, create a plain text file named requirements.txt.
|
|
@@ -507,12 +664,13 @@ pandas
|
|
| 507 |
gradio
|
| 508 |
```
|
| 509 |
|
| 510 |
-
After that, add, commit, and push your changes to the remote repository.
|
| 511 |
|
| 512 |
```bash
|
|
|
|
| 513 |
git add .
|
| 514 |
-
git commit -m "
|
| 515 |
-
git push origin
|
| 516 |
```
|
| 517 |
|
| 518 |
Once the push is complete, go to the URL of your Space and try it out!
|
|
@@ -523,11 +681,11 @@ https://huggingface.co/spaces/<username>/<space-name>
|
|
| 523 |
|
| 524 |
|
| 525 |
## Performance assessment: Anja
|
| 526 |
-
To
|
| 527 |
|
| 528 |
-
|
| 529 |
|
| 530 |
-
We then
|
| 531 |
|
| 532 |
```text
|
| 533 |
model accuracy true_ident false_pred
|
|
|
|
| 10 |
|
| 11 |
## The application: Personalized Holiday packing lists
|
| 12 |
|
| 13 |
+
The goal of the project is to develop a framework using LLM’s that automatically generates packing lists based on a textual description of a trip. This description can be really short (“a weekend camping in the mountains”) or much more elaborate (“10 days tour through Iceland in summer with hikes, camping and remote workdays. I will also do strength training, yoga and sleep outside”). The model will for instance try to understand the type, destination and duration of the trip, the season, and potential extra activities or characteristics and will then provide a list of potentially necessary items to pack for each trip.
|
| 14 |
|
| 15 |
We tested several open source NLI en zero-shot models (such as BART and DeBERTa - more about them later) to see which one performs the best on several different indicators. We have found that the standard models and options are not always directly fittable to any situation, and that when you plan to ‘use an LLM in practice’ for your own application, you might need some extra steps to make the output more reliable, consistent and usable.
|
| 16 |
|
|
|
|
| 38 |
|
| 39 |
This makes it possible to use existing models, compare them before using on the platforms and adjust them to your own application without having to train a model from scratch.
|
| 40 |
|
| 41 |
+
For our project we have used models from Hugging Face with the
|
| 42 |
- [Transformers library](https://pypi.org/project/transformers/) in Python to test and combine models, and
|
| 43 |
- [Gradio](https://www.gradio.dev/) to build a simple user interface around it.
|
| 44 |
|
|
|
|
| 47 |
## Implementation of packing list model
|
| 48 |
### Set up: Anja
|
| 49 |
|
| 50 |
+
To use the full functionality offered by Hugging Face (e.g. access to models, spaces, datasets, API access) you can create a free account on their website https://huggingface.co/.
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
To develop our model, we use the Anaconda Navigator, which includes the package and environment manager conda, as well as Jupyter Notebook for writing and running Python code. You can download the Anaconda navigator from their website https://www.anaconda.com/products/navigator. (Python is installed automatically)
|
| 53 |
|
| 54 |
+
After installation you can use the command line to create a new environment to work in and install the required packages. The following commands create a new environment called hf_env and activate it:
|
| 55 |
|
| 56 |
```bash
|
| 57 |
conda create --name hf_env
|
|
|
|
| 65 |
conda install jupyter
|
| 66 |
jupyter-notebook
|
| 67 |
```
|
| 68 |
+
Create a new Jupyter Notebook for this project and you are ready to start coding.
|
| 69 |
|
| 70 |
|
| 71 |
### Hugging face API
|
| 72 |
+
Let's first try out some Hugging Face models using their API. The main advantage of using the API is that you don't need to download the models locally - everything runs on Hugging Face’s servers.
|
| 73 |
|
| 74 |
+
To use the API, you'll first need to create an access token. Go to https://huggingface.co/settings/tokens and click *+ Create new token*. Choose *Read* as the token type and give it a name.
|
| 75 |
+
Next, save this token in your project folder using a .env file. Create a plain text file called .env, then add the following line (replacing YOUR_OWN_ACCESS_TOKEN with your actual access token) and save it:
|
| 76 |
|
| 77 |
```text
|
| 78 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 79 |
+
```
|
|
|
|
| 80 |
|
| 81 |
+
Now it's time to start coding and try out your first zero-shot-classification model. In your Jupyter Notebook, create a code cell and enter the following Python code:
|
| 82 |
|
| 83 |
```python
|
| 84 |
from dotenv import load_dotenv
|
|
|
|
| 102 |
|
| 103 |
```
|
| 104 |
|
| 105 |
+
In this code, we first load the required libraries and the .env file. Next, we then define a set of candidate labels for our zero-shot classification model. Finally, we implement a query function which takes a model name and an input text and returns the model's classification.
|
| 106 |
|
| 107 |
+
Trying the query function with the Hugging Face model "facebook/bart-large-mnli" and a short input text we get the following result:
|
| 108 |
|
| 109 |
```python
|
| 110 |
+
input_text = "I just started to play tennis, and it's so much fun!"
|
| 111 |
output = query("facebook/bart-large-mnli", input_text)
|
| 112 |
print(json.dumps(output, indent=4))
|
| 113 |
```
|
|
|
|
| 115 |
```json
|
| 116 |
[
|
| 117 |
{
|
| 118 |
+
"label": "sports",
|
| 119 |
+
"score": 0.9877110719680786
|
| 120 |
},
|
| 121 |
{
|
| 122 |
"label": "health",
|
| 123 |
+
"score": 0.006601463537663221
|
| 124 |
},
|
| 125 |
{
|
| 126 |
+
"label": "technology",
|
| 127 |
+
"score": 0.004392746835947037
|
| 128 |
},
|
| 129 |
{
|
| 130 |
"label": "politics",
|
| 131 |
+
"score": 0.0012947289505973458
|
| 132 |
}
|
| 133 |
]
|
| 134 |
```
|
| 135 |
The scores represent the probabilities of the text belonging to a particular class label.
|
| 136 |
|
| 137 |
+
This approach worked great! However, when using the API we ran into some limitations.
|
| 138 |
+
in particular, we were limited to 10 candidate labels for classification, which wasn't enough for our packing list model.
|
| 139 |
|
| 140 |
|
| 141 |
### Predefine outputs/classes: Nikky
|
| 142 |
|
| 143 |
### Model implementation: Anja
|
| 144 |
|
| 145 |
+
Now we load the model locally and work with additional functionality. We import the required libraries and load our class labels from a JSON file. The last code block prints out these labels, grouped into several *superclasses*. For each superclass, we will use a dedicated zero-shot classification model that returns the most likely class label. This gives us a list of relevant labels for our trip, which we later map to specific packing item lists.
|
| 146 |
|
| 147 |
```python
|
| 148 |
+
import json
|
| 149 |
import pandas as pd
|
|
|
|
| 150 |
from tabulate import tabulate
|
| 151 |
from transformers import pipeline
|
| 152 |
|
|
|
|
| 256 |
7+ days
|
| 257 |
```
|
| 258 |
|
| 259 |
+
Next, we use the pipeline function to load the model *facebook/bart-large-mnli* from Hugging Face. Then we pass the trip description together with the candidate labels for the *activity_type* superclass, to the classifier and print the results as a pandas DataFrame.
|
| 260 |
|
| 261 |
```python
|
| 262 |
model_name = "facebook/bart-large-mnli"
|
|
|
|
| 291 |
15 hut trek (winter) 0.002170
|
| 292 |
```
|
| 293 |
|
| 294 |
+
The model predicts "beach vacation" as most likely, which is correct! We then repeat this process for each superclass and pick the most likely class label for our trip, with one exception: the *activities* superclass. Because a trip can include more than one activity, we enable the multi_label option in the classifier function. This allows the text to be assigned to more than one class. Each label is evaluated independently, and the model returns a probability for whether the text belongs to that class or not. We treat any activity with a probability above 50% as a good match and include it in our final selection.
|
| 295 |
|
| 296 |
```python
|
| 297 |
cut_off = 0.5
|
|
|
|
| 336 |
['going to the beach', 'relaxing', 'hiking']
|
| 337 |
```
|
| 338 |
|
| 339 |
+
Next, we write a function that automatically runs the predictions for each superclass based on a given trip description and try it out.
|
| 340 |
|
| 341 |
```python
|
| 342 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
|
|
| 371 |
```
|
| 372 |
|
| 373 |
```text
|
| 374 |
+
Processing 9/9 superclass pred_class
|
| 375 |
0 activity_type beach vacation
|
| 376 |
1 activities [going to the beach, relaxing, hiking]
|
| 377 |
2 climate_or_season warm destination / summer
|
|
|
|
| 383 |
8 trip_length_days 7+ days
|
| 384 |
```
|
| 385 |
|
| 386 |
+
And just like that, we get the predicted labels for our trip description. For each label, we’ve put together a matching list of items to pack. Next, we simply connect those trip labels to the packing items by loading a file with packing items and adding some code to our pred_trip function:
|
| 387 |
+
|
| 388 |
+
```python
|
| 389 |
+
# Load packing item data
|
| 390 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 391 |
+
packing_items = json.load(file)
|
| 392 |
+
|
| 393 |
+
|
| 394 |
+
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 395 |
+
"""
|
| 396 |
+
Classifies trip
|
| 397 |
+
|
| 398 |
+
Parameters:
|
| 399 |
+
model_name: name of hugging-face model
|
| 400 |
+
trip_descr: text describing the trip
|
| 401 |
+
cut_off: cut_off for choosing activities
|
| 402 |
+
|
| 403 |
+
Returns:
|
| 404 |
+
pd Dataframe: with class predictions
|
| 405 |
+
"""
|
| 406 |
+
|
| 407 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 408 |
+
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 409 |
+
for i, key in enumerate(keys_list):
|
| 410 |
+
print(f"\rProcessing {i + 1}/{len(keys_list)}", end="", flush=True)
|
| 411 |
+
if key == 'activities':
|
| 412 |
+
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 413 |
+
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
| 414 |
+
classes = [result['labels'][i] for i in indices]
|
| 415 |
+
else:
|
| 416 |
+
result = classifier(trip_descr, candidate_labels[key])
|
| 417 |
+
classes = result["labels"][0]
|
| 418 |
+
df.loc[i] = [key, classes]
|
| 419 |
+
|
| 420 |
+
## Look up and return list of items to pack based on class predictions
|
| 421 |
+
# make list from dataframe column
|
| 422 |
+
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
| 423 |
+
# look up packing items for each class/key
|
| 424 |
+
list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]
|
| 425 |
+
# combine lists and remove doubble entries
|
| 426 |
+
flat_unique = []
|
| 427 |
+
for sublist in list_of_list_of_items:
|
| 428 |
+
for item in sublist:
|
| 429 |
+
if item not in flat_unique:
|
| 430 |
+
flat_unique.append(item)
|
| 431 |
+
# sort alphabetically and add newlines
|
| 432 |
+
sorted_list = "\n".join(sorted(flat_unique))
|
| 433 |
+
return df, sorted_list
|
| 434 |
+
|
| 435 |
+
result = pred_trip(model_name, trip_descr, cut_off = 0.5)
|
| 436 |
+
|
| 437 |
+
print(result[0])
|
| 438 |
+
print(result[1])
|
| 439 |
+
```
|
| 440 |
+
|
| 441 |
+
```text
|
| 442 |
+
superclass pred_class
|
| 443 |
+
0 activity_type beach vacation
|
| 444 |
+
1 activities [going to the beach, relaxing, hiking]
|
| 445 |
+
2 climate_or_season warm destination / summer
|
| 446 |
+
3 style_or_comfort minimalist
|
| 447 |
+
4 dress_code casual
|
| 448 |
+
5 accommodation huts with half board
|
| 449 |
+
6 transportation no own vehicle
|
| 450 |
+
7 special_conditions off-grid / no electricity
|
| 451 |
+
8 trip_length_days 7+ days
|
| 452 |
+
1 set of clothing for every situation
|
| 453 |
+
USB hub (for multiple devices)
|
| 454 |
+
all‑in‑one soap
|
| 455 |
+
backpack
|
| 456 |
+
backup lighting (e.g. small flashlight)
|
| 457 |
+
beach bag
|
| 458 |
+
beach chair
|
| 459 |
+
beach towel
|
| 460 |
+
blister plasters or tape
|
| 461 |
+
book or e‑reader
|
| 462 |
+
cap or hat
|
| 463 |
+
cash for payments
|
| 464 |
+
comfortable clothing
|
| 465 |
+
compact toothbrush
|
| 466 |
+
cooler
|
| 467 |
+
daypack
|
| 468 |
+
earplugs
|
| 469 |
+
emergency communication (e.g. GPS beacon or satellite messenger)
|
| 470 |
+
extra charger cables
|
| 471 |
+
extra clothing layer
|
| 472 |
+
first aid kit
|
| 473 |
+
flashlight or headlamp
|
| 474 |
+
flip flops
|
| 475 |
+
foldable solar panel (if on longer trips)
|
| 476 |
+
hat or cap
|
| 477 |
+
headlamp + extra batteries
|
| 478 |
+
hiking boots or trail runners
|
| 479 |
+
hiking poles
|
| 480 |
+
hiking socks (anti-blister)
|
| 481 |
+
jeans or comfortable pants
|
| 482 |
+
light pajamas or sleepwear
|
| 483 |
+
light shoes
|
| 484 |
+
light towel
|
| 485 |
+
lightweight clothing
|
| 486 |
+
lightweight towel
|
| 487 |
+
music / headphones
|
| 488 |
+
navigation (map/compass/GPS)
|
| 489 |
+
navigation device with offline maps
|
| 490 |
+
notebook + pen
|
| 491 |
+
number of meals/snacks matched to duration
|
| 492 |
+
packaging to keep electronics dry
|
| 493 |
+
paper map and compass
|
| 494 |
+
power bank (at least 10,000 mAh)
|
| 495 |
+
public transport app or ticket
|
| 496 |
+
rain jacket or poncho
|
| 497 |
+
rechargeable batteries and charger
|
| 498 |
+
reservation confirmation
|
| 499 |
+
seat cushion or beach mat
|
| 500 |
+
sheet liner (often required)
|
| 501 |
+
slippers or indoor shoes for inside
|
| 502 |
+
small backpack
|
| 503 |
+
small toiletry bag
|
| 504 |
+
snacks / energy bars
|
| 505 |
+
snacks for along the way
|
| 506 |
+
sneakers
|
| 507 |
+
socks per day
|
| 508 |
+
solar panel or portable charging system
|
| 509 |
+
sun hat
|
| 510 |
+
sunglasses
|
| 511 |
+
sunscreen
|
| 512 |
+
sunscreen and sunglasses
|
| 513 |
+
sweater or hoodie
|
| 514 |
+
swimwear
|
| 515 |
+
t-shirts
|
| 516 |
+
toiletry bag
|
| 517 |
+
underwear per day
|
| 518 |
+
water bottle
|
| 519 |
+
water bottle(s) or hydration bladder
|
| 520 |
+
```
|
| 521 |
+
We are ready! But let's make it a bit more user friendly in the next step!
|
| 522 |
|
| 523 |
### Gradio App: Anja
|
| 524 |
+
Gradio makes it easy to wrap machine learning functions into an interactive interface with inputs and outputs. All we need to do is pass our function pred_trip, along with the input and output formats and some default values into the gr.Interface function.
|
| 525 |
|
| 526 |
```python
|
| 527 |
import gradio as gr
|
|
|
|
| 529 |
demo = gr.Interface(
|
| 530 |
fn=pred_trip,
|
| 531 |
inputs=[
|
| 532 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 533 |
gr.Textbox(label="Trip description"),
|
| 534 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 535 |
],
|
| 536 |
+
outputs=[gr.Dataframe(label="Trip classification"), gr.Textbox(label="Items to pack")],
|
|
|
|
| 537 |
title="Trip classification",
|
| 538 |
description="Enter a text describing your trip",
|
| 539 |
)
|
|
|
|
| 547 |
|
| 548 |

|
| 549 |
|
| 550 |
+
And that's it. The app is now ready to take your trip description and return a packing list for your trip!
|
| 551 |
|
| 552 |
|
| 553 |
### Share your model: Anja
|
| 554 |
**Hugging Face Spaces**
|
| 555 |
+
A simple way to share your model with others is to use Hugging Face Spaces, where you can create a free Space that can be expanded later. Head to https://huggingface.co/spaces and click on "+ New Space". Choose
|
| 556 |
+
* SDK: Gradio
|
| 557 |
+
* Template: Blank
|
| 558 |
+
* Space hardware CPU Basic
|
| 559 |
+
Then click on "Create Space".
|
| 560 |
+
Connected to your space is a remote git repository which makes is easy to push your model code to the Space. Once your Space is created you'll see its URL along with instructions on how to set it up.
|
| 561 |
|
| 562 |
```bash
|
| 563 |
# When prompted for a password, use an access token with write permissions.
|
| 564 |
# Generate one from your settings: https://huggingface.co/settings/tokens
|
| 565 |
git clone https://huggingface.co/spaces/<username>/<space_name>
|
| 566 |
```
|
| 567 |
+
As prompted, go to https://huggingface.co/settings/tokens to generate an access token. Click on *+ Create new token*, set the token type to *Write*. Give your token a name and click on *Create Token*. You'll use this token as a password to push to your remote repository.
|
| 568 |
|
| 569 |
+
Next, open the command line, navigate to where you want your project folder and clone the remote repository:
|
| 570 |
|
| 571 |
```bash
|
| 572 |
+
cd path/to/where/you/want/the-folder
|
| 573 |
+
git clone https://huggingface.co/spaces/<username>/<space-name>
|
|
|
|
| 574 |
```
|
| 575 |
+
Put your project files into the newly created folder named <space-name>.
|
| 576 |
|
| 577 |
+
The Space automatically runs whatever model code you put in a file called app.py. In your project folder, create that file (e.g. on mac in command line: touch app.py) and open it. Copy all relevant code for your Gradio app into this file and save it.
|
| 578 |
|
| 579 |
```python
|
|
|
|
| 580 |
import json
|
| 581 |
import pandas as pd
|
| 582 |
+
from transformers import pipeline
|
| 583 |
import gradio as gr
|
| 584 |
|
| 585 |
+
# get candidate labels
|
| 586 |
with open("packing_label_structure.json", "r") as file:
|
| 587 |
candidate_labels = json.load(file)
|
| 588 |
keys_list = list(candidate_labels.keys())
|
| 589 |
|
| 590 |
+
# Load packing item data
|
| 591 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 592 |
+
packing_items = json.load(file)
|
| 593 |
+
|
| 594 |
+
|
| 595 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 596 |
"""
|
| 597 |
Classifies trip
|
|
|
|
| 617 |
result = classifier(trip_descr, candidate_labels[key])
|
| 618 |
classes = result["labels"][0]
|
| 619 |
df.loc[i] = [key, classes]
|
| 620 |
+
|
| 621 |
+
## Look up and return list of items to pack based on class predictions
|
| 622 |
+
# make list from dataframe column
|
| 623 |
+
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
| 624 |
+
# look up packing items for each class/key
|
| 625 |
+
list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]
|
| 626 |
+
# combine lists and remove doubble entries
|
| 627 |
+
flat_unique = []
|
| 628 |
+
for sublist in list_of_list_of_items:
|
| 629 |
+
for item in sublist:
|
| 630 |
+
if item not in flat_unique:
|
| 631 |
+
flat_unique.append(item)
|
| 632 |
+
# sort alphabetically and add newlines
|
| 633 |
+
sorted_list = "\n".join(sorted(flat_unique))
|
| 634 |
+
return df, sorted_list
|
| 635 |
+
|
| 636 |
|
| 637 |
demo = gr.Interface(
|
| 638 |
fn=pred_trip,
|
| 639 |
inputs=[
|
| 640 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 641 |
gr.Textbox(label="Trip description"),
|
| 642 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 643 |
],
|
| 644 |
+
outputs=[gr.Dataframe(label="Trip classification"), gr.Textbox(label="Items to pack")],
|
| 645 |
title="Trip classification",
|
| 646 |
description="Enter a text describing your trip",
|
| 647 |
)
|
|
|
|
| 649 |
# Launch the Gradio app
|
| 650 |
if __name__ == "__main__":
|
| 651 |
demo.launch()
|
| 652 |
+
|
| 653 |
```
|
| 654 |
|
| 655 |
Additionally, in your project folder, create a plain text file named requirements.txt.
|
|
|
|
| 664 |
gradio
|
| 665 |
```
|
| 666 |
|
| 667 |
+
After that, navigate to your project folder, add, commit, and push your changes to the remote repository.
|
| 668 |
|
| 669 |
```bash
|
| 670 |
+
cd <space-name>
|
| 671 |
git add .
|
| 672 |
+
git commit -m "initial commit"
|
| 673 |
+
git push origin master
|
| 674 |
```
|
| 675 |
|
| 676 |
Once the push is complete, go to the URL of your Space and try it out!
|
|
|
|
| 681 |
|
| 682 |
|
| 683 |
## Performance assessment: Anja
|
| 684 |
+
To see how well different zero-shot classification models perform, we put together a small test data set of 10 trip descriptions with corresponding class labels. We then compared 12 of the most popular zero-shot classification Models available on Hugging Face.
|
| 685 |
|
| 686 |
+
We assessed performance using accuracy (#correct classifications/#total classifications) across all superclasses, excluding the activities superclass. Since a single trip can include multiple activities, we measured the percentage of correctly identified activities (#correctly identified/#total correct activities) and the percentage of wrongly predicted activities (#falsly predicted/#total predicted activities).
|
| 687 |
|
| 688 |
+
We then calculated the average performance metrics across the test dataset for each model and ranked them based on their accuracy.
|
| 689 |
|
| 690 |
```text
|
| 691 |
model accuracy true_ident false_pred
|
space/blog_post/img/gradio_pred_trip.png
CHANGED
|
|
space/code_blog_post.ipynb
CHANGED
|
@@ -47,27 +47,27 @@
|
|
| 47 |
"text": [
|
| 48 |
"[\n",
|
| 49 |
" {\n",
|
| 50 |
-
" \"label\": \"
|
| 51 |
-
" \"score\": 0.
|
| 52 |
" },\n",
|
| 53 |
" {\n",
|
| 54 |
" \"label\": \"health\",\n",
|
| 55 |
-
" \"score\": 0.
|
| 56 |
" },\n",
|
| 57 |
" {\n",
|
| 58 |
-
" \"label\": \"
|
| 59 |
-
" \"score\": 0.
|
| 60 |
" },\n",
|
| 61 |
" {\n",
|
| 62 |
" \"label\": \"politics\",\n",
|
| 63 |
-
" \"score\": 0.
|
| 64 |
" }\n",
|
| 65 |
"]\n"
|
| 66 |
]
|
| 67 |
}
|
| 68 |
],
|
| 69 |
"source": [
|
| 70 |
-
"input_text = \"I just
|
| 71 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
"print(json.dumps(output, indent=4))"
|
| 73 |
]
|
|
@@ -256,26 +256,6 @@
|
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
"execution_count": 6,
|
| 259 |
-
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 260 |
-
"metadata": {},
|
| 261 |
-
"outputs": [
|
| 262 |
-
{
|
| 263 |
-
"name": "stdout",
|
| 264 |
-
"output_type": "stream",
|
| 265 |
-
"text": [
|
| 266 |
-
"beach vacation\n"
|
| 267 |
-
]
|
| 268 |
-
}
|
| 269 |
-
],
|
| 270 |
-
"source": [
|
| 271 |
-
"# the labels are sorted by score. We choose the first one as our best guess for a class label\n",
|
| 272 |
-
"class_label = result[\"labels\"][0]\n",
|
| 273 |
-
"print(class_label)"
|
| 274 |
-
]
|
| 275 |
-
},
|
| 276 |
-
{
|
| 277 |
-
"cell_type": "code",
|
| 278 |
-
"execution_count": 7,
|
| 279 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 280 |
"metadata": {
|
| 281 |
"scrolled": true
|
|
@@ -311,7 +291,7 @@
|
|
| 311 |
"22 skiing 0.001169\n",
|
| 312 |
"23 scuba diving 0.000789\n",
|
| 313 |
"24 ski touring 0.000491\n",
|
| 314 |
-
"[]\n"
|
| 315 |
]
|
| 316 |
}
|
| 317 |
],
|
|
@@ -319,22 +299,49 @@
|
|
| 319 |
"# we do things differently for \"activities\"\n",
|
| 320 |
"cut_off = 0.5\n",
|
| 321 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 322 |
-
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
| 323 |
"\n",
|
| 324 |
"df = pd.DataFrame({\n",
|
| 325 |
" \"Label\": result_activ[\"labels\"],\n",
|
| 326 |
" \"Score\": result_activ[\"scores\"]\n",
|
| 327 |
"})\n",
|
|
|
|
|
|
|
| 328 |
"print(df)\n",
|
| 329 |
"print(classes)"
|
| 330 |
]
|
| 331 |
},
|
| 332 |
{
|
| 333 |
"cell_type": "code",
|
| 334 |
-
"execution_count":
|
| 335 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 336 |
-
"metadata": {
|
| 337 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 338 |
"source": [
|
| 339 |
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 340 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
|
@@ -362,13 +369,16 @@
|
|
| 362 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 363 |
" classes = result[\"labels\"][0]\n",
|
| 364 |
" df.loc[i] = [key, classes]\n",
|
| 365 |
-
" return df"
|
|
|
|
|
|
|
|
|
|
| 366 |
]
|
| 367 |
},
|
| 368 |
{
|
| 369 |
"cell_type": "code",
|
| 370 |
-
"execution_count":
|
| 371 |
-
"id": "
|
| 372 |
"metadata": {},
|
| 373 |
"outputs": [
|
| 374 |
{
|
|
@@ -382,138 +392,175 @@
|
|
| 382 |
"name": "stdout",
|
| 383 |
"output_type": "stream",
|
| 384 |
"text": [
|
| 385 |
-
"Processing 9/9
|
| 386 |
-
"0 activity_type beach vacation\n",
|
| 387 |
-
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 388 |
-
"2 climate_or_season warm destination / summer\n",
|
| 389 |
-
"3 style_or_comfort minimalist\n",
|
| 390 |
-
"4 dress_code casual\n",
|
| 391 |
-
"5 accommodation huts with half board\n",
|
| 392 |
-
"6 transportation no own vehicle\n",
|
| 393 |
-
"7 special_conditions off-grid / no electricity\n",
|
| 394 |
-
"8 trip_length_days 7+ days\n"
|
| 395 |
-
]
|
| 396 |
-
}
|
| 397 |
-
],
|
| 398 |
-
"source": [
|
| 399 |
-
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 400 |
-
"print(result)"
|
| 401 |
-
]
|
| 402 |
-
},
|
| 403 |
-
{
|
| 404 |
-
"cell_type": "markdown",
|
| 405 |
-
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 406 |
-
"metadata": {},
|
| 407 |
-
"source": [
|
| 408 |
-
"Now use gradio app"
|
| 409 |
-
]
|
| 410 |
-
},
|
| 411 |
-
{
|
| 412 |
-
"cell_type": "code",
|
| 413 |
-
"execution_count": 3,
|
| 414 |
-
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 415 |
-
"metadata": {},
|
| 416 |
-
"outputs": [],
|
| 417 |
-
"source": [
|
| 418 |
-
"# Prerequisites\n",
|
| 419 |
-
"#from transformers import pipeline\n",
|
| 420 |
-
"#import json\n",
|
| 421 |
-
"#import pandas as pd\n"
|
| 422 |
-
]
|
| 423 |
-
},
|
| 424 |
-
{
|
| 425 |
-
"cell_type": "code",
|
| 426 |
-
"execution_count": 11,
|
| 427 |
-
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 428 |
-
"metadata": {},
|
| 429 |
-
"outputs": [
|
| 430 |
-
{
|
| 431 |
-
"name": "stdout",
|
| 432 |
-
"output_type": "stream",
|
| 433 |
-
"text": [
|
| 434 |
-
"Running on local URL: http://127.0.0.1:7861\n",
|
| 435 |
-
"\n",
|
| 436 |
-
"To create a public link, set `share=True` in `launch()`.\n"
|
| 437 |
]
|
| 438 |
-
},
|
| 439 |
-
{
|
| 440 |
-
"data": {
|
| 441 |
-
"text/html": [
|
| 442 |
-
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 443 |
-
],
|
| 444 |
-
"text/plain": [
|
| 445 |
-
"<IPython.core.display.HTML object>"
|
| 446 |
-
]
|
| 447 |
-
},
|
| 448 |
-
"metadata": {},
|
| 449 |
-
"output_type": "display_data"
|
| 450 |
}
|
| 451 |
],
|
| 452 |
"source": [
|
| 453 |
-
"
|
|
|
|
|
|
|
| 454 |
"\n",
|
| 455 |
-
"demo = gr.Interface(\n",
|
| 456 |
-
" fn=pred_trip,\n",
|
| 457 |
-
" inputs=[\n",
|
| 458 |
-
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 459 |
-
" gr.Textbox(label=\"Trip description\"),\n",
|
| 460 |
-
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 461 |
-
" ],\n",
|
| 462 |
-
" # outputs=\"dataframe\",\n",
|
| 463 |
-
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 464 |
-
" title=\"Trip classification\",\n",
|
| 465 |
-
" description=\"Enter a text describing your trip\",\n",
|
| 466 |
-
")\n",
|
| 467 |
"\n",
|
| 468 |
-
"
|
| 469 |
-
"
|
| 470 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 471 |
]
|
| 472 |
},
|
| 473 |
{
|
| 474 |
"cell_type": "code",
|
| 475 |
-
"execution_count":
|
| 476 |
-
"id": "
|
| 477 |
"metadata": {},
|
| 478 |
"outputs": [
|
| 479 |
-
{
|
| 480 |
-
"name": "stderr",
|
| 481 |
-
"output_type": "stream",
|
| 482 |
-
"text": [
|
| 483 |
-
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 484 |
-
]
|
| 485 |
-
},
|
| 486 |
{
|
| 487 |
"name": "stdout",
|
| 488 |
"output_type": "stream",
|
| 489 |
"text": [
|
| 490 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 491 |
]
|
| 492 |
}
|
| 493 |
],
|
| 494 |
"source": [
|
| 495 |
-
"
|
|
|
|
| 496 |
]
|
| 497 |
},
|
| 498 |
{
|
| 499 |
"cell_type": "markdown",
|
| 500 |
-
"id": "
|
| 501 |
"metadata": {},
|
| 502 |
"source": [
|
| 503 |
-
"
|
| 504 |
]
|
| 505 |
},
|
| 506 |
{
|
| 507 |
"cell_type": "code",
|
| 508 |
-
"execution_count":
|
| 509 |
-
"id": "
|
| 510 |
"metadata": {},
|
| 511 |
"outputs": [
|
| 512 |
{
|
| 513 |
"name": "stdout",
|
| 514 |
"output_type": "stream",
|
| 515 |
"text": [
|
| 516 |
-
"Running on local URL: http://127.0.0.1:
|
| 517 |
"\n",
|
| 518 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 519 |
]
|
|
@@ -521,7 +568,7 @@
|
|
| 521 |
{
|
| 522 |
"data": {
|
| 523 |
"text/html": [
|
| 524 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 525 |
],
|
| 526 |
"text/plain": [
|
| 527 |
"<IPython.core.display.HTML object>"
|
|
@@ -529,253 +576,35 @@
|
|
| 529 |
},
|
| 530 |
"metadata": {},
|
| 531 |
"output_type": "display_data"
|
| 532 |
-
},
|
| 533 |
-
{
|
| 534 |
-
"name": "stderr",
|
| 535 |
-
"output_type": "stream",
|
| 536 |
-
"text": [
|
| 537 |
-
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 538 |
-
]
|
| 539 |
-
},
|
| 540 |
-
{
|
| 541 |
-
"name": "stdout",
|
| 542 |
-
"output_type": "stream",
|
| 543 |
-
"text": [
|
| 544 |
-
"Processing 9/9"
|
| 545 |
-
]
|
| 546 |
}
|
| 547 |
],
|
| 548 |
"source": [
|
| 549 |
-
"from transformers import pipeline\n",
|
| 550 |
-
"import json\n",
|
| 551 |
-
"import pandas as pd\n",
|
| 552 |
"import gradio as gr\n",
|
| 553 |
"\n",
|
| 554 |
-
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 555 |
-
" candidate_labels = json.load(file)\n",
|
| 556 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 557 |
-
"\n",
|
| 558 |
-
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 559 |
-
" \"\"\"\n",
|
| 560 |
-
" Classifies trip\n",
|
| 561 |
-
" \n",
|
| 562 |
-
" Parameters:\n",
|
| 563 |
-
" model_name: name of hugging-face model\n",
|
| 564 |
-
" trip_descr: text describing the trip\n",
|
| 565 |
-
" cut_off: cut_off for choosing activities\n",
|
| 566 |
-
"\n",
|
| 567 |
-
" Returns:\n",
|
| 568 |
-
" pd Dataframe: with class predictions\n",
|
| 569 |
-
" \"\"\"\n",
|
| 570 |
-
" \n",
|
| 571 |
-
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 572 |
-
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 573 |
-
" for i, key in enumerate(keys_list):\n",
|
| 574 |
-
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 575 |
-
" if key == 'activities':\n",
|
| 576 |
-
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 577 |
-
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 578 |
-
" classes = [result['labels'][i] for i in indices]\n",
|
| 579 |
-
" else:\n",
|
| 580 |
-
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 581 |
-
" classes = result[\"labels\"][0]\n",
|
| 582 |
-
" df.loc[i] = [key, classes]\n",
|
| 583 |
-
" return df\n",
|
| 584 |
-
"\n",
|
| 585 |
"demo = gr.Interface(\n",
|
| 586 |
" fn=pred_trip,\n",
|
| 587 |
" inputs=[\n",
|
| 588 |
-
" gr.Textbox(label=\"Model name\", value = \"
|
| 589 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 590 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 591 |
" ],\n",
|
| 592 |
-
" outputs=[gr.Dataframe(label=\"
|
| 593 |
" title=\"Trip classification\",\n",
|
| 594 |
" description=\"Enter a text describing your trip\",\n",
|
| 595 |
")\n",
|
| 596 |
"\n",
|
| 597 |
"# Launch the Gradio app\n",
|
| 598 |
"if __name__ == \"__main__\":\n",
|
| 599 |
-
" demo.launch()
|
| 600 |
-
]
|
| 601 |
-
},
|
| 602 |
-
{
|
| 603 |
-
"cell_type": "code",
|
| 604 |
-
"execution_count": null,
|
| 605 |
-
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 606 |
-
"metadata": {},
|
| 607 |
-
"outputs": [],
|
| 608 |
-
"source": [
|
| 609 |
-
"Print test data set"
|
| 610 |
-
]
|
| 611 |
-
},
|
| 612 |
-
{
|
| 613 |
-
"cell_type": "code",
|
| 614 |
-
"execution_count": 4,
|
| 615 |
-
"id": "f25e1462-ca48-4853-ae44-dfc096c5011d",
|
| 616 |
-
"metadata": {},
|
| 617 |
-
"outputs": [
|
| 618 |
-
{
|
| 619 |
-
"name": "stdout",
|
| 620 |
-
"output_type": "stream",
|
| 621 |
-
"text": [
|
| 622 |
-
"0 . I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 623 |
-
"\n",
|
| 624 |
-
"beach vacation\n",
|
| 625 |
-
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 626 |
-
"warm destination / summer\n",
|
| 627 |
-
"lightweight (but comfortable)\n",
|
| 628 |
-
"casual\n",
|
| 629 |
-
"indoor\n",
|
| 630 |
-
"no own vehicle\n",
|
| 631 |
-
"no special conditions to consider\n",
|
| 632 |
-
"7+ days\n",
|
| 633 |
-
"\n",
|
| 634 |
-
"\n",
|
| 635 |
-
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 636 |
-
"\n",
|
| 637 |
-
"city trip\n",
|
| 638 |
-
"['sightseeing']\n",
|
| 639 |
-
"variable weather / spring / autumn\n",
|
| 640 |
-
"luxury (including evening wear)\n",
|
| 641 |
-
"casual\n",
|
| 642 |
-
"indoor\n",
|
| 643 |
-
"no own vehicle\n",
|
| 644 |
-
"no special conditions to consider\n",
|
| 645 |
-
"3 days\n",
|
| 646 |
-
"\n",
|
| 647 |
-
"\n",
|
| 648 |
-
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 649 |
-
"\n",
|
| 650 |
-
"city trip\n",
|
| 651 |
-
"['relaxing']\n",
|
| 652 |
-
"cold destination / winter\n",
|
| 653 |
-
"lightweight (but comfortable)\n",
|
| 654 |
-
"casual\n",
|
| 655 |
-
"indoor\n",
|
| 656 |
-
"no own vehicle\n",
|
| 657 |
-
"no special conditions to consider\n",
|
| 658 |
-
"7+ days\n",
|
| 659 |
-
"\n",
|
| 660 |
-
"\n",
|
| 661 |
-
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 662 |
-
"\n",
|
| 663 |
-
"cultural exploration\n",
|
| 664 |
-
"['sightseeing', 'hiking', 'rafting']\n",
|
| 665 |
-
"variable weather / spring / autumn\n",
|
| 666 |
-
"lightweight (but comfortable)\n",
|
| 667 |
-
"casual\n",
|
| 668 |
-
"indoor\n",
|
| 669 |
-
"no own vehicle\n",
|
| 670 |
-
"rainy climate\n",
|
| 671 |
-
"7+ days\n",
|
| 672 |
-
"\n",
|
| 673 |
-
"\n",
|
| 674 |
-
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 675 |
-
"\n",
|
| 676 |
-
"nature escape\n",
|
| 677 |
-
"['swimming', 'relaxing', 'hiking']\n",
|
| 678 |
-
"warm destination / summer\n",
|
| 679 |
-
"lightweight (but comfortable)\n",
|
| 680 |
-
"casual\n",
|
| 681 |
-
"indoor\n",
|
| 682 |
-
"no own vehicle\n",
|
| 683 |
-
"no special conditions to consider\n",
|
| 684 |
-
"7+ days\n",
|
| 685 |
-
"\n",
|
| 686 |
-
"\n",
|
| 687 |
-
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 688 |
-
"\n",
|
| 689 |
-
"long-distance hike / thru-hike\n",
|
| 690 |
-
"['going to the beach']\n",
|
| 691 |
-
"tropical / humid\n",
|
| 692 |
-
"minimalist\n",
|
| 693 |
-
"casual\n",
|
| 694 |
-
"huts with half board\n",
|
| 695 |
-
"own vehicle\n",
|
| 696 |
-
"off-grid / no electricity\n",
|
| 697 |
-
"6 days\n",
|
| 698 |
-
"\n",
|
| 699 |
-
"\n",
|
| 700 |
-
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 701 |
-
"\n",
|
| 702 |
-
"beach vacation\n",
|
| 703 |
-
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 704 |
-
"cold destination / winter\n",
|
| 705 |
-
"ultralight\n",
|
| 706 |
-
"casual\n",
|
| 707 |
-
"sleeping in a tent\n",
|
| 708 |
-
"own vehicle\n",
|
| 709 |
-
"off-grid / no electricity\n",
|
| 710 |
-
"6 days\n",
|
| 711 |
-
"\n",
|
| 712 |
-
"\n",
|
| 713 |
-
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 714 |
-
"\n",
|
| 715 |
-
"yoga / wellness retreat\n",
|
| 716 |
-
"['hiking', 'yoga']\n",
|
| 717 |
-
"cold destination / winter\n",
|
| 718 |
-
"lightweight (but comfortable)\n",
|
| 719 |
-
"casual\n",
|
| 720 |
-
"indoor\n",
|
| 721 |
-
"no own vehicle\n",
|
| 722 |
-
"snow and ice\n",
|
| 723 |
-
"7 days\n",
|
| 724 |
-
"\n",
|
| 725 |
-
"\n",
|
| 726 |
-
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 727 |
-
"\n",
|
| 728 |
-
"ski tour / skitour\n",
|
| 729 |
-
"['ski touring', 'photography']\n",
|
| 730 |
-
"cold destination / winter\n",
|
| 731 |
-
"minimalist\n",
|
| 732 |
-
"conservative\n",
|
| 733 |
-
"indoor\n",
|
| 734 |
-
"no own vehicle\n",
|
| 735 |
-
"avalanche-prone terrain\n",
|
| 736 |
-
"7+ days\n",
|
| 737 |
-
"\n",
|
| 738 |
-
"\n",
|
| 739 |
-
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 740 |
-
"\n",
|
| 741 |
-
"camping trip (wild camping)\n",
|
| 742 |
-
"['scuba diving', 'kayaking / canoeing']\n",
|
| 743 |
-
"tropical / humid\n",
|
| 744 |
-
"lightweight (but comfortable)\n",
|
| 745 |
-
"casual\n",
|
| 746 |
-
"sleeping in a tent\n",
|
| 747 |
-
"own vehicle\n",
|
| 748 |
-
"self-supported (bring your own cooking gear)\n",
|
| 749 |
-
"3 days\n",
|
| 750 |
-
"\n",
|
| 751 |
-
"\n"
|
| 752 |
-
]
|
| 753 |
-
}
|
| 754 |
-
],
|
| 755 |
-
"source": [
|
| 756 |
-
"# Load test data (list of dictionaries)\n",
|
| 757 |
-
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 758 |
-
" packing_data = json.load(file)\n",
|
| 759 |
-
" # Extract trip descriptions and classification (trip_types)\n",
|
| 760 |
-
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 761 |
-
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 762 |
-
"\n",
|
| 763 |
-
"for i, item in enumerate(trip_descriptions):\n",
|
| 764 |
-
" print(i, \".\", item, \"\\n\")\n",
|
| 765 |
-
" for elem in trip_types[i]:\n",
|
| 766 |
-
" print(elem)\n",
|
| 767 |
-
" print(\"\\n\")"
|
| 768 |
]
|
| 769 |
},
|
| 770 |
{
|
| 771 |
"cell_type": "code",
|
| 772 |
"execution_count": null,
|
| 773 |
-
"id": "
|
| 774 |
"metadata": {},
|
| 775 |
"outputs": [],
|
| 776 |
-
"source": [
|
| 777 |
-
"# "
|
| 778 |
-
]
|
| 779 |
}
|
| 780 |
],
|
| 781 |
"metadata": {
|
|
|
|
| 47 |
"text": [
|
| 48 |
"[\n",
|
| 49 |
" {\n",
|
| 50 |
+
" \"label\": \"sports\",\n",
|
| 51 |
+
" \"score\": 0.9877110719680786\n",
|
| 52 |
" },\n",
|
| 53 |
" {\n",
|
| 54 |
" \"label\": \"health\",\n",
|
| 55 |
+
" \"score\": 0.006601463537663221\n",
|
| 56 |
" },\n",
|
| 57 |
" {\n",
|
| 58 |
+
" \"label\": \"technology\",\n",
|
| 59 |
+
" \"score\": 0.004392746835947037\n",
|
| 60 |
" },\n",
|
| 61 |
" {\n",
|
| 62 |
" \"label\": \"politics\",\n",
|
| 63 |
+
" \"score\": 0.0012947289505973458\n",
|
| 64 |
" }\n",
|
| 65 |
"]\n"
|
| 66 |
]
|
| 67 |
}
|
| 68 |
],
|
| 69 |
"source": [
|
| 70 |
+
"input_text = \"I just started to play tennis, and it's so much fun!\"\n",
|
| 71 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
"print(json.dumps(output, indent=4))"
|
| 73 |
]
|
|
|
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
"execution_count": 6,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 259 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 260 |
"metadata": {
|
| 261 |
"scrolled": true
|
|
|
|
| 291 |
"22 skiing 0.001169\n",
|
| 292 |
"23 scuba diving 0.000789\n",
|
| 293 |
"24 ski touring 0.000491\n",
|
| 294 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 295 |
]
|
| 296 |
}
|
| 297 |
],
|
|
|
|
| 299 |
"# we do things differently for \"activities\"\n",
|
| 300 |
"cut_off = 0.5\n",
|
| 301 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
|
|
|
| 302 |
"\n",
|
| 303 |
"df = pd.DataFrame({\n",
|
| 304 |
" \"Label\": result_activ[\"labels\"],\n",
|
| 305 |
" \"Score\": result_activ[\"scores\"]\n",
|
| 306 |
"})\n",
|
| 307 |
+
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
| 308 |
+
"\n",
|
| 309 |
"print(df)\n",
|
| 310 |
"print(classes)"
|
| 311 |
]
|
| 312 |
},
|
| 313 |
{
|
| 314 |
"cell_type": "code",
|
| 315 |
+
"execution_count": 7,
|
| 316 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 317 |
+
"metadata": {
|
| 318 |
+
"scrolled": true
|
| 319 |
+
},
|
| 320 |
+
"outputs": [
|
| 321 |
+
{
|
| 322 |
+
"name": "stderr",
|
| 323 |
+
"output_type": "stream",
|
| 324 |
+
"text": [
|
| 325 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 326 |
+
]
|
| 327 |
+
},
|
| 328 |
+
{
|
| 329 |
+
"name": "stdout",
|
| 330 |
+
"output_type": "stream",
|
| 331 |
+
"text": [
|
| 332 |
+
"Processing 9/9 superclass pred_class\n",
|
| 333 |
+
"0 activity_type beach vacation\n",
|
| 334 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 335 |
+
"2 climate_or_season warm destination / summer\n",
|
| 336 |
+
"3 style_or_comfort minimalist\n",
|
| 337 |
+
"4 dress_code casual\n",
|
| 338 |
+
"5 accommodation huts with half board\n",
|
| 339 |
+
"6 transportation no own vehicle\n",
|
| 340 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 341 |
+
"8 trip_length_days 7+ days\n"
|
| 342 |
+
]
|
| 343 |
+
}
|
| 344 |
+
],
|
| 345 |
"source": [
|
| 346 |
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 347 |
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
|
|
|
| 369 |
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 370 |
" classes = result[\"labels\"][0]\n",
|
| 371 |
" df.loc[i] = [key, classes]\n",
|
| 372 |
+
" return df\n",
|
| 373 |
+
"\n",
|
| 374 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 375 |
+
"print(result)"
|
| 376 |
]
|
| 377 |
},
|
| 378 |
{
|
| 379 |
"cell_type": "code",
|
| 380 |
+
"execution_count": 8,
|
| 381 |
+
"id": "f8b547d4-1858-49d5-b26c-fd30e210cd58",
|
| 382 |
"metadata": {},
|
| 383 |
"outputs": [
|
| 384 |
{
|
|
|
|
| 392 |
"name": "stdout",
|
| 393 |
"output_type": "stream",
|
| 394 |
"text": [
|
| 395 |
+
"Processing 9/9"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 396 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 397 |
}
|
| 398 |
],
|
| 399 |
"source": [
|
| 400 |
+
"# Load packing item data\n",
|
| 401 |
+
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 402 |
+
" packing_items = json.load(file)\n",
|
| 403 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 404 |
"\n",
|
| 405 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 406 |
+
" \"\"\"\n",
|
| 407 |
+
" Classifies trip\n",
|
| 408 |
+
" \n",
|
| 409 |
+
" Parameters:\n",
|
| 410 |
+
" model_name: name of hugging-face model\n",
|
| 411 |
+
" trip_descr: text describing the trip\n",
|
| 412 |
+
" cut_off: cut_off for choosing activities\n",
|
| 413 |
+
"\n",
|
| 414 |
+
" Returns:\n",
|
| 415 |
+
" pd Dataframe: with class predictions\n",
|
| 416 |
+
" \"\"\"\n",
|
| 417 |
+
" \n",
|
| 418 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 419 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 420 |
+
" for i, key in enumerate(keys_list):\n",
|
| 421 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 422 |
+
" if key == 'activities':\n",
|
| 423 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 424 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 425 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 426 |
+
" else:\n",
|
| 427 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 428 |
+
" classes = result[\"labels\"][0]\n",
|
| 429 |
+
" df.loc[i] = [key, classes]\n",
|
| 430 |
+
" \n",
|
| 431 |
+
" ## Look up and return list of items to pack based on class predictions\n",
|
| 432 |
+
" # make list from dataframe column\n",
|
| 433 |
+
" all_classes = [elem for x in df[\"pred_class\"] for elem in (x if isinstance(x, list) else [x])]\n",
|
| 434 |
+
" # look up packing items for each class/key\n",
|
| 435 |
+
" list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]\n",
|
| 436 |
+
" # combine lists and remove doubble entries\n",
|
| 437 |
+
" flat_unique = []\n",
|
| 438 |
+
" for sublist in list_of_list_of_items:\n",
|
| 439 |
+
" for item in sublist:\n",
|
| 440 |
+
" if item not in flat_unique:\n",
|
| 441 |
+
" flat_unique.append(item)\n",
|
| 442 |
+
" # sort alphabetically and add newlines\n",
|
| 443 |
+
" sorted_list = \"\\n\".join(sorted(flat_unique)) \n",
|
| 444 |
+
" return df, sorted_list\n",
|
| 445 |
+
"\n",
|
| 446 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 447 |
]
|
| 448 |
},
|
| 449 |
{
|
| 450 |
"cell_type": "code",
|
| 451 |
+
"execution_count": 11,
|
| 452 |
+
"id": "162482dd-d2ce-44b7-9a6d-c34a96c1a6ed",
|
| 453 |
"metadata": {},
|
| 454 |
"outputs": [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 455 |
{
|
| 456 |
"name": "stdout",
|
| 457 |
"output_type": "stream",
|
| 458 |
"text": [
|
| 459 |
+
" superclass pred_class\n",
|
| 460 |
+
"0 activity_type beach vacation\n",
|
| 461 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 462 |
+
"2 climate_or_season warm destination / summer\n",
|
| 463 |
+
"3 style_or_comfort minimalist\n",
|
| 464 |
+
"4 dress_code casual\n",
|
| 465 |
+
"5 accommodation huts with half board\n",
|
| 466 |
+
"6 transportation no own vehicle\n",
|
| 467 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 468 |
+
"8 trip_length_days 7+ days\n",
|
| 469 |
+
"1 set of clothing for every situation\n",
|
| 470 |
+
"USB hub (for multiple devices)\n",
|
| 471 |
+
"all‑in‑one soap\n",
|
| 472 |
+
"backpack\n",
|
| 473 |
+
"backup lighting (e.g. small flashlight)\n",
|
| 474 |
+
"beach bag\n",
|
| 475 |
+
"beach chair\n",
|
| 476 |
+
"beach towel\n",
|
| 477 |
+
"blister plasters or tape\n",
|
| 478 |
+
"book or e‑reader\n",
|
| 479 |
+
"cap or hat\n",
|
| 480 |
+
"cash for payments\n",
|
| 481 |
+
"comfortable clothing\n",
|
| 482 |
+
"compact toothbrush\n",
|
| 483 |
+
"cooler\n",
|
| 484 |
+
"daypack\n",
|
| 485 |
+
"earplugs\n",
|
| 486 |
+
"emergency communication (e.g. GPS beacon or satellite messenger)\n",
|
| 487 |
+
"extra charger cables\n",
|
| 488 |
+
"extra clothing layer\n",
|
| 489 |
+
"first aid kit\n",
|
| 490 |
+
"flashlight or headlamp\n",
|
| 491 |
+
"flip flops\n",
|
| 492 |
+
"foldable solar panel (if on longer trips)\n",
|
| 493 |
+
"hat or cap\n",
|
| 494 |
+
"headlamp + extra batteries\n",
|
| 495 |
+
"hiking boots or trail runners\n",
|
| 496 |
+
"hiking poles\n",
|
| 497 |
+
"hiking socks (anti-blister)\n",
|
| 498 |
+
"jeans or comfortable pants\n",
|
| 499 |
+
"light pajamas or sleepwear\n",
|
| 500 |
+
"light shoes\n",
|
| 501 |
+
"light towel\n",
|
| 502 |
+
"lightweight clothing\n",
|
| 503 |
+
"lightweight towel\n",
|
| 504 |
+
"music / headphones\n",
|
| 505 |
+
"navigation (map/compass/GPS)\n",
|
| 506 |
+
"navigation device with offline maps\n",
|
| 507 |
+
"notebook + pen\n",
|
| 508 |
+
"number of meals/snacks matched to duration\n",
|
| 509 |
+
"packaging to keep electronics dry\n",
|
| 510 |
+
"paper map and compass\n",
|
| 511 |
+
"power bank (at least 10,000 mAh)\n",
|
| 512 |
+
"public transport app or ticket\n",
|
| 513 |
+
"rain jacket or poncho\n",
|
| 514 |
+
"rechargeable batteries and charger\n",
|
| 515 |
+
"reservation confirmation\n",
|
| 516 |
+
"seat cushion or beach mat\n",
|
| 517 |
+
"sheet liner (often required)\n",
|
| 518 |
+
"slippers or indoor shoes for inside\n",
|
| 519 |
+
"small backpack\n",
|
| 520 |
+
"small toiletry bag\n",
|
| 521 |
+
"snacks / energy bars\n",
|
| 522 |
+
"snacks for along the way\n",
|
| 523 |
+
"sneakers\n",
|
| 524 |
+
"socks per day\n",
|
| 525 |
+
"solar panel or portable charging system\n",
|
| 526 |
+
"sun hat\n",
|
| 527 |
+
"sunglasses\n",
|
| 528 |
+
"sunscreen\n",
|
| 529 |
+
"sunscreen and sunglasses\n",
|
| 530 |
+
"sweater or hoodie\n",
|
| 531 |
+
"swimwear\n",
|
| 532 |
+
"t-shirts\n",
|
| 533 |
+
"toiletry bag\n",
|
| 534 |
+
"underwear per day\n",
|
| 535 |
+
"water bottle\n",
|
| 536 |
+
"water bottle(s) or hydration bladder\n"
|
| 537 |
]
|
| 538 |
}
|
| 539 |
],
|
| 540 |
"source": [
|
| 541 |
+
"print(result[0])\n",
|
| 542 |
+
"print(result[1])"
|
| 543 |
]
|
| 544 |
},
|
| 545 |
{
|
| 546 |
"cell_type": "markdown",
|
| 547 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 548 |
"metadata": {},
|
| 549 |
"source": [
|
| 550 |
+
"Now use gradio app"
|
| 551 |
]
|
| 552 |
},
|
| 553 |
{
|
| 554 |
"cell_type": "code",
|
| 555 |
+
"execution_count": 9,
|
| 556 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 557 |
"metadata": {},
|
| 558 |
"outputs": [
|
| 559 |
{
|
| 560 |
"name": "stdout",
|
| 561 |
"output_type": "stream",
|
| 562 |
"text": [
|
| 563 |
+
"Running on local URL: http://127.0.0.1:7860\n",
|
| 564 |
"\n",
|
| 565 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 566 |
]
|
|
|
|
| 568 |
{
|
| 569 |
"data": {
|
| 570 |
"text/html": [
|
| 571 |
+
"<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 572 |
],
|
| 573 |
"text/plain": [
|
| 574 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 576 |
},
|
| 577 |
"metadata": {},
|
| 578 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 579 |
}
|
| 580 |
],
|
| 581 |
"source": [
|
|
|
|
|
|
|
|
|
|
| 582 |
"import gradio as gr\n",
|
| 583 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 584 |
"demo = gr.Interface(\n",
|
| 585 |
" fn=pred_trip,\n",
|
| 586 |
" inputs=[\n",
|
| 587 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 588 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 589 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 590 |
" ],\n",
|
| 591 |
+
" outputs=[gr.Dataframe(label=\"Trip classification\"), gr.Textbox(label=\"Items to pack\")],\n",
|
| 592 |
" title=\"Trip classification\",\n",
|
| 593 |
" description=\"Enter a text describing your trip\",\n",
|
| 594 |
")\n",
|
| 595 |
"\n",
|
| 596 |
"# Launch the Gradio app\n",
|
| 597 |
"if __name__ == \"__main__\":\n",
|
| 598 |
+
" demo.launch()"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 599 |
]
|
| 600 |
},
|
| 601 |
{
|
| 602 |
"cell_type": "code",
|
| 603 |
"execution_count": null,
|
| 604 |
+
"id": "0aa38a02-266c-4628-8f90-0d198d2f9897",
|
| 605 |
"metadata": {},
|
| 606 |
"outputs": [],
|
| 607 |
+
"source": []
|
|
|
|
|
|
|
| 608 |
}
|
| 609 |
],
|
| 610 |
"metadata": {
|
space/main_model.ipynb
CHANGED
|
@@ -1561,13 +1561,7 @@
|
|
| 1561 |
"df_all = pd.concat(perf_table, ignore_index=True)\n",
|
| 1562 |
"df = df_all.sort_values(by=\"accuracy\", ascending=False).reset_index(drop=True)\n",
|
| 1563 |
"\n",
|
| 1564 |
-
"
|
| 1565 |
-
"\n",
|
| 1566 |
-
"print(df)\n",
|
| 1567 |
-
"#print(type(df_all))\n",
|
| 1568 |
-
" \n",
|
| 1569 |
-
"\n",
|
| 1570 |
-
"# rank by accuracy\n"
|
| 1571 |
]
|
| 1572 |
},
|
| 1573 |
{
|
|
@@ -1582,7 +1576,6 @@
|
|
| 1582 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 1583 |
" packing_items = json.load(file)\n",
|
| 1584 |
"\n",
|
| 1585 |
-
"# function and gradio app\n",
|
| 1586 |
"def classify(model_name, trip_descr, cut_off = 0.5):\n",
|
| 1587 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 1588 |
" ## Create and fill dataframe with class predictions\n",
|
|
|
|
| 1561 |
"df_all = pd.concat(perf_table, ignore_index=True)\n",
|
| 1562 |
"df = df_all.sort_values(by=\"accuracy\", ascending=False).reset_index(drop=True)\n",
|
| 1563 |
"\n",
|
| 1564 |
+
"print(df)"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1565 |
]
|
| 1566 |
},
|
| 1567 |
{
|
|
|
|
| 1576 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 1577 |
" packing_items = json.load(file)\n",
|
| 1578 |
"\n",
|
|
|
|
| 1579 |
"def classify(model_name, trip_descr, cut_off = 0.5):\n",
|
| 1580 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 1581 |
" ## Create and fill dataframe with class predictions\n",
|
space/space/space/blog_post/article.md
CHANGED
|
@@ -1,30 +1,60 @@
|
|
| 1 |
# Working with Large Language Models
|
| 2 |
## Introduction: Nikky
|
| 3 |
-
|
| 4 |
-
|
| 5 |
|
| 6 |
## Why bother to adapt models to your application: Nikky
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
## Implementation of packing list model
|
| 17 |
### Set up: Anja
|
| 18 |
-
**Hugging face account**
|
| 19 |
|
| 20 |
Hugging Face is a company and platform for the machine learning community to collaborate on models, datasets and applications, especially in the field of natural language processing.
|
| 21 |
-
To be able to use the full functionality offered by Hugging Face (e.g.
|
| 22 |
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 23 |
|
| 24 |
-
**Anaconda navigator**
|
| 25 |
-
To program our model, we use the anaconda navigator with package and environment manager conda, as well as Jupyter Notebook to write our python code. You can download the Anaconda navigator [here](https://www.anaconda.com/products/navigator). (python is automatically installed)
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
| 28 |
|
| 29 |
```bash
|
| 30 |
conda create --name hf_env
|
|
@@ -32,6 +62,7 @@ conda activate hf_env
|
|
| 32 |
```
|
| 33 |
|
| 34 |
Next, install the libraries used in this project and set up Jupyter Notebook.
|
|
|
|
| 35 |
```bash
|
| 36 |
pip install transformers torch numpy tabulate gradio pandas scikit-learn
|
| 37 |
conda install jupyter
|
|
@@ -39,17 +70,19 @@ jupyter-notebook
|
|
| 39 |
```
|
| 40 |
Create a new Jupyter Notebook for this project.
|
| 41 |
|
|
|
|
| 42 |
### Hugging face API
|
| 43 |
-
Let
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
|
|
|
| 47 |
```text
|
| 48 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 49 |
```
|
| 50 |
-
where you replace YOUR_OWN_ACCESS_TOKEN with your
|
| 51 |
|
| 52 |
-
Now it
|
| 53 |
|
| 54 |
```python
|
| 55 |
from dotenv import load_dotenv
|
|
@@ -57,7 +90,7 @@ import os
|
|
| 57 |
import requests
|
| 58 |
import json
|
| 59 |
|
| 60 |
-
load_dotenv()
|
| 61 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
| 62 |
|
| 63 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
|
@@ -73,7 +106,9 @@ def query(model, input_text):
|
|
| 73 |
|
| 74 |
```
|
| 75 |
|
| 76 |
-
In
|
|
|
|
|
|
|
| 77 |
|
| 78 |
```python
|
| 79 |
input_text = "I just bought a new laptop, and it works amazing!"
|
|
@@ -101,9 +136,9 @@ print(json.dumps(output, indent=4))
|
|
| 101 |
}
|
| 102 |
]
|
| 103 |
```
|
| 104 |
-
The scores
|
| 105 |
|
| 106 |
-
This worked great! However, using API the functionality is limited. We were limited to 10 candidate labels for our classification
|
| 107 |
|
| 108 |
|
| 109 |
### Predefine outputs/classes: Nikky
|
|
@@ -386,16 +421,17 @@ The app is now ready to take your trip description and return a list of predicte
|
|
| 386 |
|
| 387 |
### Share your model: Anja
|
| 388 |
**Hugging Face Spaces**
|
| 389 |
-
A simple way to share your model with others is to use Hugging Face Spaces, where you can create a free Space that can expanded later. Go to https://huggingface.co/spaces and click on "+ New Space", as SDK choose Gradio
|
| 390 |
-
Connected to your space is a remote git repository which is a smooth way to push your model code to the Space. Once the
|
| 391 |
|
| 392 |
```bash
|
| 393 |
# When prompted for a password, use an access token with write permissions.
|
| 394 |
# Generate one from your settings: https://huggingface.co/settings/tokens
|
| 395 |
git clone https://huggingface.co/spaces/<username>/<space_name>
|
| 396 |
```
|
|
|
|
| 397 |
|
| 398 |
-
|
| 399 |
|
| 400 |
```bash
|
| 401 |
cd path/to/your/project
|
|
@@ -403,18 +439,14 @@ git init
|
|
| 403 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 404 |
```
|
| 405 |
|
| 406 |
-
|
| 407 |
-
|
| 408 |
-
The Space will automatically run the model code from a app.py file. In your project folder create the file app.py (e.g. on mac in command line: touch app.py) and open it. We need to copy all relevant code for our gradio app in it:
|
| 409 |
|
| 410 |
```python
|
| 411 |
-
# Prerequisites
|
| 412 |
from transformers import pipeline
|
| 413 |
import json
|
| 414 |
import pandas as pd
|
| 415 |
import gradio as gr
|
| 416 |
|
| 417 |
-
# Get candidate labels
|
| 418 |
with open("packing_label_structure.json", "r") as file:
|
| 419 |
candidate_labels = json.load(file)
|
| 420 |
keys_list = list(candidate_labels.keys())
|
|
@@ -429,7 +461,7 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 429 |
cut_off: cut_off for choosing activities
|
| 430 |
|
| 431 |
Returns:
|
| 432 |
-
pd Dataframe: with class predictions
|
| 433 |
"""
|
| 434 |
|
| 435 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
|
@@ -449,11 +481,10 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 449 |
demo = gr.Interface(
|
| 450 |
fn=pred_trip,
|
| 451 |
inputs=[
|
| 452 |
-
gr.Textbox(label="Model name", value = "
|
| 453 |
gr.Textbox(label="Trip description"),
|
| 454 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 455 |
],
|
| 456 |
-
# outputs="dataframe",
|
| 457 |
outputs=[gr.Dataframe(label="DataFrame")],
|
| 458 |
title="Trip classification",
|
| 459 |
description="Enter a text describing your trip",
|
|
@@ -464,7 +495,9 @@ if __name__ == "__main__":
|
|
| 464 |
demo.launch()
|
| 465 |
```
|
| 466 |
|
| 467 |
-
Additionally
|
|
|
|
|
|
|
| 468 |
|
| 469 |
```text
|
| 470 |
transformers
|
|
@@ -474,37 +507,27 @@ pandas
|
|
| 474 |
gradio
|
| 475 |
```
|
| 476 |
|
| 477 |
-
After
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 478 |
|
| 479 |
```text
|
| 480 |
https://huggingface.co/spaces/<username>/<space-name>
|
| 481 |
```
|
| 482 |
|
| 483 |
|
| 484 |
-
|
| 485 |
## Performance assessment: Anja
|
| 486 |
-
To
|
| 487 |
-
|
| 488 |
-
Let's first have a look at three of our test data trip descriptions and true class labels:
|
| 489 |
-
|
| 490 |
-
|
| 491 |
-
```text
|
| 492 |
-
I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days.
|
| 493 |
-
|
| 494 |
-
long-distance hike / thru-hike
|
| 495 |
-
['going to the beach']
|
| 496 |
-
tropical / humid
|
| 497 |
-
minimalist
|
| 498 |
-
casual
|
| 499 |
-
huts with half board
|
| 500 |
-
own vehicle
|
| 501 |
-
off-grid / no electricity
|
| 502 |
-
6 days
|
| 503 |
-
```
|
| 504 |
|
| 505 |
-
|
| 506 |
|
| 507 |
-
We computed
|
| 508 |
|
| 509 |
```text
|
| 510 |
model accuracy true_ident false_pred
|
|
|
|
| 1 |
# Working with Large Language Models
|
| 2 |
## Introduction: Nikky
|
| 3 |
+
|
| 4 |
+
Nowadays many people regularly use AI chatbots. These chatbots “understand” text and talk with you like they are humans with an above average level of knowledge (…when they are not hallucinating or answering falsely ;-) ). Behind such bots and systems is usually a Large Language Model (LLM). This is an AI-model trained on enormous amount of texts, so that it can recognize the meaning and logic of language.
|
| 5 |
|
| 6 |
## Why bother to adapt models to your application: Nikky
|
| 7 |
+
### What are LLM’s used for in practice?
|
| 8 |
+
|
| 9 |
+
A cool characteristic of LLM’s, is that they can respond in a very logical and natural way. That is why they are currently used more and more to analyze text, but also to organize information and even to generate creative ideas. AI agents are now also gaining very much in popularity, these agents are using LLM’s and automatization to learn and automatize processes. Since we were interested in their potential on a real life application, we tested how we can use LLM’s in providing personalized packing lists to know what to take on your holiday, adventure travel and/or business trip.
|
| 10 |
+
|
| 11 |
+
## The application: Personalized Holiday packing lists
|
| 12 |
+
|
| 13 |
+
The goal of the project is to develop a framework using LLM’s that automatically generates packing lists based on a textual description of a trip. This description can be really short (“a weekend camping in the mountains”) or much more elaborate (“10 days tour through Iceland in summer with hikes, camping and remote workdays. I will also do strength training, yoga and sleep outside”). The model will for instance try to understand the type, destination and duration of the trip, the season, and potential extra activities or characteristics and will then provice a list of potentially necessary items to pack for each trip.
|
| 14 |
+
|
| 15 |
+
We tested several open source NLI en zero-shot models (such as BART and DeBERTa - more about them later) to see which one performs the best on several different indicators. We have found that the standard models and options are not always directly fittable to any situation, and that when you plan to ‘use an LLM in practice’ for your own application, you might need some extra steps to make the output more reliable, consistent and usable.
|
| 16 |
+
|
| 17 |
+
In this post, we hope to outline all the steps we took and share our search for the best approach and model. We will share the code we used and the considerations we had on the way, to provide you with the tools to start applying LLM’s for your own projects and applications.
|
| 18 |
+
|
| 19 |
+
### Different types of (open-source) LLM’s
|
| 20 |
+
|
| 21 |
+
So many different LLM’s exist, with each of them having strengths and weaknesses. Some well known models such as GPT, Claude, Gemeni, LlaMa and Mistral are used for text generation and chats/conversations. For applications such as our project, where a model should understand text and than provide a list based on categories that are taken from the provided description, classification and NLI models such as BART, DeBERTa and RoBERTa can be more suitable. These type of classification models are better in recognizing the meaning and then giving a fixed set of most probable categories as outputs.
|
| 22 |
+
|
| 23 |
+
### Output: Free text or classification?
|
| 24 |
+
|
| 25 |
+
When you use a normal text-generating LLM, there are many free parameters, and packing lists outcome can then become very chaotic and unstable over time. This is why a classification model can help in providing users with well structured packing lists with the necessary items for their trip. Of course, this choice highly depends on what you would like as a final output for your application. In case you want a conversation, the text generating models would be of better use. Since we really want to provide the user with a list of items based on categories derived from the trip description, using classification models as the base for the framework seemed to be more suitable for our application.
|
| 26 |
+
|
| 27 |
+
### Trainability of LLM’s
|
| 28 |
+
|
| 29 |
+
The existing LLM’s as mentioned in the above introduction can be either used “off-the-shelf” without modification, or you can provide them with extra information regarding your specific application. If you use an text-generating LLM, as a chatbot for your customer service, you can train the model by giving more information about your specific return policy, for instance. In case of the zero-shot models, you may specify categories specific to your application, so that you will always receive the estimates for that set of categories you’re interested in.
|
| 30 |
+
|
| 31 |
+
### Open-source platforms and models
|
| 32 |
+
|
| 33 |
+
Many powerful LLM’s are available on platforms such as
|
| 34 |
+
- [Hugging Face](https://huggingface.co/)
|
| 35 |
+
- [Replicate](https://replicate.com/)
|
| 36 |
+
- [ModelScope](https://modelscope.cn/)
|
| 37 |
+
- [OpenRouter](https://openrouter.ai/)
|
| 38 |
+
|
| 39 |
+
This makes it possible to use existing models, compare them before using on the platforms and adjust them to your own application without having to train a model from scratch.
|
| 40 |
+
|
| 41 |
+
For our project we have used the
|
| 42 |
+
- [Transformers library](https://pypi.org/project/transformers/) in Python to test and combine models, and
|
| 43 |
+
- [Gradio](https://www.gradio.dev/) to build a simple user interface around it.
|
| 44 |
+
|
| 45 |
+
This means we can experiment with input and output in a visual interface, and observe the output of our models directly in an app.
|
| 46 |
|
| 47 |
## Implementation of packing list model
|
| 48 |
### Set up: Anja
|
|
|
|
| 49 |
|
| 50 |
Hugging Face is a company and platform for the machine learning community to collaborate on models, datasets and applications, especially in the field of natural language processing.
|
| 51 |
+
To be able to use the full functionality offered by Hugging Face (e.g. access to models, spaces, datasets, API access) you can create a free account on their website https://huggingface.co/.
|
| 52 |
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 53 |
|
|
|
|
|
|
|
| 54 |
|
| 55 |
+
To develop our model, we use the Anaconda Navigator, which includes the package and environment manager conda, as well as Jupyter Notebook for writing and running Python code. You can download the Anaconda navigator from their website https://www.anaconda.com/products/navigator. (Python is installed automatically)
|
| 56 |
+
|
| 57 |
+
Using the command line, you can create a new environment to work in and install the required packages. The following commands create a new environment called hf_env and activate it:
|
| 58 |
|
| 59 |
```bash
|
| 60 |
conda create --name hf_env
|
|
|
|
| 62 |
```
|
| 63 |
|
| 64 |
Next, install the libraries used in this project and set up Jupyter Notebook.
|
| 65 |
+
|
| 66 |
```bash
|
| 67 |
pip install transformers torch numpy tabulate gradio pandas scikit-learn
|
| 68 |
conda install jupyter
|
|
|
|
| 70 |
```
|
| 71 |
Create a new Jupyter Notebook for this project.
|
| 72 |
|
| 73 |
+
|
| 74 |
### Hugging face API
|
| 75 |
+
Let us first try out some Hugging Face models using their API. The main advantage of using the API is that you do not need to download the models locally and all computations are handled on Hugging Face servers.
|
| 76 |
+
|
| 77 |
+
To use their API you first need to create an access token. Go to https://huggingface.co/settings/tokens and click on *+ Create new token*. Select as token type *Read* and give your token a name.
|
| 78 |
+
Next, save this access token in your project folder within a .env file. Create a plain text file named .env, then add and save the following line inside it:
|
| 79 |
+
|
| 80 |
```text
|
| 81 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 82 |
```
|
| 83 |
+
, where you replace YOUR_OWN_ACCESS_TOKEN with your actual access token.
|
| 84 |
|
| 85 |
+
Now it is time to start coding and try out your first zero-shot-classification model. In your Jupyter Notebook, create a code cell and enter the following Python code:
|
| 86 |
|
| 87 |
```python
|
| 88 |
from dotenv import load_dotenv
|
|
|
|
| 90 |
import requests
|
| 91 |
import json
|
| 92 |
|
| 93 |
+
load_dotenv()
|
| 94 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
| 95 |
|
| 96 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
|
|
|
| 106 |
|
| 107 |
```
|
| 108 |
|
| 109 |
+
In this code, we first load the necessary libraries and then the .env file. Next, we then define a set of candidate labels for our zero-shot classification model and create a query function which receives a model name and an input text and returns the model's classification.
|
| 110 |
+
|
| 111 |
+
Trying the query function with the model "facebook/bart-large-mnli" from Hugging Face and a short input text we get the following result:
|
| 112 |
|
| 113 |
```python
|
| 114 |
input_text = "I just bought a new laptop, and it works amazing!"
|
|
|
|
| 136 |
}
|
| 137 |
]
|
| 138 |
```
|
| 139 |
+
The scores represent the probabilities of the text belonging to a particular class label.
|
| 140 |
|
| 141 |
+
This approach worked great! However, using the API the functionality is limited. We were limited to 10 candidate labels for our classification, which was not sufficient for our packing list model.
|
| 142 |
|
| 143 |
|
| 144 |
### Predefine outputs/classes: Nikky
|
|
|
|
| 421 |
|
| 422 |
### Share your model: Anja
|
| 423 |
**Hugging Face Spaces**
|
| 424 |
+
A simple way to share your model with others is to use Hugging Face Spaces, where you can create a free Space that can be expanded later. Go to https://huggingface.co/spaces and click on "+ New Space", as SDK choose Gradio, as template Blank, as Space hardware choose "CPU Basic", and click on "Create Space" to create your Space.
|
| 425 |
+
Connected to your space is a remote git repository which is a smooth way to push your model code to the Space. Once the Space is created you will see the url of your Space and some instructions of how to set it up.
|
| 426 |
|
| 427 |
```bash
|
| 428 |
# When prompted for a password, use an access token with write permissions.
|
| 429 |
# Generate one from your settings: https://huggingface.co/settings/tokens
|
| 430 |
git clone https://huggingface.co/spaces/<username>/<space_name>
|
| 431 |
```
|
| 432 |
+
As prompted, go to https://huggingface.co/settings/tokens to generate an access token. Click on *+ Create new token*, set the token type to *Write*. Give your token a name and click on *Create Token*. You will use this token as a password to push to your remote repository.
|
| 433 |
|
| 434 |
+
Next, open the command line, navigate to your project folder, initialize git and connect it to the remote repository.
|
| 435 |
|
| 436 |
```bash
|
| 437 |
cd path/to/your/project
|
|
|
|
| 439 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 440 |
```
|
| 441 |
|
| 442 |
+
The Space will automatically run the model code from a file named app.py. In your project folder, create this file (e.g. on mac in command line: touch app.py) and open it. Copy all relevant code for your Gradio app into this file and save it.
|
|
|
|
|
|
|
| 443 |
|
| 444 |
```python
|
|
|
|
| 445 |
from transformers import pipeline
|
| 446 |
import json
|
| 447 |
import pandas as pd
|
| 448 |
import gradio as gr
|
| 449 |
|
|
|
|
| 450 |
with open("packing_label_structure.json", "r") as file:
|
| 451 |
candidate_labels = json.load(file)
|
| 452 |
keys_list = list(candidate_labels.keys())
|
|
|
|
| 461 |
cut_off: cut_off for choosing activities
|
| 462 |
|
| 463 |
Returns:
|
| 464 |
+
pd Dataframe: with class predictions
|
| 465 |
"""
|
| 466 |
|
| 467 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
|
|
|
| 481 |
demo = gr.Interface(
|
| 482 |
fn=pred_trip,
|
| 483 |
inputs=[
|
| 484 |
+
gr.Textbox(label="Model name", value = "facebook/bart-large-mnli"),
|
| 485 |
gr.Textbox(label="Trip description"),
|
| 486 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 487 |
],
|
|
|
|
| 488 |
outputs=[gr.Dataframe(label="DataFrame")],
|
| 489 |
title="Trip classification",
|
| 490 |
description="Enter a text describing your trip",
|
|
|
|
| 495 |
demo.launch()
|
| 496 |
```
|
| 497 |
|
| 498 |
+
Additionally, in your project folder, create a plain text file named requirements.txt.
|
| 499 |
+
This file tells Hugging Face which Python dependencies to install before running your app.
|
| 500 |
+
Add the following lines to your requirements.txt file and save it:
|
| 501 |
|
| 502 |
```text
|
| 503 |
transformers
|
|
|
|
| 507 |
gradio
|
| 508 |
```
|
| 509 |
|
| 510 |
+
After that, add, commit, and push your changes to the remote repository.
|
| 511 |
+
|
| 512 |
+
```bash
|
| 513 |
+
git add .
|
| 514 |
+
git commit -m "Initial commit"
|
| 515 |
+
git push origin main
|
| 516 |
+
```
|
| 517 |
+
|
| 518 |
+
Once the push is complete, go to the URL of your Space and try it out!
|
| 519 |
|
| 520 |
```text
|
| 521 |
https://huggingface.co/spaces/<username>/<space-name>
|
| 522 |
```
|
| 523 |
|
| 524 |
|
|
|
|
| 525 |
## Performance assessment: Anja
|
| 526 |
+
To evaluate the performance of different zero-shot classification models, we manually created a small test data set of 10 trip descriptions with corresponding class labels. We compared 12 of the most popular zero-shot classification Models available on Hugging Face.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 527 |
|
| 528 |
+
Performance was assessed in terms of accuracy (#correct classifications/#total classifications) for all superclasses, excluding the activities superclass. Since more than one type of activity can be correct for a single trip, we use the percentage of correctly identified activities (#correctly identified/#total correct activities) and the percentage of wrongly predicted activities (#falsly predicted/#total predicted activities) to asses its performance.
|
| 529 |
|
| 530 |
+
We then computed the average performance measures (across the test dataset) for each model and ranked them by accuracy.
|
| 531 |
|
| 532 |
```text
|
| 533 |
model accuracy true_ident false_pred
|
space/space/space/code_blog_post.ipynb
CHANGED
|
@@ -20,7 +20,7 @@
|
|
| 20 |
"import requests\n",
|
| 21 |
"import json\n",
|
| 22 |
"\n",
|
| 23 |
-
"load_dotenv()
|
| 24 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 25 |
"\n",
|
| 26 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
|
@@ -546,13 +546,11 @@
|
|
| 546 |
}
|
| 547 |
],
|
| 548 |
"source": [
|
| 549 |
-
"# Prerequisites\n",
|
| 550 |
"from transformers import pipeline\n",
|
| 551 |
"import json\n",
|
| 552 |
"import pandas as pd\n",
|
| 553 |
"import gradio as gr\n",
|
| 554 |
"\n",
|
| 555 |
-
"# Get candidate labels\n",
|
| 556 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 557 |
" candidate_labels = json.load(file)\n",
|
| 558 |
"keys_list = list(candidate_labels.keys())\n",
|
|
@@ -567,7 +565,7 @@
|
|
| 567 |
" cut_off: cut_off for choosing activities\n",
|
| 568 |
"\n",
|
| 569 |
" Returns:\n",
|
| 570 |
-
" pd Dataframe: with class predictions
|
| 571 |
" \"\"\"\n",
|
| 572 |
" \n",
|
| 573 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
|
@@ -587,11 +585,10 @@
|
|
| 587 |
"demo = gr.Interface(\n",
|
| 588 |
" fn=pred_trip,\n",
|
| 589 |
" inputs=[\n",
|
| 590 |
-
" gr.Textbox(label=\"Model name\", value = \"
|
| 591 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 592 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 593 |
" ],\n",
|
| 594 |
-
" # outputs=\"dataframe\",\n",
|
| 595 |
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 596 |
" title=\"Trip classification\",\n",
|
| 597 |
" description=\"Enter a text describing your trip\",\n",
|
|
|
|
| 20 |
"import requests\n",
|
| 21 |
"import json\n",
|
| 22 |
"\n",
|
| 23 |
+
"load_dotenv() \n",
|
| 24 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 25 |
"\n",
|
| 26 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
|
|
|
| 546 |
}
|
| 547 |
],
|
| 548 |
"source": [
|
|
|
|
| 549 |
"from transformers import pipeline\n",
|
| 550 |
"import json\n",
|
| 551 |
"import pandas as pd\n",
|
| 552 |
"import gradio as gr\n",
|
| 553 |
"\n",
|
|
|
|
| 554 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 555 |
" candidate_labels = json.load(file)\n",
|
| 556 |
"keys_list = list(candidate_labels.keys())\n",
|
|
|
|
| 565 |
" cut_off: cut_off for choosing activities\n",
|
| 566 |
"\n",
|
| 567 |
" Returns:\n",
|
| 568 |
+
" pd Dataframe: with class predictions\n",
|
| 569 |
" \"\"\"\n",
|
| 570 |
" \n",
|
| 571 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
|
|
|
| 585 |
"demo = gr.Interface(\n",
|
| 586 |
" fn=pred_trip,\n",
|
| 587 |
" inputs=[\n",
|
| 588 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 589 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 590 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 591 |
" ],\n",
|
|
|
|
| 592 |
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 593 |
" title=\"Trip classification\",\n",
|
| 594 |
" description=\"Enter a text describing your trip\",\n",
|
space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
CHANGED
|
@@ -5,44 +5,85 @@
|
|
| 5 |
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
"metadata": {},
|
| 7 |
"source": [
|
| 8 |
-
"
|
| 9 |
]
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
-
"id": "
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
-
"import
|
| 19 |
-
"import pickle\n",
|
| 20 |
"import os\n",
|
| 21 |
-
"import
|
| 22 |
-
"import matplotlib.pyplot as plt\n",
|
| 23 |
-
"from tabulate import tabulate\n",
|
| 24 |
-
"\n",
|
| 25 |
-
"from transformers import pipeline\n",
|
| 26 |
"import json\n",
|
| 27 |
-
"import pandas as pd\n",
|
| 28 |
"\n",
|
| 29 |
-
"#
|
| 30 |
-
"
|
| 31 |
-
" candidate_labels = json.load(file)\n",
|
| 32 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 33 |
"\n",
|
| 34 |
-
"
|
| 35 |
-
"
|
| 36 |
-
"
|
| 37 |
-
"
|
| 38 |
-
"
|
| 39 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
]
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
-
"execution_count":
|
| 45 |
-
"id": "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
| 48 |
{
|
|
@@ -147,15 +188,24 @@
|
|
| 147 |
}
|
| 148 |
],
|
| 149 |
"source": [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 150 |
"for key in candidate_labels:\n",
|
| 151 |
" print(\"\\n\", key, \":\")\n",
|
| 152 |
" for item in candidate_labels[key]:\n",
|
| 153 |
-
" print(\"\\t\", item)
|
| 154 |
]
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
@@ -191,12 +241,11 @@
|
|
| 191 |
}
|
| 192 |
],
|
| 193 |
"source": [
|
| 194 |
-
"key = keys_list[0]\n",
|
| 195 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 196 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 197 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 198 |
-
"result = classifier(trip_descr, candidate_labels[
|
| 199 |
-
"
|
| 200 |
"df = pd.DataFrame({\n",
|
| 201 |
" \"Label\": result[\"labels\"],\n",
|
| 202 |
" \"Score\": result[\"scores\"]\n",
|
|
@@ -206,7 +255,7 @@
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
-
"execution_count":
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
@@ -226,9 +275,11 @@
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
-
"execution_count":
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
-
"metadata": {
|
|
|
|
|
|
|
| 232 |
"outputs": [
|
| 233 |
{
|
| 234 |
"name": "stdout",
|
|
@@ -260,16 +311,15 @@
|
|
| 260 |
"22 skiing 0.001169\n",
|
| 261 |
"23 scuba diving 0.000789\n",
|
| 262 |
"24 ski touring 0.000491\n",
|
| 263 |
-
"[
|
| 264 |
]
|
| 265 |
}
|
| 266 |
],
|
| 267 |
"source": [
|
| 268 |
-
"# we do
|
| 269 |
"cut_off = 0.5\n",
|
| 270 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 271 |
-
"
|
| 272 |
-
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 273 |
"\n",
|
| 274 |
"df = pd.DataFrame({\n",
|
| 275 |
" \"Label\": result_activ[\"labels\"],\n",
|
|
@@ -281,7 +331,7 @@
|
|
| 281 |
},
|
| 282 |
{
|
| 283 |
"cell_type": "code",
|
| 284 |
-
"execution_count":
|
| 285 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 286 |
"metadata": {},
|
| 287 |
"outputs": [],
|
|
@@ -297,13 +347,13 @@
|
|
| 297 |
" cut_off: cut_off for choosing activities\n",
|
| 298 |
"\n",
|
| 299 |
" Returns:\n",
|
| 300 |
-
" pd Dataframe: with class predictions
|
| 301 |
" \"\"\"\n",
|
| 302 |
" \n",
|
| 303 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 304 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 305 |
" for i, key in enumerate(keys_list):\n",
|
| 306 |
-
"
|
| 307 |
" if key == 'activities':\n",
|
| 308 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 309 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
@@ -317,7 +367,7 @@
|
|
| 317 |
},
|
| 318 |
{
|
| 319 |
"cell_type": "code",
|
| 320 |
-
"execution_count":
|
| 321 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 322 |
"metadata": {},
|
| 323 |
"outputs": [
|
|
@@ -332,7 +382,7 @@
|
|
| 332 |
"name": "stdout",
|
| 333 |
"output_type": "stream",
|
| 334 |
"text": [
|
| 335 |
-
" superclass pred_class\n",
|
| 336 |
"0 activity_type beach vacation\n",
|
| 337 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 338 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -360,21 +410,20 @@
|
|
| 360 |
},
|
| 361 |
{
|
| 362 |
"cell_type": "code",
|
| 363 |
-
"execution_count":
|
| 364 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 365 |
"metadata": {},
|
| 366 |
"outputs": [],
|
| 367 |
"source": [
|
| 368 |
"# Prerequisites\n",
|
| 369 |
-
"from transformers import pipeline\n",
|
| 370 |
-
"import json\n",
|
| 371 |
-
"import pandas as pd\n"
|
| 372 |
-
"import gradio as gr"
|
| 373 |
]
|
| 374 |
},
|
| 375 |
{
|
| 376 |
"cell_type": "code",
|
| 377 |
-
"execution_count":
|
| 378 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 379 |
"metadata": {},
|
| 380 |
"outputs": [
|
|
@@ -382,7 +431,7 @@
|
|
| 382 |
"name": "stdout",
|
| 383 |
"output_type": "stream",
|
| 384 |
"text": [
|
| 385 |
-
"Running on local URL: http://127.0.0.1:
|
| 386 |
"\n",
|
| 387 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 388 |
]
|
|
@@ -390,7 +439,7 @@
|
|
| 390 |
{
|
| 391 |
"data": {
|
| 392 |
"text/html": [
|
| 393 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 394 |
],
|
| 395 |
"text/plain": [
|
| 396 |
"<IPython.core.display.HTML object>"
|
|
@@ -401,10 +450,12 @@
|
|
| 401 |
}
|
| 402 |
],
|
| 403 |
"source": [
|
|
|
|
|
|
|
| 404 |
"demo = gr.Interface(\n",
|
| 405 |
" fn=pred_trip,\n",
|
| 406 |
" inputs=[\n",
|
| 407 |
-
" gr.Textbox(label=\"Model name\", value = \"
|
| 408 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 409 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 410 |
" ],\n",
|
|
@@ -495,13 +546,11 @@
|
|
| 495 |
}
|
| 496 |
],
|
| 497 |
"source": [
|
| 498 |
-
"# Prerequisites\n",
|
| 499 |
"from transformers import pipeline\n",
|
| 500 |
"import json\n",
|
| 501 |
"import pandas as pd\n",
|
| 502 |
"import gradio as gr\n",
|
| 503 |
"\n",
|
| 504 |
-
"# Get candidate labels\n",
|
| 505 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 506 |
" candidate_labels = json.load(file)\n",
|
| 507 |
"keys_list = list(candidate_labels.keys())\n",
|
|
@@ -516,7 +565,7 @@
|
|
| 516 |
" cut_off: cut_off for choosing activities\n",
|
| 517 |
"\n",
|
| 518 |
" Returns:\n",
|
| 519 |
-
" pd Dataframe: with class predictions
|
| 520 |
" \"\"\"\n",
|
| 521 |
" \n",
|
| 522 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
|
@@ -536,11 +585,10 @@
|
|
| 536 |
"demo = gr.Interface(\n",
|
| 537 |
" fn=pred_trip,\n",
|
| 538 |
" inputs=[\n",
|
| 539 |
-
" gr.Textbox(label=\"Model name\", value = \"
|
| 540 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 541 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 542 |
" ],\n",
|
| 543 |
-
" # outputs=\"dataframe\",\n",
|
| 544 |
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 545 |
" title=\"Trip classification\",\n",
|
| 546 |
" description=\"Enter a text describing your trip\",\n",
|
|
@@ -725,7 +773,9 @@
|
|
| 725 |
"id": "23a306bc-5dac-4569-a727-d140e1b8da7a",
|
| 726 |
"metadata": {},
|
| 727 |
"outputs": [],
|
| 728 |
-
"source": [
|
|
|
|
|
|
|
| 729 |
}
|
| 730 |
],
|
| 731 |
"metadata": {
|
|
|
|
| 5 |
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
"metadata": {},
|
| 7 |
"source": [
|
| 8 |
+
"**Hugging Face API**"
|
| 9 |
]
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 2,
|
| 14 |
+
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
+
"from dotenv import load_dotenv\n",
|
|
|
|
| 19 |
"import os\n",
|
| 20 |
+
"import requests\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
"import json\n",
|
|
|
|
| 22 |
"\n",
|
| 23 |
+
"load_dotenv() # Load environment variables from .env file\n",
|
| 24 |
+
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
|
|
|
|
|
|
| 25 |
"\n",
|
| 26 |
+
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 27 |
+
"\n",
|
| 28 |
+
"def query(model, input_text):\n",
|
| 29 |
+
" API_URL = f\"https://router.huggingface.co/hf-inference/models/{model}\"\n",
|
| 30 |
+
" payload = {\n",
|
| 31 |
+
" \"inputs\": input_text,\n",
|
| 32 |
+
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
| 33 |
+
" }\n",
|
| 34 |
+
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 35 |
+
" return response.json()"
|
| 36 |
]
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
+
"execution_count": 3,
|
| 41 |
+
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
+
"metadata": {},
|
| 43 |
+
"outputs": [
|
| 44 |
+
{
|
| 45 |
+
"name": "stdout",
|
| 46 |
+
"output_type": "stream",
|
| 47 |
+
"text": [
|
| 48 |
+
"[\n",
|
| 49 |
+
" {\n",
|
| 50 |
+
" \"label\": \"technology\",\n",
|
| 51 |
+
" \"score\": 0.970917284488678\n",
|
| 52 |
+
" },\n",
|
| 53 |
+
" {\n",
|
| 54 |
+
" \"label\": \"health\",\n",
|
| 55 |
+
" \"score\": 0.014999152161180973\n",
|
| 56 |
+
" },\n",
|
| 57 |
+
" {\n",
|
| 58 |
+
" \"label\": \"sports\",\n",
|
| 59 |
+
" \"score\": 0.008272469975054264\n",
|
| 60 |
+
" },\n",
|
| 61 |
+
" {\n",
|
| 62 |
+
" \"label\": \"politics\",\n",
|
| 63 |
+
" \"score\": 0.005811101291328669\n",
|
| 64 |
+
" }\n",
|
| 65 |
+
"]\n"
|
| 66 |
+
]
|
| 67 |
+
}
|
| 68 |
+
],
|
| 69 |
+
"source": [
|
| 70 |
+
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 71 |
+
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
+
"print(json.dumps(output, indent=4))"
|
| 73 |
+
]
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"cell_type": "markdown",
|
| 77 |
+
"id": "01dc84d7-6d0f-4409-8966-c1adac5fd0c8",
|
| 78 |
+
"metadata": {},
|
| 79 |
+
"source": [
|
| 80 |
+
"**Model implementation**"
|
| 81 |
+
]
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"cell_type": "code",
|
| 85 |
+
"execution_count": 4,
|
| 86 |
+
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
| 89 |
{
|
|
|
|
| 188 |
}
|
| 189 |
],
|
| 190 |
"source": [
|
| 191 |
+
"import json\n",
|
| 192 |
+
"import pandas as pd\n",
|
| 193 |
+
"from tabulate import tabulate\n",
|
| 194 |
+
"from transformers import pipeline\n",
|
| 195 |
+
"\n",
|
| 196 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 197 |
+
" candidate_labels = json.load(file)\n",
|
| 198 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 199 |
+
"\n",
|
| 200 |
"for key in candidate_labels:\n",
|
| 201 |
" print(\"\\n\", key, \":\")\n",
|
| 202 |
" for item in candidate_labels[key]:\n",
|
| 203 |
+
" print(\"\\t\", item)"
|
| 204 |
]
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
+
"execution_count": 5,
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
|
|
| 241 |
}
|
| 242 |
],
|
| 243 |
"source": [
|
|
|
|
| 244 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 245 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 246 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 247 |
+
"result = classifier(trip_descr, candidate_labels[\"activity_type\"])\n",
|
| 248 |
+
"\n",
|
| 249 |
"df = pd.DataFrame({\n",
|
| 250 |
" \"Label\": result[\"labels\"],\n",
|
| 251 |
" \"Score\": result[\"scores\"]\n",
|
|
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
+
"execution_count": 6,
|
| 259 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 260 |
"metadata": {},
|
| 261 |
"outputs": [
|
|
|
|
| 275 |
},
|
| 276 |
{
|
| 277 |
"cell_type": "code",
|
| 278 |
+
"execution_count": 7,
|
| 279 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 280 |
+
"metadata": {
|
| 281 |
+
"scrolled": true
|
| 282 |
+
},
|
| 283 |
"outputs": [
|
| 284 |
{
|
| 285 |
"name": "stdout",
|
|
|
|
| 311 |
"22 skiing 0.001169\n",
|
| 312 |
"23 scuba diving 0.000789\n",
|
| 313 |
"24 ski touring 0.000491\n",
|
| 314 |
+
"[]\n"
|
| 315 |
]
|
| 316 |
}
|
| 317 |
],
|
| 318 |
"source": [
|
| 319 |
+
"# we do things differently for \"activities\"\n",
|
| 320 |
"cut_off = 0.5\n",
|
| 321 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 322 |
+
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
|
|
|
| 323 |
"\n",
|
| 324 |
"df = pd.DataFrame({\n",
|
| 325 |
" \"Label\": result_activ[\"labels\"],\n",
|
|
|
|
| 331 |
},
|
| 332 |
{
|
| 333 |
"cell_type": "code",
|
| 334 |
+
"execution_count": 8,
|
| 335 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 336 |
"metadata": {},
|
| 337 |
"outputs": [],
|
|
|
|
| 347 |
" cut_off: cut_off for choosing activities\n",
|
| 348 |
"\n",
|
| 349 |
" Returns:\n",
|
| 350 |
+
" pd Dataframe: with class predictions\n",
|
| 351 |
" \"\"\"\n",
|
| 352 |
" \n",
|
| 353 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 354 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 355 |
" for i, key in enumerate(keys_list):\n",
|
| 356 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 357 |
" if key == 'activities':\n",
|
| 358 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 359 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
|
|
| 367 |
},
|
| 368 |
{
|
| 369 |
"cell_type": "code",
|
| 370 |
+
"execution_count": 9,
|
| 371 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 372 |
"metadata": {},
|
| 373 |
"outputs": [
|
|
|
|
| 382 |
"name": "stdout",
|
| 383 |
"output_type": "stream",
|
| 384 |
"text": [
|
| 385 |
+
"Processing 9/9 superclass pred_class\n",
|
| 386 |
"0 activity_type beach vacation\n",
|
| 387 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 388 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 410 |
},
|
| 411 |
{
|
| 412 |
"cell_type": "code",
|
| 413 |
+
"execution_count": 3,
|
| 414 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 415 |
"metadata": {},
|
| 416 |
"outputs": [],
|
| 417 |
"source": [
|
| 418 |
"# Prerequisites\n",
|
| 419 |
+
"#from transformers import pipeline\n",
|
| 420 |
+
"#import json\n",
|
| 421 |
+
"#import pandas as pd\n"
|
|
|
|
| 422 |
]
|
| 423 |
},
|
| 424 |
{
|
| 425 |
"cell_type": "code",
|
| 426 |
+
"execution_count": 11,
|
| 427 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 428 |
"metadata": {},
|
| 429 |
"outputs": [
|
|
|
|
| 431 |
"name": "stdout",
|
| 432 |
"output_type": "stream",
|
| 433 |
"text": [
|
| 434 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 435 |
"\n",
|
| 436 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 437 |
]
|
|
|
|
| 439 |
{
|
| 440 |
"data": {
|
| 441 |
"text/html": [
|
| 442 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 443 |
],
|
| 444 |
"text/plain": [
|
| 445 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 450 |
}
|
| 451 |
],
|
| 452 |
"source": [
|
| 453 |
+
"import gradio as gr\n",
|
| 454 |
+
"\n",
|
| 455 |
"demo = gr.Interface(\n",
|
| 456 |
" fn=pred_trip,\n",
|
| 457 |
" inputs=[\n",
|
| 458 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 459 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 460 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 461 |
" ],\n",
|
|
|
|
| 546 |
}
|
| 547 |
],
|
| 548 |
"source": [
|
|
|
|
| 549 |
"from transformers import pipeline\n",
|
| 550 |
"import json\n",
|
| 551 |
"import pandas as pd\n",
|
| 552 |
"import gradio as gr\n",
|
| 553 |
"\n",
|
|
|
|
| 554 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 555 |
" candidate_labels = json.load(file)\n",
|
| 556 |
"keys_list = list(candidate_labels.keys())\n",
|
|
|
|
| 565 |
" cut_off: cut_off for choosing activities\n",
|
| 566 |
"\n",
|
| 567 |
" Returns:\n",
|
| 568 |
+
" pd Dataframe: with class predictions\n",
|
| 569 |
" \"\"\"\n",
|
| 570 |
" \n",
|
| 571 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
|
|
|
| 585 |
"demo = gr.Interface(\n",
|
| 586 |
" fn=pred_trip,\n",
|
| 587 |
" inputs=[\n",
|
| 588 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 589 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 590 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 591 |
" ],\n",
|
|
|
|
| 592 |
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 593 |
" title=\"Trip classification\",\n",
|
| 594 |
" description=\"Enter a text describing your trip\",\n",
|
|
|
|
| 773 |
"id": "23a306bc-5dac-4569-a727-d140e1b8da7a",
|
| 774 |
"metadata": {},
|
| 775 |
"outputs": [],
|
| 776 |
+
"source": [
|
| 777 |
+
"# "
|
| 778 |
+
]
|
| 779 |
}
|
| 780 |
],
|
| 781 |
"metadata": {
|
space/space/space/space/blog_post/img/gradio_pred_trip.png
CHANGED
|
|
space/space/space/space/space/blog_post/article.md
CHANGED
|
@@ -73,7 +73,7 @@ def query(model, input_text):
|
|
| 73 |
|
| 74 |
```
|
| 75 |
|
| 76 |
-
In it we first load necessary libraries
|
| 77 |
|
| 78 |
```python
|
| 79 |
input_text = "I just bought a new laptop, and it works amazing!"
|
|
@@ -82,45 +82,42 @@ print(json.dumps(output, indent=4))
|
|
| 82 |
```
|
| 83 |
|
| 84 |
```json
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
"
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
"
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
0.008272469975054264
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
|
|
|
|
|
|
|
|
|
| 100 |
```
|
| 101 |
-
The scores contain
|
| 102 |
|
| 103 |
-
This worked great! However, using API the functionality is limited. We were limited to 10 candidate labels for our
|
| 104 |
|
| 105 |
|
| 106 |
### Predefine outputs/classes: Nikky
|
| 107 |
|
| 108 |
### Model implementation: Anja
|
| 109 |
|
| 110 |
-
Now we
|
| 111 |
|
| 112 |
```python
|
| 113 |
-
import math
|
| 114 |
-
import json
|
| 115 |
-
import pickle
|
| 116 |
-
import os
|
| 117 |
-
import time
|
| 118 |
import pandas as pd
|
| 119 |
import matplotlib.pyplot as plt
|
| 120 |
from tabulate import tabulate
|
| 121 |
from transformers import pipeline
|
| 122 |
|
| 123 |
-
# Get candidate labels
|
| 124 |
with open("packing_label_structure.json", "r") as file:
|
| 125 |
candidate_labels = json.load(file)
|
| 126 |
keys_list = list(candidate_labels.keys())
|
|
@@ -227,14 +224,13 @@ trip_length_days :
|
|
| 227 |
7+ days
|
| 228 |
```
|
| 229 |
|
| 230 |
-
|
| 231 |
|
| 232 |
```python
|
| 233 |
-
key = keys_list[0]
|
| 234 |
model_name = "facebook/bart-large-mnli"
|
| 235 |
trip_descr = "I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands."
|
| 236 |
classifier = pipeline("zero-shot-classification", model = model_name)
|
| 237 |
-
result = classifier(trip_descr, candidate_labels[
|
| 238 |
|
| 239 |
df = pd.DataFrame({
|
| 240 |
"Label": result["labels"],
|
|
@@ -263,17 +259,16 @@ print(df)
|
|
| 263 |
15 hut trek (winter) 0.002170
|
| 264 |
```
|
| 265 |
|
| 266 |
-
The most likely activity type our model predicted is beach vacation, which is correct! Now we will do this for every superclass and
|
| 267 |
|
| 268 |
```python
|
| 269 |
cut_off = 0.5
|
| 270 |
result_activ = classifier(trip_descr, candidate_labels["activities"], multi_label=True)
|
| 271 |
-
|
| 272 |
-
classes = [result_activ['labels'][i] for i in indices]
|
| 273 |
|
| 274 |
df = pd.DataFrame({
|
| 275 |
-
"Label":
|
| 276 |
-
"Score":
|
| 277 |
})
|
| 278 |
print(df)
|
| 279 |
print(classes)
|
|
@@ -309,10 +304,9 @@ print(classes)
|
|
| 309 |
['going to the beach', 'relaxing', 'hiking']
|
| 310 |
```
|
| 311 |
|
| 312 |
-
|
| 313 |
|
| 314 |
```python
|
| 315 |
-
# doing this for all superclasses, depending on local machine this might take a while
|
| 316 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 317 |
"""
|
| 318 |
Classifies trip
|
|
@@ -323,7 +317,7 @@ def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
|
| 323 |
cut_off: cut_off for choosing activities
|
| 324 |
|
| 325 |
Returns:
|
| 326 |
-
pd Dataframe: with class predictions
|
| 327 |
"""
|
| 328 |
|
| 329 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
|
@@ -357,27 +351,18 @@ print(result)
|
|
| 357 |
8 trip_length_days 7+ days
|
| 358 |
```
|
| 359 |
|
| 360 |
-
And
|
| 361 |
|
| 362 |
-
###
|
| 363 |
-
|
| 364 |
|
| 365 |
```python
|
| 366 |
-
# Prerequisites
|
| 367 |
import gradio as gr
|
| 368 |
|
| 369 |
-
|
| 370 |
-
with open("packing_label_structure.json", "r") as file:
|
| 371 |
-
candidate_labels = json.load(file)
|
| 372 |
-
keys_list = list(candidate_labels.keys())
|
| 373 |
-
|
| 374 |
-
```
|
| 375 |
-
|
| 376 |
-
```python
|
| 377 |
-
emo = gr.Interface(
|
| 378 |
fn=pred_trip,
|
| 379 |
inputs=[
|
| 380 |
-
gr.Textbox(label="Model name", value = "
|
| 381 |
gr.Textbox(label="Trip description"),
|
| 382 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 383 |
],
|
|
@@ -393,12 +378,15 @@ if __name__ == "__main__":
|
|
| 393 |
|
| 394 |
```
|
| 395 |
|
|
|
|
| 396 |

|
| 397 |
|
|
|
|
|
|
|
| 398 |
|
| 399 |
### Share your model: Anja
|
| 400 |
**Hugging Face Spaces**
|
| 401 |
-
A
|
| 402 |
Connected to your space is a remote git repository which is a smooth way to push your model code to the Space. Once the Sapce is created you will see the url of your Space and some instructions of how to set it up.
|
| 403 |
|
| 404 |
```bash
|
|
|
|
| 73 |
|
| 74 |
```
|
| 75 |
|
| 76 |
+
In it we first load necessary libraries, then the .env file. We then create some candidate labels for our zero-shot-classification model and write a query function which receives a model name and an input text and returns its classification. Trying the query function with the model "facebook/bart-large-mnli" from Hugging Face and a short input text we get the following:
|
| 77 |
|
| 78 |
```python
|
| 79 |
input_text = "I just bought a new laptop, and it works amazing!"
|
|
|
|
| 82 |
```
|
| 83 |
|
| 84 |
```json
|
| 85 |
+
[
|
| 86 |
+
{
|
| 87 |
+
"label": "technology",
|
| 88 |
+
"score": 0.970917284488678
|
| 89 |
+
},
|
| 90 |
+
{
|
| 91 |
+
"label": "health",
|
| 92 |
+
"score": 0.014999152161180973
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"label": "sports",
|
| 96 |
+
"score": 0.008272469975054264
|
| 97 |
+
},
|
| 98 |
+
{
|
| 99 |
+
"label": "politics",
|
| 100 |
+
"score": 0.005811101291328669
|
| 101 |
+
}
|
| 102 |
+
]
|
| 103 |
```
|
| 104 |
+
The scores contain probabilities of the text belonging to a particular class label.
|
| 105 |
|
| 106 |
+
This worked great! However, using API the functionality is limited. We were limited to 10 candidate labels for our classification. This was not sufficient for our packing list model.
|
| 107 |
|
| 108 |
|
| 109 |
### Predefine outputs/classes: Nikky
|
| 110 |
|
| 111 |
### Model implementation: Anja
|
| 112 |
|
| 113 |
+
Now we load the model locally and work with additional functionality. We import the required libraries and load our class labels from a JSON file. The last code block prints out these classes, sorted into several *superclasses*. For each superclass, we will use a dedicated zero-shot classification model and therefore get a list of relevant class labels for out trip.
|
| 114 |
|
| 115 |
```python
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
import pandas as pd
|
| 117 |
import matplotlib.pyplot as plt
|
| 118 |
from tabulate import tabulate
|
| 119 |
from transformers import pipeline
|
| 120 |
|
|
|
|
| 121 |
with open("packing_label_structure.json", "r") as file:
|
| 122 |
candidate_labels = json.load(file)
|
| 123 |
keys_list = list(candidate_labels.keys())
|
|
|
|
| 224 |
7+ days
|
| 225 |
```
|
| 226 |
|
| 227 |
+
Next, we use the pipeline function to load the model *facebook/bart-large-mnli* from Hugging Face. After that, we pass the trip description, along with the candidate labels for the *activity_type* superclass, to the classifier and print the output as a pandas DataFrame.
|
| 228 |
|
| 229 |
```python
|
|
|
|
| 230 |
model_name = "facebook/bart-large-mnli"
|
| 231 |
trip_descr = "I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands."
|
| 232 |
classifier = pipeline("zero-shot-classification", model = model_name)
|
| 233 |
+
result = classifier(trip_descr, candidate_labels["activity_type"])
|
| 234 |
|
| 235 |
df = pd.DataFrame({
|
| 236 |
"Label": result["labels"],
|
|
|
|
| 259 |
15 hut trek (winter) 0.002170
|
| 260 |
```
|
| 261 |
|
| 262 |
+
The most likely activity type our model predicted is "beach vacation", which is correct! Now we will do this for every superclass and choose the most likely class label for our trip, except for the *activities* superclass. Because it is possible and likely to engaeg in more than one activity during a trip, we enable the multi_label otion within the classifier function. This means that the text can belong to more than one class. For this, each class label is evaluated independently and a probability of belonging to that class (vs not belonging) is returned. The activities that we select as our best guess are those with a probability of more than 50 percent.
|
| 263 |
|
| 264 |
```python
|
| 265 |
cut_off = 0.5
|
| 266 |
result_activ = classifier(trip_descr, candidate_labels["activities"], multi_label=True)
|
| 267 |
+
classes = df.loc[df["Score"] > 0.5, "Label"].tolist()
|
|
|
|
| 268 |
|
| 269 |
df = pd.DataFrame({
|
| 270 |
+
"Label": result_activ["labels"],
|
| 271 |
+
"Score": result_activ["scores"]
|
| 272 |
})
|
| 273 |
print(df)
|
| 274 |
print(classes)
|
|
|
|
| 304 |
['going to the beach', 'relaxing', 'hiking']
|
| 305 |
```
|
| 306 |
|
| 307 |
+
We now write a function that automatically performs all predictions for each superclass based on a given trip description and try it out.
|
| 308 |
|
| 309 |
```python
|
|
|
|
| 310 |
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 311 |
"""
|
| 312 |
Classifies trip
|
|
|
|
| 317 |
cut_off: cut_off for choosing activities
|
| 318 |
|
| 319 |
Returns:
|
| 320 |
+
pd Dataframe: with class predictions
|
| 321 |
"""
|
| 322 |
|
| 323 |
classifier = pipeline("zero-shot-classification", model=model_name)
|
|
|
|
| 351 |
8 trip_length_days 7+ days
|
| 352 |
```
|
| 353 |
|
| 354 |
+
And with that, we obtain the predicted labels for our trip description.
|
| 355 |
|
| 356 |
+
### Gradio App: Anja
|
| 357 |
+
Next, let's use the Gradio library to wrap our classification function in an interactive interface with inputs and outputs. We pass our function pred_trip, along with the input and output formats and some default values, to the gr.Interface function.
|
| 358 |
|
| 359 |
```python
|
|
|
|
| 360 |
import gradio as gr
|
| 361 |
|
| 362 |
+
demo = gr.Interface(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 363 |
fn=pred_trip,
|
| 364 |
inputs=[
|
| 365 |
+
gr.Textbox(label="Model name", value = "facebook/bart-large-mnli"),
|
| 366 |
gr.Textbox(label="Trip description"),
|
| 367 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 368 |
],
|
|
|
|
| 378 |
|
| 379 |
```
|
| 380 |
|
| 381 |
+
|
| 382 |

|
| 383 |
|
| 384 |
+
The app is now ready to take your trip description and return a list of predicted class labels for your trip.
|
| 385 |
+
|
| 386 |
|
| 387 |
### Share your model: Anja
|
| 388 |
**Hugging Face Spaces**
|
| 389 |
+
A simple way to share your model with others is to use Hugging Face Spaces, where you can create a free Space that can expanded later. Go to https://huggingface.co/spaces and click on "+ New Space", as SDK choose Gradio and as template Blank, as Space hardware choose "CPU Basic", and click on "Create Space" to create your Space.
|
| 390 |
Connected to your space is a remote git repository which is a smooth way to push your model code to the Space. Once the Sapce is created you will see the url of your Space and some instructions of how to set it up.
|
| 391 |
|
| 392 |
```bash
|
space/space/space/space/space/code_blog_post.ipynb
CHANGED
|
@@ -5,44 +5,85 @@
|
|
| 5 |
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
"metadata": {},
|
| 7 |
"source": [
|
| 8 |
-
"
|
| 9 |
]
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
-
"id": "
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
-
"import
|
| 19 |
-
"import pickle\n",
|
| 20 |
"import os\n",
|
| 21 |
-
"import
|
| 22 |
-
"import matplotlib.pyplot as plt\n",
|
| 23 |
-
"from tabulate import tabulate\n",
|
| 24 |
-
"\n",
|
| 25 |
-
"from transformers import pipeline\n",
|
| 26 |
"import json\n",
|
| 27 |
-
"import pandas as pd\n",
|
| 28 |
"\n",
|
| 29 |
-
"#
|
| 30 |
-
"
|
| 31 |
-
" candidate_labels = json.load(file)\n",
|
| 32 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 33 |
"\n",
|
| 34 |
-
"
|
| 35 |
-
"
|
| 36 |
-
"
|
| 37 |
-
"
|
| 38 |
-
"
|
| 39 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
]
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
-
"execution_count":
|
| 45 |
-
"id": "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
| 48 |
{
|
|
@@ -147,15 +188,24 @@
|
|
| 147 |
}
|
| 148 |
],
|
| 149 |
"source": [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 150 |
"for key in candidate_labels:\n",
|
| 151 |
" print(\"\\n\", key, \":\")\n",
|
| 152 |
" for item in candidate_labels[key]:\n",
|
| 153 |
-
" print(\"\\t\", item)
|
| 154 |
]
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
@@ -191,12 +241,11 @@
|
|
| 191 |
}
|
| 192 |
],
|
| 193 |
"source": [
|
| 194 |
-
"key = keys_list[0]\n",
|
| 195 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 196 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 197 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 198 |
-
"result = classifier(trip_descr, candidate_labels[
|
| 199 |
-
"
|
| 200 |
"df = pd.DataFrame({\n",
|
| 201 |
" \"Label\": result[\"labels\"],\n",
|
| 202 |
" \"Score\": result[\"scores\"]\n",
|
|
@@ -206,7 +255,7 @@
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
-
"execution_count":
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
@@ -226,9 +275,11 @@
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
-
"execution_count":
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
-
"metadata": {
|
|
|
|
|
|
|
| 232 |
"outputs": [
|
| 233 |
{
|
| 234 |
"name": "stdout",
|
|
@@ -260,16 +311,15 @@
|
|
| 260 |
"22 skiing 0.001169\n",
|
| 261 |
"23 scuba diving 0.000789\n",
|
| 262 |
"24 ski touring 0.000491\n",
|
| 263 |
-
"[
|
| 264 |
]
|
| 265 |
}
|
| 266 |
],
|
| 267 |
"source": [
|
| 268 |
-
"# we do
|
| 269 |
"cut_off = 0.5\n",
|
| 270 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 271 |
-
"
|
| 272 |
-
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 273 |
"\n",
|
| 274 |
"df = pd.DataFrame({\n",
|
| 275 |
" \"Label\": result_activ[\"labels\"],\n",
|
|
@@ -281,7 +331,7 @@
|
|
| 281 |
},
|
| 282 |
{
|
| 283 |
"cell_type": "code",
|
| 284 |
-
"execution_count":
|
| 285 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 286 |
"metadata": {},
|
| 287 |
"outputs": [],
|
|
@@ -297,13 +347,13 @@
|
|
| 297 |
" cut_off: cut_off for choosing activities\n",
|
| 298 |
"\n",
|
| 299 |
" Returns:\n",
|
| 300 |
-
" pd Dataframe: with class predictions
|
| 301 |
" \"\"\"\n",
|
| 302 |
" \n",
|
| 303 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 304 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 305 |
" for i, key in enumerate(keys_list):\n",
|
| 306 |
-
"
|
| 307 |
" if key == 'activities':\n",
|
| 308 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 309 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
@@ -317,7 +367,7 @@
|
|
| 317 |
},
|
| 318 |
{
|
| 319 |
"cell_type": "code",
|
| 320 |
-
"execution_count":
|
| 321 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 322 |
"metadata": {},
|
| 323 |
"outputs": [
|
|
@@ -332,7 +382,7 @@
|
|
| 332 |
"name": "stdout",
|
| 333 |
"output_type": "stream",
|
| 334 |
"text": [
|
| 335 |
-
" superclass pred_class\n",
|
| 336 |
"0 activity_type beach vacation\n",
|
| 337 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 338 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -360,21 +410,20 @@
|
|
| 360 |
},
|
| 361 |
{
|
| 362 |
"cell_type": "code",
|
| 363 |
-
"execution_count":
|
| 364 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 365 |
"metadata": {},
|
| 366 |
"outputs": [],
|
| 367 |
"source": [
|
| 368 |
"# Prerequisites\n",
|
| 369 |
-
"from transformers import pipeline\n",
|
| 370 |
-
"import json\n",
|
| 371 |
-
"import pandas as pd\n"
|
| 372 |
-
"import gradio as gr"
|
| 373 |
]
|
| 374 |
},
|
| 375 |
{
|
| 376 |
"cell_type": "code",
|
| 377 |
-
"execution_count":
|
| 378 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 379 |
"metadata": {},
|
| 380 |
"outputs": [
|
|
@@ -382,7 +431,7 @@
|
|
| 382 |
"name": "stdout",
|
| 383 |
"output_type": "stream",
|
| 384 |
"text": [
|
| 385 |
-
"Running on local URL: http://127.0.0.1:
|
| 386 |
"\n",
|
| 387 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 388 |
]
|
|
@@ -390,7 +439,7 @@
|
|
| 390 |
{
|
| 391 |
"data": {
|
| 392 |
"text/html": [
|
| 393 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 394 |
],
|
| 395 |
"text/plain": [
|
| 396 |
"<IPython.core.display.HTML object>"
|
|
@@ -401,10 +450,12 @@
|
|
| 401 |
}
|
| 402 |
],
|
| 403 |
"source": [
|
|
|
|
|
|
|
| 404 |
"demo = gr.Interface(\n",
|
| 405 |
" fn=pred_trip,\n",
|
| 406 |
" inputs=[\n",
|
| 407 |
-
" gr.Textbox(label=\"Model name\", value = \"
|
| 408 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 409 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 410 |
" ],\n",
|
|
|
|
| 5 |
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
"metadata": {},
|
| 7 |
"source": [
|
| 8 |
+
"**Hugging Face API**"
|
| 9 |
]
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 2,
|
| 14 |
+
"id": "1c3b8678-d1fd-46a6-8633-fe0b34393d51",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
+
"from dotenv import load_dotenv\n",
|
|
|
|
| 19 |
"import os\n",
|
| 20 |
+
"import requests\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
"import json\n",
|
|
|
|
| 22 |
"\n",
|
| 23 |
+
"load_dotenv() # Load environment variables from .env file\n",
|
| 24 |
+
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
|
|
|
|
|
|
| 25 |
"\n",
|
| 26 |
+
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 27 |
+
"\n",
|
| 28 |
+
"def query(model, input_text):\n",
|
| 29 |
+
" API_URL = f\"https://router.huggingface.co/hf-inference/models/{model}\"\n",
|
| 30 |
+
" payload = {\n",
|
| 31 |
+
" \"inputs\": input_text,\n",
|
| 32 |
+
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
| 33 |
+
" }\n",
|
| 34 |
+
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 35 |
+
" return response.json()"
|
| 36 |
]
|
| 37 |
},
|
| 38 |
{
|
| 39 |
"cell_type": "code",
|
| 40 |
+
"execution_count": 3,
|
| 41 |
+
"id": "9a5fbe4a-c092-4d9b-b797-9104823b71cc",
|
| 42 |
+
"metadata": {},
|
| 43 |
+
"outputs": [
|
| 44 |
+
{
|
| 45 |
+
"name": "stdout",
|
| 46 |
+
"output_type": "stream",
|
| 47 |
+
"text": [
|
| 48 |
+
"[\n",
|
| 49 |
+
" {\n",
|
| 50 |
+
" \"label\": \"technology\",\n",
|
| 51 |
+
" \"score\": 0.970917284488678\n",
|
| 52 |
+
" },\n",
|
| 53 |
+
" {\n",
|
| 54 |
+
" \"label\": \"health\",\n",
|
| 55 |
+
" \"score\": 0.014999152161180973\n",
|
| 56 |
+
" },\n",
|
| 57 |
+
" {\n",
|
| 58 |
+
" \"label\": \"sports\",\n",
|
| 59 |
+
" \"score\": 0.008272469975054264\n",
|
| 60 |
+
" },\n",
|
| 61 |
+
" {\n",
|
| 62 |
+
" \"label\": \"politics\",\n",
|
| 63 |
+
" \"score\": 0.005811101291328669\n",
|
| 64 |
+
" }\n",
|
| 65 |
+
"]\n"
|
| 66 |
+
]
|
| 67 |
+
}
|
| 68 |
+
],
|
| 69 |
+
"source": [
|
| 70 |
+
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 71 |
+
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 72 |
+
"print(json.dumps(output, indent=4))"
|
| 73 |
+
]
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"cell_type": "markdown",
|
| 77 |
+
"id": "01dc84d7-6d0f-4409-8966-c1adac5fd0c8",
|
| 78 |
+
"metadata": {},
|
| 79 |
+
"source": [
|
| 80 |
+
"**Model implementation**"
|
| 81 |
+
]
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"cell_type": "code",
|
| 85 |
+
"execution_count": 4,
|
| 86 |
+
"id": "49355cad-771a-4080-8b67-f995c3de0110",
|
| 87 |
"metadata": {},
|
| 88 |
"outputs": [
|
| 89 |
{
|
|
|
|
| 188 |
}
|
| 189 |
],
|
| 190 |
"source": [
|
| 191 |
+
"import json\n",
|
| 192 |
+
"import pandas as pd\n",
|
| 193 |
+
"from tabulate import tabulate\n",
|
| 194 |
+
"from transformers import pipeline\n",
|
| 195 |
+
"\n",
|
| 196 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 197 |
+
" candidate_labels = json.load(file)\n",
|
| 198 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 199 |
+
"\n",
|
| 200 |
"for key in candidate_labels:\n",
|
| 201 |
" print(\"\\n\", key, \":\")\n",
|
| 202 |
" for item in candidate_labels[key]:\n",
|
| 203 |
+
" print(\"\\t\", item)"
|
| 204 |
]
|
| 205 |
},
|
| 206 |
{
|
| 207 |
"cell_type": "code",
|
| 208 |
+
"execution_count": 5,
|
| 209 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 210 |
"metadata": {},
|
| 211 |
"outputs": [
|
|
|
|
| 241 |
}
|
| 242 |
],
|
| 243 |
"source": [
|
|
|
|
| 244 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 245 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 246 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 247 |
+
"result = classifier(trip_descr, candidate_labels[\"activity_type\"])\n",
|
| 248 |
+
"\n",
|
| 249 |
"df = pd.DataFrame({\n",
|
| 250 |
" \"Label\": result[\"labels\"],\n",
|
| 251 |
" \"Score\": result[\"scores\"]\n",
|
|
|
|
| 255 |
},
|
| 256 |
{
|
| 257 |
"cell_type": "code",
|
| 258 |
+
"execution_count": 6,
|
| 259 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 260 |
"metadata": {},
|
| 261 |
"outputs": [
|
|
|
|
| 275 |
},
|
| 276 |
{
|
| 277 |
"cell_type": "code",
|
| 278 |
+
"execution_count": 7,
|
| 279 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 280 |
+
"metadata": {
|
| 281 |
+
"scrolled": true
|
| 282 |
+
},
|
| 283 |
"outputs": [
|
| 284 |
{
|
| 285 |
"name": "stdout",
|
|
|
|
| 311 |
"22 skiing 0.001169\n",
|
| 312 |
"23 scuba diving 0.000789\n",
|
| 313 |
"24 ski touring 0.000491\n",
|
| 314 |
+
"[]\n"
|
| 315 |
]
|
| 316 |
}
|
| 317 |
],
|
| 318 |
"source": [
|
| 319 |
+
"# we do things differently for \"activities\"\n",
|
| 320 |
"cut_off = 0.5\n",
|
| 321 |
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 322 |
+
"classes = df.loc[df[\"Score\"] > 0.5, \"Label\"].tolist()\n",
|
|
|
|
| 323 |
"\n",
|
| 324 |
"df = pd.DataFrame({\n",
|
| 325 |
" \"Label\": result_activ[\"labels\"],\n",
|
|
|
|
| 331 |
},
|
| 332 |
{
|
| 333 |
"cell_type": "code",
|
| 334 |
+
"execution_count": 8,
|
| 335 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 336 |
"metadata": {},
|
| 337 |
"outputs": [],
|
|
|
|
| 347 |
" cut_off: cut_off for choosing activities\n",
|
| 348 |
"\n",
|
| 349 |
" Returns:\n",
|
| 350 |
+
" pd Dataframe: with class predictions\n",
|
| 351 |
" \"\"\"\n",
|
| 352 |
" \n",
|
| 353 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 354 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 355 |
" for i, key in enumerate(keys_list):\n",
|
| 356 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 357 |
" if key == 'activities':\n",
|
| 358 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 359 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
|
|
| 367 |
},
|
| 368 |
{
|
| 369 |
"cell_type": "code",
|
| 370 |
+
"execution_count": 9,
|
| 371 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 372 |
"metadata": {},
|
| 373 |
"outputs": [
|
|
|
|
| 382 |
"name": "stdout",
|
| 383 |
"output_type": "stream",
|
| 384 |
"text": [
|
| 385 |
+
"Processing 9/9 superclass pred_class\n",
|
| 386 |
"0 activity_type beach vacation\n",
|
| 387 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 388 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 410 |
},
|
| 411 |
{
|
| 412 |
"cell_type": "code",
|
| 413 |
+
"execution_count": 3,
|
| 414 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 415 |
"metadata": {},
|
| 416 |
"outputs": [],
|
| 417 |
"source": [
|
| 418 |
"# Prerequisites\n",
|
| 419 |
+
"#from transformers import pipeline\n",
|
| 420 |
+
"#import json\n",
|
| 421 |
+
"#import pandas as pd\n"
|
|
|
|
| 422 |
]
|
| 423 |
},
|
| 424 |
{
|
| 425 |
"cell_type": "code",
|
| 426 |
+
"execution_count": 11,
|
| 427 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 428 |
"metadata": {},
|
| 429 |
"outputs": [
|
|
|
|
| 431 |
"name": "stdout",
|
| 432 |
"output_type": "stream",
|
| 433 |
"text": [
|
| 434 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 435 |
"\n",
|
| 436 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 437 |
]
|
|
|
|
| 439 |
{
|
| 440 |
"data": {
|
| 441 |
"text/html": [
|
| 442 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 443 |
],
|
| 444 |
"text/plain": [
|
| 445 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 450 |
}
|
| 451 |
],
|
| 452 |
"source": [
|
| 453 |
+
"import gradio as gr\n",
|
| 454 |
+
"\n",
|
| 455 |
"demo = gr.Interface(\n",
|
| 456 |
" fn=pred_trip,\n",
|
| 457 |
" inputs=[\n",
|
| 458 |
+
" gr.Textbox(label=\"Model name\", value = \"facebook/bart-large-mnli\"),\n",
|
| 459 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 460 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 461 |
" ],\n",
|
space/space/space/space/space/space/packing_list_api.ipynb
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
"id": "966c6d6a-c2d7-405c-bf9b-1e1f1415234e",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [
|
|
@@ -18,7 +18,21 @@
|
|
| 18 |
"name": "stdout",
|
| 19 |
"output_type": "stream",
|
| 20 |
"text": [
|
| 21 |
-
"{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
]
|
| 23 |
}
|
| 24 |
],
|
|
@@ -26,6 +40,7 @@
|
|
| 26 |
"from dotenv import load_dotenv\n",
|
| 27 |
"import os\n",
|
| 28 |
"import requests\n",
|
|
|
|
| 29 |
"\n",
|
| 30 |
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
| 31 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
|
@@ -33,7 +48,7 @@
|
|
| 33 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 34 |
"\n",
|
| 35 |
"def query(model, input_text):\n",
|
| 36 |
-
" API_URL = f\"https://
|
| 37 |
" payload = {\n",
|
| 38 |
" \"inputs\": input_text,\n",
|
| 39 |
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
|
@@ -44,7 +59,7 @@
|
|
| 44 |
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 45 |
"\n",
|
| 46 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 47 |
-
"print(output)"
|
| 48 |
]
|
| 49 |
},
|
| 50 |
{
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 7,
|
| 14 |
"id": "966c6d6a-c2d7-405c-bf9b-1e1f1415234e",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [
|
|
|
|
| 18 |
"name": "stdout",
|
| 19 |
"output_type": "stream",
|
| 20 |
"text": [
|
| 21 |
+
"{\n",
|
| 22 |
+
" \"sequence\": \"I just bought a new laptop, and it works amazing!\",\n",
|
| 23 |
+
" \"labels\": [\n",
|
| 24 |
+
" \"technology\",\n",
|
| 25 |
+
" \"health\",\n",
|
| 26 |
+
" \"sports\",\n",
|
| 27 |
+
" \"politics\"\n",
|
| 28 |
+
" ],\n",
|
| 29 |
+
" \"scores\": [\n",
|
| 30 |
+
" 0.970917284488678,\n",
|
| 31 |
+
" 0.014999152161180973,\n",
|
| 32 |
+
" 0.008272469975054264,\n",
|
| 33 |
+
" 0.005811101291328669\n",
|
| 34 |
+
" ]\n",
|
| 35 |
+
"}\n"
|
| 36 |
]
|
| 37 |
}
|
| 38 |
],
|
|
|
|
| 40 |
"from dotenv import load_dotenv\n",
|
| 41 |
"import os\n",
|
| 42 |
"import requests\n",
|
| 43 |
+
"import json\n",
|
| 44 |
"\n",
|
| 45 |
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
| 46 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
|
|
|
| 48 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 49 |
"\n",
|
| 50 |
"def query(model, input_text):\n",
|
| 51 |
+
" API_URL = f\"https://router.huggingface.co/hf-inference/models/{model}\"\n",
|
| 52 |
" payload = {\n",
|
| 53 |
" \"inputs\": input_text,\n",
|
| 54 |
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
|
|
|
| 59 |
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 60 |
"\n",
|
| 61 |
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 62 |
+
"print(json.dumps(output, indent=4))"
|
| 63 |
]
|
| 64 |
},
|
| 65 |
{
|
space/space/space/space/space/space/results/MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7_results.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bd05abe4d3647964350c919d22895e5dfc74b26d3dd556a0fe25bf1ad1916e9c
|
| 3 |
+
size 9583
|
space/space/space/space/space/space/results/pongjin-roberta_with_kornli_results.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b4f1e9f28492b70511b428bad73858426133cda8943a29a6b4176198aa1df08f
|
| 3 |
+
size 9497
|
space/space/space/space/space/space/results/sileod-deberta-v3-base-tasksource-nli_results.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:60efca8b4b3a509443eda845e60fa53aa60d7cb7c859b1846e532953e4a6c02f
|
| 3 |
+
size 9481
|
space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
CHANGED
|
@@ -155,7 +155,7 @@
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
@@ -206,7 +206,7 @@
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
-
"execution_count":
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
@@ -226,7 +226,7 @@
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
-
"execution_count":
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
@@ -234,23 +234,33 @@
|
|
| 234 |
"name": "stdout",
|
| 235 |
"output_type": "stream",
|
| 236 |
"text": [
|
| 237 |
-
"
|
| 238 |
-
"0
|
| 239 |
-
"1
|
| 240 |
-
"2
|
| 241 |
-
"3
|
| 242 |
-
"4
|
| 243 |
-
"5
|
| 244 |
-
"6
|
| 245 |
-
"7
|
| 246 |
-
"8
|
| 247 |
-
"9
|
| 248 |
-
"10
|
| 249 |
-
"11
|
| 250 |
-
"12
|
| 251 |
-
"13
|
| 252 |
-
"14
|
| 253 |
-
"15
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 254 |
]
|
| 255 |
}
|
| 256 |
],
|
|
@@ -262,8 +272,8 @@
|
|
| 262 |
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 263 |
"\n",
|
| 264 |
"df = pd.DataFrame({\n",
|
| 265 |
-
" \"Label\":
|
| 266 |
-
" \"Score\":
|
| 267 |
"})\n",
|
| 268 |
"print(df)\n",
|
| 269 |
"print(classes)"
|
|
@@ -271,23 +281,7 @@
|
|
| 271 |
},
|
| 272 |
{
|
| 273 |
"cell_type": "code",
|
| 274 |
-
"execution_count":
|
| 275 |
-
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 276 |
-
"metadata": {},
|
| 277 |
-
"outputs": [
|
| 278 |
-
{
|
| 279 |
-
"name": "stdout",
|
| 280 |
-
"output_type": "stream",
|
| 281 |
-
"text": [
|
| 282 |
-
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 283 |
-
]
|
| 284 |
-
}
|
| 285 |
-
],
|
| 286 |
-
"source": []
|
| 287 |
-
},
|
| 288 |
-
{
|
| 289 |
-
"cell_type": "code",
|
| 290 |
-
"execution_count": 8,
|
| 291 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 292 |
"metadata": {},
|
| 293 |
"outputs": [],
|
|
@@ -309,7 +303,7 @@
|
|
| 309 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 310 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 311 |
" for i, key in enumerate(keys_list):\n",
|
| 312 |
-
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 313 |
" if key == 'activities':\n",
|
| 314 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 315 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
@@ -323,7 +317,7 @@
|
|
| 323 |
},
|
| 324 |
{
|
| 325 |
"cell_type": "code",
|
| 326 |
-
"execution_count":
|
| 327 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 328 |
"metadata": {},
|
| 329 |
"outputs": [
|
|
@@ -338,7 +332,7 @@
|
|
| 338 |
"name": "stdout",
|
| 339 |
"output_type": "stream",
|
| 340 |
"text": [
|
| 341 |
-
"
|
| 342 |
"0 activity_type beach vacation\n",
|
| 343 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 344 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -366,7 +360,7 @@
|
|
| 366 |
},
|
| 367 |
{
|
| 368 |
"cell_type": "code",
|
| 369 |
-
"execution_count":
|
| 370 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 371 |
"metadata": {},
|
| 372 |
"outputs": [],
|
|
@@ -375,21 +369,12 @@
|
|
| 375 |
"from transformers import pipeline\n",
|
| 376 |
"import json\n",
|
| 377 |
"import pandas as pd\n",
|
| 378 |
-
"import gradio as gr
|
| 379 |
-
"\n",
|
| 380 |
-
"# get candidate labels\n",
|
| 381 |
-
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 382 |
-
" candidate_labels = json.load(file)\n",
|
| 383 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 384 |
-
"\n",
|
| 385 |
-
"# Load packing item data\n",
|
| 386 |
-
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 387 |
-
" packing_items = json.load(file)"
|
| 388 |
]
|
| 389 |
},
|
| 390 |
{
|
| 391 |
"cell_type": "code",
|
| 392 |
-
"execution_count":
|
| 393 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 394 |
"metadata": {},
|
| 395 |
"outputs": [
|
|
@@ -424,7 +409,7 @@
|
|
| 424 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 425 |
" ],\n",
|
| 426 |
" # outputs=\"dataframe\",\n",
|
| 427 |
-
" outputs=[gr.Dataframe(label=\"DataFrame\")
|
| 428 |
" title=\"Trip classification\",\n",
|
| 429 |
" description=\"Enter a text describing your trip\",\n",
|
| 430 |
")\n",
|
|
@@ -459,40 +444,285 @@
|
|
| 459 |
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 460 |
]
|
| 461 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 462 |
{
|
| 463 |
"cell_type": "code",
|
| 464 |
"execution_count": null,
|
| 465 |
-
"id": "
|
| 466 |
"metadata": {},
|
| 467 |
"outputs": [],
|
| 468 |
"source": [
|
| 469 |
-
"
|
| 470 |
]
|
| 471 |
},
|
| 472 |
{
|
| 473 |
"cell_type": "code",
|
| 474 |
-
"execution_count":
|
| 475 |
-
"id": "
|
| 476 |
"metadata": {},
|
| 477 |
"outputs": [
|
| 478 |
{
|
| 479 |
"name": "stdout",
|
| 480 |
"output_type": "stream",
|
| 481 |
"text": [
|
| 482 |
-
"
|
| 483 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 484 |
]
|
| 485 |
}
|
| 486 |
],
|
| 487 |
"source": [
|
| 488 |
-
"
|
| 489 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 490 |
]
|
| 491 |
},
|
| 492 |
{
|
| 493 |
"cell_type": "code",
|
| 494 |
"execution_count": null,
|
| 495 |
-
"id": "
|
| 496 |
"metadata": {},
|
| 497 |
"outputs": [],
|
| 498 |
"source": []
|
|
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
+
"execution_count": 3,
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
+
"execution_count": 4,
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
+
"execution_count": 5,
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
|
|
| 234 |
"name": "stdout",
|
| 235 |
"output_type": "stream",
|
| 236 |
"text": [
|
| 237 |
+
" Label Score\n",
|
| 238 |
+
"0 going to the beach 0.991486\n",
|
| 239 |
+
"1 relaxing 0.977136\n",
|
| 240 |
+
"2 hiking 0.942628\n",
|
| 241 |
+
"3 swimming 0.219020\n",
|
| 242 |
+
"4 sightseeing 0.175862\n",
|
| 243 |
+
"5 running 0.098545\n",
|
| 244 |
+
"6 hut-to-hut hiking 0.083704\n",
|
| 245 |
+
"7 biking 0.036792\n",
|
| 246 |
+
"8 photography 0.036690\n",
|
| 247 |
+
"9 surfing 0.030993\n",
|
| 248 |
+
"10 stand-up paddleboarding (SUP) 0.025300\n",
|
| 249 |
+
"11 snorkeling 0.021451\n",
|
| 250 |
+
"12 yoga 0.011070\n",
|
| 251 |
+
"13 kayaking / canoeing 0.007511\n",
|
| 252 |
+
"14 rock climbing 0.006307\n",
|
| 253 |
+
"15 fishing 0.003497\n",
|
| 254 |
+
"16 paragliding 0.002656\n",
|
| 255 |
+
"17 rafting 0.001970\n",
|
| 256 |
+
"18 horseback riding 0.001560\n",
|
| 257 |
+
"19 snowshoe hiking 0.001528\n",
|
| 258 |
+
"20 cross-country skiing 0.001502\n",
|
| 259 |
+
"21 ice climbing 0.001434\n",
|
| 260 |
+
"22 skiing 0.001169\n",
|
| 261 |
+
"23 scuba diving 0.000789\n",
|
| 262 |
+
"24 ski touring 0.000491\n",
|
| 263 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 264 |
]
|
| 265 |
}
|
| 266 |
],
|
|
|
|
| 272 |
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 273 |
"\n",
|
| 274 |
"df = pd.DataFrame({\n",
|
| 275 |
+
" \"Label\": result_activ[\"labels\"],\n",
|
| 276 |
+
" \"Score\": result_activ[\"scores\"]\n",
|
| 277 |
"})\n",
|
| 278 |
"print(df)\n",
|
| 279 |
"print(classes)"
|
|
|
|
| 281 |
},
|
| 282 |
{
|
| 283 |
"cell_type": "code",
|
| 284 |
+
"execution_count": 6,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 286 |
"metadata": {},
|
| 287 |
"outputs": [],
|
|
|
|
| 303 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 304 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 305 |
" for i, key in enumerate(keys_list):\n",
|
| 306 |
+
" # print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 307 |
" if key == 'activities':\n",
|
| 308 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 309 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
|
|
| 317 |
},
|
| 318 |
{
|
| 319 |
"cell_type": "code",
|
| 320 |
+
"execution_count": 7,
|
| 321 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 322 |
"metadata": {},
|
| 323 |
"outputs": [
|
|
|
|
| 332 |
"name": "stdout",
|
| 333 |
"output_type": "stream",
|
| 334 |
"text": [
|
| 335 |
+
" superclass pred_class\n",
|
| 336 |
"0 activity_type beach vacation\n",
|
| 337 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 338 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 360 |
},
|
| 361 |
{
|
| 362 |
"cell_type": "code",
|
| 363 |
+
"execution_count": 9,
|
| 364 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 365 |
"metadata": {},
|
| 366 |
"outputs": [],
|
|
|
|
| 369 |
"from transformers import pipeline\n",
|
| 370 |
"import json\n",
|
| 371 |
"import pandas as pd\n",
|
| 372 |
+
"import gradio as gr"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 373 |
]
|
| 374 |
},
|
| 375 |
{
|
| 376 |
"cell_type": "code",
|
| 377 |
+
"execution_count": 10,
|
| 378 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 379 |
"metadata": {},
|
| 380 |
"outputs": [
|
|
|
|
| 409 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 410 |
" ],\n",
|
| 411 |
" # outputs=\"dataframe\",\n",
|
| 412 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 413 |
" title=\"Trip classification\",\n",
|
| 414 |
" description=\"Enter a text describing your trip\",\n",
|
| 415 |
")\n",
|
|
|
|
| 444 |
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 445 |
]
|
| 446 |
},
|
| 447 |
+
{
|
| 448 |
+
"cell_type": "markdown",
|
| 449 |
+
"id": "39920553-27d3-4a63-8381-8310566c4874",
|
| 450 |
+
"metadata": {},
|
| 451 |
+
"source": [
|
| 452 |
+
"All code for gradio app file"
|
| 453 |
+
]
|
| 454 |
+
},
|
| 455 |
+
{
|
| 456 |
+
"cell_type": "code",
|
| 457 |
+
"execution_count": 2,
|
| 458 |
+
"id": "4ffc76d5-60c3-4bc8-bd4f-a9636077f01d",
|
| 459 |
+
"metadata": {},
|
| 460 |
+
"outputs": [
|
| 461 |
+
{
|
| 462 |
+
"name": "stdout",
|
| 463 |
+
"output_type": "stream",
|
| 464 |
+
"text": [
|
| 465 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 466 |
+
"\n",
|
| 467 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 468 |
+
]
|
| 469 |
+
},
|
| 470 |
+
{
|
| 471 |
+
"data": {
|
| 472 |
+
"text/html": [
|
| 473 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 474 |
+
],
|
| 475 |
+
"text/plain": [
|
| 476 |
+
"<IPython.core.display.HTML object>"
|
| 477 |
+
]
|
| 478 |
+
},
|
| 479 |
+
"metadata": {},
|
| 480 |
+
"output_type": "display_data"
|
| 481 |
+
},
|
| 482 |
+
{
|
| 483 |
+
"name": "stderr",
|
| 484 |
+
"output_type": "stream",
|
| 485 |
+
"text": [
|
| 486 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 487 |
+
]
|
| 488 |
+
},
|
| 489 |
+
{
|
| 490 |
+
"name": "stdout",
|
| 491 |
+
"output_type": "stream",
|
| 492 |
+
"text": [
|
| 493 |
+
"Processing 9/9"
|
| 494 |
+
]
|
| 495 |
+
}
|
| 496 |
+
],
|
| 497 |
+
"source": [
|
| 498 |
+
"# Prerequisites\n",
|
| 499 |
+
"from transformers import pipeline\n",
|
| 500 |
+
"import json\n",
|
| 501 |
+
"import pandas as pd\n",
|
| 502 |
+
"import gradio as gr\n",
|
| 503 |
+
"\n",
|
| 504 |
+
"# Get candidate labels\n",
|
| 505 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 506 |
+
" candidate_labels = json.load(file)\n",
|
| 507 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 508 |
+
"\n",
|
| 509 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 510 |
+
" \"\"\"\n",
|
| 511 |
+
" Classifies trip\n",
|
| 512 |
+
" \n",
|
| 513 |
+
" Parameters:\n",
|
| 514 |
+
" model_name: name of hugging-face model\n",
|
| 515 |
+
" trip_descr: text describing the trip\n",
|
| 516 |
+
" cut_off: cut_off for choosing activities\n",
|
| 517 |
+
"\n",
|
| 518 |
+
" Returns:\n",
|
| 519 |
+
" pd Dataframe: with class predictions and true values\n",
|
| 520 |
+
" \"\"\"\n",
|
| 521 |
+
" \n",
|
| 522 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 523 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 524 |
+
" for i, key in enumerate(keys_list):\n",
|
| 525 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 526 |
+
" if key == 'activities':\n",
|
| 527 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 528 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 529 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 530 |
+
" else:\n",
|
| 531 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 532 |
+
" classes = result[\"labels\"][0]\n",
|
| 533 |
+
" df.loc[i] = [key, classes]\n",
|
| 534 |
+
" return df\n",
|
| 535 |
+
"\n",
|
| 536 |
+
"demo = gr.Interface(\n",
|
| 537 |
+
" fn=pred_trip,\n",
|
| 538 |
+
" inputs=[\n",
|
| 539 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 540 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 541 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 542 |
+
" ],\n",
|
| 543 |
+
" # outputs=\"dataframe\",\n",
|
| 544 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 545 |
+
" title=\"Trip classification\",\n",
|
| 546 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 547 |
+
")\n",
|
| 548 |
+
"\n",
|
| 549 |
+
"# Launch the Gradio app\n",
|
| 550 |
+
"if __name__ == \"__main__\":\n",
|
| 551 |
+
" demo.launch()\n"
|
| 552 |
+
]
|
| 553 |
+
},
|
| 554 |
{
|
| 555 |
"cell_type": "code",
|
| 556 |
"execution_count": null,
|
| 557 |
+
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 558 |
"metadata": {},
|
| 559 |
"outputs": [],
|
| 560 |
"source": [
|
| 561 |
+
"Print test data set"
|
| 562 |
]
|
| 563 |
},
|
| 564 |
{
|
| 565 |
"cell_type": "code",
|
| 566 |
+
"execution_count": 4,
|
| 567 |
+
"id": "f25e1462-ca48-4853-ae44-dfc096c5011d",
|
| 568 |
"metadata": {},
|
| 569 |
"outputs": [
|
| 570 |
{
|
| 571 |
"name": "stdout",
|
| 572 |
"output_type": "stream",
|
| 573 |
"text": [
|
| 574 |
+
"0 . I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 575 |
+
"\n",
|
| 576 |
+
"beach vacation\n",
|
| 577 |
+
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 578 |
+
"warm destination / summer\n",
|
| 579 |
+
"lightweight (but comfortable)\n",
|
| 580 |
+
"casual\n",
|
| 581 |
+
"indoor\n",
|
| 582 |
+
"no own vehicle\n",
|
| 583 |
+
"no special conditions to consider\n",
|
| 584 |
+
"7+ days\n",
|
| 585 |
+
"\n",
|
| 586 |
+
"\n",
|
| 587 |
+
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 588 |
+
"\n",
|
| 589 |
+
"city trip\n",
|
| 590 |
+
"['sightseeing']\n",
|
| 591 |
+
"variable weather / spring / autumn\n",
|
| 592 |
+
"luxury (including evening wear)\n",
|
| 593 |
+
"casual\n",
|
| 594 |
+
"indoor\n",
|
| 595 |
+
"no own vehicle\n",
|
| 596 |
+
"no special conditions to consider\n",
|
| 597 |
+
"3 days\n",
|
| 598 |
+
"\n",
|
| 599 |
+
"\n",
|
| 600 |
+
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 601 |
+
"\n",
|
| 602 |
+
"city trip\n",
|
| 603 |
+
"['relaxing']\n",
|
| 604 |
+
"cold destination / winter\n",
|
| 605 |
+
"lightweight (but comfortable)\n",
|
| 606 |
+
"casual\n",
|
| 607 |
+
"indoor\n",
|
| 608 |
+
"no own vehicle\n",
|
| 609 |
+
"no special conditions to consider\n",
|
| 610 |
+
"7+ days\n",
|
| 611 |
+
"\n",
|
| 612 |
+
"\n",
|
| 613 |
+
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 614 |
+
"\n",
|
| 615 |
+
"cultural exploration\n",
|
| 616 |
+
"['sightseeing', 'hiking', 'rafting']\n",
|
| 617 |
+
"variable weather / spring / autumn\n",
|
| 618 |
+
"lightweight (but comfortable)\n",
|
| 619 |
+
"casual\n",
|
| 620 |
+
"indoor\n",
|
| 621 |
+
"no own vehicle\n",
|
| 622 |
+
"rainy climate\n",
|
| 623 |
+
"7+ days\n",
|
| 624 |
+
"\n",
|
| 625 |
+
"\n",
|
| 626 |
+
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 627 |
+
"\n",
|
| 628 |
+
"nature escape\n",
|
| 629 |
+
"['swimming', 'relaxing', 'hiking']\n",
|
| 630 |
+
"warm destination / summer\n",
|
| 631 |
+
"lightweight (but comfortable)\n",
|
| 632 |
+
"casual\n",
|
| 633 |
+
"indoor\n",
|
| 634 |
+
"no own vehicle\n",
|
| 635 |
+
"no special conditions to consider\n",
|
| 636 |
+
"7+ days\n",
|
| 637 |
+
"\n",
|
| 638 |
+
"\n",
|
| 639 |
+
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 640 |
+
"\n",
|
| 641 |
+
"long-distance hike / thru-hike\n",
|
| 642 |
+
"['going to the beach']\n",
|
| 643 |
+
"tropical / humid\n",
|
| 644 |
+
"minimalist\n",
|
| 645 |
+
"casual\n",
|
| 646 |
+
"huts with half board\n",
|
| 647 |
+
"own vehicle\n",
|
| 648 |
+
"off-grid / no electricity\n",
|
| 649 |
+
"6 days\n",
|
| 650 |
+
"\n",
|
| 651 |
+
"\n",
|
| 652 |
+
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 653 |
+
"\n",
|
| 654 |
+
"beach vacation\n",
|
| 655 |
+
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 656 |
+
"cold destination / winter\n",
|
| 657 |
+
"ultralight\n",
|
| 658 |
+
"casual\n",
|
| 659 |
+
"sleeping in a tent\n",
|
| 660 |
+
"own vehicle\n",
|
| 661 |
+
"off-grid / no electricity\n",
|
| 662 |
+
"6 days\n",
|
| 663 |
+
"\n",
|
| 664 |
+
"\n",
|
| 665 |
+
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 666 |
+
"\n",
|
| 667 |
+
"yoga / wellness retreat\n",
|
| 668 |
+
"['hiking', 'yoga']\n",
|
| 669 |
+
"cold destination / winter\n",
|
| 670 |
+
"lightweight (but comfortable)\n",
|
| 671 |
+
"casual\n",
|
| 672 |
+
"indoor\n",
|
| 673 |
+
"no own vehicle\n",
|
| 674 |
+
"snow and ice\n",
|
| 675 |
+
"7 days\n",
|
| 676 |
+
"\n",
|
| 677 |
+
"\n",
|
| 678 |
+
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 679 |
+
"\n",
|
| 680 |
+
"ski tour / skitour\n",
|
| 681 |
+
"['ski touring', 'photography']\n",
|
| 682 |
+
"cold destination / winter\n",
|
| 683 |
+
"minimalist\n",
|
| 684 |
+
"conservative\n",
|
| 685 |
+
"indoor\n",
|
| 686 |
+
"no own vehicle\n",
|
| 687 |
+
"avalanche-prone terrain\n",
|
| 688 |
+
"7+ days\n",
|
| 689 |
+
"\n",
|
| 690 |
+
"\n",
|
| 691 |
+
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 692 |
+
"\n",
|
| 693 |
+
"camping trip (wild camping)\n",
|
| 694 |
+
"['scuba diving', 'kayaking / canoeing']\n",
|
| 695 |
+
"tropical / humid\n",
|
| 696 |
+
"lightweight (but comfortable)\n",
|
| 697 |
+
"casual\n",
|
| 698 |
+
"sleeping in a tent\n",
|
| 699 |
+
"own vehicle\n",
|
| 700 |
+
"self-supported (bring your own cooking gear)\n",
|
| 701 |
+
"3 days\n",
|
| 702 |
+
"\n",
|
| 703 |
+
"\n"
|
| 704 |
]
|
| 705 |
}
|
| 706 |
],
|
| 707 |
"source": [
|
| 708 |
+
"# Load test data (list of dictionaries)\n",
|
| 709 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 710 |
+
" packing_data = json.load(file)\n",
|
| 711 |
+
" # Extract trip descriptions and classification (trip_types)\n",
|
| 712 |
+
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 713 |
+
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 714 |
+
"\n",
|
| 715 |
+
"for i, item in enumerate(trip_descriptions):\n",
|
| 716 |
+
" print(i, \".\", item, \"\\n\")\n",
|
| 717 |
+
" for elem in trip_types[i]:\n",
|
| 718 |
+
" print(elem)\n",
|
| 719 |
+
" print(\"\\n\")"
|
| 720 |
]
|
| 721 |
},
|
| 722 |
{
|
| 723 |
"cell_type": "code",
|
| 724 |
"execution_count": null,
|
| 725 |
+
"id": "23a306bc-5dac-4569-a727-d140e1b8da7a",
|
| 726 |
"metadata": {},
|
| 727 |
"outputs": [],
|
| 728 |
"source": []
|
space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb
CHANGED
|
@@ -367,6 +367,8 @@
|
|
| 367 |
],
|
| 368 |
"source": [
|
| 369 |
"# List of Hugging Face model names\n",
|
|
|
|
|
|
|
| 370 |
"model_names = [\n",
|
| 371 |
" \"facebook/bart-large-mnli\",\n",
|
| 372 |
" \"MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli\",\n",
|
|
@@ -379,6 +381,22 @@
|
|
| 379 |
" \"valhalla/distilbart-mnli-12-1\",\n",
|
| 380 |
" #\"joeddav/xlm-roberta-large-xnli\" # keeps giving errors\n",
|
| 381 |
"]\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 382 |
"\n",
|
| 383 |
"# Apply each model to the test data\n",
|
| 384 |
"for model_name in model_names:\n",
|
|
@@ -997,6 +1015,13 @@
|
|
| 997 |
},
|
| 998 |
"metadata": {},
|
| 999 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1000 |
}
|
| 1001 |
],
|
| 1002 |
"source": [
|
|
|
|
| 367 |
],
|
| 368 |
"source": [
|
| 369 |
"# List of Hugging Face model names\n",
|
| 370 |
+
"# trending...\n",
|
| 371 |
+
"\"\"\"\n",
|
| 372 |
"model_names = [\n",
|
| 373 |
" \"facebook/bart-large-mnli\",\n",
|
| 374 |
" \"MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli\",\n",
|
|
|
|
| 381 |
" \"valhalla/distilbart-mnli-12-1\",\n",
|
| 382 |
" #\"joeddav/xlm-roberta-large-xnli\" # keeps giving errors\n",
|
| 383 |
"]\n",
|
| 384 |
+
"\"\"\"\n",
|
| 385 |
+
"\n",
|
| 386 |
+
"# most downloads\n",
|
| 387 |
+
"model_names = [\n",
|
| 388 |
+
" facebook/bart-large-mnli,\n",
|
| 389 |
+
" MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli,\n",
|
| 390 |
+
" sileod/deberta-v3-base-tasksource-nli,\n",
|
| 391 |
+
" vicgalle/xlm-roberta-large-xnli-anli,\n",
|
| 392 |
+
" joeddav/xlm-roberta-large-xnli,\n",
|
| 393 |
+
" chuhac/BiomedCLIP-vit-bert-hf,\n",
|
| 394 |
+
" pongjin/roberta_with_kornli,\n",
|
| 395 |
+
" joeddav/bart-large-mnli-yahoo-answers,\n",
|
| 396 |
+
" MoritzLaurer/mDeBERTa-v3-base-mnli-xnli,\n",
|
| 397 |
+
" valhalla/distilbart-mnli-12-1 \n",
|
| 398 |
+
"]\n",
|
| 399 |
+
"\n",
|
| 400 |
"\n",
|
| 401 |
"# Apply each model to the test data\n",
|
| 402 |
"for model_name in model_names:\n",
|
|
|
|
| 1015 |
},
|
| 1016 |
"metadata": {},
|
| 1017 |
"output_type": "display_data"
|
| 1018 |
+
},
|
| 1019 |
+
{
|
| 1020 |
+
"name": "stderr",
|
| 1021 |
+
"output_type": "stream",
|
| 1022 |
+
"text": [
|
| 1023 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 1024 |
+
]
|
| 1025 |
}
|
| 1026 |
],
|
| 1027 |
"source": [
|
space/space/space/space/space/space/space/blog_post/article.md
CHANGED
|
@@ -14,16 +14,15 @@
|
|
| 14 |
* mention gradio app to make it user friendly and spaces to share model
|
| 15 |
|
| 16 |
## Implementation of packing list model
|
| 17 |
-
###
|
| 18 |
**Hugging face account**
|
| 19 |
|
| 20 |
-
Hugging Face is a company and platform for the machine learning community to collaborate on models, datasets and applications especially in the field of natural language processing.
|
| 21 |
-
To be able to use the full
|
| 22 |
-
|
| 23 |
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 24 |
|
| 25 |
**Anaconda navigator**
|
| 26 |
-
To program our model, we use the anaconda navigator with package and environment manager conda
|
| 27 |
|
| 28 |
Using the command line, you can create a new environment to work in and install the required packages. The following commands create a new environment called hf_env and activate it ([conda cheat sheet](https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf)):
|
| 29 |
|
|
@@ -38,7 +37,7 @@ pip install transformers torch numpy tabulate gradio pandas scikit-learn
|
|
| 38 |
conda install jupyter
|
| 39 |
jupyter-notebook
|
| 40 |
```
|
| 41 |
-
Create a new Jupyter Notebook
|
| 42 |
|
| 43 |
### Hugging face API
|
| 44 |
Let's first try some Hugging Face models using their API. The advantage of using API is that you do not need to download the models locally and the computation is handled on Hugging Face servers.
|
|
@@ -50,12 +49,13 @@ HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
|
| 50 |
```
|
| 51 |
where you replace YOUR_OWN_ACCESS_TOKEN with your own access token.
|
| 52 |
|
| 53 |
-
|
| 54 |
|
| 55 |
```python
|
| 56 |
from dotenv import load_dotenv
|
| 57 |
import os
|
| 58 |
import requests
|
|
|
|
| 59 |
|
| 60 |
load_dotenv() # Load environment variables from .env file
|
| 61 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
|
@@ -63,7 +63,7 @@ headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
|
| 63 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
| 64 |
|
| 65 |
def query(model, input_text):
|
| 66 |
-
API_URL = f"https://
|
| 67 |
payload = {
|
| 68 |
"inputs": input_text,
|
| 69 |
"parameters": {"candidate_labels": candidate_labels}
|
|
@@ -71,16 +71,32 @@ def query(model, input_text):
|
|
| 71 |
response = requests.post(API_URL, headers=headers, json=payload)
|
| 72 |
return response.json()
|
| 73 |
|
| 74 |
-
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
output = query("facebook/bart-large-mnli", input_text)
|
| 77 |
-
print(output)
|
| 78 |
```
|
| 79 |
|
| 80 |
-
In it we first load some libraries and then the .env file. We then create some candidate labels for our zero-shot-classification model and write a query function which contains the model. The model will then classify a text using these labels and returns the following output:
|
| 81 |
-
|
| 82 |
```json
|
| 83 |
-
{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
```
|
| 85 |
The scores contain a probability of the text belonging to a particular label.
|
| 86 |
|
|
@@ -211,7 +227,7 @@ trip_length_days :
|
|
| 211 |
7+ days
|
| 212 |
```
|
| 213 |
|
| 214 |
-
Use the pipeline function to load the model from
|
| 215 |
|
| 216 |
```python
|
| 217 |
key = keys_list[0]
|
|
@@ -219,7 +235,7 @@ model_name = "facebook/bart-large-mnli"
|
|
| 219 |
trip_descr = "I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands."
|
| 220 |
classifier = pipeline("zero-shot-classification", model = model_name)
|
| 221 |
result = classifier(trip_descr, candidate_labels[keys_list[0]])
|
| 222 |
-
|
| 223 |
df = pd.DataFrame({
|
| 224 |
"Label": result["labels"],
|
| 225 |
"Score": result["scores"]
|
|
@@ -247,7 +263,7 @@ print(df)
|
|
| 247 |
15 hut trek (winter) 0.002170
|
| 248 |
```
|
| 249 |
|
| 250 |
-
Now we will do this for every superclass
|
| 251 |
|
| 252 |
```python
|
| 253 |
cut_off = 0.5
|
|
@@ -264,28 +280,36 @@ print(classes)
|
|
| 264 |
```
|
| 265 |
|
| 266 |
```text
|
| 267 |
-
|
| 268 |
-
0
|
| 269 |
-
1
|
| 270 |
-
2
|
| 271 |
-
3
|
| 272 |
-
4
|
| 273 |
-
5
|
| 274 |
-
6
|
| 275 |
-
7
|
| 276 |
-
8
|
| 277 |
-
9
|
| 278 |
-
10
|
| 279 |
-
11
|
| 280 |
-
12
|
| 281 |
-
13
|
| 282 |
-
14
|
| 283 |
-
15
|
| 284 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
['going to the beach', 'relaxing', 'hiking']
|
| 286 |
```
|
| 287 |
|
| 288 |
-
|
| 289 |
|
| 290 |
```python
|
| 291 |
# doing this for all superclasses, depending on local machine this might take a while
|
|
@@ -321,7 +345,7 @@ print(result)
|
|
| 321 |
```
|
| 322 |
|
| 323 |
```text
|
| 324 |
-
|
| 325 |
0 activity_type beach vacation
|
| 326 |
1 activities [going to the beach, relaxing, hiking]
|
| 327 |
2 climate_or_season warm destination / summer
|
|
@@ -333,14 +357,13 @@ Processing 9/9 superclass pred_class
|
|
| 333 |
8 trip_length_days 7+ days
|
| 334 |
```
|
| 335 |
|
|
|
|
|
|
|
| 336 |
### Using gradio app: Anja
|
| 337 |
-
Now
|
| 338 |
|
| 339 |
```python
|
| 340 |
# Prerequisites
|
| 341 |
-
from transformers import pipeline
|
| 342 |
-
import json
|
| 343 |
-
import pandas as pd
|
| 344 |
import gradio as gr
|
| 345 |
|
| 346 |
# get candidate labels
|
|
@@ -348,9 +371,6 @@ with open("packing_label_structure.json", "r") as file:
|
|
| 348 |
candidate_labels = json.load(file)
|
| 349 |
keys_list = list(candidate_labels.keys())
|
| 350 |
|
| 351 |
-
# Load packing item data
|
| 352 |
-
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 353 |
-
packing_items = json.load(file)
|
| 354 |
```
|
| 355 |
|
| 356 |
```python
|
|
@@ -362,7 +382,7 @@ emo = gr.Interface(
|
|
| 362 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 363 |
],
|
| 364 |
# outputs="dataframe",
|
| 365 |
-
outputs=[gr.Dataframe(label="DataFrame")
|
| 366 |
title="Trip classification",
|
| 367 |
description="Enter a text describing your trip",
|
| 368 |
)
|
|
@@ -377,31 +397,142 @@ if __name__ == "__main__":
|
|
| 377 |
|
| 378 |
|
| 379 |
### Share your model: Anja
|
| 380 |
-
*
|
| 381 |
-
|
| 382 |
-
|
| 383 |
-
connected to your space is a remote git repository that you can push your code to. For this navigate to your project folder in the terminal
|
| 384 |
|
| 385 |
```bash
|
| 386 |
-
|
| 387 |
-
|
|
|
|
| 388 |
```
|
| 389 |
|
| 390 |
-
|
| 391 |
|
| 392 |
```bash
|
|
|
|
|
|
|
| 393 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 394 |
```
|
| 395 |
|
| 396 |
-
generate
|
| 397 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 398 |
|
| 399 |
-
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
| 400 |
-
* create requirements.txt file with necessary libraries
|
| 401 |
|
| 402 |
|
| 403 |
## Performance assessment: Anja
|
| 404 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 405 |
|
| 406 |
|
| 407 |
## Closing
|
|
|
|
| 14 |
* mention gradio app to make it user friendly and spaces to share model
|
| 15 |
|
| 16 |
## Implementation of packing list model
|
| 17 |
+
### Set up: Anja
|
| 18 |
**Hugging face account**
|
| 19 |
|
| 20 |
+
Hugging Face is a company and platform for the machine learning community to collaborate on models, datasets and applications, especially in the field of natural language processing.
|
| 21 |
+
To be able to use the full functionality offered by Hugging Face (e.g. acces to models, spaces, datasets, API access) you can create a free account [on their website](https://huggingface.co/).
|
|
|
|
| 22 |
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 23 |
|
| 24 |
**Anaconda navigator**
|
| 25 |
+
To program our model, we use the anaconda navigator with package and environment manager conda, as well as Jupyter Notebook to write our python code. You can download the Anaconda navigator [here](https://www.anaconda.com/products/navigator). (python is automatically installed)
|
| 26 |
|
| 27 |
Using the command line, you can create a new environment to work in and install the required packages. The following commands create a new environment called hf_env and activate it ([conda cheat sheet](https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf)):
|
| 28 |
|
|
|
|
| 37 |
conda install jupyter
|
| 38 |
jupyter-notebook
|
| 39 |
```
|
| 40 |
+
Create a new Jupyter Notebook for this project.
|
| 41 |
|
| 42 |
### Hugging face API
|
| 43 |
Let's first try some Hugging Face models using their API. The advantage of using API is that you do not need to download the models locally and the computation is handled on Hugging Face servers.
|
|
|
|
| 49 |
```
|
| 50 |
where you replace YOUR_OWN_ACCESS_TOKEN with your own access token.
|
| 51 |
|
| 52 |
+
Now it's time to start coding and try a first zero-shot-classification model! In your Jupyter Notebook use a code cell to write the following python code:
|
| 53 |
|
| 54 |
```python
|
| 55 |
from dotenv import load_dotenv
|
| 56 |
import os
|
| 57 |
import requests
|
| 58 |
+
import json
|
| 59 |
|
| 60 |
load_dotenv() # Load environment variables from .env file
|
| 61 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
|
|
|
| 63 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
| 64 |
|
| 65 |
def query(model, input_text):
|
| 66 |
+
API_URL = f"https://router.huggingface.co/hf-inference/models/{model}"
|
| 67 |
payload = {
|
| 68 |
"inputs": input_text,
|
| 69 |
"parameters": {"candidate_labels": candidate_labels}
|
|
|
|
| 71 |
response = requests.post(API_URL, headers=headers, json=payload)
|
| 72 |
return response.json()
|
| 73 |
|
| 74 |
+
```
|
| 75 |
|
| 76 |
+
In it we first load necessary libraries and then the .env file. We then create some candidate labels for our zero-shot-classification model and write a query function which receives a model name and an input text that needs to be classified and returns its classification. Trying the query function with the model "facebook/bart-large-mnli" from Hugging Face and a short input text we get the following:
|
| 77 |
+
|
| 78 |
+
```python
|
| 79 |
+
input_text = "I just bought a new laptop, and it works amazing!"
|
| 80 |
output = query("facebook/bart-large-mnli", input_text)
|
| 81 |
+
print(json.dumps(output, indent=4))
|
| 82 |
```
|
| 83 |
|
|
|
|
|
|
|
| 84 |
```json
|
| 85 |
+
{
|
| 86 |
+
"sequence": "I just bought a new laptop, and it works amazing!",
|
| 87 |
+
"labels": [
|
| 88 |
+
"technology",
|
| 89 |
+
"health",
|
| 90 |
+
"sports",
|
| 91 |
+
"politics"
|
| 92 |
+
],
|
| 93 |
+
"scores": [
|
| 94 |
+
0.970917284488678,
|
| 95 |
+
0.014999152161180973,
|
| 96 |
+
0.008272469975054264,
|
| 97 |
+
0.005811101291328669
|
| 98 |
+
]
|
| 99 |
+
}
|
| 100 |
```
|
| 101 |
The scores contain a probability of the text belonging to a particular label.
|
| 102 |
|
|
|
|
| 227 |
7+ days
|
| 228 |
```
|
| 229 |
|
| 230 |
+
Use the pipeline function to load the model from Hugging Face and give the classifier function the trip description together with the candidate labels, in this case for the superclass *activity_type*.
|
| 231 |
|
| 232 |
```python
|
| 233 |
key = keys_list[0]
|
|
|
|
| 235 |
trip_descr = "I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands."
|
| 236 |
classifier = pipeline("zero-shot-classification", model = model_name)
|
| 237 |
result = classifier(trip_descr, candidate_labels[keys_list[0]])
|
| 238 |
+
|
| 239 |
df = pd.DataFrame({
|
| 240 |
"Label": result["labels"],
|
| 241 |
"Score": result["scores"]
|
|
|
|
| 263 |
15 hut trek (winter) 0.002170
|
| 264 |
```
|
| 265 |
|
| 266 |
+
The most likely activity type our model predicted is beach vacation, which is correct! Now we will do this for every superclass and we choose the most likely label to be the label for our trip, except for the *activities* superclass. Since it is possible and likely to do more than one activity during your travels within the classifier function we set the multi_label option to True. This means that the text can belong to more than one class and each label is evaluated independently and a probability of belonging to that class (vs not belonging to that class) is returned. The activities that we choose as our best guess are those with a probability of more than 50 percent (cut_off = 0.5).
|
| 267 |
|
| 268 |
```python
|
| 269 |
cut_off = 0.5
|
|
|
|
| 280 |
```
|
| 281 |
|
| 282 |
```text
|
| 283 |
+
Label Score
|
| 284 |
+
0 going to the beach 0.991486
|
| 285 |
+
1 relaxing 0.977136
|
| 286 |
+
2 hiking 0.942628
|
| 287 |
+
3 swimming 0.219020
|
| 288 |
+
4 sightseeing 0.175862
|
| 289 |
+
5 running 0.098545
|
| 290 |
+
6 hut-to-hut hiking 0.083704
|
| 291 |
+
7 biking 0.036792
|
| 292 |
+
8 photography 0.036690
|
| 293 |
+
9 surfing 0.030993
|
| 294 |
+
10 stand-up paddleboarding (SUP) 0.025300
|
| 295 |
+
11 snorkeling 0.021451
|
| 296 |
+
12 yoga 0.011070
|
| 297 |
+
13 kayaking / canoeing 0.007511
|
| 298 |
+
14 rock climbing 0.006307
|
| 299 |
+
15 fishing 0.003497
|
| 300 |
+
16 paragliding 0.002656
|
| 301 |
+
17 rafting 0.001970
|
| 302 |
+
18 horseback riding 0.001560
|
| 303 |
+
19 snowshoe hiking 0.001528
|
| 304 |
+
20 cross-country skiing 0.001502
|
| 305 |
+
21 ice climbing 0.001434
|
| 306 |
+
22 skiing 0.001169
|
| 307 |
+
23 scuba diving 0.000789
|
| 308 |
+
24 ski touring 0.000491
|
| 309 |
['going to the beach', 'relaxing', 'hiking']
|
| 310 |
```
|
| 311 |
|
| 312 |
+
Now we write a function that does all the predictions for each superclass automatically for a tripdescription.
|
| 313 |
|
| 314 |
```python
|
| 315 |
# doing this for all superclasses, depending on local machine this might take a while
|
|
|
|
| 345 |
```
|
| 346 |
|
| 347 |
```text
|
| 348 |
+
superclass pred_class
|
| 349 |
0 activity_type beach vacation
|
| 350 |
1 activities [going to the beach, relaxing, hiking]
|
| 351 |
2 climate_or_season warm destination / summer
|
|
|
|
| 357 |
8 trip_length_days 7+ days
|
| 358 |
```
|
| 359 |
|
| 360 |
+
And there we have the predicted labels for our trip description.
|
| 361 |
+
|
| 362 |
### Using gradio app: Anja
|
| 363 |
+
Now, let's use the Gradio library to wrap our classification function in an interactive interface with inputs and outputs.
|
| 364 |
|
| 365 |
```python
|
| 366 |
# Prerequisites
|
|
|
|
|
|
|
|
|
|
| 367 |
import gradio as gr
|
| 368 |
|
| 369 |
# get candidate labels
|
|
|
|
| 371 |
candidate_labels = json.load(file)
|
| 372 |
keys_list = list(candidate_labels.keys())
|
| 373 |
|
|
|
|
|
|
|
|
|
|
| 374 |
```
|
| 375 |
|
| 376 |
```python
|
|
|
|
| 382 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 383 |
],
|
| 384 |
# outputs="dataframe",
|
| 385 |
+
outputs=[gr.Dataframe(label="DataFrame")],
|
| 386 |
title="Trip classification",
|
| 387 |
description="Enter a text describing your trip",
|
| 388 |
)
|
|
|
|
| 397 |
|
| 398 |
|
| 399 |
### Share your model: Anja
|
| 400 |
+
**Hugging Face Spaces**
|
| 401 |
+
A straightforward way to share the model with other people is to use Hugging Face Spaces, where you can create a free Space which you can potentially expand on later. Go to https://huggingface.co/spaces and click on "+ New Space", as SDK choose Gradio and as template Blank, as Space hardware choose "CPU Basic", and click on "Create Space" to create your Space.
|
| 402 |
+
Connected to your space is a remote git repository which is a smooth way to push your model code to the Space. Once the Sapce is created you will see the url of your Space and some instructions of how to set it up.
|
|
|
|
| 403 |
|
| 404 |
```bash
|
| 405 |
+
# When prompted for a password, use an access token with write permissions.
|
| 406 |
+
# Generate one from your settings: https://huggingface.co/settings/tokens
|
| 407 |
+
git clone https://huggingface.co/spaces/<username>/<space_name>
|
| 408 |
```
|
| 409 |
|
| 410 |
+
In the command line navigate to your project folder, initialize git and connect the remote.
|
| 411 |
|
| 412 |
```bash
|
| 413 |
+
cd path/to/your/project
|
| 414 |
+
git init
|
| 415 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 416 |
```
|
| 417 |
|
| 418 |
+
To push to your remote first generate another access token for your Space. On the Hugging Face website click on your icon, select Access Tokens, then + Create new token. As token type select Write, give your token a name and click on Create Token.
|
| 419 |
|
| 420 |
+
The Space will automatically run the model code from a app.py file. In your project folder create the file app.py (e.g. on mac in command line: touch app.py) and open it. We need to copy all relevant code for our gradio app in it:
|
| 421 |
+
|
| 422 |
+
```python
|
| 423 |
+
# Prerequisites
|
| 424 |
+
from transformers import pipeline
|
| 425 |
+
import json
|
| 426 |
+
import pandas as pd
|
| 427 |
+
import gradio as gr
|
| 428 |
+
|
| 429 |
+
# Get candidate labels
|
| 430 |
+
with open("packing_label_structure.json", "r") as file:
|
| 431 |
+
candidate_labels = json.load(file)
|
| 432 |
+
keys_list = list(candidate_labels.keys())
|
| 433 |
+
|
| 434 |
+
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 435 |
+
"""
|
| 436 |
+
Classifies trip
|
| 437 |
+
|
| 438 |
+
Parameters:
|
| 439 |
+
model_name: name of hugging-face model
|
| 440 |
+
trip_descr: text describing the trip
|
| 441 |
+
cut_off: cut_off for choosing activities
|
| 442 |
+
|
| 443 |
+
Returns:
|
| 444 |
+
pd Dataframe: with class predictions and true values
|
| 445 |
+
"""
|
| 446 |
+
|
| 447 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 448 |
+
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 449 |
+
for i, key in enumerate(keys_list):
|
| 450 |
+
print(f"\rProcessing {i + 1}/{len(keys_list)}", end="", flush=True)
|
| 451 |
+
if key == 'activities':
|
| 452 |
+
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 453 |
+
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
| 454 |
+
classes = [result['labels'][i] for i in indices]
|
| 455 |
+
else:
|
| 456 |
+
result = classifier(trip_descr, candidate_labels[key])
|
| 457 |
+
classes = result["labels"][0]
|
| 458 |
+
df.loc[i] = [key, classes]
|
| 459 |
+
return df
|
| 460 |
+
|
| 461 |
+
demo = gr.Interface(
|
| 462 |
+
fn=pred_trip,
|
| 463 |
+
inputs=[
|
| 464 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 465 |
+
gr.Textbox(label="Trip description"),
|
| 466 |
+
gr.Number(label="Activity cut-off", value = 0.5),
|
| 467 |
+
],
|
| 468 |
+
# outputs="dataframe",
|
| 469 |
+
outputs=[gr.Dataframe(label="DataFrame")],
|
| 470 |
+
title="Trip classification",
|
| 471 |
+
description="Enter a text describing your trip",
|
| 472 |
+
)
|
| 473 |
+
|
| 474 |
+
# Launch the Gradio app
|
| 475 |
+
if __name__ == "__main__":
|
| 476 |
+
demo.launch()
|
| 477 |
+
```
|
| 478 |
+
|
| 479 |
+
Additionally we need to create a plain text requirements file called requirements.txt, which tells Hugging Face which Python dependencies to install before running your app. Write and save the following in your requirements.txt file:
|
| 480 |
+
|
| 481 |
+
```text
|
| 482 |
+
transformers
|
| 483 |
+
torch
|
| 484 |
+
numpy
|
| 485 |
+
pandas
|
| 486 |
+
gradio
|
| 487 |
+
```
|
| 488 |
+
|
| 489 |
+
After you add, commit and push these changes to your remote you can go to the url of your space and try it out!
|
| 490 |
+
|
| 491 |
+
```text
|
| 492 |
+
https://huggingface.co/spaces/<username>/<space-name>
|
| 493 |
+
```
|
| 494 |
|
|
|
|
|
|
|
| 495 |
|
| 496 |
|
| 497 |
## Performance assessment: Anja
|
| 498 |
+
To asses the performance of different zero-shot classification models we manually created a small test data set of 10 trip descriptions and correct classifications. We used 12 of the most popular zero-shot-classification Hugging Face Models to compare classification performance. Performance was assessed in terms of accuracy (percentage of correct classifications/total classifications) for all superclasses excluding for the activities class. More than one type of activity can be correct and we use the percentage of correctly identified activities (#correctly identified/#total correct activities) and the percentage of wrongly identified activities (#falsly predicted/#total predicted activities) to asses the performance of the activities classification.
|
| 499 |
+
|
| 500 |
+
Let's first have a look at three of our test data trip descriptions and true class labels:
|
| 501 |
+
|
| 502 |
+
|
| 503 |
+
```text
|
| 504 |
+
I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days.
|
| 505 |
+
|
| 506 |
+
long-distance hike / thru-hike
|
| 507 |
+
['going to the beach']
|
| 508 |
+
tropical / humid
|
| 509 |
+
minimalist
|
| 510 |
+
casual
|
| 511 |
+
huts with half board
|
| 512 |
+
own vehicle
|
| 513 |
+
off-grid / no electricity
|
| 514 |
+
6 days
|
| 515 |
+
```
|
| 516 |
+
|
| 517 |
+
Our predictions using e.g. model "" could now look like this
|
| 518 |
+
|
| 519 |
+
We computed averages of performance measures for each model sorted by accuracy and averaged over our 10 test data sets.
|
| 520 |
+
|
| 521 |
+
```text
|
| 522 |
+
model accuracy true_ident false_pred
|
| 523 |
+
0 MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli 0.611111 0.841667 0.546667
|
| 524 |
+
1 sileod-deberta-v3-base-tasksource-nli 0.566667 0.700000 0.551667
|
| 525 |
+
2 MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli 0.522222 0.841667 0.572381
|
| 526 |
+
3 MoritzLaurer-deberta-v3-large-zeroshot-v2.0 0.500000 0.325000 0.500000
|
| 527 |
+
4 valhalla-distilbart-mnli-12-1 0.500000 0.300000 0.533333
|
| 528 |
+
5 MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 0.488889 0.833333 0.688373
|
| 529 |
+
6 cross-encoder-nli-deberta-v3-large 0.466667 0.566667 0.541667
|
| 530 |
+
7 facebook-bart-large-mnli 0.466667 0.708333 0.400000
|
| 531 |
+
8 MoritzLaurer-mDeBERTa-v3-base-mnli-xnli 0.455556 0.408333 0.481250
|
| 532 |
+
9 cross-encoder-nli-deberta-v3-base 0.444444 0.533333 0.712500
|
| 533 |
+
10 joeddav-bart-large-mnli-yahoo-answers 0.355556 0.650000 0.553792
|
| 534 |
+
11 pongjin-roberta_with_kornli 0.233333 0.666667 0.452857
|
| 535 |
+
```
|
| 536 |
|
| 537 |
|
| 538 |
## Closing
|
space/space/space/space/space/space/space/blog_post/img/.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
space/space/space/space/space/space/space/blog_post/img/gradio_pred_trip.png
CHANGED
|
|
space/space/space/space/space/space/space/code_blog_post.ipynb
CHANGED
|
@@ -155,7 +155,7 @@
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
@@ -206,7 +206,7 @@
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
-
"execution_count":
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
@@ -226,7 +226,7 @@
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
-
"execution_count":
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
@@ -234,23 +234,33 @@
|
|
| 234 |
"name": "stdout",
|
| 235 |
"output_type": "stream",
|
| 236 |
"text": [
|
| 237 |
-
"
|
| 238 |
-
"0
|
| 239 |
-
"1
|
| 240 |
-
"2
|
| 241 |
-
"3
|
| 242 |
-
"4
|
| 243 |
-
"5
|
| 244 |
-
"6
|
| 245 |
-
"7
|
| 246 |
-
"8
|
| 247 |
-
"9
|
| 248 |
-
"10
|
| 249 |
-
"11
|
| 250 |
-
"12
|
| 251 |
-
"13
|
| 252 |
-
"14
|
| 253 |
-
"15
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 254 |
]
|
| 255 |
}
|
| 256 |
],
|
|
@@ -262,8 +272,8 @@
|
|
| 262 |
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 263 |
"\n",
|
| 264 |
"df = pd.DataFrame({\n",
|
| 265 |
-
" \"Label\":
|
| 266 |
-
" \"Score\":
|
| 267 |
"})\n",
|
| 268 |
"print(df)\n",
|
| 269 |
"print(classes)"
|
|
@@ -271,23 +281,7 @@
|
|
| 271 |
},
|
| 272 |
{
|
| 273 |
"cell_type": "code",
|
| 274 |
-
"execution_count":
|
| 275 |
-
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 276 |
-
"metadata": {},
|
| 277 |
-
"outputs": [
|
| 278 |
-
{
|
| 279 |
-
"name": "stdout",
|
| 280 |
-
"output_type": "stream",
|
| 281 |
-
"text": [
|
| 282 |
-
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 283 |
-
]
|
| 284 |
-
}
|
| 285 |
-
],
|
| 286 |
-
"source": []
|
| 287 |
-
},
|
| 288 |
-
{
|
| 289 |
-
"cell_type": "code",
|
| 290 |
-
"execution_count": 8,
|
| 291 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 292 |
"metadata": {},
|
| 293 |
"outputs": [],
|
|
@@ -309,7 +303,7 @@
|
|
| 309 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 310 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 311 |
" for i, key in enumerate(keys_list):\n",
|
| 312 |
-
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 313 |
" if key == 'activities':\n",
|
| 314 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 315 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
@@ -323,7 +317,7 @@
|
|
| 323 |
},
|
| 324 |
{
|
| 325 |
"cell_type": "code",
|
| 326 |
-
"execution_count":
|
| 327 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 328 |
"metadata": {},
|
| 329 |
"outputs": [
|
|
@@ -338,7 +332,7 @@
|
|
| 338 |
"name": "stdout",
|
| 339 |
"output_type": "stream",
|
| 340 |
"text": [
|
| 341 |
-
"
|
| 342 |
"0 activity_type beach vacation\n",
|
| 343 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 344 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -366,7 +360,7 @@
|
|
| 366 |
},
|
| 367 |
{
|
| 368 |
"cell_type": "code",
|
| 369 |
-
"execution_count":
|
| 370 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 371 |
"metadata": {},
|
| 372 |
"outputs": [],
|
|
@@ -375,21 +369,12 @@
|
|
| 375 |
"from transformers import pipeline\n",
|
| 376 |
"import json\n",
|
| 377 |
"import pandas as pd\n",
|
| 378 |
-
"import gradio as gr
|
| 379 |
-
"\n",
|
| 380 |
-
"# get candidate labels\n",
|
| 381 |
-
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 382 |
-
" candidate_labels = json.load(file)\n",
|
| 383 |
-
"keys_list = list(candidate_labels.keys())\n",
|
| 384 |
-
"\n",
|
| 385 |
-
"# Load packing item data\n",
|
| 386 |
-
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 387 |
-
" packing_items = json.load(file)"
|
| 388 |
]
|
| 389 |
},
|
| 390 |
{
|
| 391 |
"cell_type": "code",
|
| 392 |
-
"execution_count":
|
| 393 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 394 |
"metadata": {},
|
| 395 |
"outputs": [
|
|
@@ -424,7 +409,7 @@
|
|
| 424 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 425 |
" ],\n",
|
| 426 |
" # outputs=\"dataframe\",\n",
|
| 427 |
-
" outputs=[gr.Dataframe(label=\"DataFrame\")
|
| 428 |
" title=\"Trip classification\",\n",
|
| 429 |
" description=\"Enter a text describing your trip\",\n",
|
| 430 |
")\n",
|
|
@@ -459,43 +444,290 @@
|
|
| 459 |
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 460 |
]
|
| 461 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 462 |
{
|
| 463 |
"cell_type": "code",
|
| 464 |
"execution_count": null,
|
| 465 |
-
"id": "
|
| 466 |
"metadata": {},
|
| 467 |
"outputs": [],
|
| 468 |
"source": [
|
| 469 |
-
"
|
| 470 |
]
|
| 471 |
},
|
| 472 |
{
|
| 473 |
"cell_type": "code",
|
| 474 |
-
"execution_count":
|
| 475 |
-
"id": "
|
| 476 |
"metadata": {},
|
| 477 |
"outputs": [
|
| 478 |
{
|
| 479 |
"name": "stdout",
|
| 480 |
"output_type": "stream",
|
| 481 |
"text": [
|
| 482 |
-
"
|
| 483 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 484 |
]
|
| 485 |
}
|
| 486 |
],
|
| 487 |
"source": [
|
| 488 |
-
"
|
| 489 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 490 |
]
|
| 491 |
},
|
| 492 |
{
|
| 493 |
"cell_type": "code",
|
| 494 |
"execution_count": null,
|
| 495 |
-
"id": "
|
| 496 |
"metadata": {},
|
| 497 |
"outputs": [],
|
| 498 |
-
"source": [
|
|
|
|
|
|
|
| 499 |
}
|
| 500 |
],
|
| 501 |
"metadata": {
|
|
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
+
"execution_count": 3,
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
+
"execution_count": 4,
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
+
"execution_count": 5,
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
|
|
| 234 |
"name": "stdout",
|
| 235 |
"output_type": "stream",
|
| 236 |
"text": [
|
| 237 |
+
" Label Score\n",
|
| 238 |
+
"0 going to the beach 0.991486\n",
|
| 239 |
+
"1 relaxing 0.977136\n",
|
| 240 |
+
"2 hiking 0.942628\n",
|
| 241 |
+
"3 swimming 0.219020\n",
|
| 242 |
+
"4 sightseeing 0.175862\n",
|
| 243 |
+
"5 running 0.098545\n",
|
| 244 |
+
"6 hut-to-hut hiking 0.083704\n",
|
| 245 |
+
"7 biking 0.036792\n",
|
| 246 |
+
"8 photography 0.036690\n",
|
| 247 |
+
"9 surfing 0.030993\n",
|
| 248 |
+
"10 stand-up paddleboarding (SUP) 0.025300\n",
|
| 249 |
+
"11 snorkeling 0.021451\n",
|
| 250 |
+
"12 yoga 0.011070\n",
|
| 251 |
+
"13 kayaking / canoeing 0.007511\n",
|
| 252 |
+
"14 rock climbing 0.006307\n",
|
| 253 |
+
"15 fishing 0.003497\n",
|
| 254 |
+
"16 paragliding 0.002656\n",
|
| 255 |
+
"17 rafting 0.001970\n",
|
| 256 |
+
"18 horseback riding 0.001560\n",
|
| 257 |
+
"19 snowshoe hiking 0.001528\n",
|
| 258 |
+
"20 cross-country skiing 0.001502\n",
|
| 259 |
+
"21 ice climbing 0.001434\n",
|
| 260 |
+
"22 skiing 0.001169\n",
|
| 261 |
+
"23 scuba diving 0.000789\n",
|
| 262 |
+
"24 ski touring 0.000491\n",
|
| 263 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 264 |
]
|
| 265 |
}
|
| 266 |
],
|
|
|
|
| 272 |
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 273 |
"\n",
|
| 274 |
"df = pd.DataFrame({\n",
|
| 275 |
+
" \"Label\": result_activ[\"labels\"],\n",
|
| 276 |
+
" \"Score\": result_activ[\"scores\"]\n",
|
| 277 |
"})\n",
|
| 278 |
"print(df)\n",
|
| 279 |
"print(classes)"
|
|
|
|
| 281 |
},
|
| 282 |
{
|
| 283 |
"cell_type": "code",
|
| 284 |
+
"execution_count": 6,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 286 |
"metadata": {},
|
| 287 |
"outputs": [],
|
|
|
|
| 303 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 304 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 305 |
" for i, key in enumerate(keys_list):\n",
|
| 306 |
+
" # print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 307 |
" if key == 'activities':\n",
|
| 308 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 309 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
|
|
| 317 |
},
|
| 318 |
{
|
| 319 |
"cell_type": "code",
|
| 320 |
+
"execution_count": 7,
|
| 321 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 322 |
"metadata": {},
|
| 323 |
"outputs": [
|
|
|
|
| 332 |
"name": "stdout",
|
| 333 |
"output_type": "stream",
|
| 334 |
"text": [
|
| 335 |
+
" superclass pred_class\n",
|
| 336 |
"0 activity_type beach vacation\n",
|
| 337 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 338 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 360 |
},
|
| 361 |
{
|
| 362 |
"cell_type": "code",
|
| 363 |
+
"execution_count": 9,
|
| 364 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 365 |
"metadata": {},
|
| 366 |
"outputs": [],
|
|
|
|
| 369 |
"from transformers import pipeline\n",
|
| 370 |
"import json\n",
|
| 371 |
"import pandas as pd\n",
|
| 372 |
+
"import gradio as gr"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 373 |
]
|
| 374 |
},
|
| 375 |
{
|
| 376 |
"cell_type": "code",
|
| 377 |
+
"execution_count": 10,
|
| 378 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 379 |
"metadata": {},
|
| 380 |
"outputs": [
|
|
|
|
| 409 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 410 |
" ],\n",
|
| 411 |
" # outputs=\"dataframe\",\n",
|
| 412 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 413 |
" title=\"Trip classification\",\n",
|
| 414 |
" description=\"Enter a text describing your trip\",\n",
|
| 415 |
")\n",
|
|
|
|
| 444 |
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 445 |
]
|
| 446 |
},
|
| 447 |
+
{
|
| 448 |
+
"cell_type": "markdown",
|
| 449 |
+
"id": "39920553-27d3-4a63-8381-8310566c4874",
|
| 450 |
+
"metadata": {},
|
| 451 |
+
"source": [
|
| 452 |
+
"All code for gradio app file"
|
| 453 |
+
]
|
| 454 |
+
},
|
| 455 |
+
{
|
| 456 |
+
"cell_type": "code",
|
| 457 |
+
"execution_count": 2,
|
| 458 |
+
"id": "4ffc76d5-60c3-4bc8-bd4f-a9636077f01d",
|
| 459 |
+
"metadata": {},
|
| 460 |
+
"outputs": [
|
| 461 |
+
{
|
| 462 |
+
"name": "stdout",
|
| 463 |
+
"output_type": "stream",
|
| 464 |
+
"text": [
|
| 465 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 466 |
+
"\n",
|
| 467 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 468 |
+
]
|
| 469 |
+
},
|
| 470 |
+
{
|
| 471 |
+
"data": {
|
| 472 |
+
"text/html": [
|
| 473 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 474 |
+
],
|
| 475 |
+
"text/plain": [
|
| 476 |
+
"<IPython.core.display.HTML object>"
|
| 477 |
+
]
|
| 478 |
+
},
|
| 479 |
+
"metadata": {},
|
| 480 |
+
"output_type": "display_data"
|
| 481 |
+
},
|
| 482 |
+
{
|
| 483 |
+
"name": "stderr",
|
| 484 |
+
"output_type": "stream",
|
| 485 |
+
"text": [
|
| 486 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 487 |
+
]
|
| 488 |
+
},
|
| 489 |
+
{
|
| 490 |
+
"name": "stdout",
|
| 491 |
+
"output_type": "stream",
|
| 492 |
+
"text": [
|
| 493 |
+
"Processing 9/9"
|
| 494 |
+
]
|
| 495 |
+
}
|
| 496 |
+
],
|
| 497 |
+
"source": [
|
| 498 |
+
"# Prerequisites\n",
|
| 499 |
+
"from transformers import pipeline\n",
|
| 500 |
+
"import json\n",
|
| 501 |
+
"import pandas as pd\n",
|
| 502 |
+
"import gradio as gr\n",
|
| 503 |
+
"\n",
|
| 504 |
+
"# Get candidate labels\n",
|
| 505 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 506 |
+
" candidate_labels = json.load(file)\n",
|
| 507 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 508 |
+
"\n",
|
| 509 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 510 |
+
" \"\"\"\n",
|
| 511 |
+
" Classifies trip\n",
|
| 512 |
+
" \n",
|
| 513 |
+
" Parameters:\n",
|
| 514 |
+
" model_name: name of hugging-face model\n",
|
| 515 |
+
" trip_descr: text describing the trip\n",
|
| 516 |
+
" cut_off: cut_off for choosing activities\n",
|
| 517 |
+
"\n",
|
| 518 |
+
" Returns:\n",
|
| 519 |
+
" pd Dataframe: with class predictions and true values\n",
|
| 520 |
+
" \"\"\"\n",
|
| 521 |
+
" \n",
|
| 522 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 523 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 524 |
+
" for i, key in enumerate(keys_list):\n",
|
| 525 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 526 |
+
" if key == 'activities':\n",
|
| 527 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 528 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 529 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 530 |
+
" else:\n",
|
| 531 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 532 |
+
" classes = result[\"labels\"][0]\n",
|
| 533 |
+
" df.loc[i] = [key, classes]\n",
|
| 534 |
+
" return df\n",
|
| 535 |
+
"\n",
|
| 536 |
+
"demo = gr.Interface(\n",
|
| 537 |
+
" fn=pred_trip,\n",
|
| 538 |
+
" inputs=[\n",
|
| 539 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 540 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 541 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 542 |
+
" ],\n",
|
| 543 |
+
" # outputs=\"dataframe\",\n",
|
| 544 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\")],\n",
|
| 545 |
+
" title=\"Trip classification\",\n",
|
| 546 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 547 |
+
")\n",
|
| 548 |
+
"\n",
|
| 549 |
+
"# Launch the Gradio app\n",
|
| 550 |
+
"if __name__ == \"__main__\":\n",
|
| 551 |
+
" demo.launch()\n"
|
| 552 |
+
]
|
| 553 |
+
},
|
| 554 |
{
|
| 555 |
"cell_type": "code",
|
| 556 |
"execution_count": null,
|
| 557 |
+
"id": "fb115832-d9c9-4cfd-8a2a-85916ce3a04a",
|
| 558 |
"metadata": {},
|
| 559 |
"outputs": [],
|
| 560 |
"source": [
|
| 561 |
+
"Print test data set"
|
| 562 |
]
|
| 563 |
},
|
| 564 |
{
|
| 565 |
"cell_type": "code",
|
| 566 |
+
"execution_count": 4,
|
| 567 |
+
"id": "f25e1462-ca48-4853-ae44-dfc096c5011d",
|
| 568 |
"metadata": {},
|
| 569 |
"outputs": [
|
| 570 |
{
|
| 571 |
"name": "stdout",
|
| 572 |
"output_type": "stream",
|
| 573 |
"text": [
|
| 574 |
+
"0 . I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 575 |
+
"\n",
|
| 576 |
+
"beach vacation\n",
|
| 577 |
+
"['swimming', 'going to the beach', 'relaxing', 'hiking']\n",
|
| 578 |
+
"warm destination / summer\n",
|
| 579 |
+
"lightweight (but comfortable)\n",
|
| 580 |
+
"casual\n",
|
| 581 |
+
"indoor\n",
|
| 582 |
+
"no own vehicle\n",
|
| 583 |
+
"no special conditions to consider\n",
|
| 584 |
+
"7+ days\n",
|
| 585 |
+
"\n",
|
| 586 |
+
"\n",
|
| 587 |
+
"1 . We are a couple in our thirties traveling to Vienna for a three-day city trip. We’ll be staying at a friend’s house and plan to explore the city by sightseeing, strolling through the streets, visiting markets, and trying out great restaurants and cafés. We also hope to attend a classical music concert. Our journey to Vienna will be by train. \n",
|
| 588 |
+
"\n",
|
| 589 |
+
"city trip\n",
|
| 590 |
+
"['sightseeing']\n",
|
| 591 |
+
"variable weather / spring / autumn\n",
|
| 592 |
+
"luxury (including evening wear)\n",
|
| 593 |
+
"casual\n",
|
| 594 |
+
"indoor\n",
|
| 595 |
+
"no own vehicle\n",
|
| 596 |
+
"no special conditions to consider\n",
|
| 597 |
+
"3 days\n",
|
| 598 |
+
"\n",
|
| 599 |
+
"\n",
|
| 600 |
+
"2 . My partner and I are traveling to the Netherlands and Germany to spend Christmas with our family. We are in our late twenties and will start our journey with a two-hour flight to the Netherlands. From there, we will take a 5.5-hour train ride to northern Germany. \n",
|
| 601 |
+
"\n",
|
| 602 |
+
"city trip\n",
|
| 603 |
+
"['relaxing']\n",
|
| 604 |
+
"cold destination / winter\n",
|
| 605 |
+
"lightweight (but comfortable)\n",
|
| 606 |
+
"casual\n",
|
| 607 |
+
"indoor\n",
|
| 608 |
+
"no own vehicle\n",
|
| 609 |
+
"no special conditions to consider\n",
|
| 610 |
+
"7+ days\n",
|
| 611 |
+
"\n",
|
| 612 |
+
"\n",
|
| 613 |
+
"3 . I’m in my twenties and will be traveling to Peru for three weeks. I’m going solo but will meet up with a friend to explore the Sacred Valley and take part in a Machu Picchu tour. We plan to hike, go rafting, and explore the remnants of the ancient Inca Empire. We’re also excited to try Peruvian cuisine and immerse ourselves in the local culture. Depending on our plans, we might also visit the rainforest region, such as Tarapoto. I’ll be flying to Peru on a long-haul flight and will be traveling in August. \n",
|
| 614 |
+
"\n",
|
| 615 |
+
"cultural exploration\n",
|
| 616 |
+
"['sightseeing', 'hiking', 'rafting']\n",
|
| 617 |
+
"variable weather / spring / autumn\n",
|
| 618 |
+
"lightweight (but comfortable)\n",
|
| 619 |
+
"casual\n",
|
| 620 |
+
"indoor\n",
|
| 621 |
+
"no own vehicle\n",
|
| 622 |
+
"rainy climate\n",
|
| 623 |
+
"7+ days\n",
|
| 624 |
+
"\n",
|
| 625 |
+
"\n",
|
| 626 |
+
"4 . We’re planning a 10-day trip to Austria in the summer, combining hiking with relaxation by the lake. We love exploring scenic trails and enjoying the outdoors, but we also want to unwind and swim in the lake. It’s the perfect mix of adventure and relaxation. \n",
|
| 627 |
+
"\n",
|
| 628 |
+
"nature escape\n",
|
| 629 |
+
"['swimming', 'relaxing', 'hiking']\n",
|
| 630 |
+
"warm destination / summer\n",
|
| 631 |
+
"lightweight (but comfortable)\n",
|
| 632 |
+
"casual\n",
|
| 633 |
+
"indoor\n",
|
| 634 |
+
"no own vehicle\n",
|
| 635 |
+
"no special conditions to consider\n",
|
| 636 |
+
"7+ days\n",
|
| 637 |
+
"\n",
|
| 638 |
+
"\n",
|
| 639 |
+
"5 . I am going on a multiple day hike and passing though mountains and the beach in Croatia. I like to pack light and will stay in refugios/huts with half board and travel to the start of the hike by car. It will be 6-7 days. \n",
|
| 640 |
+
"\n",
|
| 641 |
+
"long-distance hike / thru-hike\n",
|
| 642 |
+
"['going to the beach']\n",
|
| 643 |
+
"tropical / humid\n",
|
| 644 |
+
"minimalist\n",
|
| 645 |
+
"casual\n",
|
| 646 |
+
"huts with half board\n",
|
| 647 |
+
"own vehicle\n",
|
| 648 |
+
"off-grid / no electricity\n",
|
| 649 |
+
"6 days\n",
|
| 650 |
+
"\n",
|
| 651 |
+
"\n",
|
| 652 |
+
"6 . I will go with a friend on a beach holiday and we will do stand-up paddling, and surfing in the North of Spain. The destination is windy and can get cold, but is generally sunny. We will go by car and bring a tent to sleep in. It will be two weeks. \n",
|
| 653 |
+
"\n",
|
| 654 |
+
"beach vacation\n",
|
| 655 |
+
"['stand-up paddleboarding (SUP)', 'surfing']\n",
|
| 656 |
+
"cold destination / winter\n",
|
| 657 |
+
"ultralight\n",
|
| 658 |
+
"casual\n",
|
| 659 |
+
"sleeping in a tent\n",
|
| 660 |
+
"own vehicle\n",
|
| 661 |
+
"off-grid / no electricity\n",
|
| 662 |
+
"6 days\n",
|
| 663 |
+
"\n",
|
| 664 |
+
"\n",
|
| 665 |
+
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 666 |
+
"\n",
|
| 667 |
+
"yoga / wellness retreat\n",
|
| 668 |
+
"['hiking', 'yoga']\n",
|
| 669 |
+
"cold destination / winter\n",
|
| 670 |
+
"lightweight (but comfortable)\n",
|
| 671 |
+
"casual\n",
|
| 672 |
+
"indoor\n",
|
| 673 |
+
"no own vehicle\n",
|
| 674 |
+
"snow and ice\n",
|
| 675 |
+
"7 days\n",
|
| 676 |
+
"\n",
|
| 677 |
+
"\n",
|
| 678 |
+
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 679 |
+
"\n",
|
| 680 |
+
"ski tour / skitour\n",
|
| 681 |
+
"['ski touring', 'photography']\n",
|
| 682 |
+
"cold destination / winter\n",
|
| 683 |
+
"minimalist\n",
|
| 684 |
+
"conservative\n",
|
| 685 |
+
"indoor\n",
|
| 686 |
+
"no own vehicle\n",
|
| 687 |
+
"avalanche-prone terrain\n",
|
| 688 |
+
"7+ days\n",
|
| 689 |
+
"\n",
|
| 690 |
+
"\n",
|
| 691 |
+
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
| 692 |
+
"\n",
|
| 693 |
+
"camping trip (wild camping)\n",
|
| 694 |
+
"['scuba diving', 'kayaking / canoeing']\n",
|
| 695 |
+
"tropical / humid\n",
|
| 696 |
+
"lightweight (but comfortable)\n",
|
| 697 |
+
"casual\n",
|
| 698 |
+
"sleeping in a tent\n",
|
| 699 |
+
"own vehicle\n",
|
| 700 |
+
"self-supported (bring your own cooking gear)\n",
|
| 701 |
+
"3 days\n",
|
| 702 |
+
"\n",
|
| 703 |
+
"\n"
|
| 704 |
]
|
| 705 |
}
|
| 706 |
],
|
| 707 |
"source": [
|
| 708 |
+
"# Load test data (list of dictionaries)\n",
|
| 709 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 710 |
+
" packing_data = json.load(file)\n",
|
| 711 |
+
" # Extract trip descriptions and classification (trip_types)\n",
|
| 712 |
+
"trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 713 |
+
"trip_types = [trip['trip_types'] for trip in packing_data]\n",
|
| 714 |
+
"\n",
|
| 715 |
+
"for i, item in enumerate(trip_descriptions):\n",
|
| 716 |
+
" print(i, \".\", item, \"\\n\")\n",
|
| 717 |
+
" for elem in trip_types[i]:\n",
|
| 718 |
+
" print(elem)\n",
|
| 719 |
+
" print(\"\\n\")"
|
| 720 |
]
|
| 721 |
},
|
| 722 |
{
|
| 723 |
"cell_type": "code",
|
| 724 |
"execution_count": null,
|
| 725 |
+
"id": "23a306bc-5dac-4569-a727-d140e1b8da7a",
|
| 726 |
"metadata": {},
|
| 727 |
"outputs": [],
|
| 728 |
+
"source": [
|
| 729 |
+
"# "
|
| 730 |
+
]
|
| 731 |
}
|
| 732 |
],
|
| 733 |
"metadata": {
|
space/space/space/space/space/space/space/main_model.ipynb
CHANGED
|
@@ -307,7 +307,7 @@
|
|
| 307 |
},
|
| 308 |
{
|
| 309 |
"cell_type": "code",
|
| 310 |
-
"execution_count":
|
| 311 |
"id": "dd7869a8-b436-40de-9ea0-28eb4b7d3248",
|
| 312 |
"metadata": {
|
| 313 |
"scrolled": true
|
|
@@ -318,9 +318,501 @@
|
|
| 318 |
"output_type": "stream",
|
| 319 |
"text": [
|
| 320 |
"\n",
|
| 321 |
-
"Using model:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 322 |
]
|
| 323 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 324 |
{
|
| 325 |
"name": "stderr",
|
| 326 |
"output_type": "stream",
|
|
@@ -329,44 +821,223 @@
|
|
| 329 |
]
|
| 330 |
},
|
| 331 |
{
|
| 332 |
-
"
|
| 333 |
-
"
|
| 334 |
-
"
|
| 335 |
-
|
| 336 |
-
"\
|
| 337 |
-
"\
|
| 338 |
-
"
|
| 339 |
-
"
|
| 340 |
-
"
|
| 341 |
-
"
|
| 342 |
-
"
|
| 343 |
-
"
|
| 344 |
-
|
| 345 |
-
|
| 346 |
-
|
| 347 |
-
|
| 348 |
-
|
| 349 |
-
|
| 350 |
-
"
|
| 351 |
-
|
| 352 |
-
|
| 353 |
-
|
| 354 |
-
|
| 355 |
-
|
| 356 |
-
|
| 357 |
-
"
|
| 358 |
-
"
|
| 359 |
-
"
|
| 360 |
-
"
|
| 361 |
-
"
|
| 362 |
-
"
|
| 363 |
-
"
|
| 364 |
-
"\
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 365 |
]
|
| 366 |
}
|
| 367 |
],
|
| 368 |
"source": [
|
| 369 |
"# List of Hugging Face model names\n",
|
|
|
|
|
|
|
| 370 |
"model_names = [\n",
|
| 371 |
" \"facebook/bart-large-mnli\",\n",
|
| 372 |
" \"MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli\",\n",
|
|
@@ -379,6 +1050,23 @@
|
|
| 379 |
" \"valhalla/distilbart-mnli-12-1\",\n",
|
| 380 |
" #\"joeddav/xlm-roberta-large-xnli\" # keeps giving errors\n",
|
| 381 |
"]\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 382 |
"\n",
|
| 383 |
"# Apply each model to the test data\n",
|
| 384 |
"for model_name in model_names:\n",
|
|
@@ -436,7 +1124,7 @@
|
|
| 436 |
},
|
| 437 |
{
|
| 438 |
"cell_type": "code",
|
| 439 |
-
"execution_count":
|
| 440 |
"id": "37849e0b-864e-4377-b06c-0ac70c3861f9",
|
| 441 |
"metadata": {
|
| 442 |
"scrolled": true
|
|
@@ -488,6 +1176,27 @@
|
|
| 488 |
"false_pred 0.500\n",
|
| 489 |
"dtype: float64\n",
|
| 490 |
"----------------------------------------\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 491 |
"Model: facebook-bart-large-mnli\n",
|
| 492 |
"Performance Summary:\n",
|
| 493 |
"accuracy 0.466667\n",
|
|
@@ -537,6 +1246,24 @@
|
|
| 537 |
" print(\"-\" * 40)\n"
|
| 538 |
]
|
| 539 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 540 |
{
|
| 541 |
"cell_type": "markdown",
|
| 542 |
"id": "2f65e5b1-bc32-42c2-bbe9-9e3a6ffc72c1",
|
|
@@ -577,6 +1304,12 @@
|
|
| 577 |
"\n",
|
| 578 |
"MoritzLaurer-deberta-v3-large-zeroshot-v2.0: Index([1, 2, 3, 5, 6, 7, 9], dtype='int64')\n",
|
| 579 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 580 |
"facebook-bart-large-mnli: RangeIndex(start=0, stop=10, step=1)\n",
|
| 581 |
"\n",
|
| 582 |
"valhalla-distilbart-mnli-12-1: Index([0, 1, 2, 3, 4, 7, 9], dtype='int64')\n",
|
|
@@ -652,19 +1385,6 @@
|
|
| 652 |
"rainy climate\n",
|
| 653 |
"7+ days\n",
|
| 654 |
"\n",
|
| 655 |
-
"\n",
|
| 656 |
-
"7 . We will go to Sweden in the winter, to go for a yoga and sauna/wellness retreat. I prefer lightweight packing and also want clothes to go for fancy dinners and maybe on a winter hike. We stay in hotels. \n",
|
| 657 |
-
"\n",
|
| 658 |
-
"yoga / wellness retreat\n",
|
| 659 |
-
"['hiking', 'yoga']\n",
|
| 660 |
-
"cold destination / winter\n",
|
| 661 |
-
"lightweight (but comfortable)\n",
|
| 662 |
-
"casual\n",
|
| 663 |
-
"indoor\n",
|
| 664 |
-
"no own vehicle\n",
|
| 665 |
-
"snow and ice\n",
|
| 666 |
-
"7 days\n",
|
| 667 |
-
"\n",
|
| 668 |
"\n"
|
| 669 |
]
|
| 670 |
}
|
|
@@ -717,6 +1437,12 @@
|
|
| 717 |
"\n",
|
| 718 |
"MoritzLaurer-deberta-v3-large-zeroshot-v2.0: ['activities', 'climate_or_season', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
| 719 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 720 |
"facebook-bart-large-mnli: ['activities', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
| 721 |
"\n",
|
| 722 |
"valhalla-distilbart-mnli-12-1: ['activities', 'climate_or_season', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
|
@@ -762,7 +1488,7 @@
|
|
| 762 |
"name": "stdout",
|
| 763 |
"output_type": "stream",
|
| 764 |
"text": [
|
| 765 |
-
"{'
|
| 766 |
]
|
| 767 |
}
|
| 768 |
],
|
|
@@ -793,7 +1519,7 @@
|
|
| 793 |
},
|
| 794 |
{
|
| 795 |
"cell_type": "code",
|
| 796 |
-
"execution_count":
|
| 797 |
"id": "b020f584-1468-4c84-9dac-7ca7fac6e8ca",
|
| 798 |
"metadata": {
|
| 799 |
"scrolled": true
|
|
@@ -803,57 +1529,45 @@
|
|
| 803 |
"name": "stdout",
|
| 804 |
"output_type": "stream",
|
| 805 |
"text": [
|
| 806 |
-
"
|
| 807 |
-
"
|
| 808 |
-
"
|
| 809 |
-
"
|
| 810 |
-
"
|
| 811 |
-
"
|
| 812 |
-
"
|
| 813 |
-
"
|
| 814 |
-
"
|
| 815 |
-
"
|
| 816 |
-
"
|
| 817 |
-
"\n",
|
| 818 |
-
"
|
| 819 |
-
"0 cross-encoder-nli-deberta-v3-base \n",
|
| 820 |
-
"1 joeddav-bart-large-mnli-yahoo-answers \n",
|
| 821 |
-
"2 cross-encoder-nli-deberta-v3-large \n",
|
| 822 |
-
"3 MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-... \n",
|
| 823 |
-
"4 MoritzLaurer-mDeBERTa-v3-base-mnli-xnli \n",
|
| 824 |
-
"5 MoritzLaurer-deberta-v3-large-zeroshot-v2.0 \n",
|
| 825 |
-
"6 facebook-bart-large-mnli \n",
|
| 826 |
-
"7 valhalla-distilbart-mnli-12-1 \n",
|
| 827 |
-
"8 MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli \n"
|
| 828 |
]
|
| 829 |
}
|
| 830 |
],
|
| 831 |
"source": [
|
| 832 |
-
"
|
| 833 |
-
"
|
| 834 |
-
"
|
| 835 |
-
"#print(all_results[\"MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli\"].keys())\n",
|
| 836 |
-
"#print(type(all_results[\"MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli\"][\"perf_summary\"]))\n",
|
| 837 |
-
"#print(all_results[\"MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli\"][\"perf_summary\"])\n",
|
| 838 |
-
"#print(all_results[\"MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli\"][\"perf_summary\"][\"accuracy\"])\n",
|
| 839 |
-
"# make empty data frame\n",
|
| 840 |
-
"perf_table = []\n",
|
| 841 |
-
"print(perf_table)\n",
|
| 842 |
"\n",
|
|
|
|
| 843 |
"# fill in for loop with perf_summary per model\n",
|
| 844 |
"for model, result in all_results.items():\n",
|
| 845 |
" row = pd.DataFrame(result[\"perf_summary\"]).T\n",
|
| 846 |
-
" #
|
| 847 |
-
" row
|
| 848 |
" perf_table.append(row)\n",
|
| 849 |
"# Concatenate all into one table\n",
|
| 850 |
"df_all = pd.concat(perf_table, ignore_index=True)\n",
|
|
|
|
|
|
|
|
|
|
| 851 |
"\n",
|
| 852 |
-
"print(
|
| 853 |
"#print(type(df_all))\n",
|
| 854 |
" \n",
|
| 855 |
"\n",
|
| 856 |
-
"#
|
| 857 |
]
|
| 858 |
},
|
| 859 |
{
|
|
|
|
| 307 |
},
|
| 308 |
{
|
| 309 |
"cell_type": "code",
|
| 310 |
+
"execution_count": 22,
|
| 311 |
"id": "dd7869a8-b436-40de-9ea0-28eb4b7d3248",
|
| 312 |
"metadata": {
|
| 313 |
"scrolled": true
|
|
|
|
| 318 |
"output_type": "stream",
|
| 319 |
"text": [
|
| 320 |
"\n",
|
| 321 |
+
"Using model: pongjin/roberta_with_kornli\n"
|
| 322 |
+
]
|
| 323 |
+
},
|
| 324 |
+
{
|
| 325 |
+
"data": {
|
| 326 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 327 |
+
"model_id": "a44415d353264dda885031d6570e21a7",
|
| 328 |
+
"version_major": 2,
|
| 329 |
+
"version_minor": 0
|
| 330 |
+
},
|
| 331 |
+
"text/plain": [
|
| 332 |
+
"config.json: 0%| | 0.00/985 [00:00<?, ?B/s]"
|
| 333 |
+
]
|
| 334 |
+
},
|
| 335 |
+
"metadata": {},
|
| 336 |
+
"output_type": "display_data"
|
| 337 |
+
},
|
| 338 |
+
{
|
| 339 |
+
"data": {
|
| 340 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 341 |
+
"model_id": "d31b49452fe64db39745a833a832a340",
|
| 342 |
+
"version_major": 2,
|
| 343 |
+
"version_minor": 0
|
| 344 |
+
},
|
| 345 |
+
"text/plain": [
|
| 346 |
+
"pytorch_model.bin: 0%| | 0.00/443M [00:00<?, ?B/s]"
|
| 347 |
+
]
|
| 348 |
+
},
|
| 349 |
+
"metadata": {},
|
| 350 |
+
"output_type": "display_data"
|
| 351 |
+
},
|
| 352 |
+
{
|
| 353 |
+
"data": {
|
| 354 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 355 |
+
"model_id": "a4c31853bc854b8d91dca8689b7cdc18",
|
| 356 |
+
"version_major": 2,
|
| 357 |
+
"version_minor": 0
|
| 358 |
+
},
|
| 359 |
+
"text/plain": [
|
| 360 |
+
"tokenizer_config.json: 0%| | 0.00/415 [00:00<?, ?B/s]"
|
| 361 |
+
]
|
| 362 |
+
},
|
| 363 |
+
"metadata": {},
|
| 364 |
+
"output_type": "display_data"
|
| 365 |
+
},
|
| 366 |
+
{
|
| 367 |
+
"data": {
|
| 368 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 369 |
+
"model_id": "048df7d028224b5bb36b5b8042c8579b",
|
| 370 |
+
"version_major": 2,
|
| 371 |
+
"version_minor": 0
|
| 372 |
+
},
|
| 373 |
+
"text/plain": [
|
| 374 |
+
"vocab.txt: 0.00B [00:00, ?B/s]"
|
| 375 |
+
]
|
| 376 |
+
},
|
| 377 |
+
"metadata": {},
|
| 378 |
+
"output_type": "display_data"
|
| 379 |
+
},
|
| 380 |
+
{
|
| 381 |
+
"data": {
|
| 382 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 383 |
+
"model_id": "b9a39be1b8024e27aee465f0253c8274",
|
| 384 |
+
"version_major": 2,
|
| 385 |
+
"version_minor": 0
|
| 386 |
+
},
|
| 387 |
+
"text/plain": [
|
| 388 |
+
"tokenizer.json: 0.00B [00:00, ?B/s]"
|
| 389 |
+
]
|
| 390 |
+
},
|
| 391 |
+
"metadata": {},
|
| 392 |
+
"output_type": "display_data"
|
| 393 |
+
},
|
| 394 |
+
{
|
| 395 |
+
"data": {
|
| 396 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 397 |
+
"model_id": "c4170138359149728e468730ee237f8e",
|
| 398 |
+
"version_major": 2,
|
| 399 |
+
"version_minor": 0
|
| 400 |
+
},
|
| 401 |
+
"text/plain": [
|
| 402 |
+
"special_tokens_map.json: 0%| | 0.00/173 [00:00<?, ?B/s]"
|
| 403 |
+
]
|
| 404 |
+
},
|
| 405 |
+
"metadata": {},
|
| 406 |
+
"output_type": "display_data"
|
| 407 |
+
},
|
| 408 |
+
{
|
| 409 |
+
"name": "stderr",
|
| 410 |
+
"output_type": "stream",
|
| 411 |
+
"text": [
|
| 412 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 413 |
+
]
|
| 414 |
+
},
|
| 415 |
+
{
|
| 416 |
+
"name": "stdout",
|
| 417 |
+
"output_type": "stream",
|
| 418 |
+
"text": [
|
| 419 |
+
"0\n",
|
| 420 |
+
"1\n",
|
| 421 |
+
"2\n",
|
| 422 |
+
"3\n",
|
| 423 |
+
"4\n",
|
| 424 |
+
"5\n",
|
| 425 |
+
"6\n",
|
| 426 |
+
"7\n",
|
| 427 |
+
"8\n"
|
| 428 |
+
]
|
| 429 |
+
},
|
| 430 |
+
{
|
| 431 |
+
"name": "stderr",
|
| 432 |
+
"output_type": "stream",
|
| 433 |
+
"text": [
|
| 434 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 435 |
+
]
|
| 436 |
+
},
|
| 437 |
+
{
|
| 438 |
+
"name": "stdout",
|
| 439 |
+
"output_type": "stream",
|
| 440 |
+
"text": [
|
| 441 |
+
"0\n",
|
| 442 |
+
"1\n",
|
| 443 |
+
"2\n",
|
| 444 |
+
"3\n",
|
| 445 |
+
"4\n",
|
| 446 |
+
"5\n",
|
| 447 |
+
"6\n",
|
| 448 |
+
"7\n",
|
| 449 |
+
"8\n"
|
| 450 |
+
]
|
| 451 |
+
},
|
| 452 |
+
{
|
| 453 |
+
"name": "stderr",
|
| 454 |
+
"output_type": "stream",
|
| 455 |
+
"text": [
|
| 456 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 457 |
+
]
|
| 458 |
+
},
|
| 459 |
+
{
|
| 460 |
+
"name": "stdout",
|
| 461 |
+
"output_type": "stream",
|
| 462 |
+
"text": [
|
| 463 |
+
"0\n",
|
| 464 |
+
"1\n",
|
| 465 |
+
"2\n",
|
| 466 |
+
"3\n",
|
| 467 |
+
"4\n",
|
| 468 |
+
"5\n",
|
| 469 |
+
"6\n",
|
| 470 |
+
"7\n",
|
| 471 |
+
"8\n"
|
| 472 |
+
]
|
| 473 |
+
},
|
| 474 |
+
{
|
| 475 |
+
"name": "stderr",
|
| 476 |
+
"output_type": "stream",
|
| 477 |
+
"text": [
|
| 478 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 479 |
+
]
|
| 480 |
+
},
|
| 481 |
+
{
|
| 482 |
+
"name": "stdout",
|
| 483 |
+
"output_type": "stream",
|
| 484 |
+
"text": [
|
| 485 |
+
"0\n",
|
| 486 |
+
"1\n",
|
| 487 |
+
"2\n",
|
| 488 |
+
"3\n",
|
| 489 |
+
"4\n",
|
| 490 |
+
"5\n",
|
| 491 |
+
"6\n",
|
| 492 |
+
"7\n",
|
| 493 |
+
"8\n"
|
| 494 |
+
]
|
| 495 |
+
},
|
| 496 |
+
{
|
| 497 |
+
"name": "stderr",
|
| 498 |
+
"output_type": "stream",
|
| 499 |
+
"text": [
|
| 500 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 501 |
+
]
|
| 502 |
+
},
|
| 503 |
+
{
|
| 504 |
+
"name": "stdout",
|
| 505 |
+
"output_type": "stream",
|
| 506 |
+
"text": [
|
| 507 |
+
"0\n",
|
| 508 |
+
"1\n",
|
| 509 |
+
"2\n",
|
| 510 |
+
"3\n",
|
| 511 |
+
"4\n",
|
| 512 |
+
"5\n",
|
| 513 |
+
"6\n",
|
| 514 |
+
"7\n",
|
| 515 |
+
"8\n"
|
| 516 |
+
]
|
| 517 |
+
},
|
| 518 |
+
{
|
| 519 |
+
"name": "stderr",
|
| 520 |
+
"output_type": "stream",
|
| 521 |
+
"text": [
|
| 522 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 523 |
+
]
|
| 524 |
+
},
|
| 525 |
+
{
|
| 526 |
+
"name": "stdout",
|
| 527 |
+
"output_type": "stream",
|
| 528 |
+
"text": [
|
| 529 |
+
"0\n",
|
| 530 |
+
"1\n",
|
| 531 |
+
"2\n",
|
| 532 |
+
"3\n",
|
| 533 |
+
"4\n",
|
| 534 |
+
"5\n",
|
| 535 |
+
"6\n",
|
| 536 |
+
"7\n",
|
| 537 |
+
"8\n"
|
| 538 |
+
]
|
| 539 |
+
},
|
| 540 |
+
{
|
| 541 |
+
"name": "stderr",
|
| 542 |
+
"output_type": "stream",
|
| 543 |
+
"text": [
|
| 544 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 545 |
+
]
|
| 546 |
+
},
|
| 547 |
+
{
|
| 548 |
+
"name": "stdout",
|
| 549 |
+
"output_type": "stream",
|
| 550 |
+
"text": [
|
| 551 |
+
"0\n",
|
| 552 |
+
"1\n",
|
| 553 |
+
"2\n",
|
| 554 |
+
"3\n",
|
| 555 |
+
"4\n",
|
| 556 |
+
"5\n",
|
| 557 |
+
"6\n",
|
| 558 |
+
"7\n",
|
| 559 |
+
"8\n"
|
| 560 |
+
]
|
| 561 |
+
},
|
| 562 |
+
{
|
| 563 |
+
"name": "stderr",
|
| 564 |
+
"output_type": "stream",
|
| 565 |
+
"text": [
|
| 566 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 567 |
+
]
|
| 568 |
+
},
|
| 569 |
+
{
|
| 570 |
+
"name": "stdout",
|
| 571 |
+
"output_type": "stream",
|
| 572 |
+
"text": [
|
| 573 |
+
"0\n",
|
| 574 |
+
"1\n",
|
| 575 |
+
"2\n",
|
| 576 |
+
"3\n",
|
| 577 |
+
"4\n",
|
| 578 |
+
"5\n",
|
| 579 |
+
"6\n",
|
| 580 |
+
"7\n",
|
| 581 |
+
"8\n"
|
| 582 |
+
]
|
| 583 |
+
},
|
| 584 |
+
{
|
| 585 |
+
"name": "stderr",
|
| 586 |
+
"output_type": "stream",
|
| 587 |
+
"text": [
|
| 588 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 589 |
+
]
|
| 590 |
+
},
|
| 591 |
+
{
|
| 592 |
+
"name": "stdout",
|
| 593 |
+
"output_type": "stream",
|
| 594 |
+
"text": [
|
| 595 |
+
"0\n",
|
| 596 |
+
"1\n",
|
| 597 |
+
"2\n",
|
| 598 |
+
"3\n",
|
| 599 |
+
"4\n",
|
| 600 |
+
"5\n",
|
| 601 |
+
"6\n",
|
| 602 |
+
"7\n",
|
| 603 |
+
"8\n"
|
| 604 |
+
]
|
| 605 |
+
},
|
| 606 |
+
{
|
| 607 |
+
"name": "stderr",
|
| 608 |
+
"output_type": "stream",
|
| 609 |
+
"text": [
|
| 610 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 611 |
+
]
|
| 612 |
+
},
|
| 613 |
+
{
|
| 614 |
+
"name": "stdout",
|
| 615 |
+
"output_type": "stream",
|
| 616 |
+
"text": [
|
| 617 |
+
"0\n",
|
| 618 |
+
"1\n",
|
| 619 |
+
"2\n",
|
| 620 |
+
"3\n",
|
| 621 |
+
"4\n",
|
| 622 |
+
"5\n",
|
| 623 |
+
"6\n",
|
| 624 |
+
"7\n",
|
| 625 |
+
"8\n",
|
| 626 |
+
"\n",
|
| 627 |
+
"Using model: MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7\n"
|
| 628 |
+
]
|
| 629 |
+
},
|
| 630 |
+
{
|
| 631 |
+
"data": {
|
| 632 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 633 |
+
"model_id": "c1b011a630894a23b58f4a88e97a849d",
|
| 634 |
+
"version_major": 2,
|
| 635 |
+
"version_minor": 0
|
| 636 |
+
},
|
| 637 |
+
"text/plain": [
|
| 638 |
+
"config.json: 0.00B [00:00, ?B/s]"
|
| 639 |
+
]
|
| 640 |
+
},
|
| 641 |
+
"metadata": {},
|
| 642 |
+
"output_type": "display_data"
|
| 643 |
+
},
|
| 644 |
+
{
|
| 645 |
+
"data": {
|
| 646 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 647 |
+
"model_id": "5fa889524f9c439d9bbd79426e2eefa6",
|
| 648 |
+
"version_major": 2,
|
| 649 |
+
"version_minor": 0
|
| 650 |
+
},
|
| 651 |
+
"text/plain": [
|
| 652 |
+
"model.safetensors: 0%| | 0.00/558M [00:00<?, ?B/s]"
|
| 653 |
+
]
|
| 654 |
+
},
|
| 655 |
+
"metadata": {},
|
| 656 |
+
"output_type": "display_data"
|
| 657 |
+
},
|
| 658 |
+
{
|
| 659 |
+
"name": "stderr",
|
| 660 |
+
"output_type": "stream",
|
| 661 |
+
"text": [
|
| 662 |
+
"Error while downloading from https://cas-bridge.xethub.hf.co/xet-bridge-us/6303b5f71dd5d3c62482f3e9/11250a388e769f8d819bc3d9b55590ab43bde78b2d4f37df57bc5603c5dea0b0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=cas%2F20251104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251104T105145Z&X-Amz-Expires=3600&X-Amz-Signature=603bdf8dd3ba17470e9f30bd3b31141b09a02d57e6cf00b63996b798d985c323&X-Amz-SignedHeaders=host&X-Xet-Cas-Uid=public&response-content-disposition=inline%3B+filename*%3DUTF-8%27%27model.safetensors%3B+filename%3D%22model.safetensors%22%3B&x-id=GetObject&Expires=1762257105&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc2MjI1NzEwNX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2FzLWJyaWRnZS54ZXRodWIuaGYuY28veGV0LWJyaWRnZS11cy82MzAzYjVmNzFkZDVkM2M2MjQ4MmYzZTkvMTEyNTBhMzg4ZTc2OWY4ZDgxOWJjM2Q5YjU1NTkwYWI0M2JkZTc4YjJkNGYzN2RmNTdiYzU2MDNjNWRlYTBiMCoifV19&Signature=s7MbLznU9vhypkS2WueXkH%7E41QifR0qWT8YHXKdMNPheh483kMtmwoaCCqvVlcukJygS0K2zvyynfo8zpNM%7EqiH8ZI1k3E7B2f6UH97HlJA48Th3pabY4zi1WYI2IeZDnBdPlreb3qTCkCZw1l11fyt9ri2QOaxuRrT6jcxPOquIFzz5lZ0sZNjngA0rg8gtCGi89qsWATMV3Oh45cRpO%7E6DrmmHfP8LEk8sZOiu9Zi-33x-ZoO4oayJSvzxOw8y8TWs%7E5q3cD9geNVIToO-PJPHQJ6qk9FybNeN75zhWtBtERN77Ie2mf9mGEcnZVqBd%7EH4kyAyYsWUcwviGjJSxA__&Key-Pair-Id=K2L8F4GPSG1IFC: HTTPSConnectionPool(host='cas-bridge.xethub.hf.co', port=443): Read timed out.\n",
|
| 663 |
+
"Trying to resume download...\n"
|
| 664 |
+
]
|
| 665 |
+
},
|
| 666 |
+
{
|
| 667 |
+
"data": {
|
| 668 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 669 |
+
"model_id": "137e0628b2464c4ba34304ff81db5cc3",
|
| 670 |
+
"version_major": 2,
|
| 671 |
+
"version_minor": 0
|
| 672 |
+
},
|
| 673 |
+
"text/plain": [
|
| 674 |
+
"model.safetensors: 26%|##6 | 147M/558M [00:00<?, ?B/s]"
|
| 675 |
+
]
|
| 676 |
+
},
|
| 677 |
+
"metadata": {},
|
| 678 |
+
"output_type": "display_data"
|
| 679 |
+
},
|
| 680 |
+
{
|
| 681 |
+
"name": "stderr",
|
| 682 |
+
"output_type": "stream",
|
| 683 |
+
"text": [
|
| 684 |
+
"Error while downloading from https://cas-bridge.xethub.hf.co/xet-bridge-us/6303b5f71dd5d3c62482f3e9/11250a388e769f8d819bc3d9b55590ab43bde78b2d4f37df57bc5603c5dea0b0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=cas%2F20251104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251104T105145Z&X-Amz-Expires=3600&X-Amz-Signature=603bdf8dd3ba17470e9f30bd3b31141b09a02d57e6cf00b63996b798d985c323&X-Amz-SignedHeaders=host&X-Xet-Cas-Uid=public&response-content-disposition=inline%3B+filename*%3DUTF-8%27%27model.safetensors%3B+filename%3D%22model.safetensors%22%3B&x-id=GetObject&Expires=1762257105&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc2MjI1NzEwNX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2FzLWJyaWRnZS54ZXRodWIuaGYuY28veGV0LWJyaWRnZS11cy82MzAzYjVmNzFkZDVkM2M2MjQ4MmYzZTkvMTEyNTBhMzg4ZTc2OWY4ZDgxOWJjM2Q5YjU1NTkwYWI0M2JkZTc4YjJkNGYzN2RmNTdiYzU2MDNjNWRlYTBiMCoifV19&Signature=s7MbLznU9vhypkS2WueXkH%7E41QifR0qWT8YHXKdMNPheh483kMtmwoaCCqvVlcukJygS0K2zvyynfo8zpNM%7EqiH8ZI1k3E7B2f6UH97HlJA48Th3pabY4zi1WYI2IeZDnBdPlreb3qTCkCZw1l11fyt9ri2QOaxuRrT6jcxPOquIFzz5lZ0sZNjngA0rg8gtCGi89qsWATMV3Oh45cRpO%7E6DrmmHfP8LEk8sZOiu9Zi-33x-ZoO4oayJSvzxOw8y8TWs%7E5q3cD9geNVIToO-PJPHQJ6qk9FybNeN75zhWtBtERN77Ie2mf9mGEcnZVqBd%7EH4kyAyYsWUcwviGjJSxA__&Key-Pair-Id=K2L8F4GPSG1IFC: HTTPSConnectionPool(host='cas-bridge.xethub.hf.co', port=443): Read timed out.\n",
|
| 685 |
+
"Trying to resume download...\n"
|
| 686 |
+
]
|
| 687 |
+
},
|
| 688 |
+
{
|
| 689 |
+
"data": {
|
| 690 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 691 |
+
"model_id": "9a7e7a8614d14b789f54c4e61b526bf8",
|
| 692 |
+
"version_major": 2,
|
| 693 |
+
"version_minor": 0
|
| 694 |
+
},
|
| 695 |
+
"text/plain": [
|
| 696 |
+
"model.safetensors: 26%|##6 | 147M/558M [00:00<?, ?B/s]"
|
| 697 |
+
]
|
| 698 |
+
},
|
| 699 |
+
"metadata": {},
|
| 700 |
+
"output_type": "display_data"
|
| 701 |
+
},
|
| 702 |
+
{
|
| 703 |
+
"name": "stderr",
|
| 704 |
+
"output_type": "stream",
|
| 705 |
+
"text": [
|
| 706 |
+
"Error while downloading from https://cas-bridge.xethub.hf.co/xet-bridge-us/6303b5f71dd5d3c62482f3e9/11250a388e769f8d819bc3d9b55590ab43bde78b2d4f37df57bc5603c5dea0b0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=cas%2F20251104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251104T105145Z&X-Amz-Expires=3600&X-Amz-Signature=603bdf8dd3ba17470e9f30bd3b31141b09a02d57e6cf00b63996b798d985c323&X-Amz-SignedHeaders=host&X-Xet-Cas-Uid=public&response-content-disposition=inline%3B+filename*%3DUTF-8%27%27model.safetensors%3B+filename%3D%22model.safetensors%22%3B&x-id=GetObject&Expires=1762257105&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc2MjI1NzEwNX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2FzLWJyaWRnZS54ZXRodWIuaGYuY28veGV0LWJyaWRnZS11cy82MzAzYjVmNzFkZDVkM2M2MjQ4MmYzZTkvMTEyNTBhMzg4ZTc2OWY4ZDgxOWJjM2Q5YjU1NTkwYWI0M2JkZTc4YjJkNGYzN2RmNTdiYzU2MDNjNWRlYTBiMCoifV19&Signature=s7MbLznU9vhypkS2WueXkH%7E41QifR0qWT8YHXKdMNPheh483kMtmwoaCCqvVlcukJygS0K2zvyynfo8zpNM%7EqiH8ZI1k3E7B2f6UH97HlJA48Th3pabY4zi1WYI2IeZDnBdPlreb3qTCkCZw1l11fyt9ri2QOaxuRrT6jcxPOquIFzz5lZ0sZNjngA0rg8gtCGi89qsWATMV3Oh45cRpO%7E6DrmmHfP8LEk8sZOiu9Zi-33x-ZoO4oayJSvzxOw8y8TWs%7E5q3cD9geNVIToO-PJPHQJ6qk9FybNeN75zhWtBtERN77Ie2mf9mGEcnZVqBd%7EH4kyAyYsWUcwviGjJSxA__&Key-Pair-Id=K2L8F4GPSG1IFC: HTTPSConnectionPool(host='cas-bridge.xethub.hf.co', port=443): Read timed out.\n",
|
| 707 |
+
"Trying to resume download...\n"
|
| 708 |
+
]
|
| 709 |
+
},
|
| 710 |
+
{
|
| 711 |
+
"data": {
|
| 712 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 713 |
+
"model_id": "517575609add45b3a2494deae2cdacc3",
|
| 714 |
+
"version_major": 2,
|
| 715 |
+
"version_minor": 0
|
| 716 |
+
},
|
| 717 |
+
"text/plain": [
|
| 718 |
+
"model.safetensors: 26%|##6 | 147M/558M [00:00<?, ?B/s]"
|
| 719 |
+
]
|
| 720 |
+
},
|
| 721 |
+
"metadata": {},
|
| 722 |
+
"output_type": "display_data"
|
| 723 |
+
},
|
| 724 |
+
{
|
| 725 |
+
"name": "stderr",
|
| 726 |
+
"output_type": "stream",
|
| 727 |
+
"text": [
|
| 728 |
+
"Error while downloading from https://cas-bridge.xethub.hf.co/xet-bridge-us/6303b5f71dd5d3c62482f3e9/11250a388e769f8d819bc3d9b55590ab43bde78b2d4f37df57bc5603c5dea0b0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=cas%2F20251104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251104T105145Z&X-Amz-Expires=3600&X-Amz-Signature=603bdf8dd3ba17470e9f30bd3b31141b09a02d57e6cf00b63996b798d985c323&X-Amz-SignedHeaders=host&X-Xet-Cas-Uid=public&response-content-disposition=inline%3B+filename*%3DUTF-8%27%27model.safetensors%3B+filename%3D%22model.safetensors%22%3B&x-id=GetObject&Expires=1762257105&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc2MjI1NzEwNX19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2FzLWJyaWRnZS54ZXRodWIuaGYuY28veGV0LWJyaWRnZS11cy82MzAzYjVmNzFkZDVkM2M2MjQ4MmYzZTkvMTEyNTBhMzg4ZTc2OWY4ZDgxOWJjM2Q5YjU1NTkwYWI0M2JkZTc4YjJkNGYzN2RmNTdiYzU2MDNjNWRlYTBiMCoifV19&Signature=s7MbLznU9vhypkS2WueXkH%7E41QifR0qWT8YHXKdMNPheh483kMtmwoaCCqvVlcukJygS0K2zvyynfo8zpNM%7EqiH8ZI1k3E7B2f6UH97HlJA48Th3pabY4zi1WYI2IeZDnBdPlreb3qTCkCZw1l11fyt9ri2QOaxuRrT6jcxPOquIFzz5lZ0sZNjngA0rg8gtCGi89qsWATMV3Oh45cRpO%7E6DrmmHfP8LEk8sZOiu9Zi-33x-ZoO4oayJSvzxOw8y8TWs%7E5q3cD9geNVIToO-PJPHQJ6qk9FybNeN75zhWtBtERN77Ie2mf9mGEcnZVqBd%7EH4kyAyYsWUcwviGjJSxA__&Key-Pair-Id=K2L8F4GPSG1IFC: HTTPSConnectionPool(host='cas-bridge.xethub.hf.co', port=443): Read timed out.\n",
|
| 729 |
+
"Trying to resume download...\n"
|
| 730 |
]
|
| 731 |
},
|
| 732 |
+
{
|
| 733 |
+
"data": {
|
| 734 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 735 |
+
"model_id": "91efbf29652b4f77be230c371d1a515b",
|
| 736 |
+
"version_major": 2,
|
| 737 |
+
"version_minor": 0
|
| 738 |
+
},
|
| 739 |
+
"text/plain": [
|
| 740 |
+
"model.safetensors: 26%|##6 | 147M/558M [00:00<?, ?B/s]"
|
| 741 |
+
]
|
| 742 |
+
},
|
| 743 |
+
"metadata": {},
|
| 744 |
+
"output_type": "display_data"
|
| 745 |
+
},
|
| 746 |
+
{
|
| 747 |
+
"data": {
|
| 748 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 749 |
+
"model_id": "b57255653a394073ad9f6a10ae142d31",
|
| 750 |
+
"version_major": 2,
|
| 751 |
+
"version_minor": 0
|
| 752 |
+
},
|
| 753 |
+
"text/plain": [
|
| 754 |
+
"tokenizer_config.json: 0%| | 0.00/467 [00:00<?, ?B/s]"
|
| 755 |
+
]
|
| 756 |
+
},
|
| 757 |
+
"metadata": {},
|
| 758 |
+
"output_type": "display_data"
|
| 759 |
+
},
|
| 760 |
+
{
|
| 761 |
+
"data": {
|
| 762 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 763 |
+
"model_id": "4269962d819c4a06a6fd38069293eef2",
|
| 764 |
+
"version_major": 2,
|
| 765 |
+
"version_minor": 0
|
| 766 |
+
},
|
| 767 |
+
"text/plain": [
|
| 768 |
+
"spm.model: 0%| | 0.00/4.31M [00:00<?, ?B/s]"
|
| 769 |
+
]
|
| 770 |
+
},
|
| 771 |
+
"metadata": {},
|
| 772 |
+
"output_type": "display_data"
|
| 773 |
+
},
|
| 774 |
+
{
|
| 775 |
+
"data": {
|
| 776 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 777 |
+
"model_id": "d209ea62c4bb421ebe01856271b32bf5",
|
| 778 |
+
"version_major": 2,
|
| 779 |
+
"version_minor": 0
|
| 780 |
+
},
|
| 781 |
+
"text/plain": [
|
| 782 |
+
"tokenizer.json: 0%| | 0.00/16.3M [00:00<?, ?B/s]"
|
| 783 |
+
]
|
| 784 |
+
},
|
| 785 |
+
"metadata": {},
|
| 786 |
+
"output_type": "display_data"
|
| 787 |
+
},
|
| 788 |
+
{
|
| 789 |
+
"data": {
|
| 790 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 791 |
+
"model_id": "574f0405e8c2410d9659b60b263dab81",
|
| 792 |
+
"version_major": 2,
|
| 793 |
+
"version_minor": 0
|
| 794 |
+
},
|
| 795 |
+
"text/plain": [
|
| 796 |
+
"added_tokens.json: 0%| | 0.00/23.0 [00:00<?, ?B/s]"
|
| 797 |
+
]
|
| 798 |
+
},
|
| 799 |
+
"metadata": {},
|
| 800 |
+
"output_type": "display_data"
|
| 801 |
+
},
|
| 802 |
+
{
|
| 803 |
+
"data": {
|
| 804 |
+
"application/vnd.jupyter.widget-view+json": {
|
| 805 |
+
"model_id": "09d43e8d85214c0096ff7836934d23ac",
|
| 806 |
+
"version_major": 2,
|
| 807 |
+
"version_minor": 0
|
| 808 |
+
},
|
| 809 |
+
"text/plain": [
|
| 810 |
+
"special_tokens_map.json: 0%| | 0.00/173 [00:00<?, ?B/s]"
|
| 811 |
+
]
|
| 812 |
+
},
|
| 813 |
+
"metadata": {},
|
| 814 |
+
"output_type": "display_data"
|
| 815 |
+
},
|
| 816 |
{
|
| 817 |
"name": "stderr",
|
| 818 |
"output_type": "stream",
|
|
|
|
| 821 |
]
|
| 822 |
},
|
| 823 |
{
|
| 824 |
+
"name": "stdout",
|
| 825 |
+
"output_type": "stream",
|
| 826 |
+
"text": [
|
| 827 |
+
"0\n",
|
| 828 |
+
"1\n",
|
| 829 |
+
"2\n",
|
| 830 |
+
"3\n",
|
| 831 |
+
"4\n",
|
| 832 |
+
"5\n",
|
| 833 |
+
"6\n",
|
| 834 |
+
"7\n",
|
| 835 |
+
"8\n"
|
| 836 |
+
]
|
| 837 |
+
},
|
| 838 |
+
{
|
| 839 |
+
"name": "stderr",
|
| 840 |
+
"output_type": "stream",
|
| 841 |
+
"text": [
|
| 842 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 843 |
+
]
|
| 844 |
+
},
|
| 845 |
+
{
|
| 846 |
+
"name": "stdout",
|
| 847 |
+
"output_type": "stream",
|
| 848 |
+
"text": [
|
| 849 |
+
"0\n",
|
| 850 |
+
"1\n",
|
| 851 |
+
"2\n",
|
| 852 |
+
"3\n",
|
| 853 |
+
"4\n",
|
| 854 |
+
"5\n",
|
| 855 |
+
"6\n",
|
| 856 |
+
"7\n",
|
| 857 |
+
"8\n"
|
| 858 |
+
]
|
| 859 |
+
},
|
| 860 |
+
{
|
| 861 |
+
"name": "stderr",
|
| 862 |
+
"output_type": "stream",
|
| 863 |
+
"text": [
|
| 864 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 865 |
+
]
|
| 866 |
+
},
|
| 867 |
+
{
|
| 868 |
+
"name": "stdout",
|
| 869 |
+
"output_type": "stream",
|
| 870 |
+
"text": [
|
| 871 |
+
"0\n",
|
| 872 |
+
"1\n",
|
| 873 |
+
"2\n",
|
| 874 |
+
"3\n",
|
| 875 |
+
"4\n",
|
| 876 |
+
"5\n",
|
| 877 |
+
"6\n",
|
| 878 |
+
"7\n",
|
| 879 |
+
"8\n"
|
| 880 |
+
]
|
| 881 |
+
},
|
| 882 |
+
{
|
| 883 |
+
"name": "stderr",
|
| 884 |
+
"output_type": "stream",
|
| 885 |
+
"text": [
|
| 886 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 887 |
+
]
|
| 888 |
+
},
|
| 889 |
+
{
|
| 890 |
+
"name": "stdout",
|
| 891 |
+
"output_type": "stream",
|
| 892 |
+
"text": [
|
| 893 |
+
"0\n",
|
| 894 |
+
"1\n",
|
| 895 |
+
"2\n",
|
| 896 |
+
"3\n",
|
| 897 |
+
"4\n",
|
| 898 |
+
"5\n",
|
| 899 |
+
"6\n",
|
| 900 |
+
"7\n",
|
| 901 |
+
"8\n"
|
| 902 |
+
]
|
| 903 |
+
},
|
| 904 |
+
{
|
| 905 |
+
"name": "stderr",
|
| 906 |
+
"output_type": "stream",
|
| 907 |
+
"text": [
|
| 908 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 909 |
+
]
|
| 910 |
+
},
|
| 911 |
+
{
|
| 912 |
+
"name": "stdout",
|
| 913 |
+
"output_type": "stream",
|
| 914 |
+
"text": [
|
| 915 |
+
"0\n",
|
| 916 |
+
"1\n",
|
| 917 |
+
"2\n",
|
| 918 |
+
"3\n",
|
| 919 |
+
"4\n",
|
| 920 |
+
"5\n",
|
| 921 |
+
"6\n",
|
| 922 |
+
"7\n",
|
| 923 |
+
"8\n"
|
| 924 |
+
]
|
| 925 |
+
},
|
| 926 |
+
{
|
| 927 |
+
"name": "stderr",
|
| 928 |
+
"output_type": "stream",
|
| 929 |
+
"text": [
|
| 930 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 931 |
+
]
|
| 932 |
+
},
|
| 933 |
+
{
|
| 934 |
+
"name": "stdout",
|
| 935 |
+
"output_type": "stream",
|
| 936 |
+
"text": [
|
| 937 |
+
"0\n",
|
| 938 |
+
"1\n",
|
| 939 |
+
"2\n",
|
| 940 |
+
"3\n",
|
| 941 |
+
"4\n",
|
| 942 |
+
"5\n",
|
| 943 |
+
"6\n",
|
| 944 |
+
"7\n",
|
| 945 |
+
"8\n"
|
| 946 |
+
]
|
| 947 |
+
},
|
| 948 |
+
{
|
| 949 |
+
"name": "stderr",
|
| 950 |
+
"output_type": "stream",
|
| 951 |
+
"text": [
|
| 952 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 953 |
+
]
|
| 954 |
+
},
|
| 955 |
+
{
|
| 956 |
+
"name": "stdout",
|
| 957 |
+
"output_type": "stream",
|
| 958 |
+
"text": [
|
| 959 |
+
"0\n",
|
| 960 |
+
"1\n",
|
| 961 |
+
"2\n",
|
| 962 |
+
"3\n",
|
| 963 |
+
"4\n",
|
| 964 |
+
"5\n",
|
| 965 |
+
"6\n",
|
| 966 |
+
"7\n",
|
| 967 |
+
"8\n"
|
| 968 |
+
]
|
| 969 |
+
},
|
| 970 |
+
{
|
| 971 |
+
"name": "stderr",
|
| 972 |
+
"output_type": "stream",
|
| 973 |
+
"text": [
|
| 974 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 975 |
+
]
|
| 976 |
+
},
|
| 977 |
+
{
|
| 978 |
+
"name": "stdout",
|
| 979 |
+
"output_type": "stream",
|
| 980 |
+
"text": [
|
| 981 |
+
"0\n",
|
| 982 |
+
"1\n",
|
| 983 |
+
"2\n",
|
| 984 |
+
"3\n",
|
| 985 |
+
"4\n",
|
| 986 |
+
"5\n",
|
| 987 |
+
"6\n",
|
| 988 |
+
"7\n",
|
| 989 |
+
"8\n"
|
| 990 |
+
]
|
| 991 |
+
},
|
| 992 |
+
{
|
| 993 |
+
"name": "stderr",
|
| 994 |
+
"output_type": "stream",
|
| 995 |
+
"text": [
|
| 996 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 997 |
+
]
|
| 998 |
+
},
|
| 999 |
+
{
|
| 1000 |
+
"name": "stdout",
|
| 1001 |
+
"output_type": "stream",
|
| 1002 |
+
"text": [
|
| 1003 |
+
"0\n",
|
| 1004 |
+
"1\n",
|
| 1005 |
+
"2\n",
|
| 1006 |
+
"3\n",
|
| 1007 |
+
"4\n",
|
| 1008 |
+
"5\n",
|
| 1009 |
+
"6\n",
|
| 1010 |
+
"7\n",
|
| 1011 |
+
"8\n"
|
| 1012 |
+
]
|
| 1013 |
+
},
|
| 1014 |
+
{
|
| 1015 |
+
"name": "stderr",
|
| 1016 |
+
"output_type": "stream",
|
| 1017 |
+
"text": [
|
| 1018 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 1019 |
+
]
|
| 1020 |
+
},
|
| 1021 |
+
{
|
| 1022 |
+
"name": "stdout",
|
| 1023 |
+
"output_type": "stream",
|
| 1024 |
+
"text": [
|
| 1025 |
+
"0\n",
|
| 1026 |
+
"1\n",
|
| 1027 |
+
"2\n",
|
| 1028 |
+
"3\n",
|
| 1029 |
+
"4\n",
|
| 1030 |
+
"5\n",
|
| 1031 |
+
"6\n",
|
| 1032 |
+
"7\n",
|
| 1033 |
+
"8\n"
|
| 1034 |
]
|
| 1035 |
}
|
| 1036 |
],
|
| 1037 |
"source": [
|
| 1038 |
"# List of Hugging Face model names\n",
|
| 1039 |
+
"# trending...\n",
|
| 1040 |
+
"\"\"\"\n",
|
| 1041 |
"model_names = [\n",
|
| 1042 |
" \"facebook/bart-large-mnli\",\n",
|
| 1043 |
" \"MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli\",\n",
|
|
|
|
| 1050 |
" \"valhalla/distilbart-mnli-12-1\",\n",
|
| 1051 |
" #\"joeddav/xlm-roberta-large-xnli\" # keeps giving errors\n",
|
| 1052 |
"]\n",
|
| 1053 |
+
"\"\"\"\n",
|
| 1054 |
+
"\n",
|
| 1055 |
+
"# most downloads\n",
|
| 1056 |
+
"model_names = [\n",
|
| 1057 |
+
" #\"facebook/bart-large-mnli\",\n",
|
| 1058 |
+
" #\"MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli\",\n",
|
| 1059 |
+
" #\"sileod/deberta-v3-base-tasksource-nli\",\n",
|
| 1060 |
+
" #\"vicgalle/xlm-roberta-large-xnli-anli\", # gives errors\n",
|
| 1061 |
+
" #\"joeddav/xlm-roberta-large-xnli\",# errors\n",
|
| 1062 |
+
" #\"chuhac/BiomedCLIP-vit-bert-hf\",# errors\n",
|
| 1063 |
+
" \"pongjin/roberta_with_kornli\",\n",
|
| 1064 |
+
" #\"joeddav/bart-large-mnli-yahoo-answers\",\n",
|
| 1065 |
+
" #\"MoritzLaurer/mDeBERTa-v3-base-mnli-xnli\",\n",
|
| 1066 |
+
" #\"valhalla/distilbart-mnli-12-1\",\n",
|
| 1067 |
+
" \"MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7\"\n",
|
| 1068 |
+
"]\n",
|
| 1069 |
+
"\n",
|
| 1070 |
"\n",
|
| 1071 |
"# Apply each model to the test data\n",
|
| 1072 |
"for model_name in model_names:\n",
|
|
|
|
| 1124 |
},
|
| 1125 |
{
|
| 1126 |
"cell_type": "code",
|
| 1127 |
+
"execution_count": 5,
|
| 1128 |
"id": "37849e0b-864e-4377-b06c-0ac70c3861f9",
|
| 1129 |
"metadata": {
|
| 1130 |
"scrolled": true
|
|
|
|
| 1176 |
"false_pred 0.500\n",
|
| 1177 |
"dtype: float64\n",
|
| 1178 |
"----------------------------------------\n",
|
| 1179 |
+
"Model: pongjin-roberta_with_kornli\n",
|
| 1180 |
+
"Performance Summary:\n",
|
| 1181 |
+
"accuracy 0.233333\n",
|
| 1182 |
+
"true_ident 0.666667\n",
|
| 1183 |
+
"false_pred 0.452857\n",
|
| 1184 |
+
"dtype: float64\n",
|
| 1185 |
+
"----------------------------------------\n",
|
| 1186 |
+
"Model: sileod-deberta-v3-base-tasksource-nli\n",
|
| 1187 |
+
"Performance Summary:\n",
|
| 1188 |
+
"accuracy 0.566667\n",
|
| 1189 |
+
"true_ident 0.700000\n",
|
| 1190 |
+
"false_pred 0.551667\n",
|
| 1191 |
+
"dtype: float64\n",
|
| 1192 |
+
"----------------------------------------\n",
|
| 1193 |
+
"Model: MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7\n",
|
| 1194 |
+
"Performance Summary:\n",
|
| 1195 |
+
"accuracy 0.488889\n",
|
| 1196 |
+
"true_ident 0.833333\n",
|
| 1197 |
+
"false_pred 0.688373\n",
|
| 1198 |
+
"dtype: float64\n",
|
| 1199 |
+
"----------------------------------------\n",
|
| 1200 |
"Model: facebook-bart-large-mnli\n",
|
| 1201 |
"Performance Summary:\n",
|
| 1202 |
"accuracy 0.466667\n",
|
|
|
|
| 1246 |
" print(\"-\" * 40)\n"
|
| 1247 |
]
|
| 1248 |
},
|
| 1249 |
+
{
|
| 1250 |
+
"cell_type": "code",
|
| 1251 |
+
"execution_count": 6,
|
| 1252 |
+
"id": "3f1951b1-884d-49ab-985d-ab1779c6f71d",
|
| 1253 |
+
"metadata": {},
|
| 1254 |
+
"outputs": [
|
| 1255 |
+
{
|
| 1256 |
+
"name": "stdout",
|
| 1257 |
+
"output_type": "stream",
|
| 1258 |
+
"text": [
|
| 1259 |
+
"12\n"
|
| 1260 |
+
]
|
| 1261 |
+
}
|
| 1262 |
+
],
|
| 1263 |
+
"source": [
|
| 1264 |
+
"print(len(all_results))"
|
| 1265 |
+
]
|
| 1266 |
+
},
|
| 1267 |
{
|
| 1268 |
"cell_type": "markdown",
|
| 1269 |
"id": "2f65e5b1-bc32-42c2-bbe9-9e3a6ffc72c1",
|
|
|
|
| 1304 |
"\n",
|
| 1305 |
"MoritzLaurer-deberta-v3-large-zeroshot-v2.0: Index([1, 2, 3, 5, 6, 7, 9], dtype='int64')\n",
|
| 1306 |
"\n",
|
| 1307 |
+
"pongjin-roberta_with_kornli: RangeIndex(start=0, stop=10, step=1)\n",
|
| 1308 |
+
"\n",
|
| 1309 |
+
"sileod-deberta-v3-base-tasksource-nli: Index([0, 2, 3, 5, 6], dtype='int64')\n",
|
| 1310 |
+
"\n",
|
| 1311 |
+
"MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7: Index([0, 2, 3, 4, 5, 6, 7, 8, 9], dtype='int64')\n",
|
| 1312 |
+
"\n",
|
| 1313 |
"facebook-bart-large-mnli: RangeIndex(start=0, stop=10, step=1)\n",
|
| 1314 |
"\n",
|
| 1315 |
"valhalla-distilbart-mnli-12-1: Index([0, 1, 2, 3, 4, 7, 9], dtype='int64')\n",
|
|
|
|
| 1385 |
"rainy climate\n",
|
| 1386 |
"7+ days\n",
|
| 1387 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1388 |
"\n"
|
| 1389 |
]
|
| 1390 |
}
|
|
|
|
| 1437 |
"\n",
|
| 1438 |
"MoritzLaurer-deberta-v3-large-zeroshot-v2.0: ['activities', 'climate_or_season', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
| 1439 |
"\n",
|
| 1440 |
+
"pongjin-roberta_with_kornli: ['activity_type', 'activities', 'climate_or_season', 'style_or_comfort', 'dress_code', 'accommodation', 'transportation', 'special_conditions', 'trip_length_days']\n",
|
| 1441 |
+
"\n",
|
| 1442 |
+
"sileod-deberta-v3-base-tasksource-nli: ['activities', 'style_or_comfort', 'special_conditions']\n",
|
| 1443 |
+
"\n",
|
| 1444 |
+
"MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7: ['activity_type', 'activities', 'style_or_comfort', 'special_conditions']\n",
|
| 1445 |
+
"\n",
|
| 1446 |
"facebook-bart-large-mnli: ['activities', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
| 1447 |
"\n",
|
| 1448 |
"valhalla-distilbart-mnli-12-1: ['activities', 'climate_or_season', 'style_or_comfort', 'accommodation', 'special_conditions']\n",
|
|
|
|
| 1488 |
"name": "stdout",
|
| 1489 |
"output_type": "stream",
|
| 1490 |
"text": [
|
| 1491 |
+
"{'activities', 'style_or_comfort'}\n"
|
| 1492 |
]
|
| 1493 |
}
|
| 1494 |
],
|
|
|
|
| 1519 |
},
|
| 1520 |
{
|
| 1521 |
"cell_type": "code",
|
| 1522 |
+
"execution_count": 12,
|
| 1523 |
"id": "b020f584-1468-4c84-9dac-7ca7fac6e8ca",
|
| 1524 |
"metadata": {
|
| 1525 |
"scrolled": true
|
|
|
|
| 1529 |
"name": "stdout",
|
| 1530 |
"output_type": "stream",
|
| 1531 |
"text": [
|
| 1532 |
+
" model accuracy true_ident false_pred\n",
|
| 1533 |
+
"0 MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli 0.611111 0.841667 0.546667\n",
|
| 1534 |
+
"1 sileod-deberta-v3-base-tasksource-nli 0.566667 0.700000 0.551667\n",
|
| 1535 |
+
"2 MoritzLaurer-DeBERTa-v3-base-mnli-fever-anli 0.522222 0.841667 0.572381\n",
|
| 1536 |
+
"3 MoritzLaurer-deberta-v3-large-zeroshot-v2.0 0.500000 0.325000 0.500000\n",
|
| 1537 |
+
"4 valhalla-distilbart-mnli-12-1 0.500000 0.300000 0.533333\n",
|
| 1538 |
+
"5 MoritzLaurer-mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 0.488889 0.833333 0.688373\n",
|
| 1539 |
+
"6 cross-encoder-nli-deberta-v3-large 0.466667 0.566667 0.541667\n",
|
| 1540 |
+
"7 facebook-bart-large-mnli 0.466667 0.708333 0.400000\n",
|
| 1541 |
+
"8 MoritzLaurer-mDeBERTa-v3-base-mnli-xnli 0.455556 0.408333 0.481250\n",
|
| 1542 |
+
"9 cross-encoder-nli-deberta-v3-base 0.444444 0.533333 0.712500\n",
|
| 1543 |
+
"10 joeddav-bart-large-mnli-yahoo-answers 0.355556 0.650000 0.553792\n",
|
| 1544 |
+
"11 pongjin-roberta_with_kornli 0.233333 0.666667 0.452857\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1545 |
]
|
| 1546 |
}
|
| 1547 |
],
|
| 1548 |
"source": [
|
| 1549 |
+
"pd.set_option('display.max_columns', None) # show all columns\n",
|
| 1550 |
+
"pd.set_option('display.max_colwidth', None) # do not truncate cell contents\n",
|
| 1551 |
+
"pd.set_option('display.width', 200) \n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1552 |
"\n",
|
| 1553 |
+
"perf_table = []\n",
|
| 1554 |
"# fill in for loop with perf_summary per model\n",
|
| 1555 |
"for model, result in all_results.items():\n",
|
| 1556 |
" row = pd.DataFrame(result[\"perf_summary\"]).T\n",
|
| 1557 |
+
" # row[\"model\"] = model\n",
|
| 1558 |
+
" row.insert(0, \"model\", model) # insert as first column\n",
|
| 1559 |
" perf_table.append(row)\n",
|
| 1560 |
"# Concatenate all into one table\n",
|
| 1561 |
"df_all = pd.concat(perf_table, ignore_index=True)\n",
|
| 1562 |
+
"df = df_all.sort_values(by=\"accuracy\", ascending=False).reset_index(drop=True)\n",
|
| 1563 |
+
"\n",
|
| 1564 |
+
"\n",
|
| 1565 |
"\n",
|
| 1566 |
+
"print(df)\n",
|
| 1567 |
"#print(type(df_all))\n",
|
| 1568 |
" \n",
|
| 1569 |
"\n",
|
| 1570 |
+
"# rank by accuracy\n"
|
| 1571 |
]
|
| 1572 |
},
|
| 1573 |
{
|
space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
@@ -41,7 +41,7 @@
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
-
"execution_count":
|
| 45 |
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
|
@@ -155,26 +155,7 @@
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
-
"id": "e114d7ab-9bd3-4214-a630-7f9166c3b8a5",
|
| 160 |
-
"metadata": {},
|
| 161 |
-
"outputs": [
|
| 162 |
-
{
|
| 163 |
-
"name": "stdout",
|
| 164 |
-
"output_type": "stream",
|
| 165 |
-
"text": [
|
| 166 |
-
"activity_type\n"
|
| 167 |
-
]
|
| 168 |
-
}
|
| 169 |
-
],
|
| 170 |
-
"source": [
|
| 171 |
-
"# First classify using first dimension of classes ()\n",
|
| 172 |
-
"print(keys_list[0])"
|
| 173 |
-
]
|
| 174 |
-
},
|
| 175 |
-
{
|
| 176 |
-
"cell_type": "code",
|
| 177 |
-
"execution_count": 40,
|
| 178 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 179 |
"metadata": {},
|
| 180 |
"outputs": [
|
|
@@ -210,6 +191,7 @@
|
|
| 210 |
}
|
| 211 |
],
|
| 212 |
"source": [
|
|
|
|
| 213 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 214 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 215 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
|
@@ -224,7 +206,7 @@
|
|
| 224 |
},
|
| 225 |
{
|
| 226 |
"cell_type": "code",
|
| 227 |
-
"execution_count":
|
| 228 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 229 |
"metadata": {},
|
| 230 |
"outputs": [
|
|
@@ -244,7 +226,7 @@
|
|
| 244 |
},
|
| 245 |
{
|
| 246 |
"cell_type": "code",
|
| 247 |
-
"execution_count":
|
| 248 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 249 |
"metadata": {},
|
| 250 |
"outputs": [
|
|
@@ -283,12 +265,13 @@
|
|
| 283 |
" \"Label\": result[\"labels\"],\n",
|
| 284 |
" \"Score\": result[\"scores\"]\n",
|
| 285 |
"})\n",
|
| 286 |
-
"print(df)"
|
|
|
|
| 287 |
]
|
| 288 |
},
|
| 289 |
{
|
| 290 |
"cell_type": "code",
|
| 291 |
-
"execution_count":
|
| 292 |
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 293 |
"metadata": {},
|
| 294 |
"outputs": [
|
|
@@ -300,13 +283,11 @@
|
|
| 300 |
]
|
| 301 |
}
|
| 302 |
],
|
| 303 |
-
"source": [
|
| 304 |
-
"print(classes)"
|
| 305 |
-
]
|
| 306 |
},
|
| 307 |
{
|
| 308 |
"cell_type": "code",
|
| 309 |
-
"execution_count":
|
| 310 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 311 |
"metadata": {},
|
| 312 |
"outputs": [],
|
|
@@ -342,7 +323,7 @@
|
|
| 342 |
},
|
| 343 |
{
|
| 344 |
"cell_type": "code",
|
| 345 |
-
"execution_count":
|
| 346 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 347 |
"metadata": {},
|
| 348 |
"outputs": [
|
|
@@ -357,7 +338,7 @@
|
|
| 357 |
"name": "stdout",
|
| 358 |
"output_type": "stream",
|
| 359 |
"text": [
|
| 360 |
-
"Processing
|
| 361 |
"0 activity_type beach vacation\n",
|
| 362 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 363 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -385,7 +366,7 @@
|
|
| 385 |
},
|
| 386 |
{
|
| 387 |
"cell_type": "code",
|
| 388 |
-
"execution_count":
|
| 389 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 390 |
"metadata": {},
|
| 391 |
"outputs": [],
|
|
@@ -401,10 +382,6 @@
|
|
| 401 |
" candidate_labels = json.load(file)\n",
|
| 402 |
"keys_list = list(candidate_labels.keys())\n",
|
| 403 |
"\n",
|
| 404 |
-
"# Load test data (in list of dictionaries)\n",
|
| 405 |
-
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 406 |
-
" packing_data = json.load(file)\n",
|
| 407 |
-
"\n",
|
| 408 |
"# Load packing item data\n",
|
| 409 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 410 |
" packing_items = json.load(file)"
|
|
@@ -412,7 +389,7 @@
|
|
| 412 |
},
|
| 413 |
{
|
| 414 |
"cell_type": "code",
|
| 415 |
-
"execution_count":
|
| 416 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 417 |
"metadata": {},
|
| 418 |
"outputs": [
|
|
@@ -420,9 +397,7 @@
|
|
| 420 |
"name": "stdout",
|
| 421 |
"output_type": "stream",
|
| 422 |
"text": [
|
| 423 |
-
"Running on local URL: http://127.0.0.1:
|
| 424 |
-
"\n",
|
| 425 |
-
"Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB\n",
|
| 426 |
"\n",
|
| 427 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 428 |
]
|
|
@@ -430,7 +405,7 @@
|
|
| 430 |
{
|
| 431 |
"data": {
|
| 432 |
"text/html": [
|
| 433 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 434 |
],
|
| 435 |
"text/plain": [
|
| 436 |
"<IPython.core.display.HTML object>"
|
|
@@ -438,78 +413,6 @@
|
|
| 438 |
},
|
| 439 |
"metadata": {},
|
| 440 |
"output_type": "display_data"
|
| 441 |
-
},
|
| 442 |
-
{
|
| 443 |
-
"name": "stderr",
|
| 444 |
-
"output_type": "stream",
|
| 445 |
-
"text": [
|
| 446 |
-
"Traceback (most recent call last):\n",
|
| 447 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 406, in hf_raise_for_status\n",
|
| 448 |
-
" response.raise_for_status()\n",
|
| 449 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/requests/models.py\", line 1024, in raise_for_status\n",
|
| 450 |
-
" raise HTTPError(http_error_msg, response=self)\n",
|
| 451 |
-
"requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json\n",
|
| 452 |
-
"\n",
|
| 453 |
-
"The above exception was the direct cause of the following exception:\n",
|
| 454 |
-
"\n",
|
| 455 |
-
"Traceback (most recent call last):\n",
|
| 456 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 403, in cached_file\n",
|
| 457 |
-
" resolved_file = hf_hub_download(\n",
|
| 458 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 459 |
-
" return fn(*args, **kwargs)\n",
|
| 460 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 860, in hf_hub_download\n",
|
| 461 |
-
" return _hf_hub_download_to_cache_dir(\n",
|
| 462 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 967, in _hf_hub_download_to_cache_dir\n",
|
| 463 |
-
" _raise_on_head_call_error(head_call_error, force_download, local_files_only)\n",
|
| 464 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1482, in _raise_on_head_call_error\n",
|
| 465 |
-
" raise head_call_error\n",
|
| 466 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1374, in _get_metadata_or_catch_error\n",
|
| 467 |
-
" metadata = get_hf_file_metadata(\n",
|
| 468 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 469 |
-
" return fn(*args, **kwargs)\n",
|
| 470 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1294, in get_hf_file_metadata\n",
|
| 471 |
-
" r = _request_wrapper(\n",
|
| 472 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 278, in _request_wrapper\n",
|
| 473 |
-
" response = _request_wrapper(\n",
|
| 474 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 302, in _request_wrapper\n",
|
| 475 |
-
" hf_raise_for_status(response)\n",
|
| 476 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 454, in hf_raise_for_status\n",
|
| 477 |
-
" raise _format(RepositoryNotFoundError, message, response) from e\n",
|
| 478 |
-
"huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-68fa0a8b-0d531523246f6614068c359e;d99d5b5d-694d-4d4b-b774-bb70066228d2)\n",
|
| 479 |
-
"\n",
|
| 480 |
-
"Repository Not Found for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json.\n",
|
| 481 |
-
"Please make sure you specified the correct `repo_id` and `repo_type`.\n",
|
| 482 |
-
"If you are trying to access a private or gated repo, make sure you are authenticated.\n",
|
| 483 |
-
"Invalid username or password.\n",
|
| 484 |
-
"\n",
|
| 485 |
-
"The above exception was the direct cause of the following exception:\n",
|
| 486 |
-
"\n",
|
| 487 |
-
"Traceback (most recent call last):\n",
|
| 488 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/queueing.py\", line 536, in process_events\n",
|
| 489 |
-
" response = await route_utils.call_process_api(\n",
|
| 490 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/route_utils.py\", line 322, in call_process_api\n",
|
| 491 |
-
" output = await app.get_blocks().process_api(\n",
|
| 492 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1935, in process_api\n",
|
| 493 |
-
" result = await self.call_function(\n",
|
| 494 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1520, in call_function\n",
|
| 495 |
-
" prediction = await anyio.to_thread.run_sync( # type: ignore\n",
|
| 496 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/to_thread.py\", line 56, in run_sync\n",
|
| 497 |
-
" return await get_async_backend().run_sync_in_worker_thread(\n",
|
| 498 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 2134, in run_sync_in_worker_thread\n",
|
| 499 |
-
" return await future\n",
|
| 500 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 851, in run\n",
|
| 501 |
-
" result = context.run(func, *args)\n",
|
| 502 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/utils.py\", line 826, in wrapper\n",
|
| 503 |
-
" response = f(*args, **kwargs)\n",
|
| 504 |
-
" File \"/var/folders/r5/y63bxy5d4_zfxf6cgwtwysbc0000gn/T/ipykernel_22464/1188444081.py\", line 3, in classify\n",
|
| 505 |
-
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 506 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py\", line 798, in pipeline\n",
|
| 507 |
-
" resolved_config_file = cached_file(\n",
|
| 508 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 426, in cached_file\n",
|
| 509 |
-
" raise EnvironmentError(\n",
|
| 510 |
-
"OSError: MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\n",
|
| 511 |
-
"If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`\n"
|
| 512 |
-
]
|
| 513 |
}
|
| 514 |
],
|
| 515 |
"source": [
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 1,
|
| 14 |
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
+
"execution_count": 2,
|
| 45 |
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
|
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
+
"execution_count": 12,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
|
|
| 191 |
}
|
| 192 |
],
|
| 193 |
"source": [
|
| 194 |
+
"key = keys_list[0]\n",
|
| 195 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 196 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 197 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
|
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
+
"execution_count": 5,
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
+
"execution_count": 6,
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
|
|
| 265 |
" \"Label\": result[\"labels\"],\n",
|
| 266 |
" \"Score\": result[\"scores\"]\n",
|
| 267 |
"})\n",
|
| 268 |
+
"print(df)\n",
|
| 269 |
+
"print(classes)"
|
| 270 |
]
|
| 271 |
},
|
| 272 |
{
|
| 273 |
"cell_type": "code",
|
| 274 |
+
"execution_count": 7,
|
| 275 |
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 276 |
"metadata": {},
|
| 277 |
"outputs": [
|
|
|
|
| 283 |
]
|
| 284 |
}
|
| 285 |
],
|
| 286 |
+
"source": []
|
|
|
|
|
|
|
| 287 |
},
|
| 288 |
{
|
| 289 |
"cell_type": "code",
|
| 290 |
+
"execution_count": 8,
|
| 291 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 292 |
"metadata": {},
|
| 293 |
"outputs": [],
|
|
|
|
| 323 |
},
|
| 324 |
{
|
| 325 |
"cell_type": "code",
|
| 326 |
+
"execution_count": 9,
|
| 327 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 328 |
"metadata": {},
|
| 329 |
"outputs": [
|
|
|
|
| 338 |
"name": "stdout",
|
| 339 |
"output_type": "stream",
|
| 340 |
"text": [
|
| 341 |
+
"Processing 9/9 superclass pred_class\n",
|
| 342 |
"0 activity_type beach vacation\n",
|
| 343 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 344 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 366 |
},
|
| 367 |
{
|
| 368 |
"cell_type": "code",
|
| 369 |
+
"execution_count": 10,
|
| 370 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 371 |
"metadata": {},
|
| 372 |
"outputs": [],
|
|
|
|
| 382 |
" candidate_labels = json.load(file)\n",
|
| 383 |
"keys_list = list(candidate_labels.keys())\n",
|
| 384 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 385 |
"# Load packing item data\n",
|
| 386 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 387 |
" packing_items = json.load(file)"
|
|
|
|
| 389 |
},
|
| 390 |
{
|
| 391 |
"cell_type": "code",
|
| 392 |
+
"execution_count": 11,
|
| 393 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 394 |
"metadata": {},
|
| 395 |
"outputs": [
|
|
|
|
| 397 |
"name": "stdout",
|
| 398 |
"output_type": "stream",
|
| 399 |
"text": [
|
| 400 |
+
"Running on local URL: http://127.0.0.1:7860\n",
|
|
|
|
|
|
|
| 401 |
"\n",
|
| 402 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 403 |
]
|
|
|
|
| 405 |
{
|
| 406 |
"data": {
|
| 407 |
"text/html": [
|
| 408 |
+
"<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 409 |
],
|
| 410 |
"text/plain": [
|
| 411 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 413 |
},
|
| 414 |
"metadata": {},
|
| 415 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 416 |
}
|
| 417 |
],
|
| 418 |
"source": [
|
space/space/space/space/space/space/space/space/space/blog_post/.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
space/space/space/space/space/space/space/space/space/blog_post/article.md
CHANGED
|
@@ -15,64 +15,49 @@
|
|
| 15 |
|
| 16 |
## Implementation of packing list model
|
| 17 |
### Prerequisites before you start to code: Anja
|
| 18 |
-
*
|
| 19 |
-
Hugging face is..."The platform where the machine learning community collaborates on models, datasets, and applications."(from hugging face)
|
| 20 |
-
"Hugging Face is a company and open-source community that builds tools for machine learning (ML) and artificial intelligence (AI) — especially in the field of natural language processing (NLP)."(from the chate)
|
| 21 |
-
To be able to use the full funcitonality (e.g. models, spaces, data sets, api access) and most access you need to make a hugging face account here (https://huggingface.co/)
|
| 22 |
-
There is a new course at data camp free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses
|
| 23 |
-
|
| 24 |
-
* Use anaconda
|
| 25 |
-
"Without a package and environment manager like Conda"(anaconda site)
|
| 26 |
-
We used the anaconda navigator as a package and environment manager and used the jupyter notebook through there. python is automatically installed.
|
| 27 |
-
In the terminal activate the conda environment that you created previously by navigating (see cheat sheet here https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf)
|
| 28 |
-
|
| 29 |
-
View list of environments
|
| 30 |
-
```bash
|
| 31 |
-
conda env list
|
| 32 |
-
```
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
conda activate ENVIRONMENT_NAME
|
| 37 |
-
```
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
|
|
|
| 44 |
|
| 45 |
-
Install jupyter notebook
|
| 46 |
```bash
|
| 47 |
-
conda
|
|
|
|
| 48 |
```
|
| 49 |
|
| 50 |
-
|
| 51 |
```bash
|
|
|
|
|
|
|
| 52 |
jupyter-notebook
|
| 53 |
```
|
| 54 |
-
Create a new
|
| 55 |
|
| 56 |
-
###
|
| 57 |
-
Let
|
| 58 |
-
To
|
| 59 |
-
Log in
|
| 60 |
-
|
| 61 |
-
* how to implement the token in your code
|
| 62 |
-
This access token now has to be saved in you projecct folder in a .env file. Create a plan text file that you call .env. Within it you write:
|
| 63 |
```text
|
| 64 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 65 |
```
|
| 66 |
-
|
| 67 |
-
|
|
|
|
| 68 |
|
| 69 |
-
Now we load a zero-shot-classification model using API and male a simple classification.
|
| 70 |
```python
|
| 71 |
from dotenv import load_dotenv
|
| 72 |
import os
|
| 73 |
import requests
|
| 74 |
|
| 75 |
-
load_dotenv() # Load environment variables from .env file
|
| 76 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
| 77 |
|
| 78 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
|
@@ -92,17 +77,22 @@ output = query("facebook/bart-large-mnli", input_text)
|
|
| 92 |
print(output)
|
| 93 |
```
|
| 94 |
|
|
|
|
|
|
|
| 95 |
```json
|
| 96 |
{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.970917284488678, 0.014999152161180973, 0.008272469975054264, 0.005811101291328669]}
|
| 97 |
```
|
|
|
|
| 98 |
|
| 99 |
-
|
| 100 |
|
| 101 |
|
| 102 |
### Predefine outputs/classes: Nikky
|
| 103 |
|
| 104 |
### Model implementation: Anja
|
| 105 |
-
|
|
|
|
|
|
|
| 106 |
```python
|
| 107 |
import math
|
| 108 |
import json
|
|
@@ -119,21 +109,272 @@ with open("packing_label_structure.json", "r") as file:
|
|
| 119 |
candidate_labels = json.load(file)
|
| 120 |
keys_list = list(candidate_labels.keys())
|
| 121 |
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
```
|
| 129 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
```python
|
| 131 |
-
|
| 132 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
print(result)
|
| 134 |
```
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
### Using gradio app: Anja
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 137 |
|
| 138 |
### Share your model: Anja
|
| 139 |
* created space
|
|
@@ -152,7 +393,7 @@ And connect your hugging face space as a remote repository
|
|
| 152 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 153 |
```
|
| 154 |
|
| 155 |
-
generate an access token for your space
|
| 156 |
|
| 157 |
|
| 158 |
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
|
|
|
| 15 |
|
| 16 |
## Implementation of packing list model
|
| 17 |
### Prerequisites before you start to code: Anja
|
| 18 |
+
**Hugging face account**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
+
Hugging Face is a company and platform for the machine learning community to collaborate on models, datasets and applications especially in the field of natural language processing.
|
| 21 |
+
To be able to use the full funcitonality offered by Hugging Face (e.g. acces to models, spaces, datasets, api access) and you can make a free account [on their website](https://huggingface.co/).
|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
(There is a new course at data camp, which is free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses)
|
| 24 |
+
|
| 25 |
+
**Anaconda navigator**
|
| 26 |
+
To program our model, we use the anaconda navigator with package and environment manager conda and use jupyter notebook to write our python code. You can download the Anaconda navigator [here](https://www.anaconda.com/products/navigator). (python is automatically installed)
|
| 27 |
+
|
| 28 |
+
Using the command line, you can create a new environment to work in and install the required packages. The following commands create a new environment called hf_env and activate it ([conda cheat sheet](https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf)):
|
| 29 |
|
|
|
|
| 30 |
```bash
|
| 31 |
+
conda create --name hf_env
|
| 32 |
+
conda activate hf_env
|
| 33 |
```
|
| 34 |
|
| 35 |
+
Next, install the libraries used in this project and set up Jupyter Notebook.
|
| 36 |
```bash
|
| 37 |
+
pip install transformers torch numpy tabulate gradio pandas scikit-learn
|
| 38 |
+
conda install jupyter
|
| 39 |
jupyter-notebook
|
| 40 |
```
|
| 41 |
+
Create a new Jupyter Notebook to write your code.
|
| 42 |
|
| 43 |
+
### Hugging face API
|
| 44 |
+
Let's first try some Hugging Face models using their API. The advantage of using API is that you do not need to download the models locally and the computation is handled on Hugging Face servers.
|
| 45 |
+
To use their API you need to first create an access token.
|
| 46 |
+
Log in to your Hugging Face account and go to Settings > Access Tokens > + Create new token. Select as token type *Read* and give your token a name.
|
| 47 |
+
This access token now has to be saved in you project folder in an .env file. Create a plain text file that you call .env. Within it you write and save:
|
|
|
|
|
|
|
| 48 |
```text
|
| 49 |
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 50 |
```
|
| 51 |
+
where you replace YOUR_OWN_ACCESS_TOKEN with your own access token.
|
| 52 |
+
|
| 53 |
+
Nos it's time to start coding and using a first zero-shot-classification model! In your Jupyter Notebook use a code cell to write the follwing python code:
|
| 54 |
|
|
|
|
| 55 |
```python
|
| 56 |
from dotenv import load_dotenv
|
| 57 |
import os
|
| 58 |
import requests
|
| 59 |
|
| 60 |
+
load_dotenv() # Load environment variables from .env file
|
| 61 |
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
| 62 |
|
| 63 |
candidate_labels = ["technology", "sports", "politics", "health"]
|
|
|
|
| 77 |
print(output)
|
| 78 |
```
|
| 79 |
|
| 80 |
+
In it we first load some libraries and then the .env file. We then create some candidate labels for our zero-shot-classification model and write a query function which contains the model. The model will then classify a text using these labels and returns the following output:
|
| 81 |
+
|
| 82 |
```json
|
| 83 |
{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.970917284488678, 0.014999152161180973, 0.008272469975054264, 0.005811101291328669]}
|
| 84 |
```
|
| 85 |
+
The scores contain a probability of the text belonging to a particular label.
|
| 86 |
|
| 87 |
+
This worked great! However, using API the functionality is limited. We were limited to 10 candidate labels for our model. This was not sufficient for our packing list model.
|
| 88 |
|
| 89 |
|
| 90 |
### Predefine outputs/classes: Nikky
|
| 91 |
|
| 92 |
### Model implementation: Anja
|
| 93 |
+
|
| 94 |
+
Now we try to load the model locally and work with some more functionality. We load some libraries and also our clsss labels for our packing list model wich we saved in a json file and print out for your appreciation. We created several *superclasses* which each contain a list of possible class labels for our trip and will use a different zero-shot-classification model for each superclass.
|
| 95 |
+
|
| 96 |
```python
|
| 97 |
import math
|
| 98 |
import json
|
|
|
|
| 109 |
candidate_labels = json.load(file)
|
| 110 |
keys_list = list(candidate_labels.keys())
|
| 111 |
|
| 112 |
+
for key in candidate_labels:
|
| 113 |
+
print("\n", key, ":")
|
| 114 |
+
for item in candidate_labels[key]:
|
| 115 |
+
print("\t", item)
|
| 116 |
+
```
|
| 117 |
+
|
| 118 |
+
```text
|
| 119 |
+
activity_type :
|
| 120 |
+
hut trek (summer)
|
| 121 |
+
hut trek (winter)
|
| 122 |
+
camping trip (wild camping)
|
| 123 |
+
camping trip (campground)
|
| 124 |
+
ski tour / skitour
|
| 125 |
+
snowboard / splitboard trip
|
| 126 |
+
long-distance hike / thru-hike
|
| 127 |
+
digital nomad trip
|
| 128 |
+
city trip
|
| 129 |
+
road trip (car/camper)
|
| 130 |
+
festival trip
|
| 131 |
+
yoga / wellness retreat
|
| 132 |
+
micro-adventure / weekend trip
|
| 133 |
+
beach vacation
|
| 134 |
+
cultural exploration
|
| 135 |
+
nature escape
|
| 136 |
+
|
| 137 |
+
activities :
|
| 138 |
+
swimming
|
| 139 |
+
going to the beach
|
| 140 |
+
relaxing
|
| 141 |
+
sightseeing
|
| 142 |
+
biking
|
| 143 |
+
running
|
| 144 |
+
skiing
|
| 145 |
+
cross-country skiing
|
| 146 |
+
ski touring
|
| 147 |
+
hiking
|
| 148 |
+
hut-to-hut hiking
|
| 149 |
+
rock climbing
|
| 150 |
+
ice climbing
|
| 151 |
+
snowshoe hiking
|
| 152 |
+
kayaking / canoeing
|
| 153 |
+
stand-up paddleboarding (SUP)
|
| 154 |
+
snorkeling
|
| 155 |
+
scuba diving
|
| 156 |
+
surfing
|
| 157 |
+
paragliding
|
| 158 |
+
horseback riding
|
| 159 |
+
photography
|
| 160 |
+
fishing
|
| 161 |
+
rafting
|
| 162 |
+
yoga
|
| 163 |
+
|
| 164 |
+
climate_or_season :
|
| 165 |
+
cold destination / winter
|
| 166 |
+
warm destination / summer
|
| 167 |
+
variable weather / spring / autumn
|
| 168 |
+
tropical / humid
|
| 169 |
+
dry / desert-like
|
| 170 |
+
rainy climate
|
| 171 |
+
|
| 172 |
+
style_or_comfort :
|
| 173 |
+
ultralight
|
| 174 |
+
lightweight (but comfortable)
|
| 175 |
+
luxury (including evening wear)
|
| 176 |
+
minimalist
|
| 177 |
+
|
| 178 |
+
dress_code :
|
| 179 |
+
casual
|
| 180 |
+
formal (business trip)
|
| 181 |
+
conservative
|
| 182 |
+
|
| 183 |
+
accommodation :
|
| 184 |
+
indoor
|
| 185 |
+
huts with half board
|
| 186 |
+
sleeping in a tent
|
| 187 |
+
sleeping in a car
|
| 188 |
+
|
| 189 |
+
transportation :
|
| 190 |
+
own vehicle
|
| 191 |
+
no own vehicle
|
| 192 |
+
|
| 193 |
+
special_conditions :
|
| 194 |
+
off-grid / no electricity
|
| 195 |
+
self-supported (bring your own cooking gear)
|
| 196 |
+
travel with children
|
| 197 |
+
pet-friendly
|
| 198 |
+
snow and ice
|
| 199 |
+
high alpine terrain
|
| 200 |
+
snow, ice and avalanche-prone terrain
|
| 201 |
+
no special conditions to consider
|
| 202 |
+
|
| 203 |
+
trip_length_days :
|
| 204 |
+
1 day
|
| 205 |
+
2 days
|
| 206 |
+
3 days
|
| 207 |
+
4 days
|
| 208 |
+
5 days
|
| 209 |
+
6 days
|
| 210 |
+
7 days
|
| 211 |
+
7+ days
|
| 212 |
+
```
|
| 213 |
+
|
| 214 |
+
Use the pipeline function to load the model from hugging face locally and give the classifier function the trip description together with the candidate labels.
|
| 215 |
+
|
| 216 |
+
```python
|
| 217 |
+
key = keys_list[0]
|
| 218 |
+
model_name = "facebook/bart-large-mnli"
|
| 219 |
+
trip_descr = "I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands."
|
| 220 |
+
classifier = pipeline("zero-shot-classification", model = model_name)
|
| 221 |
+
result = classifier(trip_descr, candidate_labels[keys_list[0]])
|
| 222 |
+
# Create DataFrame
|
| 223 |
+
df = pd.DataFrame({
|
| 224 |
+
"Label": result["labels"],
|
| 225 |
+
"Score": result["scores"]
|
| 226 |
+
})
|
| 227 |
+
print(df)
|
| 228 |
+
```
|
| 229 |
+
|
| 230 |
+
```text
|
| 231 |
+
Label Score
|
| 232 |
+
0 beach vacation 0.376311
|
| 233 |
+
1 micro-adventure / weekend trip 0.350168
|
| 234 |
+
2 nature escape 0.133974
|
| 235 |
+
3 digital nomad trip 0.031636
|
| 236 |
+
4 cultural exploration 0.031271
|
| 237 |
+
5 yoga / wellness retreat 0.012846
|
| 238 |
+
6 festival trip 0.012700
|
| 239 |
+
7 long-distance hike / thru-hike 0.009527
|
| 240 |
+
8 hut trek (summer) 0.008148
|
| 241 |
+
9 city trip 0.007793
|
| 242 |
+
10 road trip (car/camper) 0.006512
|
| 243 |
+
11 ski tour / skitour 0.005670
|
| 244 |
+
12 camping trip (campground) 0.004448
|
| 245 |
+
13 snowboard / splitboard trip 0.004113
|
| 246 |
+
14 camping trip (wild camping) 0.002714
|
| 247 |
+
15 hut trek (winter) 0.002170
|
| 248 |
+
```
|
| 249 |
+
|
| 250 |
+
Now we will do this for every superclass. We do something slightly different for the activities superclass since it is possible and likely to do more than one activity during your travels. Within the classifier function we set the multi_label option to True, which means that the text can belong to more than one class and each label is evaluated independently and a probability of belonging to that class is returned. We choose a
|
| 251 |
+
|
| 252 |
+
```python
|
| 253 |
+
cut_off = 0.5
|
| 254 |
+
result_activ = classifier(trip_descr, candidate_labels["activities"], multi_label=True)
|
| 255 |
+
indices = [i for i, score in enumerate(result_activ['scores']) if score > cut_off]
|
| 256 |
+
classes = [result_activ['labels'][i] for i in indices]
|
| 257 |
+
|
| 258 |
+
df = pd.DataFrame({
|
| 259 |
+
"Label": result["labels"],
|
| 260 |
+
"Score": result["scores"]
|
| 261 |
+
})
|
| 262 |
+
print(df)
|
| 263 |
+
print(classes)
|
| 264 |
```
|
| 265 |
|
| 266 |
+
```text
|
| 267 |
+
Label Score
|
| 268 |
+
0 beach vacation 0.376311
|
| 269 |
+
1 micro-adventure / weekend trip 0.350168
|
| 270 |
+
2 nature escape 0.133974
|
| 271 |
+
3 digital nomad trip 0.031636
|
| 272 |
+
4 cultural exploration 0.031271
|
| 273 |
+
5 yoga / wellness retreat 0.012846
|
| 274 |
+
6 festival trip 0.012700
|
| 275 |
+
7 long-distance hike / thru-hike 0.009527
|
| 276 |
+
8 hut trek (summer) 0.008148
|
| 277 |
+
9 city trip 0.007793
|
| 278 |
+
10 road trip (car/camper) 0.006512
|
| 279 |
+
11 ski tour / skitour 0.005670
|
| 280 |
+
12 camping trip (campground) 0.004448
|
| 281 |
+
13 snowboard / splitboard trip 0.004113
|
| 282 |
+
14 camping trip (wild camping) 0.002714
|
| 283 |
+
15 hut trek (winter) 0.002170
|
| 284 |
+
|
| 285 |
+
['going to the beach', 'relaxing', 'hiking']
|
| 286 |
+
```
|
| 287 |
+
|
| 288 |
+
To do this for all superclasses we use the following function
|
| 289 |
+
|
| 290 |
```python
|
| 291 |
+
# doing this for all superclasses, depending on local machine this might take a while
|
| 292 |
+
def pred_trip(model_name, trip_descr, cut_off = 0.5):
|
| 293 |
+
"""
|
| 294 |
+
Classifies trip
|
| 295 |
+
|
| 296 |
+
Parameters:
|
| 297 |
+
model_name: name of hugging-face model
|
| 298 |
+
trip_descr: text describing the trip
|
| 299 |
+
cut_off: cut_off for choosing activities
|
| 300 |
+
|
| 301 |
+
Returns:
|
| 302 |
+
pd Dataframe: with class predictions and true values
|
| 303 |
+
"""
|
| 304 |
+
|
| 305 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 306 |
+
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 307 |
+
for i, key in enumerate(keys_list):
|
| 308 |
+
print(f"\rProcessing {i + 1}/{len(keys_list)}", end="", flush=True)
|
| 309 |
+
if key == 'activities':
|
| 310 |
+
result = classifier(trip_descr, candidate_labels[key], multi_label=True)
|
| 311 |
+
indices = [i for i, score in enumerate(result['scores']) if score > cut_off]
|
| 312 |
+
classes = [result['labels'][i] for i in indices]
|
| 313 |
+
else:
|
| 314 |
+
result = classifier(trip_descr, candidate_labels[key])
|
| 315 |
+
classes = result["labels"][0]
|
| 316 |
+
df.loc[i] = [key, classes]
|
| 317 |
+
return df
|
| 318 |
+
|
| 319 |
+
result = pred_trip(model_name, trip_descr, cut_off = 0.5)
|
| 320 |
print(result)
|
| 321 |
```
|
| 322 |
|
| 323 |
+
```text
|
| 324 |
+
Processing 9/9 superclass pred_class
|
| 325 |
+
0 activity_type beach vacation
|
| 326 |
+
1 activities [going to the beach, relaxing, hiking]
|
| 327 |
+
2 climate_or_season warm destination / summer
|
| 328 |
+
3 style_or_comfort minimalist
|
| 329 |
+
4 dress_code casual
|
| 330 |
+
5 accommodation huts with half board
|
| 331 |
+
6 transportation no own vehicle
|
| 332 |
+
7 special_conditions off-grid / no electricity
|
| 333 |
+
8 trip_length_days 7+ days
|
| 334 |
+
```
|
| 335 |
+
|
| 336 |
### Using gradio app: Anja
|
| 337 |
+
Now we want to make it more user friendly using hte gradio app. You can for now run the app in your jupyter notebook.
|
| 338 |
+
|
| 339 |
+
```python
|
| 340 |
+
# Prerequisites
|
| 341 |
+
from transformers import pipeline
|
| 342 |
+
import json
|
| 343 |
+
import pandas as pd
|
| 344 |
+
import gradio as gr
|
| 345 |
+
|
| 346 |
+
# get candidate labels
|
| 347 |
+
with open("packing_label_structure.json", "r") as file:
|
| 348 |
+
candidate_labels = json.load(file)
|
| 349 |
+
keys_list = list(candidate_labels.keys())
|
| 350 |
+
|
| 351 |
+
# Load packing item data
|
| 352 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 353 |
+
packing_items = json.load(file)
|
| 354 |
+
```
|
| 355 |
+
|
| 356 |
+
```python
|
| 357 |
+
emo = gr.Interface(
|
| 358 |
+
fn=pred_trip,
|
| 359 |
+
inputs=[
|
| 360 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 361 |
+
gr.Textbox(label="Trip description"),
|
| 362 |
+
gr.Number(label="Activity cut-off", value = 0.5),
|
| 363 |
+
],
|
| 364 |
+
# outputs="dataframe",
|
| 365 |
+
outputs=[gr.Dataframe(label="DataFrame"), gr.Textbox(label="List of words")],
|
| 366 |
+
title="Trip classification",
|
| 367 |
+
description="Enter a text describing your trip",
|
| 368 |
+
)
|
| 369 |
+
|
| 370 |
+
# Launch the Gradio app
|
| 371 |
+
if __name__ == "__main__":
|
| 372 |
+
demo.launch()
|
| 373 |
+
|
| 374 |
+
```
|
| 375 |
+
|
| 376 |
+

|
| 377 |
+
|
| 378 |
|
| 379 |
### Share your model: Anja
|
| 380 |
* created space
|
|
|
|
| 393 |
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 394 |
```
|
| 395 |
|
| 396 |
+
generate an access token for your space by clicking on your icon and then selecting Access Tokens > + Create new token and as token type select Write. Give your tokena. name and click on Creat Token. Store your token securely. Use it
|
| 397 |
|
| 398 |
|
| 399 |
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
space/space/space/space/space/space/space/space/space/blog_post/img/gradio_pred_trip.png
ADDED
|
space/space/space/space/space/space/space/space/space/code_blog_post.ipynb
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
@@ -41,7 +41,7 @@
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
-
"execution_count":
|
| 45 |
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
|
@@ -155,26 +155,7 @@
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
-
"execution_count":
|
| 159 |
-
"id": "e114d7ab-9bd3-4214-a630-7f9166c3b8a5",
|
| 160 |
-
"metadata": {},
|
| 161 |
-
"outputs": [
|
| 162 |
-
{
|
| 163 |
-
"name": "stdout",
|
| 164 |
-
"output_type": "stream",
|
| 165 |
-
"text": [
|
| 166 |
-
"activity_type\n"
|
| 167 |
-
]
|
| 168 |
-
}
|
| 169 |
-
],
|
| 170 |
-
"source": [
|
| 171 |
-
"# First classify using first dimension of classes ()\n",
|
| 172 |
-
"print(keys_list[0])"
|
| 173 |
-
]
|
| 174 |
-
},
|
| 175 |
-
{
|
| 176 |
-
"cell_type": "code",
|
| 177 |
-
"execution_count": 40,
|
| 178 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 179 |
"metadata": {},
|
| 180 |
"outputs": [
|
|
@@ -210,6 +191,7 @@
|
|
| 210 |
}
|
| 211 |
],
|
| 212 |
"source": [
|
|
|
|
| 213 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 214 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 215 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
|
@@ -224,7 +206,7 @@
|
|
| 224 |
},
|
| 225 |
{
|
| 226 |
"cell_type": "code",
|
| 227 |
-
"execution_count":
|
| 228 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 229 |
"metadata": {},
|
| 230 |
"outputs": [
|
|
@@ -244,7 +226,7 @@
|
|
| 244 |
},
|
| 245 |
{
|
| 246 |
"cell_type": "code",
|
| 247 |
-
"execution_count":
|
| 248 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 249 |
"metadata": {},
|
| 250 |
"outputs": [
|
|
@@ -283,12 +265,13 @@
|
|
| 283 |
" \"Label\": result[\"labels\"],\n",
|
| 284 |
" \"Score\": result[\"scores\"]\n",
|
| 285 |
"})\n",
|
| 286 |
-
"print(df)"
|
|
|
|
| 287 |
]
|
| 288 |
},
|
| 289 |
{
|
| 290 |
"cell_type": "code",
|
| 291 |
-
"execution_count":
|
| 292 |
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 293 |
"metadata": {},
|
| 294 |
"outputs": [
|
|
@@ -300,13 +283,11 @@
|
|
| 300 |
]
|
| 301 |
}
|
| 302 |
],
|
| 303 |
-
"source": [
|
| 304 |
-
"print(classes)"
|
| 305 |
-
]
|
| 306 |
},
|
| 307 |
{
|
| 308 |
"cell_type": "code",
|
| 309 |
-
"execution_count":
|
| 310 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 311 |
"metadata": {},
|
| 312 |
"outputs": [],
|
|
@@ -342,7 +323,7 @@
|
|
| 342 |
},
|
| 343 |
{
|
| 344 |
"cell_type": "code",
|
| 345 |
-
"execution_count":
|
| 346 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 347 |
"metadata": {},
|
| 348 |
"outputs": [
|
|
@@ -357,7 +338,7 @@
|
|
| 357 |
"name": "stdout",
|
| 358 |
"output_type": "stream",
|
| 359 |
"text": [
|
| 360 |
-
"Processing
|
| 361 |
"0 activity_type beach vacation\n",
|
| 362 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 363 |
"2 climate_or_season warm destination / summer\n",
|
|
@@ -385,7 +366,7 @@
|
|
| 385 |
},
|
| 386 |
{
|
| 387 |
"cell_type": "code",
|
| 388 |
-
"execution_count":
|
| 389 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 390 |
"metadata": {},
|
| 391 |
"outputs": [],
|
|
@@ -401,10 +382,6 @@
|
|
| 401 |
" candidate_labels = json.load(file)\n",
|
| 402 |
"keys_list = list(candidate_labels.keys())\n",
|
| 403 |
"\n",
|
| 404 |
-
"# Load test data (in list of dictionaries)\n",
|
| 405 |
-
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 406 |
-
" packing_data = json.load(file)\n",
|
| 407 |
-
"\n",
|
| 408 |
"# Load packing item data\n",
|
| 409 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 410 |
" packing_items = json.load(file)"
|
|
@@ -412,7 +389,7 @@
|
|
| 412 |
},
|
| 413 |
{
|
| 414 |
"cell_type": "code",
|
| 415 |
-
"execution_count":
|
| 416 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 417 |
"metadata": {},
|
| 418 |
"outputs": [
|
|
@@ -420,9 +397,7 @@
|
|
| 420 |
"name": "stdout",
|
| 421 |
"output_type": "stream",
|
| 422 |
"text": [
|
| 423 |
-
"Running on local URL: http://127.0.0.1:
|
| 424 |
-
"\n",
|
| 425 |
-
"Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB\n",
|
| 426 |
"\n",
|
| 427 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 428 |
]
|
|
@@ -430,7 +405,7 @@
|
|
| 430 |
{
|
| 431 |
"data": {
|
| 432 |
"text/html": [
|
| 433 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 434 |
],
|
| 435 |
"text/plain": [
|
| 436 |
"<IPython.core.display.HTML object>"
|
|
@@ -438,85 +413,13 @@
|
|
| 438 |
},
|
| 439 |
"metadata": {},
|
| 440 |
"output_type": "display_data"
|
| 441 |
-
},
|
| 442 |
-
{
|
| 443 |
-
"name": "stderr",
|
| 444 |
-
"output_type": "stream",
|
| 445 |
-
"text": [
|
| 446 |
-
"Traceback (most recent call last):\n",
|
| 447 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 406, in hf_raise_for_status\n",
|
| 448 |
-
" response.raise_for_status()\n",
|
| 449 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/requests/models.py\", line 1024, in raise_for_status\n",
|
| 450 |
-
" raise HTTPError(http_error_msg, response=self)\n",
|
| 451 |
-
"requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json\n",
|
| 452 |
-
"\n",
|
| 453 |
-
"The above exception was the direct cause of the following exception:\n",
|
| 454 |
-
"\n",
|
| 455 |
-
"Traceback (most recent call last):\n",
|
| 456 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 403, in cached_file\n",
|
| 457 |
-
" resolved_file = hf_hub_download(\n",
|
| 458 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 459 |
-
" return fn(*args, **kwargs)\n",
|
| 460 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 860, in hf_hub_download\n",
|
| 461 |
-
" return _hf_hub_download_to_cache_dir(\n",
|
| 462 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 967, in _hf_hub_download_to_cache_dir\n",
|
| 463 |
-
" _raise_on_head_call_error(head_call_error, force_download, local_files_only)\n",
|
| 464 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1482, in _raise_on_head_call_error\n",
|
| 465 |
-
" raise head_call_error\n",
|
| 466 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1374, in _get_metadata_or_catch_error\n",
|
| 467 |
-
" metadata = get_hf_file_metadata(\n",
|
| 468 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 469 |
-
" return fn(*args, **kwargs)\n",
|
| 470 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1294, in get_hf_file_metadata\n",
|
| 471 |
-
" r = _request_wrapper(\n",
|
| 472 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 278, in _request_wrapper\n",
|
| 473 |
-
" response = _request_wrapper(\n",
|
| 474 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 302, in _request_wrapper\n",
|
| 475 |
-
" hf_raise_for_status(response)\n",
|
| 476 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 454, in hf_raise_for_status\n",
|
| 477 |
-
" raise _format(RepositoryNotFoundError, message, response) from e\n",
|
| 478 |
-
"huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-68fa0a8b-0d531523246f6614068c359e;d99d5b5d-694d-4d4b-b774-bb70066228d2)\n",
|
| 479 |
-
"\n",
|
| 480 |
-
"Repository Not Found for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json.\n",
|
| 481 |
-
"Please make sure you specified the correct `repo_id` and `repo_type`.\n",
|
| 482 |
-
"If you are trying to access a private or gated repo, make sure you are authenticated.\n",
|
| 483 |
-
"Invalid username or password.\n",
|
| 484 |
-
"\n",
|
| 485 |
-
"The above exception was the direct cause of the following exception:\n",
|
| 486 |
-
"\n",
|
| 487 |
-
"Traceback (most recent call last):\n",
|
| 488 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/queueing.py\", line 536, in process_events\n",
|
| 489 |
-
" response = await route_utils.call_process_api(\n",
|
| 490 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/route_utils.py\", line 322, in call_process_api\n",
|
| 491 |
-
" output = await app.get_blocks().process_api(\n",
|
| 492 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1935, in process_api\n",
|
| 493 |
-
" result = await self.call_function(\n",
|
| 494 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1520, in call_function\n",
|
| 495 |
-
" prediction = await anyio.to_thread.run_sync( # type: ignore\n",
|
| 496 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/to_thread.py\", line 56, in run_sync\n",
|
| 497 |
-
" return await get_async_backend().run_sync_in_worker_thread(\n",
|
| 498 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 2134, in run_sync_in_worker_thread\n",
|
| 499 |
-
" return await future\n",
|
| 500 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 851, in run\n",
|
| 501 |
-
" result = context.run(func, *args)\n",
|
| 502 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/utils.py\", line 826, in wrapper\n",
|
| 503 |
-
" response = f(*args, **kwargs)\n",
|
| 504 |
-
" File \"/var/folders/r5/y63bxy5d4_zfxf6cgwtwysbc0000gn/T/ipykernel_22464/1188444081.py\", line 3, in classify\n",
|
| 505 |
-
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 506 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py\", line 798, in pipeline\n",
|
| 507 |
-
" resolved_config_file = cached_file(\n",
|
| 508 |
-
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 426, in cached_file\n",
|
| 509 |
-
" raise EnvironmentError(\n",
|
| 510 |
-
"OSError: MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\n",
|
| 511 |
-
"If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`\n"
|
| 512 |
-
]
|
| 513 |
}
|
| 514 |
],
|
| 515 |
"source": [
|
| 516 |
"demo = gr.Interface(\n",
|
| 517 |
" fn=pred_trip,\n",
|
| 518 |
" inputs=[\n",
|
| 519 |
-
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer
|
| 520 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 521 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 522 |
" ],\n",
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 1,
|
| 14 |
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [],
|
|
|
|
| 41 |
},
|
| 42 |
{
|
| 43 |
"cell_type": "code",
|
| 44 |
+
"execution_count": 2,
|
| 45 |
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 46 |
"metadata": {},
|
| 47 |
"outputs": [
|
|
|
|
| 155 |
},
|
| 156 |
{
|
| 157 |
"cell_type": "code",
|
| 158 |
+
"execution_count": 12,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 160 |
"metadata": {},
|
| 161 |
"outputs": [
|
|
|
|
| 191 |
}
|
| 192 |
],
|
| 193 |
"source": [
|
| 194 |
+
"key = keys_list[0]\n",
|
| 195 |
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 196 |
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 197 |
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
|
|
|
| 206 |
},
|
| 207 |
{
|
| 208 |
"cell_type": "code",
|
| 209 |
+
"execution_count": 5,
|
| 210 |
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 211 |
"metadata": {},
|
| 212 |
"outputs": [
|
|
|
|
| 226 |
},
|
| 227 |
{
|
| 228 |
"cell_type": "code",
|
| 229 |
+
"execution_count": 6,
|
| 230 |
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 231 |
"metadata": {},
|
| 232 |
"outputs": [
|
|
|
|
| 265 |
" \"Label\": result[\"labels\"],\n",
|
| 266 |
" \"Score\": result[\"scores\"]\n",
|
| 267 |
"})\n",
|
| 268 |
+
"print(df)\n",
|
| 269 |
+
"print(classes)"
|
| 270 |
]
|
| 271 |
},
|
| 272 |
{
|
| 273 |
"cell_type": "code",
|
| 274 |
+
"execution_count": 7,
|
| 275 |
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 276 |
"metadata": {},
|
| 277 |
"outputs": [
|
|
|
|
| 283 |
]
|
| 284 |
}
|
| 285 |
],
|
| 286 |
+
"source": []
|
|
|
|
|
|
|
| 287 |
},
|
| 288 |
{
|
| 289 |
"cell_type": "code",
|
| 290 |
+
"execution_count": 8,
|
| 291 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 292 |
"metadata": {},
|
| 293 |
"outputs": [],
|
|
|
|
| 323 |
},
|
| 324 |
{
|
| 325 |
"cell_type": "code",
|
| 326 |
+
"execution_count": 9,
|
| 327 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 328 |
"metadata": {},
|
| 329 |
"outputs": [
|
|
|
|
| 338 |
"name": "stdout",
|
| 339 |
"output_type": "stream",
|
| 340 |
"text": [
|
| 341 |
+
"Processing 9/9 superclass pred_class\n",
|
| 342 |
"0 activity_type beach vacation\n",
|
| 343 |
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 344 |
"2 climate_or_season warm destination / summer\n",
|
|
|
|
| 366 |
},
|
| 367 |
{
|
| 368 |
"cell_type": "code",
|
| 369 |
+
"execution_count": 10,
|
| 370 |
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 371 |
"metadata": {},
|
| 372 |
"outputs": [],
|
|
|
|
| 382 |
" candidate_labels = json.load(file)\n",
|
| 383 |
"keys_list = list(candidate_labels.keys())\n",
|
| 384 |
"\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 385 |
"# Load packing item data\n",
|
| 386 |
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 387 |
" packing_items = json.load(file)"
|
|
|
|
| 389 |
},
|
| 390 |
{
|
| 391 |
"cell_type": "code",
|
| 392 |
+
"execution_count": 11,
|
| 393 |
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 394 |
"metadata": {},
|
| 395 |
"outputs": [
|
|
|
|
| 397 |
"name": "stdout",
|
| 398 |
"output_type": "stream",
|
| 399 |
"text": [
|
| 400 |
+
"Running on local URL: http://127.0.0.1:7860\n",
|
|
|
|
|
|
|
| 401 |
"\n",
|
| 402 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 403 |
]
|
|
|
|
| 405 |
{
|
| 406 |
"data": {
|
| 407 |
"text/html": [
|
| 408 |
+
"<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 409 |
],
|
| 410 |
"text/plain": [
|
| 411 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 413 |
},
|
| 414 |
"metadata": {},
|
| 415 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 416 |
}
|
| 417 |
],
|
| 418 |
"source": [
|
| 419 |
"demo = gr.Interface(\n",
|
| 420 |
" fn=pred_trip,\n",
|
| 421 |
" inputs=[\n",
|
| 422 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 423 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 424 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 425 |
" ],\n",
|
space/space/space/space/space/space/space/space/space/space/main_model.ipynb
CHANGED
|
@@ -946,7 +946,7 @@
|
|
| 946 |
},
|
| 947 |
{
|
| 948 |
"cell_type": "code",
|
| 949 |
-
"execution_count":
|
| 950 |
"id": "5bf23e10-0a93-4b2f-9508-34bb0974d24c",
|
| 951 |
"metadata": {},
|
| 952 |
"outputs": [],
|
|
@@ -973,7 +973,7 @@
|
|
| 973 |
},
|
| 974 |
{
|
| 975 |
"cell_type": "code",
|
| 976 |
-
"execution_count":
|
| 977 |
"id": "61ebbe99-2563-4c99-ba65-d2312c9d5844",
|
| 978 |
"metadata": {},
|
| 979 |
"outputs": [
|
|
@@ -981,7 +981,7 @@
|
|
| 981 |
"name": "stdout",
|
| 982 |
"output_type": "stream",
|
| 983 |
"text": [
|
| 984 |
-
"Running on local URL: http://127.0.0.1:
|
| 985 |
"\n",
|
| 986 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 987 |
]
|
|
@@ -989,7 +989,7 @@
|
|
| 989 |
{
|
| 990 |
"data": {
|
| 991 |
"text/html": [
|
| 992 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 993 |
],
|
| 994 |
"text/plain": [
|
| 995 |
"<IPython.core.display.HTML object>"
|
|
@@ -997,6 +997,13 @@
|
|
| 997 |
},
|
| 998 |
"metadata": {},
|
| 999 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1000 |
}
|
| 1001 |
],
|
| 1002 |
"source": [
|
|
@@ -1033,7 +1040,7 @@
|
|
| 1033 |
"demo = gr.Interface(\n",
|
| 1034 |
" fn=classify,\n",
|
| 1035 |
" inputs=[\n",
|
| 1036 |
-
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer
|
| 1037 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 1038 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 1039 |
" ],\n",
|
|
@@ -1482,7 +1489,7 @@
|
|
| 1482 |
{
|
| 1483 |
"cell_type": "code",
|
| 1484 |
"execution_count": null,
|
| 1485 |
-
"id": "
|
| 1486 |
"metadata": {},
|
| 1487 |
"outputs": [],
|
| 1488 |
"source": []
|
|
|
|
| 946 |
},
|
| 947 |
{
|
| 948 |
"cell_type": "code",
|
| 949 |
+
"execution_count": 2,
|
| 950 |
"id": "5bf23e10-0a93-4b2f-9508-34bb0974d24c",
|
| 951 |
"metadata": {},
|
| 952 |
"outputs": [],
|
|
|
|
| 973 |
},
|
| 974 |
{
|
| 975 |
"cell_type": "code",
|
| 976 |
+
"execution_count": 3,
|
| 977 |
"id": "61ebbe99-2563-4c99-ba65-d2312c9d5844",
|
| 978 |
"metadata": {},
|
| 979 |
"outputs": [
|
|
|
|
| 981 |
"name": "stdout",
|
| 982 |
"output_type": "stream",
|
| 983 |
"text": [
|
| 984 |
+
"Running on local URL: http://127.0.0.1:7862\n",
|
| 985 |
"\n",
|
| 986 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 987 |
]
|
|
|
|
| 989 |
{
|
| 990 |
"data": {
|
| 991 |
"text/html": [
|
| 992 |
+
"<div><iframe src=\"http://127.0.0.1:7862/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 993 |
],
|
| 994 |
"text/plain": [
|
| 995 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 997 |
},
|
| 998 |
"metadata": {},
|
| 999 |
"output_type": "display_data"
|
| 1000 |
+
},
|
| 1001 |
+
{
|
| 1002 |
+
"name": "stderr",
|
| 1003 |
+
"output_type": "stream",
|
| 1004 |
+
"text": [
|
| 1005 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 1006 |
+
]
|
| 1007 |
}
|
| 1008 |
],
|
| 1009 |
"source": [
|
|
|
|
| 1040 |
"demo = gr.Interface(\n",
|
| 1041 |
" fn=classify,\n",
|
| 1042 |
" inputs=[\n",
|
| 1043 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 1044 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 1045 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 1046 |
" ],\n",
|
|
|
|
| 1489 |
{
|
| 1490 |
"cell_type": "code",
|
| 1491 |
"execution_count": null,
|
| 1492 |
+
"id": "e300e3f3-93e0-457b-b2f0-e05cc5c2cafb",
|
| 1493 |
"metadata": {},
|
| 1494 |
"outputs": [],
|
| 1495 |
"source": []
|
space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/app-checkpoint.py
CHANGED
|
@@ -1,4 +1,7 @@
|
|
| 1 |
# Prerequisites
|
|
|
|
|
|
|
|
|
|
| 2 |
from transformers import pipeline
|
| 3 |
import json
|
| 4 |
import pandas as pd
|
|
@@ -9,10 +12,6 @@ with open("packing_label_structure.json", "r") as file:
|
|
| 9 |
candidate_labels = json.load(file)
|
| 10 |
keys_list = list(candidate_labels.keys())
|
| 11 |
|
| 12 |
-
# Load test data (in list of dictionaries)
|
| 13 |
-
with open("test_data.json", "r") as file:
|
| 14 |
-
packing_data = json.load(file)
|
| 15 |
-
|
| 16 |
# Load packing item data
|
| 17 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 18 |
packing_items = json.load(file)
|
|
@@ -50,7 +49,7 @@ def classify(model_name, trip_descr, cut_off = 0.5):
|
|
| 50 |
demo = gr.Interface(
|
| 51 |
fn=classify,
|
| 52 |
inputs=[
|
| 53 |
-
gr.Textbox(label="Model name", value = "MoritzLaurer
|
| 54 |
gr.Textbox(label="Trip description"),
|
| 55 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 56 |
],
|
|
|
|
| 1 |
# Prerequisites
|
| 2 |
+
import os
|
| 3 |
+
os.environ["OMP_NUM_THREADS"] = "1" # Set 1, 2, or 4 depending on CPU usage
|
| 4 |
+
|
| 5 |
from transformers import pipeline
|
| 6 |
import json
|
| 7 |
import pandas as pd
|
|
|
|
| 12 |
candidate_labels = json.load(file)
|
| 13 |
keys_list = list(candidate_labels.keys())
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
# Load packing item data
|
| 16 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 17 |
packing_items = json.load(file)
|
|
|
|
| 49 |
demo = gr.Interface(
|
| 50 |
fn=classify,
|
| 51 |
inputs=[
|
| 52 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 53 |
gr.Textbox(label="Trip description"),
|
| 54 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 55 |
],
|
space/space/space/space/space/space/space/space/space/space/space/app.py
CHANGED
|
@@ -1,4 +1,7 @@
|
|
| 1 |
# Prerequisites
|
|
|
|
|
|
|
|
|
|
| 2 |
from transformers import pipeline
|
| 3 |
import json
|
| 4 |
import pandas as pd
|
|
@@ -9,10 +12,6 @@ with open("packing_label_structure.json", "r") as file:
|
|
| 9 |
candidate_labels = json.load(file)
|
| 10 |
keys_list = list(candidate_labels.keys())
|
| 11 |
|
| 12 |
-
# Load test data (in list of dictionaries)
|
| 13 |
-
with open("test_data.json", "r") as file:
|
| 14 |
-
packing_data = json.load(file)
|
| 15 |
-
|
| 16 |
# Load packing item data
|
| 17 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 18 |
packing_items = json.load(file)
|
|
@@ -50,7 +49,7 @@ def classify(model_name, trip_descr, cut_off = 0.5):
|
|
| 50 |
demo = gr.Interface(
|
| 51 |
fn=classify,
|
| 52 |
inputs=[
|
| 53 |
-
gr.Textbox(label="Model name", value = "MoritzLaurer
|
| 54 |
gr.Textbox(label="Trip description"),
|
| 55 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 56 |
],
|
|
|
|
| 1 |
# Prerequisites
|
| 2 |
+
import os
|
| 3 |
+
os.environ["OMP_NUM_THREADS"] = "1" # Set 1, 2, or 4 depending on CPU usage
|
| 4 |
+
|
| 5 |
from transformers import pipeline
|
| 6 |
import json
|
| 7 |
import pandas as pd
|
|
|
|
| 12 |
candidate_labels = json.load(file)
|
| 13 |
keys_list = list(candidate_labels.keys())
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
# Load packing item data
|
| 16 |
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 17 |
packing_items = json.load(file)
|
|
|
|
| 49 |
demo = gr.Interface(
|
| 50 |
fn=classify,
|
| 51 |
inputs=[
|
| 52 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 53 |
gr.Textbox(label="Trip description"),
|
| 54 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 55 |
],
|
space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
CHANGED
|
@@ -16,14 +16,15 @@
|
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
"import math\n",
|
| 19 |
-
"import json\n",
|
| 20 |
"import pickle\n",
|
| 21 |
"import os\n",
|
| 22 |
"import time\n",
|
| 23 |
-
"import pandas as pd\n",
|
| 24 |
"import matplotlib.pyplot as plt\n",
|
| 25 |
"from tabulate import tabulate\n",
|
|
|
|
| 26 |
"from transformers import pipeline\n",
|
|
|
|
|
|
|
| 27 |
"\n",
|
| 28 |
"# Get candidate labels\n",
|
| 29 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
|
@@ -173,7 +174,7 @@
|
|
| 173 |
},
|
| 174 |
{
|
| 175 |
"cell_type": "code",
|
| 176 |
-
"execution_count":
|
| 177 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 178 |
"metadata": {},
|
| 179 |
"outputs": [
|
|
@@ -305,7 +306,7 @@
|
|
| 305 |
},
|
| 306 |
{
|
| 307 |
"cell_type": "code",
|
| 308 |
-
"execution_count":
|
| 309 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 310 |
"metadata": {},
|
| 311 |
"outputs": [],
|
|
@@ -327,7 +328,7 @@
|
|
| 327 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 328 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 329 |
" for i, key in enumerate(keys_list):\n",
|
| 330 |
-
" print(f\"\\rProcessing {i}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 331 |
" if key == 'activities':\n",
|
| 332 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 333 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
@@ -341,7 +342,7 @@
|
|
| 341 |
},
|
| 342 |
{
|
| 343 |
"cell_type": "code",
|
| 344 |
-
"execution_count":
|
| 345 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 346 |
"metadata": {},
|
| 347 |
"outputs": [
|
|
@@ -356,7 +357,16 @@
|
|
| 356 |
"name": "stdout",
|
| 357 |
"output_type": "stream",
|
| 358 |
"text": [
|
| 359 |
-
"Processing
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 360 |
]
|
| 361 |
}
|
| 362 |
],
|
|
@@ -365,10 +375,221 @@
|
|
| 365 |
"print(result)"
|
| 366 |
]
|
| 367 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 368 |
{
|
| 369 |
"cell_type": "code",
|
| 370 |
"execution_count": null,
|
| 371 |
-
"id": "
|
| 372 |
"metadata": {},
|
| 373 |
"outputs": [],
|
| 374 |
"source": []
|
|
|
|
| 16 |
"outputs": [],
|
| 17 |
"source": [
|
| 18 |
"import math\n",
|
|
|
|
| 19 |
"import pickle\n",
|
| 20 |
"import os\n",
|
| 21 |
"import time\n",
|
|
|
|
| 22 |
"import matplotlib.pyplot as plt\n",
|
| 23 |
"from tabulate import tabulate\n",
|
| 24 |
+
"\n",
|
| 25 |
"from transformers import pipeline\n",
|
| 26 |
+
"import json\n",
|
| 27 |
+
"import pandas as pd\n",
|
| 28 |
"\n",
|
| 29 |
"# Get candidate labels\n",
|
| 30 |
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
|
|
|
| 174 |
},
|
| 175 |
{
|
| 176 |
"cell_type": "code",
|
| 177 |
+
"execution_count": 40,
|
| 178 |
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 179 |
"metadata": {},
|
| 180 |
"outputs": [
|
|
|
|
| 306 |
},
|
| 307 |
{
|
| 308 |
"cell_type": "code",
|
| 309 |
+
"execution_count": 45,
|
| 310 |
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 311 |
"metadata": {},
|
| 312 |
"outputs": [],
|
|
|
|
| 328 |
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 329 |
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 330 |
" for i, key in enumerate(keys_list):\n",
|
| 331 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 332 |
" if key == 'activities':\n",
|
| 333 |
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 334 |
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
|
|
|
| 342 |
},
|
| 343 |
{
|
| 344 |
"cell_type": "code",
|
| 345 |
+
"execution_count": 35,
|
| 346 |
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 347 |
"metadata": {},
|
| 348 |
"outputs": [
|
|
|
|
| 357 |
"name": "stdout",
|
| 358 |
"output_type": "stream",
|
| 359 |
"text": [
|
| 360 |
+
"Processing 8/9... superclass pred_class\n",
|
| 361 |
+
"0 activity_type beach vacation\n",
|
| 362 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 363 |
+
"2 climate_or_season warm destination / summer\n",
|
| 364 |
+
"3 style_or_comfort minimalist\n",
|
| 365 |
+
"4 dress_code casual\n",
|
| 366 |
+
"5 accommodation huts with half board\n",
|
| 367 |
+
"6 transportation no own vehicle\n",
|
| 368 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 369 |
+
"8 trip_length_days 7+ days\n"
|
| 370 |
]
|
| 371 |
}
|
| 372 |
],
|
|
|
|
| 375 |
"print(result)"
|
| 376 |
]
|
| 377 |
},
|
| 378 |
+
{
|
| 379 |
+
"cell_type": "markdown",
|
| 380 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 381 |
+
"metadata": {},
|
| 382 |
+
"source": [
|
| 383 |
+
"Now use gradio app"
|
| 384 |
+
]
|
| 385 |
+
},
|
| 386 |
+
{
|
| 387 |
+
"cell_type": "code",
|
| 388 |
+
"execution_count": 37,
|
| 389 |
+
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 390 |
+
"metadata": {},
|
| 391 |
+
"outputs": [],
|
| 392 |
+
"source": [
|
| 393 |
+
"# Prerequisites\n",
|
| 394 |
+
"from transformers import pipeline\n",
|
| 395 |
+
"import json\n",
|
| 396 |
+
"import pandas as pd\n",
|
| 397 |
+
"import gradio as gr\n",
|
| 398 |
+
"\n",
|
| 399 |
+
"# get candidate labels\n",
|
| 400 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 401 |
+
" candidate_labels = json.load(file)\n",
|
| 402 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 403 |
+
"\n",
|
| 404 |
+
"# Load test data (in list of dictionaries)\n",
|
| 405 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 406 |
+
" packing_data = json.load(file)\n",
|
| 407 |
+
"\n",
|
| 408 |
+
"# Load packing item data\n",
|
| 409 |
+
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 410 |
+
" packing_items = json.load(file)"
|
| 411 |
+
]
|
| 412 |
+
},
|
| 413 |
+
{
|
| 414 |
+
"cell_type": "code",
|
| 415 |
+
"execution_count": 39,
|
| 416 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 417 |
+
"metadata": {},
|
| 418 |
+
"outputs": [
|
| 419 |
+
{
|
| 420 |
+
"name": "stdout",
|
| 421 |
+
"output_type": "stream",
|
| 422 |
+
"text": [
|
| 423 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 424 |
+
"\n",
|
| 425 |
+
"Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB\n",
|
| 426 |
+
"\n",
|
| 427 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 428 |
+
]
|
| 429 |
+
},
|
| 430 |
+
{
|
| 431 |
+
"data": {
|
| 432 |
+
"text/html": [
|
| 433 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 434 |
+
],
|
| 435 |
+
"text/plain": [
|
| 436 |
+
"<IPython.core.display.HTML object>"
|
| 437 |
+
]
|
| 438 |
+
},
|
| 439 |
+
"metadata": {},
|
| 440 |
+
"output_type": "display_data"
|
| 441 |
+
},
|
| 442 |
+
{
|
| 443 |
+
"name": "stderr",
|
| 444 |
+
"output_type": "stream",
|
| 445 |
+
"text": [
|
| 446 |
+
"Traceback (most recent call last):\n",
|
| 447 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 406, in hf_raise_for_status\n",
|
| 448 |
+
" response.raise_for_status()\n",
|
| 449 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/requests/models.py\", line 1024, in raise_for_status\n",
|
| 450 |
+
" raise HTTPError(http_error_msg, response=self)\n",
|
| 451 |
+
"requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json\n",
|
| 452 |
+
"\n",
|
| 453 |
+
"The above exception was the direct cause of the following exception:\n",
|
| 454 |
+
"\n",
|
| 455 |
+
"Traceback (most recent call last):\n",
|
| 456 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 403, in cached_file\n",
|
| 457 |
+
" resolved_file = hf_hub_download(\n",
|
| 458 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 459 |
+
" return fn(*args, **kwargs)\n",
|
| 460 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 860, in hf_hub_download\n",
|
| 461 |
+
" return _hf_hub_download_to_cache_dir(\n",
|
| 462 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 967, in _hf_hub_download_to_cache_dir\n",
|
| 463 |
+
" _raise_on_head_call_error(head_call_error, force_download, local_files_only)\n",
|
| 464 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1482, in _raise_on_head_call_error\n",
|
| 465 |
+
" raise head_call_error\n",
|
| 466 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1374, in _get_metadata_or_catch_error\n",
|
| 467 |
+
" metadata = get_hf_file_metadata(\n",
|
| 468 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 469 |
+
" return fn(*args, **kwargs)\n",
|
| 470 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1294, in get_hf_file_metadata\n",
|
| 471 |
+
" r = _request_wrapper(\n",
|
| 472 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 278, in _request_wrapper\n",
|
| 473 |
+
" response = _request_wrapper(\n",
|
| 474 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 302, in _request_wrapper\n",
|
| 475 |
+
" hf_raise_for_status(response)\n",
|
| 476 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 454, in hf_raise_for_status\n",
|
| 477 |
+
" raise _format(RepositoryNotFoundError, message, response) from e\n",
|
| 478 |
+
"huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-68fa0a8b-0d531523246f6614068c359e;d99d5b5d-694d-4d4b-b774-bb70066228d2)\n",
|
| 479 |
+
"\n",
|
| 480 |
+
"Repository Not Found for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json.\n",
|
| 481 |
+
"Please make sure you specified the correct `repo_id` and `repo_type`.\n",
|
| 482 |
+
"If you are trying to access a private or gated repo, make sure you are authenticated.\n",
|
| 483 |
+
"Invalid username or password.\n",
|
| 484 |
+
"\n",
|
| 485 |
+
"The above exception was the direct cause of the following exception:\n",
|
| 486 |
+
"\n",
|
| 487 |
+
"Traceback (most recent call last):\n",
|
| 488 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/queueing.py\", line 536, in process_events\n",
|
| 489 |
+
" response = await route_utils.call_process_api(\n",
|
| 490 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/route_utils.py\", line 322, in call_process_api\n",
|
| 491 |
+
" output = await app.get_blocks().process_api(\n",
|
| 492 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1935, in process_api\n",
|
| 493 |
+
" result = await self.call_function(\n",
|
| 494 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1520, in call_function\n",
|
| 495 |
+
" prediction = await anyio.to_thread.run_sync( # type: ignore\n",
|
| 496 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/to_thread.py\", line 56, in run_sync\n",
|
| 497 |
+
" return await get_async_backend().run_sync_in_worker_thread(\n",
|
| 498 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 2134, in run_sync_in_worker_thread\n",
|
| 499 |
+
" return await future\n",
|
| 500 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 851, in run\n",
|
| 501 |
+
" result = context.run(func, *args)\n",
|
| 502 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/utils.py\", line 826, in wrapper\n",
|
| 503 |
+
" response = f(*args, **kwargs)\n",
|
| 504 |
+
" File \"/var/folders/r5/y63bxy5d4_zfxf6cgwtwysbc0000gn/T/ipykernel_22464/1188444081.py\", line 3, in classify\n",
|
| 505 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 506 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py\", line 798, in pipeline\n",
|
| 507 |
+
" resolved_config_file = cached_file(\n",
|
| 508 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 426, in cached_file\n",
|
| 509 |
+
" raise EnvironmentError(\n",
|
| 510 |
+
"OSError: MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\n",
|
| 511 |
+
"If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`\n"
|
| 512 |
+
]
|
| 513 |
+
}
|
| 514 |
+
],
|
| 515 |
+
"source": [
|
| 516 |
+
"demo = gr.Interface(\n",
|
| 517 |
+
" fn=pred_trip,\n",
|
| 518 |
+
" inputs=[\n",
|
| 519 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 520 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 521 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 522 |
+
" ],\n",
|
| 523 |
+
" # outputs=\"dataframe\",\n",
|
| 524 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\"), gr.Textbox(label=\"List of words\")],\n",
|
| 525 |
+
" title=\"Trip classification\",\n",
|
| 526 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 527 |
+
")\n",
|
| 528 |
+
"\n",
|
| 529 |
+
"# Launch the Gradio app\n",
|
| 530 |
+
"if __name__ == \"__main__\":\n",
|
| 531 |
+
" demo.launch()\n"
|
| 532 |
+
]
|
| 533 |
+
},
|
| 534 |
+
{
|
| 535 |
+
"cell_type": "code",
|
| 536 |
+
"execution_count": 46,
|
| 537 |
+
"id": "11006b67-bfd5-42a7-99c4-36c3db3affac",
|
| 538 |
+
"metadata": {},
|
| 539 |
+
"outputs": [
|
| 540 |
+
{
|
| 541 |
+
"name": "stderr",
|
| 542 |
+
"output_type": "stream",
|
| 543 |
+
"text": [
|
| 544 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 545 |
+
]
|
| 546 |
+
},
|
| 547 |
+
{
|
| 548 |
+
"name": "stdout",
|
| 549 |
+
"output_type": "stream",
|
| 550 |
+
"text": [
|
| 551 |
+
"Processing 9/9"
|
| 552 |
+
]
|
| 553 |
+
}
|
| 554 |
+
],
|
| 555 |
+
"source": [
|
| 556 |
+
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 557 |
+
]
|
| 558 |
+
},
|
| 559 |
+
{
|
| 560 |
+
"cell_type": "code",
|
| 561 |
+
"execution_count": null,
|
| 562 |
+
"id": "0f6cd8d1-b742-4034-a3e2-e55c3ddd2904",
|
| 563 |
+
"metadata": {},
|
| 564 |
+
"outputs": [],
|
| 565 |
+
"source": [
|
| 566 |
+
"print(test)"
|
| 567 |
+
]
|
| 568 |
+
},
|
| 569 |
+
{
|
| 570 |
+
"cell_type": "code",
|
| 571 |
+
"execution_count": 41,
|
| 572 |
+
"id": "5db16415-47c1-42c6-861c-5d26f4b3bb03",
|
| 573 |
+
"metadata": {},
|
| 574 |
+
"outputs": [
|
| 575 |
+
{
|
| 576 |
+
"name": "stdout",
|
| 577 |
+
"output_type": "stream",
|
| 578 |
+
"text": [
|
| 579 |
+
"facebook/bart-large-mnli\n",
|
| 580 |
+
"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\n"
|
| 581 |
+
]
|
| 582 |
+
}
|
| 583 |
+
],
|
| 584 |
+
"source": [
|
| 585 |
+
"print(model_name)\n",
|
| 586 |
+
"print(trip_descr)"
|
| 587 |
+
]
|
| 588 |
+
},
|
| 589 |
{
|
| 590 |
"cell_type": "code",
|
| 591 |
"execution_count": null,
|
| 592 |
+
"id": "bbca6bc6-bd97-4d43-8a6e-fe338f2735f2",
|
| 593 |
"metadata": {},
|
| 594 |
"outputs": [],
|
| 595 |
"source": []
|
space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb
CHANGED
|
@@ -18,7 +18,7 @@
|
|
| 18 |
},
|
| 19 |
{
|
| 20 |
"cell_type": "code",
|
| 21 |
-
"execution_count":
|
| 22 |
"id": "f8c28d2d-8458-49fd-8ebf-5e729d6e861f",
|
| 23 |
"metadata": {},
|
| 24 |
"outputs": [],
|
|
@@ -59,9 +59,6 @@
|
|
| 59 |
"execution_count": 2,
|
| 60 |
"id": "89d42ca7-e871-4eda-b428-69e9bd965428",
|
| 61 |
"metadata": {
|
| 62 |
-
"jupyter": {
|
| 63 |
-
"source_hidden": true
|
| 64 |
-
},
|
| 65 |
"scrolled": true
|
| 66 |
},
|
| 67 |
"outputs": [
|
|
@@ -176,14 +173,14 @@
|
|
| 176 |
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 177 |
"\n",
|
| 178 |
"ski tour / skitour\n",
|
| 179 |
-
"['ski touring', 'photography'
|
| 180 |
"cold destination / winter\n",
|
| 181 |
"minimalist\n",
|
| 182 |
"conservative\n",
|
| 183 |
"indoor\n",
|
| 184 |
"no own vehicle\n",
|
| 185 |
"avalanche-prone terrain\n",
|
| 186 |
-
"
|
| 187 |
"\n",
|
| 188 |
"\n",
|
| 189 |
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
|
@@ -192,10 +189,10 @@
|
|
| 192 |
"['scuba diving', 'kayaking / canoeing']\n",
|
| 193 |
"tropical / humid\n",
|
| 194 |
"lightweight (but comfortable)\n",
|
| 195 |
-
"
|
| 196 |
"sleeping in a tent\n",
|
| 197 |
"own vehicle\n",
|
| 198 |
-
"
|
| 199 |
"3 days\n",
|
| 200 |
"\n",
|
| 201 |
"\n"
|
|
@@ -949,7 +946,7 @@
|
|
| 949 |
},
|
| 950 |
{
|
| 951 |
"cell_type": "code",
|
| 952 |
-
"execution_count":
|
| 953 |
"id": "5bf23e10-0a93-4b2f-9508-34bb0974d24c",
|
| 954 |
"metadata": {},
|
| 955 |
"outputs": [],
|
|
@@ -976,7 +973,7 @@
|
|
| 976 |
},
|
| 977 |
{
|
| 978 |
"cell_type": "code",
|
| 979 |
-
"execution_count":
|
| 980 |
"id": "61ebbe99-2563-4c99-ba65-d2312c9d5844",
|
| 981 |
"metadata": {},
|
| 982 |
"outputs": [
|
|
@@ -984,7 +981,7 @@
|
|
| 984 |
"name": "stdout",
|
| 985 |
"output_type": "stream",
|
| 986 |
"text": [
|
| 987 |
-
"Running on local URL: http://127.0.0.1:
|
| 988 |
"\n",
|
| 989 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 990 |
]
|
|
@@ -992,7 +989,7 @@
|
|
| 992 |
{
|
| 993 |
"data": {
|
| 994 |
"text/html": [
|
| 995 |
-
"<div><iframe src=\"http://127.0.0.1:
|
| 996 |
],
|
| 997 |
"text/plain": [
|
| 998 |
"<IPython.core.display.HTML object>"
|
|
@@ -1000,13 +997,6 @@
|
|
| 1000 |
},
|
| 1001 |
"metadata": {},
|
| 1002 |
"output_type": "display_data"
|
| 1003 |
-
},
|
| 1004 |
-
{
|
| 1005 |
-
"name": "stderr",
|
| 1006 |
-
"output_type": "stream",
|
| 1007 |
-
"text": [
|
| 1008 |
-
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 1009 |
-
]
|
| 1010 |
}
|
| 1011 |
],
|
| 1012 |
"source": [
|
|
@@ -1043,7 +1033,7 @@
|
|
| 1043 |
"demo = gr.Interface(\n",
|
| 1044 |
" fn=classify,\n",
|
| 1045 |
" inputs=[\n",
|
| 1046 |
-
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer
|
| 1047 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 1048 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 1049 |
" ],\n",
|
|
@@ -1062,7 +1052,9 @@
|
|
| 1062 |
"cell_type": "code",
|
| 1063 |
"execution_count": 32,
|
| 1064 |
"id": "1f5df949-a527-4b11-8e5e-23786e1cde12",
|
| 1065 |
-
"metadata": {
|
|
|
|
|
|
|
| 1066 |
"outputs": [
|
| 1067 |
{
|
| 1068 |
"name": "stdout",
|
|
@@ -1083,10 +1075,414 @@
|
|
| 1083 |
"print(first_trip)"
|
| 1084 |
]
|
| 1085 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1086 |
{
|
| 1087 |
"cell_type": "code",
|
| 1088 |
"execution_count": null,
|
| 1089 |
-
"id": "
|
| 1090 |
"metadata": {},
|
| 1091 |
"outputs": [],
|
| 1092 |
"source": []
|
|
|
|
| 18 |
},
|
| 19 |
{
|
| 20 |
"cell_type": "code",
|
| 21 |
+
"execution_count": 1,
|
| 22 |
"id": "f8c28d2d-8458-49fd-8ebf-5e729d6e861f",
|
| 23 |
"metadata": {},
|
| 24 |
"outputs": [],
|
|
|
|
| 59 |
"execution_count": 2,
|
| 60 |
"id": "89d42ca7-e871-4eda-b428-69e9bd965428",
|
| 61 |
"metadata": {
|
|
|
|
|
|
|
|
|
|
| 62 |
"scrolled": true
|
| 63 |
},
|
| 64 |
"outputs": [
|
|
|
|
| 173 |
"8 . I go on a skitouring trip where we also make videos/photos and the destination is Japan. Mainly sports clothes and isolation are needed (it is winter). We stay in a guesthouse. It will be 10 days. \n",
|
| 174 |
"\n",
|
| 175 |
"ski tour / skitour\n",
|
| 176 |
+
"['ski touring', 'photography']\n",
|
| 177 |
"cold destination / winter\n",
|
| 178 |
"minimalist\n",
|
| 179 |
"conservative\n",
|
| 180 |
"indoor\n",
|
| 181 |
"no own vehicle\n",
|
| 182 |
"avalanche-prone terrain\n",
|
| 183 |
+
"7+ days\n",
|
| 184 |
"\n",
|
| 185 |
"\n",
|
| 186 |
"9 . We plan a wild camping trip with activities such as snorkeling, kayaking and canoeing. It is a warm place and we want to bring little stuff. We stay in tents and hammocks and travel with a car, it will be 3 days. \n",
|
|
|
|
| 189 |
"['scuba diving', 'kayaking / canoeing']\n",
|
| 190 |
"tropical / humid\n",
|
| 191 |
"lightweight (but comfortable)\n",
|
| 192 |
+
"casual\n",
|
| 193 |
"sleeping in a tent\n",
|
| 194 |
"own vehicle\n",
|
| 195 |
+
"self-supported (bring your own cooking gear)\n",
|
| 196 |
"3 days\n",
|
| 197 |
"\n",
|
| 198 |
"\n"
|
|
|
|
| 946 |
},
|
| 947 |
{
|
| 948 |
"cell_type": "code",
|
| 949 |
+
"execution_count": 2,
|
| 950 |
"id": "5bf23e10-0a93-4b2f-9508-34bb0974d24c",
|
| 951 |
"metadata": {},
|
| 952 |
"outputs": [],
|
|
|
|
| 973 |
},
|
| 974 |
{
|
| 975 |
"cell_type": "code",
|
| 976 |
+
"execution_count": 3,
|
| 977 |
"id": "61ebbe99-2563-4c99-ba65-d2312c9d5844",
|
| 978 |
"metadata": {},
|
| 979 |
"outputs": [
|
|
|
|
| 981 |
"name": "stdout",
|
| 982 |
"output_type": "stream",
|
| 983 |
"text": [
|
| 984 |
+
"Running on local URL: http://127.0.0.1:7862\n",
|
| 985 |
"\n",
|
| 986 |
"To create a public link, set `share=True` in `launch()`.\n"
|
| 987 |
]
|
|
|
|
| 989 |
{
|
| 990 |
"data": {
|
| 991 |
"text/html": [
|
| 992 |
+
"<div><iframe src=\"http://127.0.0.1:7862/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 993 |
],
|
| 994 |
"text/plain": [
|
| 995 |
"<IPython.core.display.HTML object>"
|
|
|
|
| 997 |
},
|
| 998 |
"metadata": {},
|
| 999 |
"output_type": "display_data"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1000 |
}
|
| 1001 |
],
|
| 1002 |
"source": [
|
|
|
|
| 1033 |
"demo = gr.Interface(\n",
|
| 1034 |
" fn=classify,\n",
|
| 1035 |
" inputs=[\n",
|
| 1036 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 1037 |
" gr.Textbox(label=\"Trip description\"),\n",
|
| 1038 |
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 1039 |
" ],\n",
|
|
|
|
| 1052 |
"cell_type": "code",
|
| 1053 |
"execution_count": 32,
|
| 1054 |
"id": "1f5df949-a527-4b11-8e5e-23786e1cde12",
|
| 1055 |
+
"metadata": {
|
| 1056 |
+
"scrolled": true
|
| 1057 |
+
},
|
| 1058 |
"outputs": [
|
| 1059 |
{
|
| 1060 |
"name": "stdout",
|
|
|
|
| 1075 |
"print(first_trip)"
|
| 1076 |
]
|
| 1077 |
},
|
| 1078 |
+
{
|
| 1079 |
+
"cell_type": "markdown",
|
| 1080 |
+
"id": "4ba29d94-88e4-4fb9-b42b-4e013ec2faa0",
|
| 1081 |
+
"metadata": {},
|
| 1082 |
+
"source": [
|
| 1083 |
+
"**Check for duplicate entries, which to combine?**"
|
| 1084 |
+
]
|
| 1085 |
+
},
|
| 1086 |
+
{
|
| 1087 |
+
"cell_type": "code",
|
| 1088 |
+
"execution_count": 3,
|
| 1089 |
+
"id": "66311e68-c7ab-47a0-8d42-02991bc048f2",
|
| 1090 |
+
"metadata": {},
|
| 1091 |
+
"outputs": [
|
| 1092 |
+
{
|
| 1093 |
+
"name": "stdout",
|
| 1094 |
+
"output_type": "stream",
|
| 1095 |
+
"text": [
|
| 1096 |
+
"<class 'dict'>\n"
|
| 1097 |
+
]
|
| 1098 |
+
}
|
| 1099 |
+
],
|
| 1100 |
+
"source": [
|
| 1101 |
+
"print(type(packing_items))"
|
| 1102 |
+
]
|
| 1103 |
+
},
|
| 1104 |
+
{
|
| 1105 |
+
"cell_type": "code",
|
| 1106 |
+
"execution_count": 9,
|
| 1107 |
+
"id": "9b2a01e7-55ac-405a-bb34-2b759c1f2d8e",
|
| 1108 |
+
"metadata": {},
|
| 1109 |
+
"outputs": [
|
| 1110 |
+
{
|
| 1111 |
+
"name": "stdout",
|
| 1112 |
+
"output_type": "stream",
|
| 1113 |
+
"text": [
|
| 1114 |
+
"1 set of clothing for every situation\n",
|
| 1115 |
+
"GPS or offline maps\n",
|
| 1116 |
+
"Gore‑Tex clothing\n",
|
| 1117 |
+
"Gore‑Tex jacket and pants\n",
|
| 1118 |
+
"MiFi router or portable WiFi hotspot\n",
|
| 1119 |
+
"SUP board and paddle\n",
|
| 1120 |
+
"USB hub (for multiple devices)\n",
|
| 1121 |
+
"WiFi hotspot or local SIM card\n",
|
| 1122 |
+
"accessories\n",
|
| 1123 |
+
"activity book or tablet with films\n",
|
| 1124 |
+
"airbag backpack (if available)\n",
|
| 1125 |
+
"all‑in‑one soap\n",
|
| 1126 |
+
"at least 2 liters of water storage per person\n",
|
| 1127 |
+
"avalanche beacon (transceiver)\n",
|
| 1128 |
+
"baby monitor (for staying at location)\n",
|
| 1129 |
+
"backpack\n",
|
| 1130 |
+
"backup lighting (e.g. small flashlight)\n",
|
| 1131 |
+
"bags for waste\n",
|
| 1132 |
+
"bait / lures\n",
|
| 1133 |
+
"bank card / cash\n",
|
| 1134 |
+
"beach bag\n",
|
| 1135 |
+
"beach chair\n",
|
| 1136 |
+
"beach towel\n",
|
| 1137 |
+
"belay device\n",
|
| 1138 |
+
"bike light and lock\n",
|
| 1139 |
+
"bike or rental bike\n",
|
| 1140 |
+
"biodegradable soap + sponge\n",
|
| 1141 |
+
"bivvy bag or tarp\n",
|
| 1142 |
+
"blister plasters or tape\n",
|
| 1143 |
+
"board leash\n",
|
| 1144 |
+
"book / meditation material\n",
|
| 1145 |
+
"book or e‑reader\n",
|
| 1146 |
+
"boots or waders\n",
|
| 1147 |
+
"bottles and food (if applicable)\n",
|
| 1148 |
+
"breathable thermal clothing\n",
|
| 1149 |
+
"buff or neck warmer\n",
|
| 1150 |
+
"business cards / documents\n",
|
| 1151 |
+
"camera + lenses\n",
|
| 1152 |
+
"camera or smartphone\n",
|
| 1153 |
+
"camping gear (if staying overnight)\n",
|
| 1154 |
+
"camping table (optional)\n",
|
| 1155 |
+
"cap or hat\n",
|
| 1156 |
+
"car documents\n",
|
| 1157 |
+
"cash / card\n",
|
| 1158 |
+
"cash for hut\n",
|
| 1159 |
+
"cash for payments\n",
|
| 1160 |
+
"chair\n",
|
| 1161 |
+
"chair and table\n",
|
| 1162 |
+
"chalk bag\n",
|
| 1163 |
+
"charger\n",
|
| 1164 |
+
"child carrier or stroller\n",
|
| 1165 |
+
"child sleeping bag or pad\n",
|
| 1166 |
+
"children’s sunscreen\n",
|
| 1167 |
+
"children’s travel pharmacy\n",
|
| 1168 |
+
"chlorine drops or purification tablets\n",
|
| 1169 |
+
"city map / offline maps\n",
|
| 1170 |
+
"climbing harness\n",
|
| 1171 |
+
"climbing rope\n",
|
| 1172 |
+
"climbing shoes\n",
|
| 1173 |
+
"climbing skins\n",
|
| 1174 |
+
"closed shoes\n",
|
| 1175 |
+
"comfortable backpack or trolley\n",
|
| 1176 |
+
"comfortable clothing\n",
|
| 1177 |
+
"comfortable shoes\n",
|
| 1178 |
+
"comfortable sleeping pad\n",
|
| 1179 |
+
"compact clothing pack\n",
|
| 1180 |
+
"compact rain jacket\n",
|
| 1181 |
+
"compact sleeping gear (if overnighting)\n",
|
| 1182 |
+
"compact toothbrush\n",
|
| 1183 |
+
"cookset + stove\n",
|
| 1184 |
+
"cooler\n",
|
| 1185 |
+
"cooler box\n",
|
| 1186 |
+
"cooler box (optional)\n",
|
| 1187 |
+
"covering clothing\n",
|
| 1188 |
+
"crampons\n",
|
| 1189 |
+
"cross-country ski boots\n",
|
| 1190 |
+
"cross-country skis and poles\n",
|
| 1191 |
+
"daypack\n",
|
| 1192 |
+
"diapers or potty (depending on age)\n",
|
| 1193 |
+
"dishes & cutlery\n",
|
| 1194 |
+
"dive computer\n",
|
| 1195 |
+
"dog leash or harness\n",
|
| 1196 |
+
"down jacket or warm insulation layer\n",
|
| 1197 |
+
"dress or shirt\n",
|
| 1198 |
+
"dress shoes\n",
|
| 1199 |
+
"dried or freeze‑dried meals\n",
|
| 1200 |
+
"driver’s license\n",
|
| 1201 |
+
"dry bag\n",
|
| 1202 |
+
"earplugs\n",
|
| 1203 |
+
"emergency communication (e.g. GPS beacon or satellite messenger)\n",
|
| 1204 |
+
"energy bars or sports nutrition\n",
|
| 1205 |
+
"entertainment (book, music, games)\n",
|
| 1206 |
+
"essential oils (optional)\n",
|
| 1207 |
+
"extension cord (for powered campsites)\n",
|
| 1208 |
+
"extra batteries\n",
|
| 1209 |
+
"extra charger cables\n",
|
| 1210 |
+
"extra clothing\n",
|
| 1211 |
+
"extra clothing layer\n",
|
| 1212 |
+
"extra clothing or gear if needed\n",
|
| 1213 |
+
"extra clothing set per day\n",
|
| 1214 |
+
"extra food\n",
|
| 1215 |
+
"extra snacks for children\n",
|
| 1216 |
+
"favorite toy or stuffed animal\n",
|
| 1217 |
+
"fins\n",
|
| 1218 |
+
"first aid kit\n",
|
| 1219 |
+
"fishing license\n",
|
| 1220 |
+
"fishing rod\n",
|
| 1221 |
+
"flashlight or headlamp\n",
|
| 1222 |
+
"flip flops\n",
|
| 1223 |
+
"foldable cutting board (optional)\n",
|
| 1224 |
+
"foldable solar panel (if on longer trips)\n",
|
| 1225 |
+
"food and snacks\n",
|
| 1226 |
+
"food and water bowl\n",
|
| 1227 |
+
"food bag or hanging bag (wild-safe)\n",
|
| 1228 |
+
"food supply\n",
|
| 1229 |
+
"friend meetups\n",
|
| 1230 |
+
"fuel (enough for several days)\n",
|
| 1231 |
+
"gaiters (in deep snow)\n",
|
| 1232 |
+
"general items for this situation\n",
|
| 1233 |
+
"glitter / outfit\n",
|
| 1234 |
+
"gloves\n",
|
| 1235 |
+
"gloves (2 pairs)\n",
|
| 1236 |
+
"groceries\n",
|
| 1237 |
+
"groundsheet\n",
|
| 1238 |
+
"guidebook or highlights list\n",
|
| 1239 |
+
"hat and gloves\n",
|
| 1240 |
+
"hat or cap\n",
|
| 1241 |
+
"hat or headband\n",
|
| 1242 |
+
"head cover\n",
|
| 1243 |
+
"head protection\n",
|
| 1244 |
+
"headband or cap\n",
|
| 1245 |
+
"headlamp\n",
|
| 1246 |
+
"headlamp + extra batteries\n",
|
| 1247 |
+
"headlamp or flashlight\n",
|
| 1248 |
+
"helmet\n",
|
| 1249 |
+
"hiking boots\n",
|
| 1250 |
+
"hiking boots or trail runners\n",
|
| 1251 |
+
"hiking poles\n",
|
| 1252 |
+
"hiking socks (anti-blister)\n",
|
| 1253 |
+
"hut slippers / Crocs\n",
|
| 1254 |
+
"hydrating cream (for sensitive skin)\n",
|
| 1255 |
+
"ice axes\n",
|
| 1256 |
+
"identity document or passport\n",
|
| 1257 |
+
"indoor hut clothing (thermo / fleece)\n",
|
| 1258 |
+
"insect repellent\n",
|
| 1259 |
+
"insurance card / travel insurance info\n",
|
| 1260 |
+
"jeans or comfortable pants\n",
|
| 1261 |
+
"journal / pen\n",
|
| 1262 |
+
"kayak or canoe\n",
|
| 1263 |
+
"kids first aid kit (including thermometer and bandages)\n",
|
| 1264 |
+
"knife or multitool\n",
|
| 1265 |
+
"knowledge of avalanche safety / course\n",
|
| 1266 |
+
"lamp or lantern\n",
|
| 1267 |
+
"laptop and charger\n",
|
| 1268 |
+
"layered clothing\n",
|
| 1269 |
+
"layers for temperature control\n",
|
| 1270 |
+
"layers of clothing\n",
|
| 1271 |
+
"lens cloth\n",
|
| 1272 |
+
"life jacket\n",
|
| 1273 |
+
"light backpack with water and snacks\n",
|
| 1274 |
+
"light clothing\n",
|
| 1275 |
+
"light down jacket or warm layer\n",
|
| 1276 |
+
"light gloves for climbing\n",
|
| 1277 |
+
"light jacket or raincoat\n",
|
| 1278 |
+
"light long sleeves\n",
|
| 1279 |
+
"light pajamas or sleepwear\n",
|
| 1280 |
+
"light shoes\n",
|
| 1281 |
+
"light tent or tarp\n",
|
| 1282 |
+
"light towel\n",
|
| 1283 |
+
"lighter\n",
|
| 1284 |
+
"lighter + matches (waterproof packed)\n",
|
| 1285 |
+
"lightweight backpack\n",
|
| 1286 |
+
"lightweight backpack (< 1kg)\n",
|
| 1287 |
+
"lightweight clothing\n",
|
| 1288 |
+
"lightweight cookset\n",
|
| 1289 |
+
"lightweight sleeping pad\n",
|
| 1290 |
+
"lightweight stove (gas, petrol or alcohol)\n",
|
| 1291 |
+
"lightweight towel\n",
|
| 1292 |
+
"lightweight trekking backpack (30–45 liters)\n",
|
| 1293 |
+
"limited clothing (layers!)\n",
|
| 1294 |
+
"lip balm\n",
|
| 1295 |
+
"long pants or skirt\n",
|
| 1296 |
+
"lots of water\n",
|
| 1297 |
+
"map and compass\n",
|
| 1298 |
+
"map and compass / GPS\n",
|
| 1299 |
+
"map or GPS\n",
|
| 1300 |
+
"map or offline maps\n",
|
| 1301 |
+
"mask and snorkel\n",
|
| 1302 |
+
"memory card(s)\n",
|
| 1303 |
+
"minimalist shelter (tarp or tent)\n",
|
| 1304 |
+
"music / headphones\n",
|
| 1305 |
+
"navigation\n",
|
| 1306 |
+
"navigation (map/compass/GPS)\n",
|
| 1307 |
+
"navigation device with offline maps\n",
|
| 1308 |
+
"navigation or smartphone\n",
|
| 1309 |
+
"noise‑cancelling headphones\n",
|
| 1310 |
+
"notebook + pen\n",
|
| 1311 |
+
"number of meals/snacks matched to duration\n",
|
| 1312 |
+
"optional own saddle or stirrups\n",
|
| 1313 |
+
"pacifier or dummy\n",
|
| 1314 |
+
"packaging to keep electronics dry\n",
|
| 1315 |
+
"pad and sleeping bag\n",
|
| 1316 |
+
"paddle\n",
|
| 1317 |
+
"pajamas\n",
|
| 1318 |
+
"pan or small pot\n",
|
| 1319 |
+
"paper map and compass\n",
|
| 1320 |
+
"paraglider\n",
|
| 1321 |
+
"partner check before departure\n",
|
| 1322 |
+
"payment methods (debit card / cash)\n",
|
| 1323 |
+
"perfume / deodorant\n",
|
| 1324 |
+
"phone + charger\n",
|
| 1325 |
+
"phone charger\n",
|
| 1326 |
+
"phone holder\n",
|
| 1327 |
+
"phone holder / navigation\n",
|
| 1328 |
+
"pillow or inflatable pillow\n",
|
| 1329 |
+
"poncho or rain jacket\n",
|
| 1330 |
+
"poncho or towel\n",
|
| 1331 |
+
"poop bags\n",
|
| 1332 |
+
"power bank\n",
|
| 1333 |
+
"power bank (at least 10,000 mAh)\n",
|
| 1334 |
+
"power bank or 12V charger\n",
|
| 1335 |
+
"press‑on bowl or mug\n",
|
| 1336 |
+
"probe\n",
|
| 1337 |
+
"probe and shovel\n",
|
| 1338 |
+
"public transport app or ticket\n",
|
| 1339 |
+
"quick snacks for en route\n",
|
| 1340 |
+
"quick‑dry base layers\n",
|
| 1341 |
+
"quick‑dry clothing\n",
|
| 1342 |
+
"quick‑dry towel\n",
|
| 1343 |
+
"quilt or down blanket\n",
|
| 1344 |
+
"rain cover for stroller or carrier\n",
|
| 1345 |
+
"rain jacket\n",
|
| 1346 |
+
"rain jacket or poncho\n",
|
| 1347 |
+
"rain jacket or windbreaker\n",
|
| 1348 |
+
"rain poncho\n",
|
| 1349 |
+
"rain protection\n",
|
| 1350 |
+
"rechargeable batteries and charger\n",
|
| 1351 |
+
"regulator (if own)\n",
|
| 1352 |
+
"repair kit\n",
|
| 1353 |
+
"reservation confirmation\n",
|
| 1354 |
+
"reusable bag\n",
|
| 1355 |
+
"reusable cup\n",
|
| 1356 |
+
"riding boots or shoes with heel\n",
|
| 1357 |
+
"riding pants\n",
|
| 1358 |
+
"rubber shoes\n",
|
| 1359 |
+
"running shoes\n",
|
| 1360 |
+
"sandals\n",
|
| 1361 |
+
"scarf or shawl\n",
|
| 1362 |
+
"seat cushion or beach mat\n",
|
| 1363 |
+
"sheet liner\n",
|
| 1364 |
+
"sheet liner (often required)\n",
|
| 1365 |
+
"shirt / blouse\n",
|
| 1366 |
+
"shovel\n",
|
| 1367 |
+
"ski boots\n",
|
| 1368 |
+
"ski goggles\n",
|
| 1369 |
+
"ski or sunglasses\n",
|
| 1370 |
+
"ski pass\n",
|
| 1371 |
+
"skis and poles\n",
|
| 1372 |
+
"sleep mask\n",
|
| 1373 |
+
"sleeping bag\n",
|
| 1374 |
+
"sleeping bag (light, warm variant)\n",
|
| 1375 |
+
"sleeping bag (suitable for temperature)\n",
|
| 1376 |
+
"sleeping pad\n",
|
| 1377 |
+
"sleeping pad that fits in car\n",
|
| 1378 |
+
"slippers\n",
|
| 1379 |
+
"slippers or indoor shoes for inside\n",
|
| 1380 |
+
"small backpack\n",
|
| 1381 |
+
"small toiletry bag\n",
|
| 1382 |
+
"smart jacket\n",
|
| 1383 |
+
"snacks\n",
|
| 1384 |
+
"snacks / emergency bars\n",
|
| 1385 |
+
"snacks / energy bars\n",
|
| 1386 |
+
"snacks and drinks\n",
|
| 1387 |
+
"snacks and toys\n",
|
| 1388 |
+
"snacks for along the way\n",
|
| 1389 |
+
"snacks for the night\n",
|
| 1390 |
+
"sneakers\n",
|
| 1391 |
+
"snorkel and mask\n",
|
| 1392 |
+
"snow goggles\n",
|
| 1393 |
+
"socks\n",
|
| 1394 |
+
"socks per day\n",
|
| 1395 |
+
"solar panel or portable charging system\n",
|
| 1396 |
+
"splitboard or snowboard\n",
|
| 1397 |
+
"spork or spoon\n",
|
| 1398 |
+
"sports clothing\n",
|
| 1399 |
+
"sports watch (optional)\n",
|
| 1400 |
+
"sun hat\n",
|
| 1401 |
+
"sun hat or cap\n",
|
| 1402 |
+
"sun protection\n",
|
| 1403 |
+
"sunglasses\n",
|
| 1404 |
+
"sunglasses or sport glasses\n",
|
| 1405 |
+
"sunglasses with strap\n",
|
| 1406 |
+
"sunscreen\n",
|
| 1407 |
+
"sunscreen and sunglasses\n",
|
| 1408 |
+
"sunshades or blackout covers\n",
|
| 1409 |
+
"surfboard\n",
|
| 1410 |
+
"sweater or hoodie\n",
|
| 1411 |
+
"swimming goggles\n",
|
| 1412 |
+
"swimwear\n",
|
| 1413 |
+
"t-shirts\n",
|
| 1414 |
+
"tent\n",
|
| 1415 |
+
"tent (1‑ or 2‑person, depending on trip)\n",
|
| 1416 |
+
"tent or tarp\n",
|
| 1417 |
+
"thermal blanket (for cold nights)\n",
|
| 1418 |
+
"thermal clothing\n",
|
| 1419 |
+
"thermos bottle\n",
|
| 1420 |
+
"thick gloves\n",
|
| 1421 |
+
"thin gloves\n",
|
| 1422 |
+
"tissues or toilet paper\n",
|
| 1423 |
+
"titanium cookset\n",
|
| 1424 |
+
"toiletry bag\n",
|
| 1425 |
+
"toiletry bag (toothpaste, brush, deodorant, soap)\n",
|
| 1426 |
+
"toiletry bag with biodegradable soap\n",
|
| 1427 |
+
"toiletry bag with essentials\n",
|
| 1428 |
+
"toothbrush (shortened ;))\n",
|
| 1429 |
+
"tour bindings (for splitboard)\n",
|
| 1430 |
+
"touring backpack with ski attachment\n",
|
| 1431 |
+
"touring skis or splitboard\n",
|
| 1432 |
+
"towel\n",
|
| 1433 |
+
"traction soles / spikes\n",
|
| 1434 |
+
"trail runners or lightweight hiking shoes\n",
|
| 1435 |
+
"travel chair or sling\n",
|
| 1436 |
+
"travel crib or mattress (for young children)\n",
|
| 1437 |
+
"travel guide or maps\n",
|
| 1438 |
+
"travel mat or blanket\n",
|
| 1439 |
+
"trekking poles\n",
|
| 1440 |
+
"tripod\n",
|
| 1441 |
+
"underwear per day\n",
|
| 1442 |
+
"vaccination booklet\n",
|
| 1443 |
+
"warm boots\n",
|
| 1444 |
+
"warm insulation layers\n",
|
| 1445 |
+
"warm jacket\n",
|
| 1446 |
+
"warm layer\n",
|
| 1447 |
+
"warm sleeping bag\n",
|
| 1448 |
+
"warm sweater\n",
|
| 1449 |
+
"warm sweater or scarf\n",
|
| 1450 |
+
"washing up supplies\n",
|
| 1451 |
+
"water bottle\n",
|
| 1452 |
+
"water bottle or belt\n",
|
| 1453 |
+
"water bottle within reach\n",
|
| 1454 |
+
"water bottle(s) or hydration bladder\n",
|
| 1455 |
+
"water filter or pump\n",
|
| 1456 |
+
"water shoes\n",
|
| 1457 |
+
"waterproof backpack cover\n",
|
| 1458 |
+
"waterproof bag\n",
|
| 1459 |
+
"waterproof shoes\n",
|
| 1460 |
+
"wax\n",
|
| 1461 |
+
"wet wipes\n",
|
| 1462 |
+
"wetsuit\n",
|
| 1463 |
+
"wind jacket\n",
|
| 1464 |
+
"windproof and water-repellent outer layer\n",
|
| 1465 |
+
"wind‑ and waterproof jacket\n",
|
| 1466 |
+
"world adapter plug\n",
|
| 1467 |
+
"yoga mat or yoga towel\n"
|
| 1468 |
+
]
|
| 1469 |
+
}
|
| 1470 |
+
],
|
| 1471 |
+
"source": [
|
| 1472 |
+
"# Load packing item data\n",
|
| 1473 |
+
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 1474 |
+
" packing_items = json.load(file)\n",
|
| 1475 |
+
"\n",
|
| 1476 |
+
"unique_sorted = sorted({item for values in packing_items.values() for item in values})\n",
|
| 1477 |
+
"\n",
|
| 1478 |
+
"for item in unique_sorted:\n",
|
| 1479 |
+
" print(item)\n"
|
| 1480 |
+
]
|
| 1481 |
+
},
|
| 1482 |
{
|
| 1483 |
"cell_type": "code",
|
| 1484 |
"execution_count": null,
|
| 1485 |
+
"id": "e300e3f3-93e0-457b-b2f0-e05cc5c2cafb",
|
| 1486 |
"metadata": {},
|
| 1487 |
"outputs": [],
|
| 1488 |
"source": []
|
space/space/space/space/space/space/space/space/space/space/space/space/space/requirements.txt
CHANGED
|
@@ -1,8 +1,5 @@
|
|
| 1 |
transformers
|
| 2 |
-
json
|
| 3 |
torch
|
| 4 |
numpy
|
| 5 |
-
tabulate
|
| 6 |
pandas
|
| 7 |
-
sklearn
|
| 8 |
gradio
|
|
|
|
| 1 |
transformers
|
|
|
|
| 2 |
torch
|
| 3 |
numpy
|
|
|
|
| 4 |
pandas
|
|
|
|
| 5 |
gradio
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/code_blog_post-checkpoint.ipynb
ADDED
|
@@ -0,0 +1,398 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cells": [
|
| 3 |
+
{
|
| 4 |
+
"cell_type": "markdown",
|
| 5 |
+
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
+
"metadata": {},
|
| 7 |
+
"source": [
|
| 8 |
+
"Implementation for blog post"
|
| 9 |
+
]
|
| 10 |
+
},
|
| 11 |
+
{
|
| 12 |
+
"cell_type": "code",
|
| 13 |
+
"execution_count": 14,
|
| 14 |
+
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
+
"metadata": {},
|
| 16 |
+
"outputs": [],
|
| 17 |
+
"source": [
|
| 18 |
+
"import math\n",
|
| 19 |
+
"import json\n",
|
| 20 |
+
"import pickle\n",
|
| 21 |
+
"import os\n",
|
| 22 |
+
"import time\n",
|
| 23 |
+
"import pandas as pd\n",
|
| 24 |
+
"import matplotlib.pyplot as plt\n",
|
| 25 |
+
"from tabulate import tabulate\n",
|
| 26 |
+
"from transformers import pipeline\n",
|
| 27 |
+
"\n",
|
| 28 |
+
"# Get candidate labels\n",
|
| 29 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 30 |
+
" candidate_labels = json.load(file)\n",
|
| 31 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 32 |
+
"\n",
|
| 33 |
+
"# Load test data (list of dictionaries)\n",
|
| 34 |
+
"# with open(\"test_data.json\", \"r\") as file:\n",
|
| 35 |
+
"# packing_data = json.load(file)\n",
|
| 36 |
+
"# Extract trip descriptions and classification (trip_types)\n",
|
| 37 |
+
"# trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 38 |
+
"# trip_types = [trip['trip_types'] for trip in packing_data]\n"
|
| 39 |
+
]
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"cell_type": "code",
|
| 43 |
+
"execution_count": 19,
|
| 44 |
+
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 45 |
+
"metadata": {},
|
| 46 |
+
"outputs": [
|
| 47 |
+
{
|
| 48 |
+
"name": "stdout",
|
| 49 |
+
"output_type": "stream",
|
| 50 |
+
"text": [
|
| 51 |
+
"\n",
|
| 52 |
+
" activity_type :\n",
|
| 53 |
+
"\t hut trek (summer)\n",
|
| 54 |
+
"\t hut trek (winter)\n",
|
| 55 |
+
"\t camping trip (wild camping)\n",
|
| 56 |
+
"\t camping trip (campground)\n",
|
| 57 |
+
"\t ski tour / skitour\n",
|
| 58 |
+
"\t snowboard / splitboard trip\n",
|
| 59 |
+
"\t long-distance hike / thru-hike\n",
|
| 60 |
+
"\t digital nomad trip\n",
|
| 61 |
+
"\t city trip\n",
|
| 62 |
+
"\t road trip (car/camper)\n",
|
| 63 |
+
"\t festival trip\n",
|
| 64 |
+
"\t yoga / wellness retreat\n",
|
| 65 |
+
"\t micro-adventure / weekend trip\n",
|
| 66 |
+
"\t beach vacation\n",
|
| 67 |
+
"\t cultural exploration\n",
|
| 68 |
+
"\t nature escape\n",
|
| 69 |
+
"\n",
|
| 70 |
+
" activities :\n",
|
| 71 |
+
"\t swimming\n",
|
| 72 |
+
"\t going to the beach\n",
|
| 73 |
+
"\t relaxing\n",
|
| 74 |
+
"\t sightseeing\n",
|
| 75 |
+
"\t biking\n",
|
| 76 |
+
"\t running\n",
|
| 77 |
+
"\t skiing\n",
|
| 78 |
+
"\t cross-country skiing\n",
|
| 79 |
+
"\t ski touring\n",
|
| 80 |
+
"\t hiking\n",
|
| 81 |
+
"\t hut-to-hut hiking\n",
|
| 82 |
+
"\t rock climbing\n",
|
| 83 |
+
"\t ice climbing\n",
|
| 84 |
+
"\t snowshoe hiking\n",
|
| 85 |
+
"\t kayaking / canoeing\n",
|
| 86 |
+
"\t stand-up paddleboarding (SUP)\n",
|
| 87 |
+
"\t snorkeling\n",
|
| 88 |
+
"\t scuba diving\n",
|
| 89 |
+
"\t surfing\n",
|
| 90 |
+
"\t paragliding\n",
|
| 91 |
+
"\t horseback riding\n",
|
| 92 |
+
"\t photography\n",
|
| 93 |
+
"\t fishing\n",
|
| 94 |
+
"\t rafting\n",
|
| 95 |
+
"\t yoga\n",
|
| 96 |
+
"\n",
|
| 97 |
+
" climate_or_season :\n",
|
| 98 |
+
"\t cold destination / winter\n",
|
| 99 |
+
"\t warm destination / summer\n",
|
| 100 |
+
"\t variable weather / spring / autumn\n",
|
| 101 |
+
"\t tropical / humid\n",
|
| 102 |
+
"\t dry / desert-like\n",
|
| 103 |
+
"\t rainy climate\n",
|
| 104 |
+
"\n",
|
| 105 |
+
" style_or_comfort :\n",
|
| 106 |
+
"\t ultralight\n",
|
| 107 |
+
"\t lightweight (but comfortable)\n",
|
| 108 |
+
"\t luxury (including evening wear)\n",
|
| 109 |
+
"\t minimalist\n",
|
| 110 |
+
"\n",
|
| 111 |
+
" dress_code :\n",
|
| 112 |
+
"\t casual\n",
|
| 113 |
+
"\t formal (business trip)\n",
|
| 114 |
+
"\t conservative\n",
|
| 115 |
+
"\n",
|
| 116 |
+
" accommodation :\n",
|
| 117 |
+
"\t indoor\n",
|
| 118 |
+
"\t huts with half board\n",
|
| 119 |
+
"\t sleeping in a tent\n",
|
| 120 |
+
"\t sleeping in a car\n",
|
| 121 |
+
"\n",
|
| 122 |
+
" transportation :\n",
|
| 123 |
+
"\t own vehicle\n",
|
| 124 |
+
"\t no own vehicle\n",
|
| 125 |
+
"\n",
|
| 126 |
+
" special_conditions :\n",
|
| 127 |
+
"\t off-grid / no electricity\n",
|
| 128 |
+
"\t self-supported (bring your own cooking gear)\n",
|
| 129 |
+
"\t travel with children\n",
|
| 130 |
+
"\t pet-friendly\n",
|
| 131 |
+
"\t snow and ice\n",
|
| 132 |
+
"\t high alpine terrain\n",
|
| 133 |
+
"\t snow, ice and avalanche-prone terrain\n",
|
| 134 |
+
"\t no special conditions to consider\n",
|
| 135 |
+
"\n",
|
| 136 |
+
" trip_length_days :\n",
|
| 137 |
+
"\t 1 day\n",
|
| 138 |
+
"\t 2 days\n",
|
| 139 |
+
"\t 3 days\n",
|
| 140 |
+
"\t 4 days\n",
|
| 141 |
+
"\t 5 days\n",
|
| 142 |
+
"\t 6 days\n",
|
| 143 |
+
"\t 7 days\n",
|
| 144 |
+
"\t 7+ days\n"
|
| 145 |
+
]
|
| 146 |
+
}
|
| 147 |
+
],
|
| 148 |
+
"source": [
|
| 149 |
+
"for key in candidate_labels:\n",
|
| 150 |
+
" print(\"\\n\", key, \":\")\n",
|
| 151 |
+
" for item in candidate_labels[key]:\n",
|
| 152 |
+
" print(\"\\t\", item)\n"
|
| 153 |
+
]
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"cell_type": "code",
|
| 157 |
+
"execution_count": 20,
|
| 158 |
+
"id": "e114d7ab-9bd3-4214-a630-7f9166c3b8a5",
|
| 159 |
+
"metadata": {},
|
| 160 |
+
"outputs": [
|
| 161 |
+
{
|
| 162 |
+
"name": "stdout",
|
| 163 |
+
"output_type": "stream",
|
| 164 |
+
"text": [
|
| 165 |
+
"activity_type\n"
|
| 166 |
+
]
|
| 167 |
+
}
|
| 168 |
+
],
|
| 169 |
+
"source": [
|
| 170 |
+
"# First classify using first dimension of classes ()\n",
|
| 171 |
+
"print(keys_list[0])"
|
| 172 |
+
]
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"cell_type": "code",
|
| 176 |
+
"execution_count": 23,
|
| 177 |
+
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 178 |
+
"metadata": {},
|
| 179 |
+
"outputs": [
|
| 180 |
+
{
|
| 181 |
+
"name": "stderr",
|
| 182 |
+
"output_type": "stream",
|
| 183 |
+
"text": [
|
| 184 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 185 |
+
]
|
| 186 |
+
},
|
| 187 |
+
{
|
| 188 |
+
"name": "stdout",
|
| 189 |
+
"output_type": "stream",
|
| 190 |
+
"text": [
|
| 191 |
+
" Label Score\n",
|
| 192 |
+
"0 beach vacation 0.376311\n",
|
| 193 |
+
"1 micro-adventure / weekend trip 0.350168\n",
|
| 194 |
+
"2 nature escape 0.133974\n",
|
| 195 |
+
"3 digital nomad trip 0.031636\n",
|
| 196 |
+
"4 cultural exploration 0.031271\n",
|
| 197 |
+
"5 yoga / wellness retreat 0.012846\n",
|
| 198 |
+
"6 festival trip 0.012700\n",
|
| 199 |
+
"7 long-distance hike / thru-hike 0.009527\n",
|
| 200 |
+
"8 hut trek (summer) 0.008148\n",
|
| 201 |
+
"9 city trip 0.007793\n",
|
| 202 |
+
"10 road trip (car/camper) 0.006512\n",
|
| 203 |
+
"11 ski tour / skitour 0.005670\n",
|
| 204 |
+
"12 camping trip (campground) 0.004448\n",
|
| 205 |
+
"13 snowboard / splitboard trip 0.004113\n",
|
| 206 |
+
"14 camping trip (wild camping) 0.002714\n",
|
| 207 |
+
"15 hut trek (winter) 0.002170\n"
|
| 208 |
+
]
|
| 209 |
+
}
|
| 210 |
+
],
|
| 211 |
+
"source": [
|
| 212 |
+
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 213 |
+
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 214 |
+
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 215 |
+
"result = classifier(trip_descr, candidate_labels[keys_list[0]])\n",
|
| 216 |
+
"# Create DataFrame\n",
|
| 217 |
+
"df = pd.DataFrame({\n",
|
| 218 |
+
" \"Label\": result[\"labels\"],\n",
|
| 219 |
+
" \"Score\": result[\"scores\"]\n",
|
| 220 |
+
"})\n",
|
| 221 |
+
"print(df)"
|
| 222 |
+
]
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"cell_type": "code",
|
| 226 |
+
"execution_count": 25,
|
| 227 |
+
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 228 |
+
"metadata": {},
|
| 229 |
+
"outputs": [
|
| 230 |
+
{
|
| 231 |
+
"name": "stdout",
|
| 232 |
+
"output_type": "stream",
|
| 233 |
+
"text": [
|
| 234 |
+
"beach vacation\n"
|
| 235 |
+
]
|
| 236 |
+
}
|
| 237 |
+
],
|
| 238 |
+
"source": [
|
| 239 |
+
"# the labels are sorted by score. We choose the first one as our best guess for a class label\n",
|
| 240 |
+
"class_label = result[\"labels\"][0]\n",
|
| 241 |
+
"print(class_label)"
|
| 242 |
+
]
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"cell_type": "code",
|
| 246 |
+
"execution_count": 30,
|
| 247 |
+
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 248 |
+
"metadata": {},
|
| 249 |
+
"outputs": [
|
| 250 |
+
{
|
| 251 |
+
"name": "stdout",
|
| 252 |
+
"output_type": "stream",
|
| 253 |
+
"text": [
|
| 254 |
+
" Label Score\n",
|
| 255 |
+
"0 beach vacation 0.376311\n",
|
| 256 |
+
"1 micro-adventure / weekend trip 0.350168\n",
|
| 257 |
+
"2 nature escape 0.133974\n",
|
| 258 |
+
"3 digital nomad trip 0.031636\n",
|
| 259 |
+
"4 cultural exploration 0.031271\n",
|
| 260 |
+
"5 yoga / wellness retreat 0.012846\n",
|
| 261 |
+
"6 festival trip 0.012700\n",
|
| 262 |
+
"7 long-distance hike / thru-hike 0.009527\n",
|
| 263 |
+
"8 hut trek (summer) 0.008148\n",
|
| 264 |
+
"9 city trip 0.007793\n",
|
| 265 |
+
"10 road trip (car/camper) 0.006512\n",
|
| 266 |
+
"11 ski tour / skitour 0.005670\n",
|
| 267 |
+
"12 camping trip (campground) 0.004448\n",
|
| 268 |
+
"13 snowboard / splitboard trip 0.004113\n",
|
| 269 |
+
"14 camping trip (wild camping) 0.002714\n",
|
| 270 |
+
"15 hut trek (winter) 0.002170\n"
|
| 271 |
+
]
|
| 272 |
+
}
|
| 273 |
+
],
|
| 274 |
+
"source": [
|
| 275 |
+
"# we do this for each superclass and receive a list of class labels for our trip. We did do things differently for activities\n",
|
| 276 |
+
"cut_off = 0.5\n",
|
| 277 |
+
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 278 |
+
"indices = [i for i, score in enumerate(result_activ['scores']) if score > cut_off]\n",
|
| 279 |
+
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 280 |
+
"\n",
|
| 281 |
+
"df = pd.DataFrame({\n",
|
| 282 |
+
" \"Label\": result[\"labels\"],\n",
|
| 283 |
+
" \"Score\": result[\"scores\"]\n",
|
| 284 |
+
"})\n",
|
| 285 |
+
"print(df)"
|
| 286 |
+
]
|
| 287 |
+
},
|
| 288 |
+
{
|
| 289 |
+
"cell_type": "code",
|
| 290 |
+
"execution_count": 31,
|
| 291 |
+
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 292 |
+
"metadata": {},
|
| 293 |
+
"outputs": [
|
| 294 |
+
{
|
| 295 |
+
"name": "stdout",
|
| 296 |
+
"output_type": "stream",
|
| 297 |
+
"text": [
|
| 298 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 299 |
+
]
|
| 300 |
+
}
|
| 301 |
+
],
|
| 302 |
+
"source": [
|
| 303 |
+
"print(classes)"
|
| 304 |
+
]
|
| 305 |
+
},
|
| 306 |
+
{
|
| 307 |
+
"cell_type": "code",
|
| 308 |
+
"execution_count": 34,
|
| 309 |
+
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 310 |
+
"metadata": {},
|
| 311 |
+
"outputs": [],
|
| 312 |
+
"source": [
|
| 313 |
+
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 314 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 315 |
+
" \"\"\"\n",
|
| 316 |
+
" Classifies trip\n",
|
| 317 |
+
" \n",
|
| 318 |
+
" Parameters:\n",
|
| 319 |
+
" model_name: name of hugging-face model\n",
|
| 320 |
+
" trip_descr: text describing the trip\n",
|
| 321 |
+
" cut_off: cut_off for choosing activities\n",
|
| 322 |
+
"\n",
|
| 323 |
+
" Returns:\n",
|
| 324 |
+
" pd Dataframe: with class predictions and true values\n",
|
| 325 |
+
" \"\"\"\n",
|
| 326 |
+
" \n",
|
| 327 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 328 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 329 |
+
" for i, key in enumerate(keys_list):\n",
|
| 330 |
+
" print(f\"\\rProcessing {i}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 331 |
+
" if key == 'activities':\n",
|
| 332 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 333 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 334 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 335 |
+
" else:\n",
|
| 336 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 337 |
+
" classes = result[\"labels\"][0]\n",
|
| 338 |
+
" df.loc[i] = [key, classes]\n",
|
| 339 |
+
" return df"
|
| 340 |
+
]
|
| 341 |
+
},
|
| 342 |
+
{
|
| 343 |
+
"cell_type": "code",
|
| 344 |
+
"execution_count": null,
|
| 345 |
+
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 346 |
+
"metadata": {},
|
| 347 |
+
"outputs": [
|
| 348 |
+
{
|
| 349 |
+
"name": "stderr",
|
| 350 |
+
"output_type": "stream",
|
| 351 |
+
"text": [
|
| 352 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 353 |
+
]
|
| 354 |
+
},
|
| 355 |
+
{
|
| 356 |
+
"name": "stdout",
|
| 357 |
+
"output_type": "stream",
|
| 358 |
+
"text": [
|
| 359 |
+
"Processing 0/9..."
|
| 360 |
+
]
|
| 361 |
+
}
|
| 362 |
+
],
|
| 363 |
+
"source": [
|
| 364 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 365 |
+
"print(result)"
|
| 366 |
+
]
|
| 367 |
+
},
|
| 368 |
+
{
|
| 369 |
+
"cell_type": "code",
|
| 370 |
+
"execution_count": null,
|
| 371 |
+
"id": "73f6a8c7-3c5f-4620-b2a5-cd31079bfeb4",
|
| 372 |
+
"metadata": {},
|
| 373 |
+
"outputs": [],
|
| 374 |
+
"source": []
|
| 375 |
+
}
|
| 376 |
+
],
|
| 377 |
+
"metadata": {
|
| 378 |
+
"kernelspec": {
|
| 379 |
+
"display_name": "Python (huggingface_env)",
|
| 380 |
+
"language": "python",
|
| 381 |
+
"name": "huggingface_env"
|
| 382 |
+
},
|
| 383 |
+
"language_info": {
|
| 384 |
+
"codemirror_mode": {
|
| 385 |
+
"name": "ipython",
|
| 386 |
+
"version": 3
|
| 387 |
+
},
|
| 388 |
+
"file_extension": ".py",
|
| 389 |
+
"mimetype": "text/x-python",
|
| 390 |
+
"name": "python",
|
| 391 |
+
"nbconvert_exporter": "python",
|
| 392 |
+
"pygments_lexer": "ipython3",
|
| 393 |
+
"version": "3.8.20"
|
| 394 |
+
}
|
| 395 |
+
},
|
| 396 |
+
"nbformat": 4,
|
| 397 |
+
"nbformat_minor": 5
|
| 398 |
+
}
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/packing_list_api-checkpoint.ipynb
CHANGED
|
@@ -10,169 +10,621 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
-
"id": "
|
| 15 |
"metadata": {},
|
| 16 |
-
"outputs": [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
"source": [
|
| 18 |
"from dotenv import load_dotenv\n",
|
| 19 |
"import os\n",
|
| 20 |
"import requests\n",
|
| 21 |
"\n",
|
| 22 |
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
| 23 |
-
"\n",
|
| 24 |
-
"API_URL = \"https://api-inference.huggingface.co/models/facebook/bart-large-mnli\"\n",
|
| 25 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/MoritzLaurer/mDeBERTa-v3-base-mnli-xnli\"\n",
|
| 26 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/cross-encoder/nli-deberta-v3-base\"\n",
|
| 27 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/valhalla/distilbart-mnli-12-3\"\n",
|
| 28 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 29 |
"\n",
|
| 30 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 32 |
" return response.json()\n",
|
| 33 |
-
"\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
]
|
| 35 |
},
|
| 36 |
{
|
| 37 |
"cell_type": "code",
|
| 38 |
-
"execution_count":
|
| 39 |
-
"id": "
|
| 40 |
"metadata": {},
|
| 41 |
"outputs": [
|
| 42 |
{
|
| 43 |
"name": "stdout",
|
| 44 |
"output_type": "stream",
|
| 45 |
"text": [
|
| 46 |
-
"
|
|
|
|
|
|
|
| 47 |
]
|
| 48 |
}
|
| 49 |
],
|
| 50 |
"source": [
|
| 51 |
-
"#
|
| 52 |
-
"
|
|
|
|
|
|
|
| 53 |
"\n",
|
| 54 |
-
"#
|
| 55 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
"\n",
|
| 57 |
-
"#
|
| 58 |
-
"
|
| 59 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
]
|
| 61 |
},
|
| 62 |
{
|
| 63 |
"cell_type": "markdown",
|
| 64 |
-
"id": "
|
| 65 |
"metadata": {},
|
| 66 |
"source": [
|
| 67 |
-
"
|
| 68 |
]
|
| 69 |
},
|
| 70 |
{
|
| 71 |
"cell_type": "code",
|
| 72 |
-
"execution_count":
|
| 73 |
-
"id": "
|
| 74 |
"metadata": {},
|
| 75 |
"outputs": [
|
| 76 |
{
|
| 77 |
-
"name": "
|
| 78 |
"output_type": "stream",
|
| 79 |
"text": [
|
| 80 |
-
"
|
|
|
|
|
|
|
|
|
|
| 81 |
]
|
| 82 |
}
|
| 83 |
],
|
| 84 |
"source": [
|
| 85 |
-
"#
|
| 86 |
-
"
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
"
|
| 105 |
-
"
|
| 106 |
-
" \"Glow sticks\", \"Festival tickets\", \"Diapers\", \"Baby wipes\", \"Baby food\",\n",
|
| 107 |
-
" \"Stroller\", \"Pacifier\", \"Baby clothes\", \"Baby blanket\", \"Travel crib\",\n",
|
| 108 |
-
" \"Toys\", \"Nursing cover\"\n",
|
| 109 |
-
"]\n",
|
| 110 |
-
"\n",
|
| 111 |
-
"\n",
|
| 112 |
-
"# Get the prediction\n",
|
| 113 |
-
"output = query({\"inputs\": input_text, \"parameters\": {\"candidate_labels\": candidate_labels}})\n",
|
| 114 |
-
"print(output)"
|
| 115 |
]
|
| 116 |
},
|
| 117 |
{
|
| 118 |
"cell_type": "markdown",
|
| 119 |
-
"id": "
|
| 120 |
"metadata": {},
|
| 121 |
"source": [
|
| 122 |
-
"
|
| 123 |
]
|
| 124 |
},
|
| 125 |
{
|
| 126 |
"cell_type": "code",
|
| 127 |
-
"execution_count":
|
| 128 |
-
"id": "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
"metadata": {},
|
| 130 |
"outputs": [
|
| 131 |
{
|
| 132 |
"name": "stdout",
|
| 133 |
"output_type": "stream",
|
| 134 |
"text": [
|
| 135 |
-
"
|
| 136 |
-
"
|
| 137 |
-
"\n",
|
| 138 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
]
|
| 140 |
}
|
| 141 |
],
|
| 142 |
"source": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
"\n",
|
| 144 |
-
"
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
"
|
| 153 |
-
|
| 154 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 155 |
"\n",
|
| 156 |
-
"#
|
| 157 |
-
"
|
| 158 |
-
"
|
| 159 |
-
" result = query({\"inputs\": input_text, \"parameters\": {\"candidate_labels\": batch}})\n",
|
| 160 |
-
" print(result)\n",
|
| 161 |
-
" for label, score in zip(result[\"labels\"], result[\"scores\"]):\n",
|
| 162 |
-
" if score > 0.1: # Adjust threshold as needed\n",
|
| 163 |
-
" packing_list.append(label)\n",
|
| 164 |
"\n",
|
| 165 |
-
"#
|
| 166 |
-
"
|
|
|
|
| 167 |
]
|
| 168 |
},
|
| 169 |
{
|
| 170 |
"cell_type": "code",
|
| 171 |
-
"execution_count":
|
| 172 |
-
"id": "
|
| 173 |
"metadata": {},
|
| 174 |
-
"outputs": [
|
| 175 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
}
|
| 177 |
],
|
| 178 |
"metadata": {
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 14,
|
| 14 |
+
"id": "966c6d6a-c2d7-405c-bf9b-1e1f1415234e",
|
| 15 |
"metadata": {},
|
| 16 |
+
"outputs": [
|
| 17 |
+
{
|
| 18 |
+
"name": "stdout",
|
| 19 |
+
"output_type": "stream",
|
| 20 |
+
"text": [
|
| 21 |
+
"{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.970917284488678, 0.014999152161180973, 0.008272469975054264, 0.005811101291328669]}\n"
|
| 22 |
+
]
|
| 23 |
+
}
|
| 24 |
+
],
|
| 25 |
"source": [
|
| 26 |
"from dotenv import load_dotenv\n",
|
| 27 |
"import os\n",
|
| 28 |
"import requests\n",
|
| 29 |
"\n",
|
| 30 |
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 32 |
"\n",
|
| 33 |
+
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 34 |
+
"\n",
|
| 35 |
+
"def query(model, input_text):\n",
|
| 36 |
+
" API_URL = f\"https://api-inference.huggingface.co/models/{model}\"\n",
|
| 37 |
+
" payload = {\n",
|
| 38 |
+
" \"inputs\": input_text,\n",
|
| 39 |
+
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
| 40 |
+
" }\n",
|
| 41 |
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 42 |
" return response.json()\n",
|
| 43 |
+
"\n",
|
| 44 |
+
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 45 |
+
"\n",
|
| 46 |
+
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 47 |
+
"print(output)"
|
| 48 |
+
]
|
| 49 |
+
},
|
| 50 |
+
{
|
| 51 |
+
"cell_type": "markdown",
|
| 52 |
+
"id": "edf44387-d166-4e0f-a8ad-621230aee115",
|
| 53 |
+
"metadata": {},
|
| 54 |
+
"source": [
|
| 55 |
+
"**Try to run a model locally**"
|
| 56 |
]
|
| 57 |
},
|
| 58 |
{
|
| 59 |
"cell_type": "code",
|
| 60 |
+
"execution_count": 35,
|
| 61 |
+
"id": "1d01a363-572b-450c-8fce-0721234f9a1a",
|
| 62 |
"metadata": {},
|
| 63 |
"outputs": [
|
| 64 |
{
|
| 65 |
"name": "stdout",
|
| 66 |
"output_type": "stream",
|
| 67 |
"text": [
|
| 68 |
+
"First trip: 7-Day Island Beach Holiday in Greece (Summer). I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands. \n",
|
| 69 |
+
"\n",
|
| 70 |
+
"Packing list: ['bathing suit', 'beach towel', 'beach bag', 'sandals', 'comfortable walking shoes', 'light jacket', 'sunscreen', 'sunglasses', 'sunhat', 'entertainment for downtime (e.g. book/ebook, games, laptop, journal)', 'short pants/skirts', 't-shirts/tops']\n"
|
| 71 |
]
|
| 72 |
}
|
| 73 |
],
|
| 74 |
"source": [
|
| 75 |
+
"# Prerequisites\n",
|
| 76 |
+
"from tabulate import tabulate\n",
|
| 77 |
+
"from transformers import pipeline\n",
|
| 78 |
+
"import json\n",
|
| 79 |
"\n",
|
| 80 |
+
"# input text\n",
|
| 81 |
+
"input_text = \"I like to cycle and I burn easily. I also love culture and like to post on social media about my food. I will go on a trip to italy in july.\"\n",
|
| 82 |
+
"\n",
|
| 83 |
+
"# Load labels from a txt file\n",
|
| 84 |
+
"with open(\"labels.txt\", \"r\", encoding=\"utf-8\") as f:\n",
|
| 85 |
+
" class_labels = [line.strip() for line in f if line.strip()]\n",
|
| 86 |
"\n",
|
| 87 |
+
"# Load test data (in dictionary)\n",
|
| 88 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 89 |
+
" packing_data = json.load(file)\n",
|
| 90 |
+
"# Get a list of trip descriptions (keys)\n",
|
| 91 |
+
"trips = list(packing_data.keys())\n",
|
| 92 |
+
"# Access the first trip description\n",
|
| 93 |
+
"first_trip = trips[0]\n",
|
| 94 |
+
"# Get the packing list for the secondfirst trip\n",
|
| 95 |
+
"first_trip_items = packing_data[first_trip]\n",
|
| 96 |
+
"\n",
|
| 97 |
+
"print(f\"First trip: {first_trip} \\n\")\n",
|
| 98 |
+
"print(f\"Packing list: {first_trip_items}\")"
|
| 99 |
]
|
| 100 |
},
|
| 101 |
{
|
| 102 |
"cell_type": "markdown",
|
| 103 |
+
"id": "88aa1d7e-8a32-4530-9ddd-60fa38e4a342",
|
| 104 |
"metadata": {},
|
| 105 |
"source": [
|
| 106 |
+
"Load classifiers"
|
| 107 |
]
|
| 108 |
},
|
| 109 |
{
|
| 110 |
"cell_type": "code",
|
| 111 |
+
"execution_count": 36,
|
| 112 |
+
"id": "d0d8f7c0-c2d9-4fbe-b1a7-699a5b99466c",
|
| 113 |
"metadata": {},
|
| 114 |
"outputs": [
|
| 115 |
{
|
| 116 |
+
"name": "stderr",
|
| 117 |
"output_type": "stream",
|
| 118 |
"text": [
|
| 119 |
+
"Some weights of BartForSequenceClassification were not initialized from the model checkpoint at facebook/bart-base and are newly initialized: ['classification_head.dense.bias', 'classification_head.dense.weight', 'classification_head.out_proj.bias', 'classification_head.out_proj.weight']\n",
|
| 120 |
+
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
|
| 121 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n",
|
| 122 |
+
"Failed to determine 'entailment' label id from the label2id mapping in the model config. Setting to -1. Define a descriptive label2id mapping in the model config to ensure correct outputs.\n"
|
| 123 |
]
|
| 124 |
}
|
| 125 |
],
|
| 126 |
"source": [
|
| 127 |
+
"# Load smaller the model and create a pipeline for zero-shot classification (1min loading + classifying with 89 labels)\n",
|
| 128 |
+
"classifier_bart_base = pipeline(\"zero-shot-classification\", model=\"facebook/bart-base\")"
|
| 129 |
+
]
|
| 130 |
+
},
|
| 131 |
+
{
|
| 132 |
+
"cell_type": "code",
|
| 133 |
+
"execution_count": 37,
|
| 134 |
+
"id": "a971ca1c-d478-489f-9592-bc243d587eb4",
|
| 135 |
+
"metadata": {},
|
| 136 |
+
"outputs": [
|
| 137 |
+
{
|
| 138 |
+
"name": "stderr",
|
| 139 |
+
"output_type": "stream",
|
| 140 |
+
"text": [
|
| 141 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 142 |
+
]
|
| 143 |
+
}
|
| 144 |
+
],
|
| 145 |
+
"source": [
|
| 146 |
+
"# Load larger the model and create a pipeline for zero-shot classification (5min loading model + classifying with 89 labels)\n",
|
| 147 |
+
"classifier_bart_large_mnli = pipeline(\"zero-shot-classification\", model=\"facebook/bart-large-mnli\")"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
]
|
| 149 |
},
|
| 150 |
{
|
| 151 |
"cell_type": "markdown",
|
| 152 |
+
"id": "38805499-9919-40fe-9d42-de6869ba01dc",
|
| 153 |
"metadata": {},
|
| 154 |
"source": [
|
| 155 |
+
"Try classifiers"
|
| 156 |
]
|
| 157 |
},
|
| 158 |
{
|
| 159 |
"cell_type": "code",
|
| 160 |
+
"execution_count": 38,
|
| 161 |
+
"id": "abb13524-71c6-448d-948d-fb22a0e0ceeb",
|
| 162 |
+
"metadata": {},
|
| 163 |
+
"outputs": [
|
| 164 |
+
{
|
| 165 |
+
"name": "stderr",
|
| 166 |
+
"output_type": "stream",
|
| 167 |
+
"text": [
|
| 168 |
+
"Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.\n"
|
| 169 |
+
]
|
| 170 |
+
}
|
| 171 |
+
],
|
| 172 |
+
"source": [
|
| 173 |
+
"# Run the classification (ca 30 seconds classifying)\n",
|
| 174 |
+
"result_bart_base = classifier_bart_base(first_trip, class_labels)"
|
| 175 |
+
]
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"cell_type": "code",
|
| 179 |
+
"execution_count": 39,
|
| 180 |
+
"id": "116c7ee3-2b59-4623-a416-162c487aab70",
|
| 181 |
+
"metadata": {},
|
| 182 |
+
"outputs": [],
|
| 183 |
+
"source": [
|
| 184 |
+
"# Run the classification (ca 1 minute classifying)\n",
|
| 185 |
+
"result_bart_large_mnli = classifier_bart_large_mnli(first_trip, class_labels)"
|
| 186 |
+
]
|
| 187 |
+
},
|
| 188 |
+
{
|
| 189 |
+
"cell_type": "code",
|
| 190 |
+
"execution_count": 40,
|
| 191 |
+
"id": "8591425b-ce55-4a36-a4b6-70974e8d4e59",
|
| 192 |
"metadata": {},
|
| 193 |
"outputs": [
|
| 194 |
{
|
| 195 |
"name": "stdout",
|
| 196 |
"output_type": "stream",
|
| 197 |
"text": [
|
| 198 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 199 |
+
"| bart_base | bart_large_mnli |\n",
|
| 200 |
+
"+======================================================================+======================================================================+\n",
|
| 201 |
+
"| bandana | travel adapter |\n",
|
| 202 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 203 |
+
"| lip balm | travel journal |\n",
|
| 204 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 205 |
+
"| hand warmers | light jacket |\n",
|
| 206 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 207 |
+
"| sim card | sim card |\n",
|
| 208 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 209 |
+
"| neck wallet | bathing suit |\n",
|
| 210 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 211 |
+
"| tent | multi-tool |\n",
|
| 212 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 213 |
+
"| poncho | dry bag |\n",
|
| 214 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 215 |
+
"| gloves | travel pillow |\n",
|
| 216 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 217 |
+
"| painkiller | base layers |\n",
|
| 218 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 219 |
+
"| pen | day pack |\n",
|
| 220 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 221 |
+
"| thin scarf | entertainment for downtime (e.g. book/ebook, games, laptop, journal) |\n",
|
| 222 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 223 |
+
"| wallet | passport |\n",
|
| 224 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 225 |
+
"| sleeping bag | thin scarf |\n",
|
| 226 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 227 |
+
"| dry bag | comfortable shoes |\n",
|
| 228 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 229 |
+
"| gifts | lightweight backpack |\n",
|
| 230 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 231 |
+
"| fanny pack | beach bag |\n",
|
| 232 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 233 |
+
"| beach towel | swimsuit |\n",
|
| 234 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 235 |
+
"| underwear | short pants/skirts |\n",
|
| 236 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 237 |
+
"| swimsuit | quick-dry towel |\n",
|
| 238 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 239 |
+
"| blanket | sunhat |\n",
|
| 240 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 241 |
+
"| laptop | local currency |\n",
|
| 242 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 243 |
+
"| quick-dry towel | tickets |\n",
|
| 244 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 245 |
+
"| ski jacket | wallet |\n",
|
| 246 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 247 |
+
"| emergency roadside kit | cardigan/sweater |\n",
|
| 248 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 249 |
+
"| passport | refillable water bottle |\n",
|
| 250 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 251 |
+
"| necessary medication | fanny pack |\n",
|
| 252 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 253 |
+
"| snacks for the journey | poncho |\n",
|
| 254 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 255 |
+
"| snow boots | thermal clothing |\n",
|
| 256 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 257 |
+
"| sunglasses | laptop |\n",
|
| 258 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 259 |
+
"| flip-flops | pen |\n",
|
| 260 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 261 |
+
"| phone and charger | big backpack/suitcase |\n",
|
| 262 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 263 |
+
"| socks | beach towel |\n",
|
| 264 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 265 |
+
"| local currency | currency |\n",
|
| 266 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 267 |
+
"| t-shirts/tops | blanket |\n",
|
| 268 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 269 |
+
"| bathing suit | compass |\n",
|
| 270 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 271 |
+
"| currency | beanie |\n",
|
| 272 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 273 |
+
"| cardigan/sweater | sunscreen |\n",
|
| 274 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 275 |
+
"| snowboard | phone and charger |\n",
|
| 276 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 277 |
+
"| visa documents | reusable coffee mug |\n",
|
| 278 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 279 |
+
"| shirts | power bank |\n",
|
| 280 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 281 |
+
"| headphones | personal toiletries (e.g. toothbrush, toothpaste) |\n",
|
| 282 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 283 |
+
"| pants | packable rain jacket |\n",
|
| 284 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 285 |
+
"| refillable water bottle | bandana |\n",
|
| 286 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 287 |
+
"| beach bag | short pants |\n",
|
| 288 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 289 |
+
"| big backpack/suitcase | business attire |\n",
|
| 290 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 291 |
+
"| multi-tool | sleeping bag |\n",
|
| 292 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 293 |
+
"| sandals | flashlight |\n",
|
| 294 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 295 |
+
"| tickets | t-shirts/tops |\n",
|
| 296 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 297 |
+
"| thermal clothing | waterproof phone case |\n",
|
| 298 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 299 |
+
"| short pants/skirts | long pants |\n",
|
| 300 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 301 |
+
"| light jacket | travel-sized toiletries |\n",
|
| 302 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 303 |
+
"| base layers | visa documents |\n",
|
| 304 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 305 |
+
"| snacks | sandals |\n",
|
| 306 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 307 |
+
"| comfortable shoes | hand warmers |\n",
|
| 308 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 309 |
+
"| lightweight backpack | hostel lock |\n",
|
| 310 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 311 |
+
"| winter shoes | headphones |\n",
|
| 312 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 313 |
+
"| dress shoes | emergency roadside kit |\n",
|
| 314 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 315 |
+
"| power bank | ski jacket |\n",
|
| 316 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 317 |
+
"| packable rain jacket | shirts |\n",
|
| 318 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 319 |
+
"| first aid kit | first aid kit |\n",
|
| 320 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 321 |
+
"| water filter | reusable shopping bags |\n",
|
| 322 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 323 |
+
"| short pants | flip-flops |\n",
|
| 324 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 325 |
+
"| ski goggles | camping stove |\n",
|
| 326 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 327 |
+
"| waterproof phone case | water filter |\n",
|
| 328 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 329 |
+
"| sunhat | gloves |\n",
|
| 330 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 331 |
+
"| flashlight | dress shoes |\n",
|
| 332 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 333 |
+
"| earplugs | tent |\n",
|
| 334 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 335 |
+
"| beanie | sunglasses |\n",
|
| 336 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 337 |
+
"| hostel lock | pants |\n",
|
| 338 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 339 |
+
"| personal toiletries (e.g. toothbrush, toothpaste) | charging cables |\n",
|
| 340 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 341 |
+
"| travel journal | snacks |\n",
|
| 342 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 343 |
+
"| reusable coffee mug | neck wallet |\n",
|
| 344 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 345 |
+
"| sunscreen | snacks for the journey |\n",
|
| 346 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 347 |
+
"| travel pillow | ski goggles |\n",
|
| 348 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 349 |
+
"| trekking poles | mosquito repellant |\n",
|
| 350 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 351 |
+
"| business attire | snorkel gear |\n",
|
| 352 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 353 |
+
"| snorkel gear | bug spray |\n",
|
| 354 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 355 |
+
"| reusable shopping bags | earplugs |\n",
|
| 356 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 357 |
+
"| sleep wear | travel insurance documents |\n",
|
| 358 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 359 |
+
"| camping stove | painkiller |\n",
|
| 360 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 361 |
+
"| travel-sized toiletries | hiking boots |\n",
|
| 362 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 363 |
+
"| hiking boots | necessary medication |\n",
|
| 364 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 365 |
+
"| travel insurance documents | socks |\n",
|
| 366 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 367 |
+
"| long pants | underwear |\n",
|
| 368 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 369 |
+
"| charging cables | trekking poles |\n",
|
| 370 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 371 |
+
"| entertainment for downtime (e.g. book/ebook, games, laptop, journal) | sleep wear |\n",
|
| 372 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 373 |
+
"| bug spray | gifts |\n",
|
| 374 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 375 |
+
"| day pack | lip balm |\n",
|
| 376 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 377 |
+
"| travel adapter | snowboard |\n",
|
| 378 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 379 |
+
"| malaria medication | malaria medication |\n",
|
| 380 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 381 |
+
"| compass | snow boots |\n",
|
| 382 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n",
|
| 383 |
+
"| mosquito repellant | winter shoes |\n",
|
| 384 |
+
"+----------------------------------------------------------------------+----------------------------------------------------------------------+\n"
|
| 385 |
]
|
| 386 |
}
|
| 387 |
],
|
| 388 |
"source": [
|
| 389 |
+
"# Creating a table\n",
|
| 390 |
+
"table = zip(result_bart_base[\"labels\"], \n",
|
| 391 |
+
" result_bart_large_mnli[\"labels\"])\n",
|
| 392 |
+
"headers = [\"bart_base\", \"bart_large_mnli\"]\n",
|
| 393 |
"\n",
|
| 394 |
+
"print(tabulate(table, headers=headers, tablefmt=\"grid\"))\n"
|
| 395 |
+
]
|
| 396 |
+
},
|
| 397 |
+
{
|
| 398 |
+
"cell_type": "markdown",
|
| 399 |
+
"id": "21a35d0c-9451-433a-b14c-87e8dac21d68",
|
| 400 |
+
"metadata": {},
|
| 401 |
+
"source": [
|
| 402 |
+
"**Try simple prompt engineering**"
|
| 403 |
+
]
|
| 404 |
+
},
|
| 405 |
+
{
|
| 406 |
+
"cell_type": "code",
|
| 407 |
+
"execution_count": 12,
|
| 408 |
+
"id": "ee734de6-bbcb-427d-8987-ab41286f7907",
|
| 409 |
+
"metadata": {},
|
| 410 |
+
"outputs": [],
|
| 411 |
+
"source": [
|
| 412 |
+
"# No prompt\n",
|
| 413 |
+
"no_prompt = input_text\n",
|
| 414 |
+
"no_result = classifier(no_prompt, class_labels)\n",
|
| 415 |
"\n",
|
| 416 |
+
"# Simple prompt\n",
|
| 417 |
+
"simple_prompt = \"Classify the following text: \" + input_text\n",
|
| 418 |
+
"simple_result = classifier(simple_prompt, class_labels)\n",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 419 |
"\n",
|
| 420 |
+
"# Primed prompt\n",
|
| 421 |
+
"primed_prompt = input_text + \"What are the most important things to pack for the trip?\"\n",
|
| 422 |
+
"primed_result = classifier(primed_prompt, class_labels)"
|
| 423 |
]
|
| 424 |
},
|
| 425 |
{
|
| 426 |
"cell_type": "code",
|
| 427 |
+
"execution_count": 13,
|
| 428 |
+
"id": "96deb877-b3b0-4048-9960-d8b0b0d56cd0",
|
| 429 |
"metadata": {},
|
| 430 |
+
"outputs": [
|
| 431 |
+
{
|
| 432 |
+
"name": "stdout",
|
| 433 |
+
"output_type": "stream",
|
| 434 |
+
"text": [
|
| 435 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 436 |
+
"| no_prompt | no_prompt | simple_prompt | simple_prompt | primed_prompt | primed_prompt |\n",
|
| 437 |
+
"+============================+=============+============================+=================+============================+=================+\n",
|
| 438 |
+
"| Travel-sized toiletries | 0.0141621 | Beanie | 0.0126489 | First aid kit | 0.0126422 |\n",
|
| 439 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 440 |
+
"| Refillable water bottle | 0.013635 | Baby wipes | 0.0125994 | Work ID badge | 0.0125781 |\n",
|
| 441 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 442 |
+
"| Aloe vera gel | 0.0135288 | Bandana | 0.0125701 | Travel insurance documents | 0.0125387 |\n",
|
| 443 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 444 |
+
"| Snorkel gear | 0.0135229 | Blanket | 0.0125266 | Business attire | 0.0124256 |\n",
|
| 445 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 446 |
+
"| Waterproof phone case | 0.0135219 | Sunglasses | 0.0123896 | Baby wipes | 0.012401 |\n",
|
| 447 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 448 |
+
"| Packable rain jacket | 0.0133905 | Sunglasses | 0.0123896 | Blanket | 0.0122619 |\n",
|
| 449 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 450 |
+
"| Reusable shopping bags | 0.0133134 | Laptop | 0.0123645 | Lightweight backpack | 0.0122291 |\n",
|
| 451 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 452 |
+
"| Reusable coffee mug | 0.012921 | Snacks | 0.0123038 | Sunglasses | 0.0121536 |\n",
|
| 453 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 454 |
+
"| Reusable water bottle | 0.012695 | Sunscreen | 0.0122985 | Sunglasses | 0.0121536 |\n",
|
| 455 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 456 |
+
"| First aid kit | 0.0124969 | Pen | 0.0122703 | Laptop | 0.0121034 |\n",
|
| 457 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 458 |
+
"| Travel insurance documents | 0.0124186 | Cooler | 0.0122299 | Passport | 0.0121023 |\n",
|
| 459 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 460 |
+
"| Work ID badge | 0.0123513 | Snowboard | 0.012205 | Beanie | 0.0120397 |\n",
|
| 461 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 462 |
+
"| Lightweight backpack | 0.0122867 | Passport | 0.0121188 | Baby clothes | 0.0120325 |\n",
|
| 463 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 464 |
+
"| Presentation materials | 0.0121707 | Visa documents | 0.0121176 | Snacks | 0.0119757 |\n",
|
| 465 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 466 |
+
"| Flip-flops | 0.0121665 | Swimsuit | 0.0120711 | Packable rain jacket | 0.011946 |\n",
|
| 467 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 468 |
+
"| Charging cables | 0.0121361 | Flashlight | 0.0120105 | Baby food | 0.0119228 |\n",
|
| 469 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 470 |
+
"| Hiking boots | 0.0121118 | Stroller | 0.0119368 | Baby blanket | 0.0118852 |\n",
|
| 471 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 472 |
+
"| Comfortable shoes | 0.0120349 | Map | 0.01193 | Dress shoes | 0.0118458 |\n",
|
| 473 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 474 |
+
"| Fanny pack | 0.011914 | First aid kit | 0.0119121 | Bug spray | 0.0118403 |\n",
|
| 475 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 476 |
+
"| Trekking poles | 0.0118607 | Notebook | 0.0118809 | Travel journal | 0.0118067 |\n",
|
| 477 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 478 |
+
"| Visa documents | 0.0118408 | Hat | 0.011833 | Travel pillow | 0.0118006 |\n",
|
| 479 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 480 |
+
"| Baby wipes | 0.0117297 | Currency | 0.0118279 | Visa documents | 0.0117734 |\n",
|
| 481 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 482 |
+
"| Quick-dry towel | 0.0116303 | Work ID badge | 0.0117867 | Emergency roadside kit | 0.0117412 |\n",
|
| 483 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 484 |
+
"| Baby blanket | 0.011586 | Travel insurance documents | 0.01168 | SIM card | 0.0117407 |\n",
|
| 485 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 486 |
+
"| Hostel lock | 0.0115573 | Business attire | 0.0116774 | Cooler | 0.0117317 |\n",
|
| 487 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 488 |
+
"| Blanket | 0.0114867 | Compass | 0.0116575 | Snowboard | 0.0117232 |\n",
|
| 489 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 490 |
+
"| Business attire | 0.0114801 | Playlist | 0.0116254 | Diapers | 0.0117056 |\n",
|
| 491 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 492 |
+
"| Laptop | 0.0112665 | Bug spray | 0.0115941 | Notebook | 0.011676 |\n",
|
| 493 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 494 |
+
"| Beanie | 0.0112438 | Tent | 0.0115531 | Bandana | 0.0116441 |\n",
|
| 495 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 496 |
+
"| Bug spray | 0.0112397 | Tent | 0.0115531 | Pen | 0.011614 |\n",
|
| 497 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 498 |
+
"| Travel pillow | 0.0111955 | Diapers | 0.0115231 | Flashlight | 0.011587 |\n",
|
| 499 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 500 |
+
"| Baby clothes | 0.0111946 | Travel journal | 0.0114808 | Playlist | 0.0115787 |\n",
|
| 501 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 502 |
+
"| Passport | 0.0111815 | Hiking boots | 0.0114734 | Sunscreen | 0.0115577 |\n",
|
| 503 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 504 |
+
"| Earplugs | 0.0111681 | Reusable shopping bags | 0.0114722 | Swimsuit | 0.0115468 |\n",
|
| 505 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 506 |
+
"| Camping stove | 0.0111419 | SIM card | 0.0114319 | Reusable coffee mug | 0.0115091 |\n",
|
| 507 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 508 |
+
"| Travel journal | 0.01114 | Toys | 0.0114257 | Trekking poles | 0.011476 |\n",
|
| 509 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 510 |
+
"| Emergency roadside kit | 0.011128 | Dress shoes | 0.0113439 | Sleeping bag | 0.0114472 |\n",
|
| 511 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 512 |
+
"| Baby food | 0.0110848 | Waterproof phone case | 0.0113438 | Hiking boots | 0.0114388 |\n",
|
| 513 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 514 |
+
"| Pen | 0.0110799 | Travel pillow | 0.0113271 | Snorkel gear | 0.0114219 |\n",
|
| 515 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 516 |
+
"| Bandana | 0.0110713 | Refillable water bottle | 0.0113269 | Reusable shopping bags | 0.0113664 |\n",
|
| 517 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 518 |
+
"| Dress shoes | 0.0110698 | Fanny pack | 0.0113193 | Portable charger | 0.0113632 |\n",
|
| 519 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 520 |
+
"| Snacks | 0.0110153 | Baby blanket | 0.0113175 | Fanny pack | 0.011333 |\n",
|
| 521 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 522 |
+
"| Travel crib | 0.0110133 | Aloe vera gel | 0.0113123 | Headphones | 0.0113156 |\n",
|
| 523 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 524 |
+
"| Sunscreen | 0.0109995 | Snorkel gear | 0.011283 | Currency | 0.0112893 |\n",
|
| 525 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 526 |
+
"| Ski goggles | 0.010981 | Pacifier | 0.0112826 | Travel adapter | 0.0112652 |\n",
|
| 527 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 528 |
+
"| Sunglasses | 0.0109759 | Headphones | 0.0112543 | Travel crib | 0.011224 |\n",
|
| 529 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 530 |
+
"| Sunglasses | 0.0109759 | Packable rain jacket | 0.0112416 | Presentation materials | 0.0112228 |\n",
|
| 531 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 532 |
+
"| Stroller | 0.0109661 | Poncho | 0.0112411 | Waterproof phone case | 0.0112181 |\n",
|
| 533 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 534 |
+
"| Lip balm | 0.0109645 | Nursing cover | 0.0112323 | Nursing cover | 0.0111811 |\n",
|
| 535 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 536 |
+
"| Notebook | 0.0109306 | Comfortable shoes | 0.0112138 | Beach bag | 0.0111739 |\n",
|
| 537 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 538 |
+
"| Glow sticks | 0.0108928 | Reusable coffee mug | 0.0112081 | Stroller | 0.0111447 |\n",
|
| 539 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 540 |
+
"| Cooler | 0.0108527 | Travel crib | 0.0111724 | Car charger | 0.0110935 |\n",
|
| 541 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 542 |
+
"| Snowboard | 0.0108444 | Baby clothes | 0.0111683 | Neck wallet | 0.0110586 |\n",
|
| 543 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 544 |
+
"| Map | 0.0108275 | Presentation materials | 0.0111555 | Lip balm | 0.0110534 |\n",
|
| 545 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 546 |
+
"| Thermal clothing | 0.0108054 | Baby food | 0.0111165 | Comfortable shoes | 0.0110398 |\n",
|
| 547 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 548 |
+
"| Neck wallet | 0.0107892 | Sleeping bag | 0.0110978 | Poncho | 0.0109919 |\n",
|
| 549 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 550 |
+
"| Water filter | 0.0107846 | Lightweight backpack | 0.011038 | Reusable water bottle | 0.0109792 |\n",
|
| 551 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 552 |
+
"| Travel adapter | 0.0107792 | Gloves | 0.010946 | Energy bars | 0.0109684 |\n",
|
| 553 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 554 |
+
"| Currency | 0.010762 | Portable charger | 0.0108962 | Map | 0.0109623 |\n",
|
| 555 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 556 |
+
"| Nursing cover | 0.0107438 | Trekking poles | 0.0108781 | Hostel lock | 0.0109603 |\n",
|
| 557 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 558 |
+
"| Snow boots | 0.0107272 | Charging cables | 0.0108504 | Power bank | 0.0109483 |\n",
|
| 559 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 560 |
+
"| Pacifier | 0.0107227 | Reusable water bottle | 0.0108255 | Thermal clothing | 0.0109311 |\n",
|
| 561 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 562 |
+
"| Sleeping bag | 0.0106966 | Neck wallet | 0.0108161 | Earplugs | 0.0109061 |\n",
|
| 563 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 564 |
+
"| Car charger | 0.0106959 | Beach bag | 0.0108042 | Charging cables | 0.0108819 |\n",
|
| 565 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 566 |
+
"| Diapers | 0.010669 | Travel-sized toiletries | 0.0107921 | Toys | 0.0108427 |\n",
|
| 567 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 568 |
+
"| Flashlight | 0.0106647 | Travel adapter | 0.0107415 | Ski jacket | 0.0108272 |\n",
|
| 569 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 570 |
+
"| Ski jacket | 0.0106418 | Hostel lock | 0.0106021 | Base layers | 0.0107343 |\n",
|
| 571 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 572 |
+
"| Portable charger | 0.0106307 | Thermal clothing | 0.0105911 | Glow sticks | 0.0106845 |\n",
|
| 573 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 574 |
+
"| Playlist | 0.0106083 | Car charger | 0.0105783 | Beach towel | 0.010634 |\n",
|
| 575 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 576 |
+
"| Swimsuit | 0.0105832 | Ski goggles | 0.0105752 | Water filter | 0.0106173 |\n",
|
| 577 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 578 |
+
"| Tent | 0.010549 | Ski jacket | 0.0105524 | Festival tickets | 0.0106124 |\n",
|
| 579 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 580 |
+
"| Tent | 0.010549 | Water filter | 0.010523 | Dry bag | 0.0105999 |\n",
|
| 581 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 582 |
+
"| SIM card | 0.0105225 | Festival tickets | 0.0105077 | Hat | 0.010555 |\n",
|
| 583 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 584 |
+
"| Compass | 0.0105095 | Dry bag | 0.0104999 | Tent | 0.0105432 |\n",
|
| 585 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 586 |
+
"| Multi-tool | 0.0104697 | Glow sticks | 0.0104861 | Tent | 0.0105432 |\n",
|
| 587 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 588 |
+
"| Hat | 0.0104314 | Beach towel | 0.0104595 | Refillable water bottle | 0.0105226 |\n",
|
| 589 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 590 |
+
"| Base layers | 0.0104078 | Earplugs | 0.0104484 | Language phrasebook | 0.0104878 |\n",
|
| 591 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 592 |
+
"| Energy bars | 0.0103765 | Emergency roadside kit | 0.01042 | Aloe vera gel | 0.0104495 |\n",
|
| 593 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 594 |
+
"| Toys | 0.0103667 | Energy bars | 0.010328 | Compass | 0.0102844 |\n",
|
| 595 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 596 |
+
"| Power bank | 0.010329 | Flip-flops | 0.010279 | Pacifier | 0.0102553 |\n",
|
| 597 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 598 |
+
"| Dry bag | 0.0102989 | Power bank | 0.0102667 | Flip-flops | 0.0102396 |\n",
|
| 599 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 600 |
+
"| Beach towel | 0.0102733 | Base layers | 0.0102346 | Ski goggles | 0.010229 |\n",
|
| 601 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 602 |
+
"| Beach bag | 0.0102259 | Multi-tool | 0.0101584 | Multi-tool | 0.0100441 |\n",
|
| 603 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 604 |
+
"| Poncho | 0.0100634 | Lip balm | 0.0101392 | Gloves | 0.0100095 |\n",
|
| 605 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 606 |
+
"| Headphones | 0.010053 | Snow boots | 0.0101161 | Hand warmers | 0.00999101 |\n",
|
| 607 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 608 |
+
"| Gloves | 0.010049 | Camping stove | 0.00999308 | Camping stove | 0.00982307 |\n",
|
| 609 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 610 |
+
"| Festival tickets | 0.00984126 | Language phrasebook | 0.00958238 | Travel-sized toiletries | 0.0097547 |\n",
|
| 611 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 612 |
+
"| Hand warmers | 0.00967844 | Quick-dry towel | 0.00957849 | Snow boots | 0.00964112 |\n",
|
| 613 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n",
|
| 614 |
+
"| Language phrasebook | 0.0093065 | Hand warmers | 0.00916433 | Quick-dry towel | 0.00960495 |\n",
|
| 615 |
+
"+----------------------------+-------------+----------------------------+-----------------+----------------------------+-----------------+\n"
|
| 616 |
+
]
|
| 617 |
+
}
|
| 618 |
+
],
|
| 619 |
+
"source": [
|
| 620 |
+
"# Creating a table\n",
|
| 621 |
+
"table = zip(no_result[\"labels\"], no_result[\"scores\"], \n",
|
| 622 |
+
" simple_result[\"labels\"], simple_result[\"scores\"], \n",
|
| 623 |
+
" primed_result[\"labels\"], primed_result[\"scores\"])\n",
|
| 624 |
+
"headers = [\"no_prompt\", \"no_prompt\", \"simple_prompt\", \"simple_prompt\", \"primed_prompt\", \"primed_prompt\"]\n",
|
| 625 |
+
"\n",
|
| 626 |
+
"print(tabulate(table, headers=headers, tablefmt=\"grid\"))\n"
|
| 627 |
+
]
|
| 628 |
}
|
| 629 |
],
|
| 630 |
"metadata": {
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/README.md
CHANGED
|
@@ -1,14 +1,14 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: yellow
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version: 5.
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
-
license:
|
| 11 |
-
short_description:
|
| 12 |
---
|
| 13 |
|
| 14 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Test Space
|
| 3 |
+
emoji: 🌍
|
| 4 |
colorFrom: yellow
|
| 5 |
+
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 5.49.1
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
short_description: Test for github access token
|
| 12 |
---
|
| 13 |
|
| 14 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/blog_post/article.md
CHANGED
|
@@ -16,21 +16,145 @@
|
|
| 16 |
## Implementation of packing list model
|
| 17 |
### Prerequisites before you start to code: Anja
|
| 18 |
* Hugging face account also use access token, mention api
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
* Use anaconda
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
### Predefine outputs/classes: Nikky
|
| 26 |
|
| 27 |
### Model implementation: Anja
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
### Using gradio app: Anja
|
| 30 |
|
| 31 |
### Share your model: Anja
|
| 32 |
* created space
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
| 35 |
* create requirements.txt file with necessary libraries
|
| 36 |
|
|
|
|
| 16 |
## Implementation of packing list model
|
| 17 |
### Prerequisites before you start to code: Anja
|
| 18 |
* Hugging face account also use access token, mention api
|
| 19 |
+
Hugging face is..."The platform where the machine learning community collaborates on models, datasets, and applications."(from hugging face)
|
| 20 |
+
"Hugging Face is a company and open-source community that builds tools for machine learning (ML) and artificial intelligence (AI) — especially in the field of natural language processing (NLP)."(from the chate)
|
| 21 |
+
To be able to use the full funcitonality (e.g. models, spaces, data sets, api access) and most access you need to make a hugging face account here (https://huggingface.co/)
|
| 22 |
+
There is a new course at data camp free for the remainder of 2025: https://huggingface.co/blog/huggingface/datacamp-ai-courses
|
| 23 |
+
|
| 24 |
* Use anaconda
|
| 25 |
+
"Without a package and environment manager like Conda"(anaconda site)
|
| 26 |
+
We used the anaconda navigator as a package and environment manager and used the jupyter notebook through there. python is automatically installed.
|
| 27 |
+
In the terminal activate the conda environment that you created previously by navigating (see cheat sheet here https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf)
|
| 28 |
+
|
| 29 |
+
View list of environments
|
| 30 |
+
```bash
|
| 31 |
+
conda env list
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
Activate correct environment
|
| 35 |
+
```bash
|
| 36 |
+
conda activate ENVIRONMENT_NAME
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
Install packages using pip
|
| 40 |
+
```bash
|
| 41 |
+
pip install transformers torch numpy tabulate gradio pandas sklearn
|
| 42 |
+
# pip install -r requirements.txt
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
Install jupyter notebook
|
| 46 |
+
```bash
|
| 47 |
+
conda install jupyter
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
Start up jupyter notebook
|
| 51 |
+
```bash
|
| 52 |
+
jupyter-notebook
|
| 53 |
+
```
|
| 54 |
+
Create a new jupyter notebook
|
| 55 |
+
|
| 56 |
+
#### Hugging face API
|
| 57 |
+
Let us first try out some hugging face models using their API. The advantage of using the API is that you do not need to download the model locally and the computation is handled by hugging face on their servers.
|
| 58 |
+
To do this you need to make an accesstoken on the hugging face website.
|
| 59 |
+
Log in > Settings (on left side) > Access Tokens (on left side) > + Create new token
|
| 60 |
+
Give token a name
|
| 61 |
+
* how to implement the token in your code
|
| 62 |
+
This access token now has to be saved in you projecct folder in a .env file. Create a plan text file that you call .env. Within it you write:
|
| 63 |
+
```text
|
| 64 |
+
HF_API_TOKEN=YOUR_OWN_ACCESS_TOKEN
|
| 65 |
+
```
|
| 66 |
+
and save it.
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
Now we load a zero-shot-classification model using API and male a simple classification.
|
| 70 |
+
```python
|
| 71 |
+
from dotenv import load_dotenv
|
| 72 |
+
import os
|
| 73 |
+
import requests
|
| 74 |
+
|
| 75 |
+
load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)
|
| 76 |
+
headers = {"Authorization": f"Bearer {os.getenv('HF_API_TOKEN')}"}
|
| 77 |
+
|
| 78 |
+
candidate_labels = ["technology", "sports", "politics", "health"]
|
| 79 |
+
|
| 80 |
+
def query(model, input_text):
|
| 81 |
+
API_URL = f"https://api-inference.huggingface.co/models/{model}"
|
| 82 |
+
payload = {
|
| 83 |
+
"inputs": input_text,
|
| 84 |
+
"parameters": {"candidate_labels": candidate_labels}
|
| 85 |
+
}
|
| 86 |
+
response = requests.post(API_URL, headers=headers, json=payload)
|
| 87 |
+
return response.json()
|
| 88 |
+
|
| 89 |
+
input_text = "I just bought a new laptop, and it works amazing!"
|
| 90 |
+
|
| 91 |
+
output = query("facebook/bart-large-mnli", input_text)
|
| 92 |
+
print(output)
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
+
```json
|
| 96 |
+
{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.970917284488678, 0.014999152161180973, 0.008272469975054264, 0.005811101291328669]}
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
However the functionality using API is limited and we were only allowed to use 10 candidate labels for our models. This was not sufficient for our packing list example.
|
| 100 |
+
|
| 101 |
|
| 102 |
### Predefine outputs/classes: Nikky
|
| 103 |
|
| 104 |
### Model implementation: Anja
|
| 105 |
+
Prerequisites
|
| 106 |
+
```python
|
| 107 |
+
import math
|
| 108 |
+
import json
|
| 109 |
+
import pickle
|
| 110 |
+
import os
|
| 111 |
+
import time
|
| 112 |
+
import pandas as pd
|
| 113 |
+
import matplotlib.pyplot as plt
|
| 114 |
+
from tabulate import tabulate
|
| 115 |
+
from transformers import pipeline
|
| 116 |
+
|
| 117 |
+
# Get candidate labels
|
| 118 |
+
with open("packing_label_structure.json", "r") as file:
|
| 119 |
+
candidate_labels = json.load(file)
|
| 120 |
+
keys_list = list(candidate_labels.keys())
|
| 121 |
+
|
| 122 |
+
# Load test data (list of dictionaries)
|
| 123 |
+
with open("test_data.json", "r") as file:
|
| 124 |
+
packing_data = json.load(file)
|
| 125 |
+
# Extract trip descriptions and classification (trip_types)
|
| 126 |
+
trip_descriptions = [trip['description'] for trip in packing_data]
|
| 127 |
+
trip_types = [trip['trip_types'] for trip in packing_data]
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
```python
|
| 131 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 132 |
+
result = classifier(trip_descr, candidate_labels[key])
|
| 133 |
+
print(result)
|
| 134 |
+
```
|
| 135 |
|
| 136 |
### Using gradio app: Anja
|
| 137 |
|
| 138 |
### Share your model: Anja
|
| 139 |
* created space
|
| 140 |
+
An easy way to share your modle is using hugging face spaces. Go to https://huggingface.co/spaces and click on "+ New Space", as SDK choose Gradio and as template Blank, as Space hardware you can choose "CPU Basic", and click on "Create Space" to create your space.
|
| 141 |
+
* space as remote repository
|
| 142 |
+
connected to your space is a remote git repository that you can push your code to. For this navigate to your project folder in the terminal
|
| 143 |
+
|
| 144 |
+
```bash
|
| 145 |
+
cd path/to/your/project
|
| 146 |
+
git init
|
| 147 |
+
```
|
| 148 |
+
|
| 149 |
+
And connect your hugging face space as a remote repository
|
| 150 |
+
|
| 151 |
+
```bash
|
| 152 |
+
git remote add origin https://huggingface.co/spaces/<username>/<space-name>
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
generate an access token for your space here
|
| 156 |
+
|
| 157 |
+
|
| 158 |
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
| 159 |
* create requirements.txt file with necessary libraries
|
| 160 |
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/code_blog_post.ipynb
ADDED
|
@@ -0,0 +1,619 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cells": [
|
| 3 |
+
{
|
| 4 |
+
"cell_type": "markdown",
|
| 5 |
+
"id": "fce70006-809b-4c98-b89c-00910b8bbea1",
|
| 6 |
+
"metadata": {},
|
| 7 |
+
"source": [
|
| 8 |
+
"Implementation for blog post"
|
| 9 |
+
]
|
| 10 |
+
},
|
| 11 |
+
{
|
| 12 |
+
"cell_type": "code",
|
| 13 |
+
"execution_count": 14,
|
| 14 |
+
"id": "1eaa3a9f-0b39-4d77-91d6-f935d226ac98",
|
| 15 |
+
"metadata": {},
|
| 16 |
+
"outputs": [],
|
| 17 |
+
"source": [
|
| 18 |
+
"import math\n",
|
| 19 |
+
"import pickle\n",
|
| 20 |
+
"import os\n",
|
| 21 |
+
"import time\n",
|
| 22 |
+
"import matplotlib.pyplot as plt\n",
|
| 23 |
+
"from tabulate import tabulate\n",
|
| 24 |
+
"\n",
|
| 25 |
+
"from transformers import pipeline\n",
|
| 26 |
+
"import json\n",
|
| 27 |
+
"import pandas as pd\n",
|
| 28 |
+
"\n",
|
| 29 |
+
"# Get candidate labels\n",
|
| 30 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 31 |
+
" candidate_labels = json.load(file)\n",
|
| 32 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 33 |
+
"\n",
|
| 34 |
+
"# Load test data (list of dictionaries)\n",
|
| 35 |
+
"# with open(\"test_data.json\", \"r\") as file:\n",
|
| 36 |
+
"# packing_data = json.load(file)\n",
|
| 37 |
+
"# Extract trip descriptions and classification (trip_types)\n",
|
| 38 |
+
"# trip_descriptions = [trip['description'] for trip in packing_data]\n",
|
| 39 |
+
"# trip_types = [trip['trip_types'] for trip in packing_data]\n"
|
| 40 |
+
]
|
| 41 |
+
},
|
| 42 |
+
{
|
| 43 |
+
"cell_type": "code",
|
| 44 |
+
"execution_count": 19,
|
| 45 |
+
"id": "bb1bc7ed-227e-4c0b-b769-ead4daf01c57",
|
| 46 |
+
"metadata": {},
|
| 47 |
+
"outputs": [
|
| 48 |
+
{
|
| 49 |
+
"name": "stdout",
|
| 50 |
+
"output_type": "stream",
|
| 51 |
+
"text": [
|
| 52 |
+
"\n",
|
| 53 |
+
" activity_type :\n",
|
| 54 |
+
"\t hut trek (summer)\n",
|
| 55 |
+
"\t hut trek (winter)\n",
|
| 56 |
+
"\t camping trip (wild camping)\n",
|
| 57 |
+
"\t camping trip (campground)\n",
|
| 58 |
+
"\t ski tour / skitour\n",
|
| 59 |
+
"\t snowboard / splitboard trip\n",
|
| 60 |
+
"\t long-distance hike / thru-hike\n",
|
| 61 |
+
"\t digital nomad trip\n",
|
| 62 |
+
"\t city trip\n",
|
| 63 |
+
"\t road trip (car/camper)\n",
|
| 64 |
+
"\t festival trip\n",
|
| 65 |
+
"\t yoga / wellness retreat\n",
|
| 66 |
+
"\t micro-adventure / weekend trip\n",
|
| 67 |
+
"\t beach vacation\n",
|
| 68 |
+
"\t cultural exploration\n",
|
| 69 |
+
"\t nature escape\n",
|
| 70 |
+
"\n",
|
| 71 |
+
" activities :\n",
|
| 72 |
+
"\t swimming\n",
|
| 73 |
+
"\t going to the beach\n",
|
| 74 |
+
"\t relaxing\n",
|
| 75 |
+
"\t sightseeing\n",
|
| 76 |
+
"\t biking\n",
|
| 77 |
+
"\t running\n",
|
| 78 |
+
"\t skiing\n",
|
| 79 |
+
"\t cross-country skiing\n",
|
| 80 |
+
"\t ski touring\n",
|
| 81 |
+
"\t hiking\n",
|
| 82 |
+
"\t hut-to-hut hiking\n",
|
| 83 |
+
"\t rock climbing\n",
|
| 84 |
+
"\t ice climbing\n",
|
| 85 |
+
"\t snowshoe hiking\n",
|
| 86 |
+
"\t kayaking / canoeing\n",
|
| 87 |
+
"\t stand-up paddleboarding (SUP)\n",
|
| 88 |
+
"\t snorkeling\n",
|
| 89 |
+
"\t scuba diving\n",
|
| 90 |
+
"\t surfing\n",
|
| 91 |
+
"\t paragliding\n",
|
| 92 |
+
"\t horseback riding\n",
|
| 93 |
+
"\t photography\n",
|
| 94 |
+
"\t fishing\n",
|
| 95 |
+
"\t rafting\n",
|
| 96 |
+
"\t yoga\n",
|
| 97 |
+
"\n",
|
| 98 |
+
" climate_or_season :\n",
|
| 99 |
+
"\t cold destination / winter\n",
|
| 100 |
+
"\t warm destination / summer\n",
|
| 101 |
+
"\t variable weather / spring / autumn\n",
|
| 102 |
+
"\t tropical / humid\n",
|
| 103 |
+
"\t dry / desert-like\n",
|
| 104 |
+
"\t rainy climate\n",
|
| 105 |
+
"\n",
|
| 106 |
+
" style_or_comfort :\n",
|
| 107 |
+
"\t ultralight\n",
|
| 108 |
+
"\t lightweight (but comfortable)\n",
|
| 109 |
+
"\t luxury (including evening wear)\n",
|
| 110 |
+
"\t minimalist\n",
|
| 111 |
+
"\n",
|
| 112 |
+
" dress_code :\n",
|
| 113 |
+
"\t casual\n",
|
| 114 |
+
"\t formal (business trip)\n",
|
| 115 |
+
"\t conservative\n",
|
| 116 |
+
"\n",
|
| 117 |
+
" accommodation :\n",
|
| 118 |
+
"\t indoor\n",
|
| 119 |
+
"\t huts with half board\n",
|
| 120 |
+
"\t sleeping in a tent\n",
|
| 121 |
+
"\t sleeping in a car\n",
|
| 122 |
+
"\n",
|
| 123 |
+
" transportation :\n",
|
| 124 |
+
"\t own vehicle\n",
|
| 125 |
+
"\t no own vehicle\n",
|
| 126 |
+
"\n",
|
| 127 |
+
" special_conditions :\n",
|
| 128 |
+
"\t off-grid / no electricity\n",
|
| 129 |
+
"\t self-supported (bring your own cooking gear)\n",
|
| 130 |
+
"\t travel with children\n",
|
| 131 |
+
"\t pet-friendly\n",
|
| 132 |
+
"\t snow and ice\n",
|
| 133 |
+
"\t high alpine terrain\n",
|
| 134 |
+
"\t snow, ice and avalanche-prone terrain\n",
|
| 135 |
+
"\t no special conditions to consider\n",
|
| 136 |
+
"\n",
|
| 137 |
+
" trip_length_days :\n",
|
| 138 |
+
"\t 1 day\n",
|
| 139 |
+
"\t 2 days\n",
|
| 140 |
+
"\t 3 days\n",
|
| 141 |
+
"\t 4 days\n",
|
| 142 |
+
"\t 5 days\n",
|
| 143 |
+
"\t 6 days\n",
|
| 144 |
+
"\t 7 days\n",
|
| 145 |
+
"\t 7+ days\n"
|
| 146 |
+
]
|
| 147 |
+
}
|
| 148 |
+
],
|
| 149 |
+
"source": [
|
| 150 |
+
"for key in candidate_labels:\n",
|
| 151 |
+
" print(\"\\n\", key, \":\")\n",
|
| 152 |
+
" for item in candidate_labels[key]:\n",
|
| 153 |
+
" print(\"\\t\", item)\n"
|
| 154 |
+
]
|
| 155 |
+
},
|
| 156 |
+
{
|
| 157 |
+
"cell_type": "code",
|
| 158 |
+
"execution_count": 20,
|
| 159 |
+
"id": "e114d7ab-9bd3-4214-a630-7f9166c3b8a5",
|
| 160 |
+
"metadata": {},
|
| 161 |
+
"outputs": [
|
| 162 |
+
{
|
| 163 |
+
"name": "stdout",
|
| 164 |
+
"output_type": "stream",
|
| 165 |
+
"text": [
|
| 166 |
+
"activity_type\n"
|
| 167 |
+
]
|
| 168 |
+
}
|
| 169 |
+
],
|
| 170 |
+
"source": [
|
| 171 |
+
"# First classify using first dimension of classes ()\n",
|
| 172 |
+
"print(keys_list[0])"
|
| 173 |
+
]
|
| 174 |
+
},
|
| 175 |
+
{
|
| 176 |
+
"cell_type": "code",
|
| 177 |
+
"execution_count": 40,
|
| 178 |
+
"id": "4b3a1bcb-3450-4128-b941-952f145baf99",
|
| 179 |
+
"metadata": {},
|
| 180 |
+
"outputs": [
|
| 181 |
+
{
|
| 182 |
+
"name": "stderr",
|
| 183 |
+
"output_type": "stream",
|
| 184 |
+
"text": [
|
| 185 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 186 |
+
]
|
| 187 |
+
},
|
| 188 |
+
{
|
| 189 |
+
"name": "stdout",
|
| 190 |
+
"output_type": "stream",
|
| 191 |
+
"text": [
|
| 192 |
+
" Label Score\n",
|
| 193 |
+
"0 beach vacation 0.376311\n",
|
| 194 |
+
"1 micro-adventure / weekend trip 0.350168\n",
|
| 195 |
+
"2 nature escape 0.133974\n",
|
| 196 |
+
"3 digital nomad trip 0.031636\n",
|
| 197 |
+
"4 cultural exploration 0.031271\n",
|
| 198 |
+
"5 yoga / wellness retreat 0.012846\n",
|
| 199 |
+
"6 festival trip 0.012700\n",
|
| 200 |
+
"7 long-distance hike / thru-hike 0.009527\n",
|
| 201 |
+
"8 hut trek (summer) 0.008148\n",
|
| 202 |
+
"9 city trip 0.007793\n",
|
| 203 |
+
"10 road trip (car/camper) 0.006512\n",
|
| 204 |
+
"11 ski tour / skitour 0.005670\n",
|
| 205 |
+
"12 camping trip (campground) 0.004448\n",
|
| 206 |
+
"13 snowboard / splitboard trip 0.004113\n",
|
| 207 |
+
"14 camping trip (wild camping) 0.002714\n",
|
| 208 |
+
"15 hut trek (winter) 0.002170\n"
|
| 209 |
+
]
|
| 210 |
+
}
|
| 211 |
+
],
|
| 212 |
+
"source": [
|
| 213 |
+
"model_name = \"facebook/bart-large-mnli\"\n",
|
| 214 |
+
"trip_descr = \"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\"\n",
|
| 215 |
+
"classifier = pipeline(\"zero-shot-classification\", model = model_name)\n",
|
| 216 |
+
"result = classifier(trip_descr, candidate_labels[keys_list[0]])\n",
|
| 217 |
+
"# Create DataFrame\n",
|
| 218 |
+
"df = pd.DataFrame({\n",
|
| 219 |
+
" \"Label\": result[\"labels\"],\n",
|
| 220 |
+
" \"Score\": result[\"scores\"]\n",
|
| 221 |
+
"})\n",
|
| 222 |
+
"print(df)"
|
| 223 |
+
]
|
| 224 |
+
},
|
| 225 |
+
{
|
| 226 |
+
"cell_type": "code",
|
| 227 |
+
"execution_count": 25,
|
| 228 |
+
"id": "04208f9e-59bb-408b-92c6-941d064bf43d",
|
| 229 |
+
"metadata": {},
|
| 230 |
+
"outputs": [
|
| 231 |
+
{
|
| 232 |
+
"name": "stdout",
|
| 233 |
+
"output_type": "stream",
|
| 234 |
+
"text": [
|
| 235 |
+
"beach vacation\n"
|
| 236 |
+
]
|
| 237 |
+
}
|
| 238 |
+
],
|
| 239 |
+
"source": [
|
| 240 |
+
"# the labels are sorted by score. We choose the first one as our best guess for a class label\n",
|
| 241 |
+
"class_label = result[\"labels\"][0]\n",
|
| 242 |
+
"print(class_label)"
|
| 243 |
+
]
|
| 244 |
+
},
|
| 245 |
+
{
|
| 246 |
+
"cell_type": "code",
|
| 247 |
+
"execution_count": 30,
|
| 248 |
+
"id": "9f5f1c45-b411-4de1-a0a6-a7ecde5d8eae",
|
| 249 |
+
"metadata": {},
|
| 250 |
+
"outputs": [
|
| 251 |
+
{
|
| 252 |
+
"name": "stdout",
|
| 253 |
+
"output_type": "stream",
|
| 254 |
+
"text": [
|
| 255 |
+
" Label Score\n",
|
| 256 |
+
"0 beach vacation 0.376311\n",
|
| 257 |
+
"1 micro-adventure / weekend trip 0.350168\n",
|
| 258 |
+
"2 nature escape 0.133974\n",
|
| 259 |
+
"3 digital nomad trip 0.031636\n",
|
| 260 |
+
"4 cultural exploration 0.031271\n",
|
| 261 |
+
"5 yoga / wellness retreat 0.012846\n",
|
| 262 |
+
"6 festival trip 0.012700\n",
|
| 263 |
+
"7 long-distance hike / thru-hike 0.009527\n",
|
| 264 |
+
"8 hut trek (summer) 0.008148\n",
|
| 265 |
+
"9 city trip 0.007793\n",
|
| 266 |
+
"10 road trip (car/camper) 0.006512\n",
|
| 267 |
+
"11 ski tour / skitour 0.005670\n",
|
| 268 |
+
"12 camping trip (campground) 0.004448\n",
|
| 269 |
+
"13 snowboard / splitboard trip 0.004113\n",
|
| 270 |
+
"14 camping trip (wild camping) 0.002714\n",
|
| 271 |
+
"15 hut trek (winter) 0.002170\n"
|
| 272 |
+
]
|
| 273 |
+
}
|
| 274 |
+
],
|
| 275 |
+
"source": [
|
| 276 |
+
"# we do this for each superclass and receive a list of class labels for our trip. We did do things differently for activities\n",
|
| 277 |
+
"cut_off = 0.5\n",
|
| 278 |
+
"result_activ = classifier(trip_descr, candidate_labels[\"activities\"], multi_label=True)\n",
|
| 279 |
+
"indices = [i for i, score in enumerate(result_activ['scores']) if score > cut_off]\n",
|
| 280 |
+
"classes = [result_activ['labels'][i] for i in indices]\n",
|
| 281 |
+
"\n",
|
| 282 |
+
"df = pd.DataFrame({\n",
|
| 283 |
+
" \"Label\": result[\"labels\"],\n",
|
| 284 |
+
" \"Score\": result[\"scores\"]\n",
|
| 285 |
+
"})\n",
|
| 286 |
+
"print(df)"
|
| 287 |
+
]
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"cell_type": "code",
|
| 291 |
+
"execution_count": 31,
|
| 292 |
+
"id": "7c5c50c4-7d58-4f5a-8f22-e4c06298a2f7",
|
| 293 |
+
"metadata": {},
|
| 294 |
+
"outputs": [
|
| 295 |
+
{
|
| 296 |
+
"name": "stdout",
|
| 297 |
+
"output_type": "stream",
|
| 298 |
+
"text": [
|
| 299 |
+
"['going to the beach', 'relaxing', 'hiking']\n"
|
| 300 |
+
]
|
| 301 |
+
}
|
| 302 |
+
],
|
| 303 |
+
"source": [
|
| 304 |
+
"print(classes)"
|
| 305 |
+
]
|
| 306 |
+
},
|
| 307 |
+
{
|
| 308 |
+
"cell_type": "code",
|
| 309 |
+
"execution_count": 45,
|
| 310 |
+
"id": "3a7287c2-78f0-4a53-af72-1bc0f62da36f",
|
| 311 |
+
"metadata": {},
|
| 312 |
+
"outputs": [],
|
| 313 |
+
"source": [
|
| 314 |
+
"# doing this for all superclasses, depending on local machine this might take a while\n",
|
| 315 |
+
"def pred_trip(model_name, trip_descr, cut_off = 0.5):\n",
|
| 316 |
+
" \"\"\"\n",
|
| 317 |
+
" Classifies trip\n",
|
| 318 |
+
" \n",
|
| 319 |
+
" Parameters:\n",
|
| 320 |
+
" model_name: name of hugging-face model\n",
|
| 321 |
+
" trip_descr: text describing the trip\n",
|
| 322 |
+
" cut_off: cut_off for choosing activities\n",
|
| 323 |
+
"\n",
|
| 324 |
+
" Returns:\n",
|
| 325 |
+
" pd Dataframe: with class predictions and true values\n",
|
| 326 |
+
" \"\"\"\n",
|
| 327 |
+
" \n",
|
| 328 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 329 |
+
" df = pd.DataFrame(columns=['superclass', 'pred_class'])\n",
|
| 330 |
+
" for i, key in enumerate(keys_list):\n",
|
| 331 |
+
" print(f\"\\rProcessing {i + 1}/{len(keys_list)}\", end=\"\", flush=True)\n",
|
| 332 |
+
" if key == 'activities':\n",
|
| 333 |
+
" result = classifier(trip_descr, candidate_labels[key], multi_label=True)\n",
|
| 334 |
+
" indices = [i for i, score in enumerate(result['scores']) if score > cut_off]\n",
|
| 335 |
+
" classes = [result['labels'][i] for i in indices]\n",
|
| 336 |
+
" else:\n",
|
| 337 |
+
" result = classifier(trip_descr, candidate_labels[key])\n",
|
| 338 |
+
" classes = result[\"labels\"][0]\n",
|
| 339 |
+
" df.loc[i] = [key, classes]\n",
|
| 340 |
+
" return df"
|
| 341 |
+
]
|
| 342 |
+
},
|
| 343 |
+
{
|
| 344 |
+
"cell_type": "code",
|
| 345 |
+
"execution_count": 35,
|
| 346 |
+
"id": "43481d4c-039a-4a37-bd6d-dfe638bf9732",
|
| 347 |
+
"metadata": {},
|
| 348 |
+
"outputs": [
|
| 349 |
+
{
|
| 350 |
+
"name": "stderr",
|
| 351 |
+
"output_type": "stream",
|
| 352 |
+
"text": [
|
| 353 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 354 |
+
]
|
| 355 |
+
},
|
| 356 |
+
{
|
| 357 |
+
"name": "stdout",
|
| 358 |
+
"output_type": "stream",
|
| 359 |
+
"text": [
|
| 360 |
+
"Processing 8/9... superclass pred_class\n",
|
| 361 |
+
"0 activity_type beach vacation\n",
|
| 362 |
+
"1 activities [going to the beach, relaxing, hiking]\n",
|
| 363 |
+
"2 climate_or_season warm destination / summer\n",
|
| 364 |
+
"3 style_or_comfort minimalist\n",
|
| 365 |
+
"4 dress_code casual\n",
|
| 366 |
+
"5 accommodation huts with half board\n",
|
| 367 |
+
"6 transportation no own vehicle\n",
|
| 368 |
+
"7 special_conditions off-grid / no electricity\n",
|
| 369 |
+
"8 trip_length_days 7+ days\n"
|
| 370 |
+
]
|
| 371 |
+
}
|
| 372 |
+
],
|
| 373 |
+
"source": [
|
| 374 |
+
"result = pred_trip(model_name, trip_descr, cut_off = 0.5)\n",
|
| 375 |
+
"print(result)"
|
| 376 |
+
]
|
| 377 |
+
},
|
| 378 |
+
{
|
| 379 |
+
"cell_type": "markdown",
|
| 380 |
+
"id": "c4799d6b-6ab5-42da-a992-afe3666d0015",
|
| 381 |
+
"metadata": {},
|
| 382 |
+
"source": [
|
| 383 |
+
"Now use gradio app"
|
| 384 |
+
]
|
| 385 |
+
},
|
| 386 |
+
{
|
| 387 |
+
"cell_type": "code",
|
| 388 |
+
"execution_count": 37,
|
| 389 |
+
"id": "35e14ac8-4445-4586-a115-081cf1ef2686",
|
| 390 |
+
"metadata": {},
|
| 391 |
+
"outputs": [],
|
| 392 |
+
"source": [
|
| 393 |
+
"# Prerequisites\n",
|
| 394 |
+
"from transformers import pipeline\n",
|
| 395 |
+
"import json\n",
|
| 396 |
+
"import pandas as pd\n",
|
| 397 |
+
"import gradio as gr\n",
|
| 398 |
+
"\n",
|
| 399 |
+
"# get candidate labels\n",
|
| 400 |
+
"with open(\"packing_label_structure.json\", \"r\") as file:\n",
|
| 401 |
+
" candidate_labels = json.load(file)\n",
|
| 402 |
+
"keys_list = list(candidate_labels.keys())\n",
|
| 403 |
+
"\n",
|
| 404 |
+
"# Load test data (in list of dictionaries)\n",
|
| 405 |
+
"with open(\"test_data.json\", \"r\") as file:\n",
|
| 406 |
+
" packing_data = json.load(file)\n",
|
| 407 |
+
"\n",
|
| 408 |
+
"# Load packing item data\n",
|
| 409 |
+
"with open(\"packing_templates_self_supported_offgrid_expanded.json\", \"r\") as file:\n",
|
| 410 |
+
" packing_items = json.load(file)"
|
| 411 |
+
]
|
| 412 |
+
},
|
| 413 |
+
{
|
| 414 |
+
"cell_type": "code",
|
| 415 |
+
"execution_count": 39,
|
| 416 |
+
"id": "8eefd4cc-c375-4cc0-956b-472b36bafdb7",
|
| 417 |
+
"metadata": {},
|
| 418 |
+
"outputs": [
|
| 419 |
+
{
|
| 420 |
+
"name": "stdout",
|
| 421 |
+
"output_type": "stream",
|
| 422 |
+
"text": [
|
| 423 |
+
"Running on local URL: http://127.0.0.1:7861\n",
|
| 424 |
+
"\n",
|
| 425 |
+
"Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB\n",
|
| 426 |
+
"\n",
|
| 427 |
+
"To create a public link, set `share=True` in `launch()`.\n"
|
| 428 |
+
]
|
| 429 |
+
},
|
| 430 |
+
{
|
| 431 |
+
"data": {
|
| 432 |
+
"text/html": [
|
| 433 |
+
"<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
|
| 434 |
+
],
|
| 435 |
+
"text/plain": [
|
| 436 |
+
"<IPython.core.display.HTML object>"
|
| 437 |
+
]
|
| 438 |
+
},
|
| 439 |
+
"metadata": {},
|
| 440 |
+
"output_type": "display_data"
|
| 441 |
+
},
|
| 442 |
+
{
|
| 443 |
+
"name": "stderr",
|
| 444 |
+
"output_type": "stream",
|
| 445 |
+
"text": [
|
| 446 |
+
"Traceback (most recent call last):\n",
|
| 447 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 406, in hf_raise_for_status\n",
|
| 448 |
+
" response.raise_for_status()\n",
|
| 449 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/requests/models.py\", line 1024, in raise_for_status\n",
|
| 450 |
+
" raise HTTPError(http_error_msg, response=self)\n",
|
| 451 |
+
"requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json\n",
|
| 452 |
+
"\n",
|
| 453 |
+
"The above exception was the direct cause of the following exception:\n",
|
| 454 |
+
"\n",
|
| 455 |
+
"Traceback (most recent call last):\n",
|
| 456 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 403, in cached_file\n",
|
| 457 |
+
" resolved_file = hf_hub_download(\n",
|
| 458 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 459 |
+
" return fn(*args, **kwargs)\n",
|
| 460 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 860, in hf_hub_download\n",
|
| 461 |
+
" return _hf_hub_download_to_cache_dir(\n",
|
| 462 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 967, in _hf_hub_download_to_cache_dir\n",
|
| 463 |
+
" _raise_on_head_call_error(head_call_error, force_download, local_files_only)\n",
|
| 464 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1482, in _raise_on_head_call_error\n",
|
| 465 |
+
" raise head_call_error\n",
|
| 466 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1374, in _get_metadata_or_catch_error\n",
|
| 467 |
+
" metadata = get_hf_file_metadata(\n",
|
| 468 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py\", line 114, in _inner_fn\n",
|
| 469 |
+
" return fn(*args, **kwargs)\n",
|
| 470 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 1294, in get_hf_file_metadata\n",
|
| 471 |
+
" r = _request_wrapper(\n",
|
| 472 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 278, in _request_wrapper\n",
|
| 473 |
+
" response = _request_wrapper(\n",
|
| 474 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/file_download.py\", line 302, in _request_wrapper\n",
|
| 475 |
+
" hf_raise_for_status(response)\n",
|
| 476 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/huggingface_hub/utils/_http.py\", line 454, in hf_raise_for_status\n",
|
| 477 |
+
" raise _format(RepositoryNotFoundError, message, response) from e\n",
|
| 478 |
+
"huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-68fa0a8b-0d531523246f6614068c359e;d99d5b5d-694d-4d4b-b774-bb70066228d2)\n",
|
| 479 |
+
"\n",
|
| 480 |
+
"Repository Not Found for url: https://huggingface.co/MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli/resolve/main/config.json.\n",
|
| 481 |
+
"Please make sure you specified the correct `repo_id` and `repo_type`.\n",
|
| 482 |
+
"If you are trying to access a private or gated repo, make sure you are authenticated.\n",
|
| 483 |
+
"Invalid username or password.\n",
|
| 484 |
+
"\n",
|
| 485 |
+
"The above exception was the direct cause of the following exception:\n",
|
| 486 |
+
"\n",
|
| 487 |
+
"Traceback (most recent call last):\n",
|
| 488 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/queueing.py\", line 536, in process_events\n",
|
| 489 |
+
" response = await route_utils.call_process_api(\n",
|
| 490 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/route_utils.py\", line 322, in call_process_api\n",
|
| 491 |
+
" output = await app.get_blocks().process_api(\n",
|
| 492 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1935, in process_api\n",
|
| 493 |
+
" result = await self.call_function(\n",
|
| 494 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/blocks.py\", line 1520, in call_function\n",
|
| 495 |
+
" prediction = await anyio.to_thread.run_sync( # type: ignore\n",
|
| 496 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/to_thread.py\", line 56, in run_sync\n",
|
| 497 |
+
" return await get_async_backend().run_sync_in_worker_thread(\n",
|
| 498 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 2134, in run_sync_in_worker_thread\n",
|
| 499 |
+
" return await future\n",
|
| 500 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py\", line 851, in run\n",
|
| 501 |
+
" result = context.run(func, *args)\n",
|
| 502 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/gradio/utils.py\", line 826, in wrapper\n",
|
| 503 |
+
" response = f(*args, **kwargs)\n",
|
| 504 |
+
" File \"/var/folders/r5/y63bxy5d4_zfxf6cgwtwysbc0000gn/T/ipykernel_22464/1188444081.py\", line 3, in classify\n",
|
| 505 |
+
" classifier = pipeline(\"zero-shot-classification\", model=model_name)\n",
|
| 506 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/pipelines/__init__.py\", line 798, in pipeline\n",
|
| 507 |
+
" resolved_config_file = cached_file(\n",
|
| 508 |
+
" File \"/Users/anjaruetenbudde/opt/anaconda3/envs/huggingface_env/lib/python3.8/site-packages/transformers/utils/hub.py\", line 426, in cached_file\n",
|
| 509 |
+
" raise EnvironmentError(\n",
|
| 510 |
+
"OSError: MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\n",
|
| 511 |
+
"If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`\n"
|
| 512 |
+
]
|
| 513 |
+
}
|
| 514 |
+
],
|
| 515 |
+
"source": [
|
| 516 |
+
"demo = gr.Interface(\n",
|
| 517 |
+
" fn=pred_trip,\n",
|
| 518 |
+
" inputs=[\n",
|
| 519 |
+
" gr.Textbox(label=\"Model name\", value = \"MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli\"),\n",
|
| 520 |
+
" gr.Textbox(label=\"Trip description\"),\n",
|
| 521 |
+
" gr.Number(label=\"Activity cut-off\", value = 0.5),\n",
|
| 522 |
+
" ],\n",
|
| 523 |
+
" # outputs=\"dataframe\",\n",
|
| 524 |
+
" outputs=[gr.Dataframe(label=\"DataFrame\"), gr.Textbox(label=\"List of words\")],\n",
|
| 525 |
+
" title=\"Trip classification\",\n",
|
| 526 |
+
" description=\"Enter a text describing your trip\",\n",
|
| 527 |
+
")\n",
|
| 528 |
+
"\n",
|
| 529 |
+
"# Launch the Gradio app\n",
|
| 530 |
+
"if __name__ == \"__main__\":\n",
|
| 531 |
+
" demo.launch()\n"
|
| 532 |
+
]
|
| 533 |
+
},
|
| 534 |
+
{
|
| 535 |
+
"cell_type": "code",
|
| 536 |
+
"execution_count": 46,
|
| 537 |
+
"id": "11006b67-bfd5-42a7-99c4-36c3db3affac",
|
| 538 |
+
"metadata": {},
|
| 539 |
+
"outputs": [
|
| 540 |
+
{
|
| 541 |
+
"name": "stderr",
|
| 542 |
+
"output_type": "stream",
|
| 543 |
+
"text": [
|
| 544 |
+
"Hardware accelerator e.g. GPU is available in the environment, but no `device` argument is passed to the `Pipeline` object. Model will be on CPU.\n"
|
| 545 |
+
]
|
| 546 |
+
},
|
| 547 |
+
{
|
| 548 |
+
"name": "stdout",
|
| 549 |
+
"output_type": "stream",
|
| 550 |
+
"text": [
|
| 551 |
+
"Processing 9/9"
|
| 552 |
+
]
|
| 553 |
+
}
|
| 554 |
+
],
|
| 555 |
+
"source": [
|
| 556 |
+
"test = pred_trip(model_name, trip_descr, cut_off = 0.5)"
|
| 557 |
+
]
|
| 558 |
+
},
|
| 559 |
+
{
|
| 560 |
+
"cell_type": "code",
|
| 561 |
+
"execution_count": null,
|
| 562 |
+
"id": "0f6cd8d1-b742-4034-a3e2-e55c3ddd2904",
|
| 563 |
+
"metadata": {},
|
| 564 |
+
"outputs": [],
|
| 565 |
+
"source": [
|
| 566 |
+
"print(test)"
|
| 567 |
+
]
|
| 568 |
+
},
|
| 569 |
+
{
|
| 570 |
+
"cell_type": "code",
|
| 571 |
+
"execution_count": 41,
|
| 572 |
+
"id": "5db16415-47c1-42c6-861c-5d26f4b3bb03",
|
| 573 |
+
"metadata": {},
|
| 574 |
+
"outputs": [
|
| 575 |
+
{
|
| 576 |
+
"name": "stdout",
|
| 577 |
+
"output_type": "stream",
|
| 578 |
+
"text": [
|
| 579 |
+
"facebook/bart-large-mnli\n",
|
| 580 |
+
"I am planning a trip to Greece with my boyfriend, where we will visit two islands. We have booked an apartment on each island for a few days and plan to spend most of our time relaxing. Our main goals are to enjoy the beach, try delicious local food, and possibly go on a hike—if it’s not too hot. We will be relying solely on public transport. We’re in our late 20s and traveling from the Netherlands.\n"
|
| 581 |
+
]
|
| 582 |
+
}
|
| 583 |
+
],
|
| 584 |
+
"source": [
|
| 585 |
+
"print(model_name)\n",
|
| 586 |
+
"print(trip_descr)"
|
| 587 |
+
]
|
| 588 |
+
},
|
| 589 |
+
{
|
| 590 |
+
"cell_type": "code",
|
| 591 |
+
"execution_count": null,
|
| 592 |
+
"id": "bbca6bc6-bd97-4d43-8a6e-fe338f2735f2",
|
| 593 |
+
"metadata": {},
|
| 594 |
+
"outputs": [],
|
| 595 |
+
"source": []
|
| 596 |
+
}
|
| 597 |
+
],
|
| 598 |
+
"metadata": {
|
| 599 |
+
"kernelspec": {
|
| 600 |
+
"display_name": "Python (huggingface_env)",
|
| 601 |
+
"language": "python",
|
| 602 |
+
"name": "huggingface_env"
|
| 603 |
+
},
|
| 604 |
+
"language_info": {
|
| 605 |
+
"codemirror_mode": {
|
| 606 |
+
"name": "ipython",
|
| 607 |
+
"version": 3
|
| 608 |
+
},
|
| 609 |
+
"file_extension": ".py",
|
| 610 |
+
"mimetype": "text/x-python",
|
| 611 |
+
"name": "python",
|
| 612 |
+
"nbconvert_exporter": "python",
|
| 613 |
+
"pygments_lexer": "ipython3",
|
| 614 |
+
"version": "3.8.20"
|
| 615 |
+
}
|
| 616 |
+
},
|
| 617 |
+
"nbformat": 4,
|
| 618 |
+
"nbformat_minor": 5
|
| 619 |
+
}
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/packing_list_api.ipynb
CHANGED
|
@@ -10,53 +10,41 @@
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
-
"execution_count":
|
| 14 |
-
"id": "
|
| 15 |
-
"metadata": {},
|
| 16 |
-
"outputs": [],
|
| 17 |
-
"source": [
|
| 18 |
-
"from dotenv import load_dotenv\n",
|
| 19 |
-
"import os\n",
|
| 20 |
-
"import requests\n",
|
| 21 |
-
"\n",
|
| 22 |
-
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
| 23 |
-
"\n",
|
| 24 |
-
"API_URL = \"https://api-inference.huggingface.co/models/facebook/bart-large-mnli\"\n",
|
| 25 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/MoritzLaurer/mDeBERTa-v3-base-mnli-xnli\"\n",
|
| 26 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/cross-encoder/nli-deberta-v3-base\"\n",
|
| 27 |
-
"# API_URL = \"https://api-inference.huggingface.co/models/valhalla/distilbart-mnli-12-3\"\n",
|
| 28 |
-
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 29 |
-
"\n",
|
| 30 |
-
"def query(payload):\n",
|
| 31 |
-
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 32 |
-
" return response.json()\n",
|
| 33 |
-
"\n"
|
| 34 |
-
]
|
| 35 |
-
},
|
| 36 |
-
{
|
| 37 |
-
"cell_type": "code",
|
| 38 |
-
"execution_count": 3,
|
| 39 |
-
"id": "21b4f8b6-e774-45ad-8054-bf5db2b7b07c",
|
| 40 |
"metadata": {},
|
| 41 |
"outputs": [
|
| 42 |
{
|
| 43 |
"name": "stdout",
|
| 44 |
"output_type": "stream",
|
| 45 |
"text": [
|
| 46 |
-
"{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.
|
| 47 |
]
|
| 48 |
}
|
| 49 |
],
|
| 50 |
"source": [
|
| 51 |
-
"
|
| 52 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
"\n",
|
| 54 |
-
"# Candidate labels\n",
|
| 55 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 56 |
"\n",
|
| 57 |
-
"
|
| 58 |
-
"
|
| 59 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
]
|
| 61 |
},
|
| 62 |
{
|
|
|
|
| 10 |
},
|
| 11 |
{
|
| 12 |
"cell_type": "code",
|
| 13 |
+
"execution_count": 14,
|
| 14 |
+
"id": "966c6d6a-c2d7-405c-bf9b-1e1f1415234e",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
"metadata": {},
|
| 16 |
"outputs": [
|
| 17 |
{
|
| 18 |
"name": "stdout",
|
| 19 |
"output_type": "stream",
|
| 20 |
"text": [
|
| 21 |
+
"{'sequence': 'I just bought a new laptop, and it works amazing!', 'labels': ['technology', 'health', 'sports', 'politics'], 'scores': [0.970917284488678, 0.014999152161180973, 0.008272469975054264, 0.005811101291328669]}\n"
|
| 22 |
]
|
| 23 |
}
|
| 24 |
],
|
| 25 |
"source": [
|
| 26 |
+
"from dotenv import load_dotenv\n",
|
| 27 |
+
"import os\n",
|
| 28 |
+
"import requests\n",
|
| 29 |
+
"\n",
|
| 30 |
+
"load_dotenv() # Load environment variables from .env file, contains personal access token (HF_API_TOKEN=your_token)\n",
|
| 31 |
+
"headers = {\"Authorization\": f\"Bearer {os.getenv('HF_API_TOKEN')}\"}\n",
|
| 32 |
"\n",
|
|
|
|
| 33 |
"candidate_labels = [\"technology\", \"sports\", \"politics\", \"health\"]\n",
|
| 34 |
"\n",
|
| 35 |
+
"def query(model, input_text):\n",
|
| 36 |
+
" API_URL = f\"https://api-inference.huggingface.co/models/{model}\"\n",
|
| 37 |
+
" payload = {\n",
|
| 38 |
+
" \"inputs\": input_text,\n",
|
| 39 |
+
" \"parameters\": {\"candidate_labels\": candidate_labels}\n",
|
| 40 |
+
" }\n",
|
| 41 |
+
" response = requests.post(API_URL, headers=headers, json=payload)\n",
|
| 42 |
+
" return response.json()\n",
|
| 43 |
+
"\n",
|
| 44 |
+
"input_text = \"I just bought a new laptop, and it works amazing!\"\n",
|
| 45 |
+
"\n",
|
| 46 |
+
"output = query(\"facebook/bart-large-mnli\", input_text)\n",
|
| 47 |
+
"print(output)"
|
| 48 |
]
|
| 49 |
},
|
| 50 |
{
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/requirements.txt
CHANGED
|
@@ -1,3 +1,8 @@
|
|
| 1 |
transformers
|
| 2 |
-
|
| 3 |
-
torch
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
transformers
|
| 2 |
+
json
|
| 3 |
+
torch
|
| 4 |
+
numpy
|
| 5 |
+
tabulate
|
| 6 |
+
pandas
|
| 7 |
+
sklearn
|
| 8 |
+
gradio
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/packing_templates_self_supported_offgrid_expanded_dutch_original.json
ADDED
|
@@ -0,0 +1,696 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"hut trek (summer)": [
|
| 3 |
+
"lichtgewicht trekkingrugzak (30\u201345 liter)",
|
| 4 |
+
"waterzak of drinkflessen",
|
| 5 |
+
"sneldrogende kleding",
|
| 6 |
+
"regenjas",
|
| 7 |
+
"snacks/energierepen",
|
| 8 |
+
"wandelschoenen",
|
| 9 |
+
"hoed of pet",
|
| 10 |
+
"zonnebrandcr\u00e8me",
|
| 11 |
+
"lakenzak",
|
| 12 |
+
"oordoppen"
|
| 13 |
+
],
|
| 14 |
+
"hut trek (winter)": [
|
| 15 |
+
"slaapzak (lichte, warme variant)",
|
| 16 |
+
"lakenzak",
|
| 17 |
+
"binnenkleding voor hut (thermo / fleece)",
|
| 18 |
+
"hutpantoffels / Crocs",
|
| 19 |
+
"oordoppen",
|
| 20 |
+
"snacks voor onderweg",
|
| 21 |
+
"contant geld voor hutbetalingen"
|
| 22 |
+
],
|
| 23 |
+
"camping trip (wild camping)": [
|
| 24 |
+
"tent of tarp",
|
| 25 |
+
"slaapmat",
|
| 26 |
+
"slaapzak",
|
| 27 |
+
"hoofdlamp",
|
| 28 |
+
"mes of multitool",
|
| 29 |
+
"waterfilter",
|
| 30 |
+
"kookset + brander",
|
| 31 |
+
"voedsel en snacks",
|
| 32 |
+
"aansteker",
|
| 33 |
+
"toilettas met biologisch afbreekbare zeep"
|
| 34 |
+
],
|
| 35 |
+
"camping trip (campground)": [
|
| 36 |
+
"tent",
|
| 37 |
+
"kussen",
|
| 38 |
+
"stoeltje",
|
| 39 |
+
"kampeertafel (optioneel)",
|
| 40 |
+
"lamp of lantaarn",
|
| 41 |
+
"verlengsnoer (voor stroomcamping)",
|
| 42 |
+
"koelbox",
|
| 43 |
+
"servies & bestek",
|
| 44 |
+
"afwasspullen",
|
| 45 |
+
"boodschappen"
|
| 46 |
+
],
|
| 47 |
+
"skitocht / skitour": [
|
| 48 |
+
"touringski's of splitboard",
|
| 49 |
+
"stijgvellen",
|
| 50 |
+
"lawinepieper (transceiver)",
|
| 51 |
+
"schep",
|
| 52 |
+
"sonde",
|
| 53 |
+
"gore-tex jas en broek",
|
| 54 |
+
"donsjas of warme isolatielaag",
|
| 55 |
+
"helm",
|
| 56 |
+
"handschoenen (2 paar)",
|
| 57 |
+
"ski- of zonnebril",
|
| 58 |
+
"thermosfles",
|
| 59 |
+
"sneldrogende baselayers",
|
| 60 |
+
"gamaschen (bij diepe sneeuw)",
|
| 61 |
+
"tourrugzak met ski-bevestiging",
|
| 62 |
+
"laagjes voor temperatuurregeling",
|
| 63 |
+
"energierepen of sportvoeding",
|
| 64 |
+
"GPS of offline kaarten",
|
| 65 |
+
"EHBO-set",
|
| 66 |
+
"lichte handschoenen voor het stijgen"
|
| 67 |
+
],
|
| 68 |
+
"snowboard / splitboard trip": [
|
| 69 |
+
"splitboard of snowboard",
|
| 70 |
+
"lawinepieper",
|
| 71 |
+
"helm",
|
| 72 |
+
"dikke handschoenen",
|
| 73 |
+
"skibril",
|
| 74 |
+
"laagjes kleding",
|
| 75 |
+
"tourbindingen (voor splitboard)",
|
| 76 |
+
"stijgvellen",
|
| 77 |
+
"sonde en schep",
|
| 78 |
+
"lichte handschoenen voor het stijgen"
|
| 79 |
+
],
|
| 80 |
+
"long-distance hike / thru-hike": [
|
| 81 |
+
"lichtgewicht rugzak",
|
| 82 |
+
"waterfilter",
|
| 83 |
+
"voedselvoorraad",
|
| 84 |
+
"navigatie (kaart/kompas/GPS)",
|
| 85 |
+
"first aid kit",
|
| 86 |
+
"trail runners of lichte wandelschoenen",
|
| 87 |
+
"zonbescherming",
|
| 88 |
+
"regenbescherming",
|
| 89 |
+
"natte doekjes",
|
| 90 |
+
"sneldrogende handdoek"
|
| 91 |
+
],
|
| 92 |
+
"digital nomad trip": [
|
| 93 |
+
"laptop en oplader",
|
| 94 |
+
"wereldstekker",
|
| 95 |
+
"noise-cancelling koptelefoon",
|
| 96 |
+
"wifi-hotspot of lokale simkaart",
|
| 97 |
+
"powerbank",
|
| 98 |
+
"notitieboekje",
|
| 99 |
+
"lichte kleding",
|
| 100 |
+
"comfortabele rugzak of trolley",
|
| 101 |
+
"mifi-router of portable wifi-hotspot"
|
| 102 |
+
],
|
| 103 |
+
"city trip": [
|
| 104 |
+
"dagrugzak",
|
| 105 |
+
"oplader voor telefoon",
|
| 106 |
+
"stadsplattegrond / offline maps",
|
| 107 |
+
"betaalkaart / contant geld",
|
| 108 |
+
"comfortabele schoenen",
|
| 109 |
+
"waterfles",
|
| 110 |
+
"reisgids of highlightslijst",
|
| 111 |
+
"lichte jas of regenjas"
|
| 112 |
+
],
|
| 113 |
+
"road trip (car/camper)": [
|
| 114 |
+
"autopapieren",
|
| 115 |
+
"EHBO-kit",
|
| 116 |
+
"snacks en drinken",
|
| 117 |
+
"telefoonhouder / navigatie",
|
| 118 |
+
"kampeeruitrusting (indien overnachten)",
|
| 119 |
+
"reisgids of kaarten",
|
| 120 |
+
"zonnebril",
|
| 121 |
+
"stoel en tafel",
|
| 122 |
+
"vermaak (boek, muziek, spelletjes)"
|
| 123 |
+
],
|
| 124 |
+
"festivaltrip": [
|
| 125 |
+
"tent",
|
| 126 |
+
"matje en slaapzak",
|
| 127 |
+
"oordoppen",
|
| 128 |
+
"poncho of regenjas",
|
| 129 |
+
"glitter / outfit",
|
| 130 |
+
"herbruikbare beker",
|
| 131 |
+
"zonnebrand",
|
| 132 |
+
"cash / pinpas",
|
| 133 |
+
"vriendenafspraken",
|
| 134 |
+
"snacks"
|
| 135 |
+
],
|
| 136 |
+
"yoga / wellness retreat": [
|
| 137 |
+
"yogamat of yogahanddoek",
|
| 138 |
+
"comfortabele kleding",
|
| 139 |
+
"waterfles",
|
| 140 |
+
"dagboek / pen",
|
| 141 |
+
"warme trui of sjaal",
|
| 142 |
+
"sandalen",
|
| 143 |
+
"etherische olie (optioneel)",
|
| 144 |
+
"boek / meditatiemateriaal"
|
| 145 |
+
],
|
| 146 |
+
"micro-adventure / weekend trip": [
|
| 147 |
+
"dagrugzak",
|
| 148 |
+
"snacks en drinken",
|
| 149 |
+
"regenjas",
|
| 150 |
+
"warme laag",
|
| 151 |
+
"kaart of GPS",
|
| 152 |
+
"toilettasje",
|
| 153 |
+
"hoofddeksel",
|
| 154 |
+
"compacte slaapuitrusting (indien overnachting)"
|
| 155 |
+
],
|
| 156 |
+
"beach vacation": [
|
| 157 |
+
"zwemkleding",
|
| 158 |
+
"strandlaken",
|
| 159 |
+
"zonnebrandcr\u00e8me",
|
| 160 |
+
"zonnebril",
|
| 161 |
+
"hoed of pet",
|
| 162 |
+
"slippers",
|
| 163 |
+
"luchtige kleding",
|
| 164 |
+
"waterfles",
|
| 165 |
+
"strandtas",
|
| 166 |
+
"boek of e-reader"
|
| 167 |
+
],
|
| 168 |
+
"cultural exploration": [
|
| 169 |
+
"dagrugzak",
|
| 170 |
+
"oplader voor telefoon",
|
| 171 |
+
"stadsplattegrond / offline maps",
|
| 172 |
+
"betaalkaart / contant geld",
|
| 173 |
+
"comfortabele schoenen",
|
| 174 |
+
"waterfles",
|
| 175 |
+
"reisgids of highlightslijst",
|
| 176 |
+
"lichte jas of regenjas"
|
| 177 |
+
],
|
| 178 |
+
"nature escape": [
|
| 179 |
+
"dagrugzak",
|
| 180 |
+
"oplader voor telefoon",
|
| 181 |
+
"betaalkaart / contant geld",
|
| 182 |
+
"comfortabele schoenen",
|
| 183 |
+
"waterfles",
|
| 184 |
+
"downtime entertainment (e.g. book/ebook, games, laptop, journal)"
|
| 185 |
+
],
|
| 186 |
+
"swimming": [
|
| 187 |
+
"badkleding",
|
| 188 |
+
"handdoek",
|
| 189 |
+
"zonnebrandcr\u00e8me",
|
| 190 |
+
"slippers",
|
| 191 |
+
"zwembril",
|
| 192 |
+
"waterfles"
|
| 193 |
+
],
|
| 194 |
+
"going to the beach": [
|
| 195 |
+
"strandlaken",
|
| 196 |
+
"zwemkleding",
|
| 197 |
+
"zonnebrand",
|
| 198 |
+
"koeltas",
|
| 199 |
+
"strandstoel",
|
| 200 |
+
"zonnehoed"
|
| 201 |
+
],
|
| 202 |
+
"relaxing": [
|
| 203 |
+
"comfortabele kleding",
|
| 204 |
+
"boek of e-reader",
|
| 205 |
+
"zitkussen of strandmat",
|
| 206 |
+
"muziek / koptelefoon"
|
| 207 |
+
],
|
| 208 |
+
"sightseeing": [
|
| 209 |
+
"dagrugzak",
|
| 210 |
+
"waterfles",
|
| 211 |
+
"camera of smartphone",
|
| 212 |
+
"kaart of offline maps",
|
| 213 |
+
"zonnebril"
|
| 214 |
+
],
|
| 215 |
+
"biking": [
|
| 216 |
+
"fiets of huurfiets",
|
| 217 |
+
"helm",
|
| 218 |
+
"sportkleding",
|
| 219 |
+
"fietslicht en slot",
|
| 220 |
+
"reparatieset"
|
| 221 |
+
],
|
| 222 |
+
"running": [
|
| 223 |
+
"hardloopschoenen",
|
| 224 |
+
"sportkleding",
|
| 225 |
+
"zweetband of pet",
|
| 226 |
+
"waterfles of belt",
|
| 227 |
+
"sporthorloge (optioneel)"
|
| 228 |
+
],
|
| 229 |
+
"skiing": [
|
| 230 |
+
"ski\u2019s en stokken",
|
| 231 |
+
"skischoenen",
|
| 232 |
+
"helm",
|
| 233 |
+
"skibril",
|
| 234 |
+
"skipas",
|
| 235 |
+
"dikke handschoenen",
|
| 236 |
+
"gore-tex kleding"
|
| 237 |
+
],
|
| 238 |
+
"cross-country skiing": [
|
| 239 |
+
"langlaufski's en stokken",
|
| 240 |
+
"langlaufschoenen",
|
| 241 |
+
"ademende thermokleding",
|
| 242 |
+
"winddichte en waterafstotende buitenlaag",
|
| 243 |
+
"dunne handschoenen",
|
| 244 |
+
"muts of hoofdband",
|
| 245 |
+
"zonnebril of sportbril",
|
| 246 |
+
"buff of nekwarmer",
|
| 247 |
+
"lichte rugzak met water en snacks",
|
| 248 |
+
"EHBO-set"
|
| 249 |
+
],
|
| 250 |
+
"ski touring": [
|
| 251 |
+
"algemene items voor deze situatie",
|
| 252 |
+
"extra kleding of uitrusting indien nodig"
|
| 253 |
+
],
|
| 254 |
+
"hiking": [
|
| 255 |
+
"wandelschoenen of trailrunners",
|
| 256 |
+
"hiking sokken (anti-blaren)",
|
| 257 |
+
"hikingstokken",
|
| 258 |
+
"dagrugzak",
|
| 259 |
+
"regenjas of poncho",
|
| 260 |
+
"waterfles of waterzak",
|
| 261 |
+
"snacks / energierepen",
|
| 262 |
+
"zonnebrand en zonnebril",
|
| 263 |
+
"pet of hoed",
|
| 264 |
+
"blarenpleisters of tape",
|
| 265 |
+
"EHBO-set",
|
| 266 |
+
"navigatie (kaart, kompas of GPS)"
|
| 267 |
+
],
|
| 268 |
+
"hut-to-hut hiking": [
|
| 269 |
+
"lichtgewicht trekkingrugzak (30\u201345 liter)",
|
| 270 |
+
"waterflessen of waterzak",
|
| 271 |
+
"snacks voor onderweg",
|
| 272 |
+
"regenjas",
|
| 273 |
+
"licht donsjack of warme laag",
|
| 274 |
+
"sneldrogende kleding",
|
| 275 |
+
"wandelschoenen",
|
| 276 |
+
"lakenzak",
|
| 277 |
+
"pantoffels of slippers voor hut",
|
| 278 |
+
"oordoppen",
|
| 279 |
+
"reserveringsbevestiging / contant geld voor hut"
|
| 280 |
+
],
|
| 281 |
+
"rock climbing": [
|
| 282 |
+
"klimschoenen",
|
| 283 |
+
"klimgordel",
|
| 284 |
+
"zekeringsapparaat",
|
| 285 |
+
"chalk bag",
|
| 286 |
+
"helm"
|
| 287 |
+
],
|
| 288 |
+
"ice climbing": [
|
| 289 |
+
"ijsbijlen",
|
| 290 |
+
"crampons",
|
| 291 |
+
"klimtouw",
|
| 292 |
+
"warme isolatielagen",
|
| 293 |
+
"klimgordel en helm"
|
| 294 |
+
],
|
| 295 |
+
"snowshoe hiking": [
|
| 296 |
+
"algemene items voor deze situatie",
|
| 297 |
+
"extra kleding of uitrusting indien nodig"
|
| 298 |
+
],
|
| 299 |
+
"kayaking / canoeing": [
|
| 300 |
+
"kayak of kano",
|
| 301 |
+
"peddel",
|
| 302 |
+
"reddingsvest",
|
| 303 |
+
"drybag",
|
| 304 |
+
"waterschoenen"
|
| 305 |
+
],
|
| 306 |
+
"stand-up paddleboarding (SUP)": [
|
| 307 |
+
"SUP-board en peddel",
|
| 308 |
+
"reddingsvest",
|
| 309 |
+
"drybag",
|
| 310 |
+
"zonnebril met koord"
|
| 311 |
+
],
|
| 312 |
+
"snorkeling": [
|
| 313 |
+
"snorkel en duikbril",
|
| 314 |
+
"zwemvliezen",
|
| 315 |
+
"badkleding",
|
| 316 |
+
"waterdichte tas"
|
| 317 |
+
],
|
| 318 |
+
"scuba diving": [
|
| 319 |
+
"duikbril en snorkel",
|
| 320 |
+
"wetsuit",
|
| 321 |
+
"vinnen",
|
| 322 |
+
"duikcomputer",
|
| 323 |
+
"ademautomaat (indien eigen)"
|
| 324 |
+
],
|
| 325 |
+
"surfing": [
|
| 326 |
+
"surfboard",
|
| 327 |
+
"wetsuit",
|
| 328 |
+
"wax",
|
| 329 |
+
"board leash",
|
| 330 |
+
"poncho of handdoek"
|
| 331 |
+
],
|
| 332 |
+
"paragliding": [
|
| 333 |
+
"paraglider",
|
| 334 |
+
"helm",
|
| 335 |
+
"handschoenen",
|
| 336 |
+
"wandelschoenen",
|
| 337 |
+
"windjack"
|
| 338 |
+
],
|
| 339 |
+
"horseback riding": [
|
| 340 |
+
"rijbroek",
|
| 341 |
+
"rijlaarzen of schoenen met hak",
|
| 342 |
+
"handschoenen",
|
| 343 |
+
"helm",
|
| 344 |
+
"eventueel eigen zadel of stijgbeugels"
|
| 345 |
+
],
|
| 346 |
+
"photography": [
|
| 347 |
+
"camera + lenzen",
|
| 348 |
+
"statief",
|
| 349 |
+
"geheugenkaart(en)",
|
| 350 |
+
"extra accu's",
|
| 351 |
+
"lensdoekje"
|
| 352 |
+
],
|
| 353 |
+
"fishing": [
|
| 354 |
+
"hengel",
|
| 355 |
+
"aas / kunstaas",
|
| 356 |
+
"visvergunning",
|
| 357 |
+
"laarzen of waadpak",
|
| 358 |
+
"koelbox (optioneel)"
|
| 359 |
+
],
|
| 360 |
+
"rafting": [
|
| 361 |
+
"reddingsvest",
|
| 362 |
+
"waterdichte tas",
|
| 363 |
+
"waterschoenen",
|
| 364 |
+
"sneldrogende kleding",
|
| 365 |
+
"helm"
|
| 366 |
+
],
|
| 367 |
+
"yoga": [
|
| 368 |
+
"algemene items voor deze situatie",
|
| 369 |
+
"extra kleding of uitrusting indien nodig"
|
| 370 |
+
],
|
| 371 |
+
"cold destination / winter": [
|
| 372 |
+
"thermokleding",
|
| 373 |
+
"muts en handschoenen",
|
| 374 |
+
"warme jas",
|
| 375 |
+
"waterdichte schoenen",
|
| 376 |
+
"lippenbalsem"
|
| 377 |
+
],
|
| 378 |
+
"warm destination / summer": [
|
| 379 |
+
"luchtige kleding",
|
| 380 |
+
"zonnebril",
|
| 381 |
+
"zonnecr\u00e8me",
|
| 382 |
+
"waterfles",
|
| 383 |
+
"hoed of pet"
|
| 384 |
+
],
|
| 385 |
+
"variable weather / spring / autumn": [
|
| 386 |
+
"regenjas",
|
| 387 |
+
"laagjeskleding",
|
| 388 |
+
"warme trui",
|
| 389 |
+
"waterdichte schoenen"
|
| 390 |
+
],
|
| 391 |
+
"tropical / humid": [
|
| 392 |
+
"lichtgewicht kleding",
|
| 393 |
+
"insectenspray",
|
| 394 |
+
"zonnecr\u00e8me",
|
| 395 |
+
"regenponcho",
|
| 396 |
+
"sneldrogende handdoek"
|
| 397 |
+
],
|
| 398 |
+
"dry / desert-like": [
|
| 399 |
+
"zonnebril",
|
| 400 |
+
"zonnecr\u00e8me",
|
| 401 |
+
"hoofdbescherming",
|
| 402 |
+
"lichte lange mouwen",
|
| 403 |
+
"veel water"
|
| 404 |
+
],
|
| 405 |
+
"ultralight": [
|
| 406 |
+
"lichtgewicht rugzak (< 1kg)",
|
| 407 |
+
"minimalistische shelter (tarp of tent)",
|
| 408 |
+
"lichtgewicht slaapmat",
|
| 409 |
+
"quilt of donsdeken",
|
| 410 |
+
"titanium kookset",
|
| 411 |
+
"beperkte kleding (laagjes!)",
|
| 412 |
+
"compacte regenjas",
|
| 413 |
+
"sneldrogende handdoek",
|
| 414 |
+
"tandenborstel (afgezaagd ;))"
|
| 415 |
+
],
|
| 416 |
+
"lightweight (but comfortable)": [
|
| 417 |
+
"lichte tent of tarp",
|
| 418 |
+
"comfortabele slaapmat",
|
| 419 |
+
"lichtgewicht kookset",
|
| 420 |
+
"compact kledingpakket",
|
| 421 |
+
"sneldrogende handdoek"
|
| 422 |
+
],
|
| 423 |
+
"luxury (including evening wear)": [
|
| 424 |
+
"nette schoenen",
|
| 425 |
+
"jurk of overhemd",
|
| 426 |
+
"parfum/deodorant",
|
| 427 |
+
"accessoires",
|
| 428 |
+
"reistoilettas met essentials"
|
| 429 |
+
],
|
| 430 |
+
"minimalist": [
|
| 431 |
+
"1 set kleding voor elke situatie",
|
| 432 |
+
"compacte tandenborstel",
|
| 433 |
+
"alles-in-\u00e9\u00e9n zeep",
|
| 434 |
+
"lichtgewicht handdoek",
|
| 435 |
+
"kleine rugzak"
|
| 436 |
+
],
|
| 437 |
+
"casual": [
|
| 438 |
+
"jeans of comfortabele broek",
|
| 439 |
+
"t-shirts",
|
| 440 |
+
"trui of hoodie",
|
| 441 |
+
"sneakers",
|
| 442 |
+
"zonnebril"
|
| 443 |
+
],
|
| 444 |
+
"formal (business trip)": [
|
| 445 |
+
"overhemd / blouse",
|
| 446 |
+
"net jasje",
|
| 447 |
+
"nette schoenen",
|
| 448 |
+
"laptop en oplader",
|
| 449 |
+
"naamkaartjes / documenten"
|
| 450 |
+
],
|
| 451 |
+
"conservative": [
|
| 452 |
+
"bedekkende kleding",
|
| 453 |
+
"sjaal of omslagdoek",
|
| 454 |
+
"lange broek of rok",
|
| 455 |
+
"gesloten schoenen"
|
| 456 |
+
],
|
| 457 |
+
"outdoor": [
|
| 458 |
+
"regenjas",
|
| 459 |
+
"waterdichte schoenen",
|
| 460 |
+
"bivakzak of tarp",
|
| 461 |
+
"hoofdlamp",
|
| 462 |
+
"navigatie"
|
| 463 |
+
],
|
| 464 |
+
"indoor": [
|
| 465 |
+
"pantoffels",
|
| 466 |
+
"comfortabele kleding",
|
| 467 |
+
"pyjama",
|
| 468 |
+
"oplader",
|
| 469 |
+
"toilettas"
|
| 470 |
+
],
|
| 471 |
+
"huts with half board": [
|
| 472 |
+
"reserveringsbevestiging",
|
| 473 |
+
"lakenzak (vaak verplicht)",
|
| 474 |
+
"pantoffels of slippers voor binnen",
|
| 475 |
+
"lichte pyjama of slaapkleding",
|
| 476 |
+
"oorstopjes",
|
| 477 |
+
"waterfles",
|
| 478 |
+
"snacks voor onderweg",
|
| 479 |
+
"kleine toilettas",
|
| 480 |
+
"contant geld voor betalingen",
|
| 481 |
+
"lichte handdoek",
|
| 482 |
+
"zaklamp of hoofdlamp"
|
| 483 |
+
],
|
| 484 |
+
"sleeping in a tent": [
|
| 485 |
+
"tent (1- of 2-persoons, afhankelijk van reis)",
|
| 486 |
+
"grondzeil",
|
| 487 |
+
"slaapmat",
|
| 488 |
+
"slaapzak (passend bij temperatuur)",
|
| 489 |
+
"hoofdlamp of zaklamp",
|
| 490 |
+
"sneldrogende handdoek",
|
| 491 |
+
"kussen of opblaasbaar hoofdkussen",
|
| 492 |
+
"oorstopjes",
|
| 493 |
+
"toilettas",
|
| 494 |
+
"powerbank",
|
| 495 |
+
"waterfles",
|
| 496 |
+
"boek of e-reader",
|
| 497 |
+
"EHBO-set"
|
| 498 |
+
],
|
| 499 |
+
"sleeping in a car": [
|
| 500 |
+
"slaapmat die past in auto",
|
| 501 |
+
"warme slaapzak",
|
| 502 |
+
"zonneschermen of verduistering",
|
| 503 |
+
"kussen",
|
| 504 |
+
"waterfles binnen handbereik",
|
| 505 |
+
"powerbank of 12V-lader",
|
| 506 |
+
"toilettas",
|
| 507 |
+
"snacks voor de nacht",
|
| 508 |
+
"thermische deken (voor koude nachten)"
|
| 509 |
+
],
|
| 510 |
+
"own vehicle": [
|
| 511 |
+
"rijbewijs",
|
| 512 |
+
"autopapieren",
|
| 513 |
+
"EHBO-set",
|
| 514 |
+
"navigatie of smartphone",
|
| 515 |
+
"telefoonhouder"
|
| 516 |
+
],
|
| 517 |
+
"no own vehicle": [
|
| 518 |
+
"rugzak",
|
| 519 |
+
"waterfles",
|
| 520 |
+
"lichte schoenen",
|
| 521 |
+
"openbaar vervoer app of ticket"
|
| 522 |
+
],
|
| 523 |
+
"off-grid / no electricity": [
|
| 524 |
+
"powerbank (minstens 10.000 mAh)",
|
| 525 |
+
"zonnepaneel of draagbaar laadsysteem",
|
| 526 |
+
"hoofdlamp + extra batterijen",
|
| 527 |
+
"oplaadbare batterijen en oplader",
|
| 528 |
+
"back-up verlichting (bijv. kleine zaklamp)",
|
| 529 |
+
"papieren kaart en kompas",
|
| 530 |
+
"notitieboekje + pen",
|
| 531 |
+
"noodcommunicatie (bijv. GPS beacon of satellietboodschapper)",
|
| 532 |
+
"opvouwbaar zonnepaneel (indien langere tochten)",
|
| 533 |
+
"navigatieapparaat met offline kaarten",
|
| 534 |
+
"extra opladerkabels",
|
| 535 |
+
"USB-hub (voor meerdere devices)",
|
| 536 |
+
"verpakking om elektronica droog te houden"
|
| 537 |
+
],
|
| 538 |
+
"self-supported (bring your own cooking gear)": [
|
| 539 |
+
"lichtgewicht kooktoestel (gas, benzine of alcohol)",
|
| 540 |
+
"brandstof (voldoende voor aantal dagen)",
|
| 541 |
+
"pan of keteltje",
|
| 542 |
+
"spork of lepel",
|
| 543 |
+
"opvouwbaar snijplankje (optioneel)",
|
| 544 |
+
"aandrukbare kom of beker",
|
| 545 |
+
"aansteker + lucifers (waterdicht verpakt)",
|
| 546 |
+
"gedroogde of gevriesdroogde maaltijden",
|
| 547 |
+
"snacks / noodbars",
|
| 548 |
+
"afwasmiddel (biologisch afbreekbaar) + sponsje",
|
| 549 |
+
"opslagzakken voor afval",
|
| 550 |
+
"waterfilter of -pomp",
|
| 551 |
+
"chloordruppels of zuiveringstabletten",
|
| 552 |
+
"minstens 2 liter wateropslag per persoon",
|
| 553 |
+
"food bag of hangzak voor voedsel (wild-safe)"
|
| 554 |
+
],
|
| 555 |
+
"travel with children": [
|
| 556 |
+
"snacks en speelgoed",
|
| 557 |
+
"EHBO-set met pleisters",
|
| 558 |
+
"extra kleding",
|
| 559 |
+
"kinderdrager of buggy",
|
| 560 |
+
"zonbescherming",
|
| 561 |
+
"extra snacks voor kinderen",
|
| 562 |
+
"favoriet speelgoed of knuffel",
|
| 563 |
+
"reisstoel of draagzak",
|
| 564 |
+
"natte doekjes",
|
| 565 |
+
"luiers of potje (afhankelijk van leeftijd)",
|
| 566 |
+
"extra set kleding per dag",
|
| 567 |
+
"reisapotheek voor kinderen",
|
| 568 |
+
"kinderzonnebrand",
|
| 569 |
+
"activiteitenboekje of tablet met filmpjes",
|
| 570 |
+
"snelle snacks voor onderweg",
|
| 571 |
+
"kinder slaapzak of slaapmat",
|
| 572 |
+
"reisbedje of matrasje (voor jonge kinderen)",
|
| 573 |
+
"luiers of oefenbroekjes",
|
| 574 |
+
"flesjes en voeding (indien van toepassing)",
|
| 575 |
+
"babyfoon (voor verblijf op locatie)",
|
| 576 |
+
"speen of fopspeen",
|
| 577 |
+
"kinder EHBO-set (inclusief thermometer)",
|
| 578 |
+
"zonnehoedje of muts",
|
| 579 |
+
"regenhoes voor kinderwagen of draagzak",
|
| 580 |
+
"hydraterende cr\u00e8me (voor gevoelige huid)"
|
| 581 |
+
],
|
| 582 |
+
"pet-friendly": [
|
| 583 |
+
"voer en waterbak",
|
| 584 |
+
"hondenriem of tuigje",
|
| 585 |
+
"poepzakjes",
|
| 586 |
+
"reismatje of deken",
|
| 587 |
+
"vaccinatieboekje"
|
| 588 |
+
],
|
| 589 |
+
"rainy climate": [
|
| 590 |
+
"regenjas of poncho",
|
| 591 |
+
"waterdichte rugzakhoes",
|
| 592 |
+
"sneldrogende kleding",
|
| 593 |
+
"rubberen schoenen"
|
| 594 |
+
],
|
| 595 |
+
"snow and ice": [
|
| 596 |
+
"warme laarzen",
|
| 597 |
+
"dikke handschoenen",
|
| 598 |
+
"sneeuwbril",
|
| 599 |
+
"thermokleding",
|
| 600 |
+
"gripzolen / spikes"
|
| 601 |
+
],
|
| 602 |
+
"high alpine terrain": [
|
| 603 |
+
"wandelstokken",
|
| 604 |
+
"kaart en kompas",
|
| 605 |
+
"wind- en waterdichte jas",
|
| 606 |
+
"extra voeding",
|
| 607 |
+
"EHBO-kit"
|
| 608 |
+
],
|
| 609 |
+
"snow, ice and avalanche-prone terrain": [
|
| 610 |
+
"lawinepieper",
|
| 611 |
+
"schep",
|
| 612 |
+
"sonde",
|
| 613 |
+
"airbag rugzak (indien beschikbaar)",
|
| 614 |
+
"kennis van lawineveiligheid / cursus",
|
| 615 |
+
"kaart en kompas / GPS",
|
| 616 |
+
"partner check voor vertrek"
|
| 617 |
+
],
|
| 618 |
+
"no special conditions": [
|
| 619 |
+
],
|
| 620 |
+
"1 day": [
|
| 621 |
+
"ondergoed per dag",
|
| 622 |
+
"sokken per dag",
|
| 623 |
+
"toilettas",
|
| 624 |
+
"extra kledinglaag",
|
| 625 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 626 |
+
],
|
| 627 |
+
"2 days": [
|
| 628 |
+
"ondergoed per dag",
|
| 629 |
+
"sokken per dag",
|
| 630 |
+
"toilettas",
|
| 631 |
+
"extra kledinglaag",
|
| 632 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 633 |
+
],
|
| 634 |
+
"3 days": [
|
| 635 |
+
"ondergoed per dag",
|
| 636 |
+
"sokken per dag",
|
| 637 |
+
"toilettas",
|
| 638 |
+
"extra kledinglaag",
|
| 639 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 640 |
+
],
|
| 641 |
+
"4 days": [
|
| 642 |
+
"ondergoed per dag",
|
| 643 |
+
"sokken per dag",
|
| 644 |
+
"toilettas",
|
| 645 |
+
"extra kledinglaag",
|
| 646 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 647 |
+
],
|
| 648 |
+
"5 days": [
|
| 649 |
+
"ondergoed per dag",
|
| 650 |
+
"sokken per dag",
|
| 651 |
+
"toilettas",
|
| 652 |
+
"extra kledinglaag",
|
| 653 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 654 |
+
],
|
| 655 |
+
"6 days": [
|
| 656 |
+
"ondergoed per dag",
|
| 657 |
+
"sokken per dag",
|
| 658 |
+
"toilettas",
|
| 659 |
+
"extra kledinglaag",
|
| 660 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 661 |
+
],
|
| 662 |
+
"7 days": [
|
| 663 |
+
"ondergoed per dag",
|
| 664 |
+
"sokken per dag",
|
| 665 |
+
"toilettas",
|
| 666 |
+
"extra kledinglaag",
|
| 667 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 668 |
+
],
|
| 669 |
+
"7+ days": [
|
| 670 |
+
"ondergoed per dag",
|
| 671 |
+
"sokken per dag",
|
| 672 |
+
"toilettas",
|
| 673 |
+
"extra kledinglaag",
|
| 674 |
+
"aantal maaltijden/snacks afgestemd op duur"
|
| 675 |
+
],
|
| 676 |
+
"default_items": [
|
| 677 |
+
"telefoon + oplader",
|
| 678 |
+
"powerbank",
|
| 679 |
+
"identiteitsbewijs of paspoort",
|
| 680 |
+
"betaalmiddelen (pinpas / contant geld)",
|
| 681 |
+
"verzekeringspas / reisverzekering-info",
|
| 682 |
+
"toilettas (tandpasta, borstel, deodorant, zeep)",
|
| 683 |
+
"zonnebrandcr\u00e8me",
|
| 684 |
+
"lippenbalsem",
|
| 685 |
+
"ondergoed",
|
| 686 |
+
"sokken",
|
| 687 |
+
"regenjas of windjack",
|
| 688 |
+
"waterfles",
|
| 689 |
+
"snacks",
|
| 690 |
+
"zonnebril",
|
| 691 |
+
"oorstopjes",
|
| 692 |
+
"slaapmasker",
|
| 693 |
+
"herbruikbare tas",
|
| 694 |
+
"zakdoekjes of toiletpapier"
|
| 695 |
+
]
|
| 696 |
+
}
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/app-checkpoint.py
CHANGED
|
@@ -13,15 +13,14 @@ keys_list = list(candidate_labels.keys())
|
|
| 13 |
with open("test_data.json", "r") as file:
|
| 14 |
packing_data = json.load(file)
|
| 15 |
|
| 16 |
-
#
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
cut_off = 0.5 # used to choose which activities are relevant
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
# Create
|
| 25 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 26 |
for i, key in enumerate(keys_list):
|
| 27 |
if key == 'activities':
|
|
@@ -32,16 +31,31 @@ def classify(#model_name,
|
|
| 32 |
result = classifier(trip_descr, candidate_labels[key])
|
| 33 |
classes = result["labels"][0]
|
| 34 |
df.loc[i] = [key, classes]
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
demo = gr.Interface(
|
| 38 |
fn=classify,
|
| 39 |
inputs=[
|
| 40 |
-
|
| 41 |
gr.Textbox(label="Trip description"),
|
| 42 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 43 |
],
|
| 44 |
-
outputs="dataframe",
|
|
|
|
| 45 |
title="Trip classification",
|
| 46 |
description="Enter a text describing your trip",
|
| 47 |
)
|
|
|
|
| 13 |
with open("test_data.json", "r") as file:
|
| 14 |
packing_data = json.load(file)
|
| 15 |
|
| 16 |
+
# Load packing item data
|
| 17 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 18 |
+
packing_items = json.load(file)
|
|
|
|
| 19 |
|
| 20 |
+
# function and gradio app
|
| 21 |
+
def classify(model_name, trip_descr, cut_off = 0.5):
|
| 22 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 23 |
+
## Create and fill dataframe with class predictions
|
| 24 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 25 |
for i, key in enumerate(keys_list):
|
| 26 |
if key == 'activities':
|
|
|
|
| 31 |
result = classifier(trip_descr, candidate_labels[key])
|
| 32 |
classes = result["labels"][0]
|
| 33 |
df.loc[i] = [key, classes]
|
| 34 |
+
|
| 35 |
+
## Look up and return list of items to pack based on class predictions
|
| 36 |
+
# make list from dataframe column
|
| 37 |
+
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
| 38 |
+
# look up packing items for each class/key
|
| 39 |
+
list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]
|
| 40 |
+
# combine lists and remove doubble entries
|
| 41 |
+
flat_unique = []
|
| 42 |
+
for sublist in list_of_list_of_items:
|
| 43 |
+
for item in sublist:
|
| 44 |
+
if item not in flat_unique:
|
| 45 |
+
flat_unique.append(item)
|
| 46 |
+
# sort alphabetically to notice duplicates
|
| 47 |
+
sorted_list = sorted(flat_unique)
|
| 48 |
+
return df, "\n".join(sorted_list)
|
| 49 |
|
| 50 |
demo = gr.Interface(
|
| 51 |
fn=classify,
|
| 52 |
inputs=[
|
| 53 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 54 |
gr.Textbox(label="Trip description"),
|
| 55 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 56 |
],
|
| 57 |
+
# outputs="dataframe",
|
| 58 |
+
outputs=[gr.Dataframe(label="DataFrame"), gr.Textbox(label="List of words")],
|
| 59 |
title="Trip classification",
|
| 60 |
description="Enter a text describing your trip",
|
| 61 |
)
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/labels-checkpoint.txt
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
sunscreen
|
| 2 |
+
sunglasses
|
| 3 |
+
sunhat
|
| 4 |
+
flip-flops
|
| 5 |
+
swimsuit
|
| 6 |
+
beach towel
|
| 7 |
+
waterproof phone case
|
| 8 |
+
beach bag
|
| 9 |
+
snorkel gear
|
| 10 |
+
tent
|
| 11 |
+
sleeping bag
|
| 12 |
+
camping stove
|
| 13 |
+
flashlight
|
| 14 |
+
hiking boots
|
| 15 |
+
water filter
|
| 16 |
+
compass
|
| 17 |
+
first aid kit
|
| 18 |
+
bug spray
|
| 19 |
+
multi-tool
|
| 20 |
+
thermal clothing
|
| 21 |
+
ski jacket
|
| 22 |
+
ski goggles
|
| 23 |
+
snow boots
|
| 24 |
+
gloves
|
| 25 |
+
hand warmers
|
| 26 |
+
beanie
|
| 27 |
+
lip balm
|
| 28 |
+
snowboard
|
| 29 |
+
base layers
|
| 30 |
+
passport
|
| 31 |
+
visa documents
|
| 32 |
+
travel adapter
|
| 33 |
+
currency
|
| 34 |
+
sim card
|
| 35 |
+
travel pillow
|
| 36 |
+
neck wallet
|
| 37 |
+
travel insurance documents
|
| 38 |
+
power bank
|
| 39 |
+
laptop
|
| 40 |
+
business attire
|
| 41 |
+
dress shoes
|
| 42 |
+
charging cables
|
| 43 |
+
pen
|
| 44 |
+
headphones
|
| 45 |
+
lightweight backpack
|
| 46 |
+
travel-sized toiletries
|
| 47 |
+
packable rain jacket
|
| 48 |
+
dry bag
|
| 49 |
+
trekking poles
|
| 50 |
+
hostel lock
|
| 51 |
+
quick-dry towel
|
| 52 |
+
travel journal
|
| 53 |
+
snacks
|
| 54 |
+
blanket
|
| 55 |
+
emergency roadside kit
|
| 56 |
+
reusable coffee mug
|
| 57 |
+
reusable shopping bags
|
| 58 |
+
earplugs
|
| 59 |
+
fanny pack
|
| 60 |
+
poncho
|
| 61 |
+
bandana
|
| 62 |
+
comfortable shoes
|
| 63 |
+
bathing suit
|
| 64 |
+
sandals
|
| 65 |
+
light jacket
|
| 66 |
+
entertainment for downtime (e.g. book/ebook, games, laptop, journal)
|
| 67 |
+
short pants/skirts
|
| 68 |
+
t-shirts/tops
|
| 69 |
+
thin scarf
|
| 70 |
+
pants
|
| 71 |
+
shirts
|
| 72 |
+
cardigan/sweater
|
| 73 |
+
gifts
|
| 74 |
+
winter shoes
|
| 75 |
+
long pants
|
| 76 |
+
short pants
|
| 77 |
+
malaria medication
|
| 78 |
+
mosquito repellant
|
| 79 |
+
local currency
|
| 80 |
+
wallet
|
| 81 |
+
tickets
|
| 82 |
+
phone and charger
|
| 83 |
+
painkiller
|
| 84 |
+
necessary medication
|
| 85 |
+
personal toiletries (e.g. toothbrush, toothpaste)
|
| 86 |
+
underwear
|
| 87 |
+
socks
|
| 88 |
+
sleep wear
|
| 89 |
+
snacks for the journey
|
| 90 |
+
refillable water bottle
|
| 91 |
+
day pack
|
| 92 |
+
big backpack/suitcase
|
| 93 |
+
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/.ipynb_checkpoints/main_model-checkpoint.ipynb
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/app.py
CHANGED
|
@@ -13,15 +13,14 @@ keys_list = list(candidate_labels.keys())
|
|
| 13 |
with open("test_data.json", "r") as file:
|
| 14 |
packing_data = json.load(file)
|
| 15 |
|
| 16 |
-
#
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
cut_off = 0.5 # used to choose which activities are relevant
|
| 20 |
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
# Create
|
| 25 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 26 |
for i, key in enumerate(keys_list):
|
| 27 |
if key == 'activities':
|
|
@@ -32,16 +31,31 @@ def classify(#model_name,
|
|
| 32 |
result = classifier(trip_descr, candidate_labels[key])
|
| 33 |
classes = result["labels"][0]
|
| 34 |
df.loc[i] = [key, classes]
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
demo = gr.Interface(
|
| 38 |
fn=classify,
|
| 39 |
inputs=[
|
| 40 |
-
|
| 41 |
gr.Textbox(label="Trip description"),
|
| 42 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 43 |
],
|
| 44 |
-
outputs="dataframe",
|
|
|
|
| 45 |
title="Trip classification",
|
| 46 |
description="Enter a text describing your trip",
|
| 47 |
)
|
|
|
|
| 13 |
with open("test_data.json", "r") as file:
|
| 14 |
packing_data = json.load(file)
|
| 15 |
|
| 16 |
+
# Load packing item data
|
| 17 |
+
with open("packing_templates_self_supported_offgrid_expanded.json", "r") as file:
|
| 18 |
+
packing_items = json.load(file)
|
|
|
|
| 19 |
|
| 20 |
+
# function and gradio app
|
| 21 |
+
def classify(model_name, trip_descr, cut_off = 0.5):
|
| 22 |
+
classifier = pipeline("zero-shot-classification", model=model_name)
|
| 23 |
+
## Create and fill dataframe with class predictions
|
| 24 |
df = pd.DataFrame(columns=['superclass', 'pred_class'])
|
| 25 |
for i, key in enumerate(keys_list):
|
| 26 |
if key == 'activities':
|
|
|
|
| 31 |
result = classifier(trip_descr, candidate_labels[key])
|
| 32 |
classes = result["labels"][0]
|
| 33 |
df.loc[i] = [key, classes]
|
| 34 |
+
|
| 35 |
+
## Look up and return list of items to pack based on class predictions
|
| 36 |
+
# make list from dataframe column
|
| 37 |
+
all_classes = [elem for x in df["pred_class"] for elem in (x if isinstance(x, list) else [x])]
|
| 38 |
+
# look up packing items for each class/key
|
| 39 |
+
list_of_list_of_items = [packing_items.get(k, []) for k in all_classes]
|
| 40 |
+
# combine lists and remove doubble entries
|
| 41 |
+
flat_unique = []
|
| 42 |
+
for sublist in list_of_list_of_items:
|
| 43 |
+
for item in sublist:
|
| 44 |
+
if item not in flat_unique:
|
| 45 |
+
flat_unique.append(item)
|
| 46 |
+
# sort alphabetically to notice duplicates
|
| 47 |
+
sorted_list = sorted(flat_unique)
|
| 48 |
+
return df, "\n".join(sorted_list)
|
| 49 |
|
| 50 |
demo = gr.Interface(
|
| 51 |
fn=classify,
|
| 52 |
inputs=[
|
| 53 |
+
gr.Textbox(label="Model name", value = "MoritzLaurer-DeBERTa-v3-large-mnli-fever-anli-ling-wanli"),
|
| 54 |
gr.Textbox(label="Trip description"),
|
| 55 |
gr.Number(label="Activity cut-off", value = 0.5),
|
| 56 |
],
|
| 57 |
+
# outputs="dataframe",
|
| 58 |
+
outputs=[gr.Dataframe(label="DataFrame"), gr.Textbox(label="List of words")],
|
| 59 |
title="Trip classification",
|
| 60 |
description="Enter a text describing your trip",
|
| 61 |
)
|
space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/space/blog_post/article.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Working with Large Language Models
|
| 2 |
+
## Introduction: Nikky
|
| 3 |
+
* What are LLMs and why are they popular
|
| 4 |
+
* How to use the open source platform to use LLMs for own application; example of creating a packing list.
|
| 5 |
+
|
| 6 |
+
## Why bother to adapt models to your application: Nikky
|
| 7 |
+
* Problems of LLMS: Hallucination and wrong outputs
|
| 8 |
+
* Controll outputs by using zero-shot-calssification
|
| 9 |
+
* briefly mention other types of classification
|
| 10 |
+
* How we do it with our packing list model
|
| 11 |
+
* Why not use packing items as classes
|
| 12 |
+
* Use superclasses to categories trip and have packing items correspond to superclasses
|
| 13 |
+
* Asses performance with small test data set
|
| 14 |
+
* mention gradio app to make it user friendly and spaces to share model
|
| 15 |
+
|
| 16 |
+
## Implementation of packing list model
|
| 17 |
+
### Prerequisites before you start to code: Anja
|
| 18 |
+
* Hugging face account also use access token, mention api
|
| 19 |
+
* Use anaconda
|
| 20 |
+
* Install python and libraries
|
| 21 |
+
* create jupyter notebook
|
| 22 |
+
* pip install stuff
|
| 23 |
+
* Hugging face API (or mention during implementation)
|
| 24 |
+
|
| 25 |
+
### Predefine outputs/classes: Nikky
|
| 26 |
+
|
| 27 |
+
### Model implementation: Anja
|
| 28 |
+
|
| 29 |
+
### Using gradio app: Anja
|
| 30 |
+
|
| 31 |
+
### Share your model: Anja
|
| 32 |
+
* created space
|
| 33 |
+
* created github access token on huffing face and then github actions workflow
|
| 34 |
+
* you need to install the required dependencies in the environment where your Hugging Face Space is running.
|
| 35 |
+
* create requirements.txt file with necessary libraries
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
## Performance assessment: Anja
|
| 39 |
+
* Test data creation
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
## Closing
|
| 43 |
+
* Summary
|
| 44 |
+
* Limitations
|