notebook.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "f12898a8d61a",
      "metadata": {},
      "source": [
        "# Chandassu: eight-class Telugu poem metre recognition\n",
        "\n",
        "The default model is the new **eight-class ordered poem CNN**. It recognizes\n",
        "ఉత్పలమాల, చంపకమాల, మత్తేభము, శార్దూలం, ఆటవెలది, తేటగీతి, కందము and సీసము.\n",
        "It predicts once from all four ordered pādas; a Seesamu prediction uses its body\n",
        "only. CPU is supported; CUDA is optional. No project checkout or login is required.\n",
        "\n",
        "**Fixed held-out benchmark:** Macro F1 94.84%, accuracy 97.90%. Ataveladi F1 is\n",
        "69.34%, the main weakness. These source-labelled tests include a historically\n",
        "inspected benchmark; they are not independent expert gold or a wholly blind test.\n",
        "Scores are **uncalibrated** and do not verify formal metre rules.\n",
        "\n",
        "Run the setup, loader, and prediction cells in order. The loader resolves `main`\n",
        "once and pins both tokenizer and model to that immutable commit. `trust_remote_code`\n",
        "loads the published, inspectable `hf_poem.py` from this repository. The optional\n",
        "original four-class example remains separate below.\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "2f83c3532128",
      "metadata": {
        "tags": [
          "dependencies"
        ]
      },
      "source": [
        "# Run once in a fresh notebook kernel. Keep Colab's preinstalled PyTorch.\n",
        "%pip install -q \"transformers==4.57.6\" \"safetensors==0.8.0\" \"huggingface-hub==0.36.2\"\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "id": "31ee51cadf3a",
      "metadata": {},
      "source": [
        "import torch\n",
        "from huggingface_hub import HfApi\n",
        "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
        "\n",
        "REPO_ID = \"sprx7767/chandassu-recognition\"\n",
        "REVISION = None  # Optional: replace with a complete 40-character release commit.\n",
        "SUBFOLDER = None  # Root defaults to eight classes; \"cnn-eight-class\" is equivalent.\n",
        "\n",
        "resolved_revision = REVISION or HfApi().model_info(REPO_ID, token=False).sha\n",
        "options = {\"revision\": resolved_revision, \"trust_remote_code\": True, \"token\": False}\n",
        "if SUBFOLDER:\n",
        "    options[\"subfolder\"] = SUBFOLDER\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(REPO_ID, **options)\n",
        "model = AutoModelForSequenceClassification.from_pretrained(REPO_ID, **options).eval()\n",
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "model = model.to(device)\n",
        "assert model.config.prediction_unit == \"poem\" and len(model.config.labels) == 8\n",
        "print(\"HF commit:\", resolved_revision)\n",
        "print(\"Device:\", device)\n",
        "print(\"Labels:\", model.config.labels)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "e2dffdfa4464",
      "metadata": {},
      "source": [
        "## Enter a complete poem\n",
        "\n",
        "Use one newline-separated string containing **four full ordered pādas**.\n",
        "For Seesamu, eight body half-lines are paired into four pādas; a 12-line input\n",
        "(eight body half-lines plus four closing geeti lines) also works, and its geeti\n",
        "is excluded. An eight-line input always means eight Seesamu body half-lines,\n",
        "not four full body pādas plus four geeti lines.\n",
        "\n",
        "The limit is 512 Unicode characters per full pāda. Input is never silently\n",
        "truncated. A batch is a list of complete poem strings, not a list of individual\n",
        "pādas. Whole-poem tensors have shape `[batch, 4, tokens]`; logits have shape\n",
        "`[batch, 8]`. There are no independent per-line scores to average.\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "adb382bb6c60",
      "metadata": {},
      "source": [
        "POEM = \"\"\"శ్రీ కైవల్య పదంబుఁ జేరుటకునై చింతించెదన్ లోక ర\n",
        "క్షైకారంభకు, భక్త పాలన కళా సంరంభకున్, దానవో\n",
        "ద్రేకస్తంభకుఁ, గేళి లోల విలసద్దృగ్జాల సంభూత నా\n",
        "నా కంజాత భవాండ కుంభకు, మహానందాంగనాడింభకున్.\"\"\"\n",
        "\n",
        "\n",
        "def predict_poems(poems):\n",
        "    inputs = tokenizer(poems, padding=True, return_tensors=\"pt\").to(device)\n",
        "    with torch.inference_mode():\n",
        "        probabilities = model(**inputs).logits.double().softmax(dim=-1).cpu()\n",
        "    results = []\n",
        "    for scores in probabilities:\n",
        "        winner = int(scores.argmax())\n",
        "        results.append({\n",
        "            \"prediction\": model.config.id2label[winner],\n",
        "            \"uncalibrated_scores\": {\n",
        "                model.config.id2label[i]: float(scores[i])\n",
        "                for i in scores.argsort(descending=True).tolist()\n",
        "            },\n",
        "        })\n",
        "    return results\n",
        "\n",
        "\n",
        "result = predict_poems(POEM)[0]\n",
        "print(\"Prediction:\", result[\"prediction\"])\n",
        "for label, score in result[\"uncalibrated_scores\"].items():\n",
        "    print(f\"{label}: {score:.2%}\")\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "id": "5b00c2754e96",
      "metadata": {},
      "source": [
        "# Optional: paste a Seesamu body as four full pādas or eight half-lines.\n",
        "# A 12-line body-plus-geeti input is supported too; the final geeti is ignored.\n",
        "SEESAMU = None\n",
        "if SEESAMU:\n",
        "    print(predict_poems(SEESAMU)[0])\n",
        "\n",
        "# For a batch, use predict_poems([POEM, ANOTHER_COMPLETE_POEM]).\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "e32c59f04656",
      "metadata": {},
      "source": [
        "## Benchmark and limits\n",
        "\n",
        "| New label | Fit | Validation | Held-out test | Test F1 |\n",
        "|---|---:|---:|---:|---:|\n",
        "| Ataveladi | 289 | 1,092 | 152 | 69.34% |\n",
        "| Tetageeti | 943 | 209 | 640 | 92.27% |\n",
        "| Kandamu | 2,571 | 721 | 1,071 | 99.63% |\n",
        "| Seesamu body | 680 | 250 | 466 | 98.50% |\n",
        "\n",
        "The complete v3 corpus has 23,029 poems: 12,058 fit, 4,131 validation, 5,282 test,\n",
        "and 1,558 reserved calibration poems (unused in this experiment). All original\n",
        "v2 roles stay frozen. Whole poems and work/edition/exact-duplicate groups stay\n",
        "together. Validation poem Macro F1 selected epoch 11; training stopped after 19.\n",
        "The CPU model has 266,056 parameters. No ByT5 retraining was performed.\n",
        "\n",
        "More diverse Ataveladi fitting data is needed. Vemana dominates its validation\n",
        "support; authors can overlap, and the exact-duplicate screen is not proof against\n",
        "fuzzy duplicates. High scores do not validate prosody or reliable rejection of\n",
        "unlisted metres. See [full reports](https://github.com/Pranav-20186017/chandassu_recognition/tree/main/results/v3).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bdddcacc85bf",
      "metadata": {},
      "source": [
        "## Optional: original four-class line models\n",
        "\n",
        "The original `cnn/` and `byt5/` packages remain available. They classify individual\n",
        "lines into the original four vruttas and average calibrated line probabilities\n",
        "for poems. Keep this calculation separate from the eight-class joint CNN above.\n",
        "ByT5 is a much larger download and is parked by default.\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "7da03b036678",
      "metadata": {},
      "source": [
        "RUN_LEGACY = False\n",
        "LEGACY_MODEL = \"cnn\"  # \"byt5\" is also retained; it downloads a much larger encoder.\n",
        "\n",
        "if RUN_LEGACY:\n",
        "    legacy_options = {\"subfolder\": LEGACY_MODEL, \"revision\": resolved_revision,\n",
        "                      \"trust_remote_code\": True, \"token\": False}\n",
        "    legacy_tokenizer = AutoTokenizer.from_pretrained(REPO_ID, **legacy_options)\n",
        "    legacy_model = AutoModelForSequenceClassification.from_pretrained(\n",
        "        REPO_ID, **legacy_options).to(device).eval()\n",
        "    lines = [line.strip() for line in POEM.splitlines() if line.strip()]\n",
        "    if len(lines) not in (1, 4):\n",
        "        raise ValueError(\"The original model accepts one line or four full pādas\")\n",
        "    legacy_inputs = legacy_tokenizer(lines, padding=True, return_tensors=\"pt\").to(device)\n",
        "    with torch.inference_mode():\n",
        "        logits = legacy_model(**legacy_inputs).logits\n",
        "        scores = (logits.double() / legacy_model.config.calibration_temperature).softmax(-1).mean(0)\n",
        "    print(\"Original four-class prediction:\", legacy_model.config.id2label[int(scores.argmax())])\n",
        "    for i, score in enumerate(scores):\n",
        "        print(f\"{legacy_model.config.id2label[i]}: {float(score):.2%}\")\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "id": "d58d1813824b",
      "metadata": {},
      "source": [
        "import platform\n",
        "import transformers\n",
        "\n",
        "print({\"python\": platform.python_version(), \"torch\": torch.__version__,\n",
        "       \"transformers\": transformers.__version__, \"huggingface_revision\": resolved_revision})\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "0208ef2a8279",
      "metadata": {},
      "source": [
        "## Sources and citation\n",
        "\n",
        "[Source and rule references](https://github.com/Pranav-20186017/chandassu_recognition/blob/main/data/SOURCES.md) ·\n",
        "[Work catalogue](https://github.com/Pranav-20186017/chandassu_recognition/blob/main/data/v3/WORKS.md) ·\n",
        "[Model card](https://huggingface.co/sprx7767/chandassu-recognition) ·\n",
        "[Project code](https://github.com/Pranav-20186017/chandassu_recognition)\n",
        "\n",
        "Author: Pranav Surampudi. Project-owned material follows the Chandassu citation\n",
        "license. Poems, editions, transcriptions and annotations retain their source\n",
        "rights. Preserve authors and source links. The examples use a pretrained\n",
        "classifier; they do not implement a formal lakshana verifier or teacher distillation.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "notebook.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}