Bhuvandesai commited on
Commit
c935f9c
·
verified ·
1 Parent(s): 9748758

Rewrite CPU banner (T4 now paid; 5-8min load, 2-4min query) and add on-page FAQ

Browse files
Files changed (1) hide show
  1. templates/index.html +77 -7
templates/index.html CHANGED
@@ -898,6 +898,38 @@
898
  0% { transform: rotate(0deg); }
899
  100% { transform: rotate(360deg); }
900
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
901
  </style>
902
  </head>
903
  <body>
@@ -935,12 +967,12 @@
935
  gap: 0.5rem;
936
  font-size: 0.85rem;
937
  ">
938
- <span style="color: #fca5a5; font-weight: 600;">CPU Demo:</span>
939
  <span style="color: #d1d5db;">
940
- Serves a 4-bit GGUF (llama.cpp) on a free 2-vCPU CPU Space — no GPU.
941
- First load takes <strong style="color: #fbbf24;">~1-3 min</strong> (it downloads the model),
942
- then each query takes <strong style="color: #fbbf24;">~30s-2min</strong> to generate.
943
- Submit a question and wait — it will complete.
944
  </span>
945
  </div>
946
 
@@ -1056,6 +1088,44 @@
1056
 
1057
  </div>
1058
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1059
  </div>
1060
 
1061
  <!-- Fine-Tuning Console removed for deployed demo -->
@@ -1416,7 +1486,7 @@
1416
  }
1417
  }
1418
 
1419
- // Query execution — runs the fine-tuned adapter and polls until done.
1420
  // (The base-model comparison was removed from the CPU demo: it relied on the
1421
  // HF Inference API for the base model, which is unreliable here and doubled
1422
  // the work, slowing every query and making the timer appear stuck.)
@@ -1458,7 +1528,7 @@
1458
  const p = await fetch(`/api/query_result/${job_id}`);
1459
  d = await p.json();
1460
  } catch (pollErr) {
1461
- continue; // transient network hiccup — keep polling
1462
  }
1463
  if (d.status === "done") {
1464
  renderOutput(d.result, "adapter");
 
898
  0% { transform: rotate(0deg); }
899
  100% { transform: rotate(360deg); }
900
  }
901
+
902
+ /* FAQ accordion */
903
+ .faq details {
904
+ border-bottom: 1px solid rgba(255, 255, 255, 0.06);
905
+ padding: 0.7rem 0;
906
+ }
907
+ .faq details:last-child { border-bottom: none; padding-bottom: 0; }
908
+ .faq summary {
909
+ cursor: pointer;
910
+ font-weight: 600;
911
+ color: #fff;
912
+ font-size: 0.92rem;
913
+ list-style: none;
914
+ display: flex;
915
+ justify-content: space-between;
916
+ align-items: center;
917
+ gap: 1rem;
918
+ }
919
+ .faq summary::-webkit-details-marker { display: none; }
920
+ .faq summary::after {
921
+ content: "+";
922
+ color: var(--accent-cyan);
923
+ font-size: 1.2rem;
924
+ line-height: 1;
925
+ }
926
+ .faq details[open] summary::after { content: "\2013"; }
927
+ .faq p {
928
+ margin-top: 0.6rem;
929
+ color: var(--text-muted);
930
+ font-size: 0.88rem;
931
+ line-height: 1.65;
932
+ }
933
  </style>
934
  </head>
935
  <body>
 
967
  gap: 0.5rem;
968
  font-size: 0.85rem;
969
  ">
970
+ <span style="color: #fca5a5; font-weight: 600;">Heads up, this runs on CPU.</span>
971
  <span style="color: #d1d5db;">
972
+ Hugging Face's free T4 GPU is a paid tier now, so to keep this demo free it runs on a plain CPU instead.
973
+ That makes it slow. The first load takes about <strong style="color: #fbbf24;">5 to 8 minutes</strong> while it pulls down the model,
974
+ and each query then takes about <strong style="color: #fbbf24;">2 to 4 minutes</strong> to answer.
975
+ Type a question, press run, and give it a minute. It does finish.
976
  </span>
977
  </div>
978
 
 
1088
 
1089
  </div>
1090
 
1091
+ <!-- FAQ -->
1092
+ <div class="glass-card cyan">
1093
+ <div class="card-header">
1094
+ <h2 class="card-title">Good to know</h2>
1095
+ <span style="font-size: 0.8rem; color: var(--text-muted);">Questions people usually ask</span>
1096
+ </div>
1097
+ <div class="faq">
1098
+ <details open>
1099
+ <summary>Why is it so slow when the database is this small?</summary>
1100
+ <p>Honestly, the database has nothing to do with it. Running the SQL on the data takes about a millisecond. The wait is the model writing the query. This is Phi-3-mini, a 3.8 billion parameter model, and on this free Space it runs on 2 CPU cores with no GPU. For every question it reads the whole schema and then writes the SQL one token at a time, and a CPU is slow at that. So the amount of data does not change anything here. The model is what takes the time.</p>
1101
+ </details>
1102
+ <details>
1103
+ <summary>How big is the database anyway?</summary>
1104
+ <p>It is small on purpose. Five departments, 13 employees, 6 products, and 75 sales rows, so a little under 100 rows in total. The point is to watch the model write correct SQL for a schema it knows, not to query a huge warehouse.</p>
1105
+ </details>
1106
+ <details>
1107
+ <summary>Does the fine-tuning actually help?</summary>
1108
+ <p>Yes, quite a bit. On the held-out questions the fine-tuned model gets about 75 percent of queries exactly right when you run them, versus roughly 42 percent for the base model on its own. It also writes valid SQL every time.</p>
1109
+ </details>
1110
+ <details>
1111
+ <summary>How was it trained?</summary>
1112
+ <p>With QLoRA, on 50 hand-written question and SQL pairs, plus 12 kept back for testing. It only updated about 0.12 percent of the model's weights, which comes out to a 9 MB adapter file. The whole run took around 3 minutes on a 6 GB laptop GPU.</p>
1113
+ </details>
1114
+ <details>
1115
+ <summary>Why not just use a GPU?</summary>
1116
+ <p>Hugging Face used to give Spaces a free T4 GPU, but that costs money now. Running on CPU keeps the demo free, and that is the trade for the speed. On a GPU this same model answers in a second or two.</p>
1117
+ </details>
1118
+ <details>
1119
+ <summary>What is actually running here?</summary>
1120
+ <p>The fine-tuned model is merged into the base weights and quantized down to a 4-bit GGUF file, then served with llama.cpp. That file is about 2.4 GB, roughly a third of the full-size model, and it loses almost no accuracy on this task.</p>
1121
+ </details>
1122
+ <details>
1123
+ <summary>Can I point it at my own database?</summary>
1124
+ <p>Not in this demo. It was trained on this one fixed schema, the tables you see in the sidebar. A different schema would need its own round of fine-tuning, or a much larger general model.</p>
1125
+ </details>
1126
+ </div>
1127
+ </div>
1128
+
1129
  </div>
1130
 
1131
  <!-- Fine-Tuning Console removed for deployed demo -->
 
1486
  }
1487
  }
1488
 
1489
+ // Query execution. Runs the fine-tuned adapter and polls until done.
1490
  // (The base-model comparison was removed from the CPU demo: it relied on the
1491
  // HF Inference API for the base model, which is unreliable here and doubled
1492
  // the work, slowing every query and making the timer appear stuck.)
 
1528
  const p = await fetch(`/api/query_result/${job_id}`);
1529
  d = await p.json();
1530
  } catch (pollErr) {
1531
+ continue; // transient network hiccup, keep polling
1532
  }
1533
  if (d.status === "done") {
1534
  renderOutput(d.result, "adapter");