Add note.md: architecture & operations guide for onboarding
Browse filesDocuments the live data flow (Power Automate + Office Script -> OneDrive Excel -> app HTTP GET), the DATA_URL secret, key files/functions, deploy procedure, gotchas, and a self-maintenance section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
note.md
ADDED
|
@@ -0,0 +1,190 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# note.md β Architecture & Operations (read this first)
|
| 2 |
+
|
| 3 |
+
> **Purpose:** Onboard a new AI/dev to this project fast. Keep it current β see
|
| 4 |
+
> **[Β§9 Maintaining this file](#9-maintaining-this-file)**. Last updated: **2026-05-23**.
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## 1. What this app is
|
| 9 |
+
|
| 10 |
+
**Auto Form Generator** (Thai: *ΰΈ£ΰΈ°ΰΈΰΈΰΈͺΰΈ£ΰΉΰΈ²ΰΈΰΈ«ΰΈΰΈ±ΰΈΰΈͺΰΈ·ΰΈΰΈ£ΰΈ²ΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈ±ΰΈΰΉΰΈΰΈ‘ΰΈ±ΰΈΰΈ΄*) β a Streamlit web
|
| 11 |
+
app that generates 5 standard Thai procurement letters (Word `.docx`) from templates.
|
| 12 |
+
The user types a **PO number**, the app looks it up in an Excel sheet and auto-fills
|
| 13 |
+
vendor name, job title, etc., then renders a downloadable document.
|
| 14 |
+
|
| 15 |
+
- **Live deployment:** Hugging Face Space **`Menixis/auto-form`** (private).
|
| 16 |
+
URL: https://huggingface.co/spaces/Menixis/auto-form
|
| 17 |
+
- **SDK:** Docker (see `Dockerfile`), Streamlit on **port 7860** (HF convention).
|
| 18 |
+
- **Theme:** "Forest Night" dark theme, no sidebar, inline data-source bar.
|
| 19 |
+
|
| 20 |
+
> β οΈ The user sometimes edits `streamlit_app.py` **directly in the HF web editor**, so
|
| 21 |
+
> the Space can diverge from this local clone / GitHub. **Always `git fetch origin` and
|
| 22 |
+
> compare before pushing. Never force-push.**
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
## 2. The data-flow architecture (the important part)
|
| 27 |
+
|
| 28 |
+
This is the part that took the most effort to get right. Read it carefully.
|
| 29 |
+
|
| 30 |
+
```
|
| 31 |
+
βββββββββββββββ Power Automate flow ββββββββββββββββββββββββββββ
|
| 32 |
+
β Power BI β ββ Office Script ββΆ β Excel file on the user's β
|
| 33 |
+
β "Data For β (cloud, on PC-off)β PERSONAL OneDrive β
|
| 34 |
+
β Usage" β β (pttplc-my.sharepointβ¦) β
|
| 35 |
+
β model β β shared "Anyone w/ link" β
|
| 36 |
+
βββββββββββββββ ββββββββββββββ¬ββββββββββββββ
|
| 37 |
+
β HTTP GET (anonymous link
|
| 38 |
+
β + &download=1 + browser UA)
|
| 39 |
+
βΌ
|
| 40 |
+
ββββββββββββββββββββββββββββ
|
| 41 |
+
β HF Space Menixis/auto-formβ
|
| 42 |
+
β load_live_data(DATA_URL) β
|
| 43 |
+
β β auto-clean β lookup β
|
| 44 |
+
β cached ~1h, sample fallbckβ
|
| 45 |
+
ββββββββββββββββββββββββββββ
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
**Division of labor:**
|
| 49 |
+
- **User side is automated:** a **Power Automate flow** refreshes the Power BI dataset
|
| 50 |
+
and runs an **Office Script (Excel script)** that pulls the latest data into the Excel
|
| 51 |
+
file on OneDrive. This runs in the Microsoft cloud on a schedule, so it happens **even
|
| 52 |
+
with the user's PC off** β no manual save step.
|
| 53 |
+
- **App** fetches that file live on each load. No upload step, no database, no dataset,
|
| 54 |
+
no Microsoft auth token, no GitHub Action, no IT involvement β and it works **with the
|
| 55 |
+
user's PC turned off** (the file lives in the cloud).
|
| 56 |
+
|
| 57 |
+
**Why this design (the hard constraints that drove it):** the data had to be
|
| 58 |
+
PC-off-accessible, free, with no admin/IT involvement and no fragile periodic re-login.
|
| 59 |
+
Those constraints ruled out service principals, Power Automate premium, scheduled
|
| 60 |
+
uploads, etc. The breakthrough: **PTT blocks anonymous share links on team SharePoint
|
| 61 |
+
sites but allows them on personal OneDrive** (`pttplc-my.sharepoint.com`). So the app
|
| 62 |
+
reads a personal-OneDrive anonymous link directly. Tradeoff the user accepted: anyone
|
| 63 |
+
with that obscure link can read the file.
|
| 64 |
+
|
| 65 |
+
**Data source upstream:** Power BI semantic model **"Data For Usage"**
|
| 66 |
+
(workspace `461e9fbe-d85e-4070-bf3b-526fbb127b4c`, dataset
|
| 67 |
+
`6fffb9e8-1be2-4091-874b-4aa8f04ea1c4`), table **"Data for Auto Form"**, lookup key
|
| 68 |
+
column **`poNo`**.
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
+
|
| 72 |
+
## 3. Configuration β the one secret that matters
|
| 73 |
+
|
| 74 |
+
| Where | Name | Value | Purpose |
|
| 75 |
+
|-------|------|-------|---------|
|
| 76 |
+
| HF Space β Settings β Variables and secrets | **`DATA_URL`** | the personal-OneDrive "Anyone with the link" URL to the Excel | Live data source. If unset/unreachable, app falls back to `sample_po_data.xlsx`. |
|
| 77 |
+
|
| 78 |
+
The code (`streamlit_app.py`) reads it via `os.environ.get("DATA_URL")`, auto-appends
|
| 79 |
+
`&download=1`, and sends a browser `User-Agent` (a plain GET 403s without it).
|
| 80 |
+
|
| 81 |
+
> The actual `DATA_URL` value is **not stored in this repo** β it lives only in the HF
|
| 82 |
+
> Space secret. Don't paste it into committed files.
|
| 83 |
+
|
| 84 |
+
**Legacy secrets that are now UNUSED** (safe to delete from the Space): `HF_TOKEN`.
|
| 85 |
+
Also unused: the `procurement-tools/ptt-auto-form-data` HF dataset and the
|
| 86 |
+
`menixis-data-read` HF token. These are leftovers from an earlier "upload to a private
|
| 87 |
+
HF dataset" approach that was replaced by the OneDrive-GET approach.
|
| 88 |
+
|
| 89 |
+
---
|
| 90 |
+
|
| 91 |
+
## 4. Key files
|
| 92 |
+
|
| 93 |
+
| File | Role |
|
| 94 |
+
|------|------|
|
| 95 |
+
| `streamlit_app.py` | **Main entry point.** Web UI + data loading + PO lookup. ~1127 lines. |
|
| 96 |
+
| `form_logic.py` | Shared doc-generation logic. Exports `generate_form`, `FORM_TYPES`, `DOC_TYPE_OPTIONS`, `DISPLAY_FIELDS`. |
|
| 97 |
+
| `auto_form_generator.py` | Desktop (tkinter) version β not used by the web app. |
|
| 98 |
+
| `excel_lookup.py` | Older standalone Excel lookup helper β the web app uses `lookup_po_in_bytes()` inside `streamlit_app.py`, not this. |
|
| 99 |
+
| `templates/form1-5.docx` | The 5 Word templates that get filled in. |
|
| 100 |
+
| `sample_po_data.xlsx` | Demo data + fallback when `DATA_URL` is unreachable. Demo POs: `4500987654 / 4500987655 / 4500987656`. |
|
| 101 |
+
| `Dockerfile` | HF Spaces container (python:3.11-slim, runs Streamlit on 7860). |
|
| 102 |
+
| `requirements.txt` | Deps. NOTE: lists `huggingface_hub` (now unused/legacy). `requests` is used by `load_live_data` and comes in transitively via Streamlit. |
|
| 103 |
+
| `README.md` | Has the HF YAML frontmatter (title/sdk/app_port) β **do not delete the frontmatter**, HF needs it. |
|
| 104 |
+
| `.streamlit/config.toml` | Theme + toolbar config. |
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
## 5. Key code (in `streamlit_app.py`)
|
| 109 |
+
|
| 110 |
+
Two functions carry the data layer. Both are robust against the **raw Power BI Excel
|
| 111 |
+
export format** (a banner row like "Data returned for β¦", a blank row, then headers
|
| 112 |
+
wrapped as `Data for Auto Form[poNo]`).
|
| 113 |
+
|
| 114 |
+
- **`_strip_col_wrapper(name)`** β turns `Data for Auto Form[poNo]` β `poNo`.
|
| 115 |
+
- **`lookup_po_in_bytes(po_number, file_bytes, sheet_name=None, po_column="poNo")`** β
|
| 116 |
+
scans the first 15 rows to auto-detect the header row (so banner/blank rows don't
|
| 117 |
+
break it), strips the wrappers, matches the PO (handles floats like `4.14e9` β int
|
| 118 |
+
string), and returns `(result_dict, available_cols)`. Returns `(None, cols)` if PO not
|
| 119 |
+
found, or `{"__error__": ...}` if the `poNo` column is missing.
|
| 120 |
+
- **`load_live_data(url)`** β `@st.cache_data(ttl=3600)`; GETs the OneDrive link,
|
| 121 |
+
validates the response starts with `PK` (a real `.xlsx` zip β catches login-redirect
|
| 122 |
+
HTML), returns `(bytes, fetched_at)`.
|
| 123 |
+
|
| 124 |
+
**Data-source selection** (~line 904): if the user uploaded a file this session, use it;
|
| 125 |
+
else `load_live_data(DATA_URL)`; else fall back to `sample_po_data.xlsx`. The upload
|
| 126 |
+
path is preserved as an override.
|
| 127 |
+
|
| 128 |
+
---
|
| 129 |
+
|
| 130 |
+
## 6. Deploying changes
|
| 131 |
+
|
| 132 |
+
The Space deploys on push to `main`.
|
| 133 |
+
|
| 134 |
+
```powershell
|
| 135 |
+
cd "C:\Users\640097\Documents\Applications\auto-form-generator"
|
| 136 |
+
git fetch origin # ALWAYS β the Space may have been edited in the web UI
|
| 137 |
+
git log --oneline origin/main -3 # compare; reconcile before pushing
|
| 138 |
+
# ... make edits, commit ...
|
| 139 |
+
git push origin main
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
**HF git auth:** there are no ambient credentials. The working pattern is: mint a
|
| 143 |
+
**temporary** fine-grained *write* token in the HF web UI, push using
|
| 144 |
+
`https://Menixis:<TOKEN>@huggingface.co/spaces/Menixis/auto-form`, then **revoke the
|
| 145 |
+
token** immediately after.
|
| 146 |
+
|
| 147 |
+
> β οΈ **Do not leave a token embedded in the git remote.** (As of 2026-05-23 the local
|
| 148 |
+
> `.git/config` remote URL still had a token baked in β that should be scrubbed; see
|
| 149 |
+
> Β§8.) Use a clean remote: `git remote set-url origin https://huggingface.co/spaces/Menixis/auto-form`.
|
| 150 |
+
|
| 151 |
+
Pushing to `main` is gated by Claude Code's permission classifier and needs a
|
| 152 |
+
`PowerShell(git -C "<repo>" push*)` allow rule in `~/.claude/settings.json` (already
|
| 153 |
+
added for this repo path).
|
| 154 |
+
|
| 155 |
+
---
|
| 156 |
+
|
| 157 |
+
## 7. Gotchas (things that already bit us)
|
| 158 |
+
|
| 159 |
+
1. **Don't keep this repo inside OneDrive-synced `Documents`.** Mid-session, OneDrive
|
| 160 |
+
dehydrated/removed the working folder and broke git. The folder is currently pinned
|
| 161 |
+
with `attrib +P -U`, but the real fix is to **move it out of OneDrive** (e.g. `C:\dev\`).
|
| 162 |
+
2. **Team-site SharePoint links don't work headless** β they redirect to login. Only the
|
| 163 |
+
**personal OneDrive** anonymous link works, and only with a browser `User-Agent`.
|
| 164 |
+
3. **The HF Space code can diverge from this clone** (web-UI edits). Always fetch first.
|
| 165 |
+
4. **`requirements.txt` lists `huggingface_hub`** but nothing imports it anymore β it's a
|
| 166 |
+
legacy dep. Harmless; remove only if doing a deliberate cleanup.
|
| 167 |
+
|
| 168 |
+
---
|
| 169 |
+
|
| 170 |
+
## 8. Known cleanup items (optional, none urgent)
|
| 171 |
+
|
| 172 |
+
- [ ] Scrub the embedded HF token from `.git/config` remote URL (see Β§6).
|
| 173 |
+
- [ ] Delete unused HF assets: `HF_TOKEN` Space secret, `menixis-data-read` token,
|
| 174 |
+
`procurement-tools/ptt-auto-form-data` dataset.
|
| 175 |
+
- [ ] Move this repo out of OneDrive-synced `Documents`.
|
| 176 |
+
- [ ] (Optional) drop `huggingface_hub` from `requirements.txt`.
|
| 177 |
+
|
| 178 |
+
---
|
| 179 |
+
|
| 180 |
+
## 9. Maintaining this file
|
| 181 |
+
|
| 182 |
+
Update `note.md` whenever any of these change, and bump the "Last updated" date at the top:
|
| 183 |
+
|
| 184 |
+
- The data-flow architecture or the data source (Β§2) β e.g. if the OneDrive approach is
|
| 185 |
+
ever replaced.
|
| 186 |
+
- The `DATA_URL` secret or any other config/secret (Β§3).
|
| 187 |
+
- Key files added/removed/renamed, or the data-layer functions (Β§4βΒ§5).
|
| 188 |
+
- The deploy target Space or the push procedure (Β§6).
|
| 189 |
+
- Any new gotcha worth warning the next person about (Β§7), or when a cleanup item in Β§8
|
| 190 |
+
is done (check it off / remove it).
|