Menixis Claude Opus 4.7 (1M context) commited on
Commit
d2d751b
Β·
1 Parent(s): d4ebc16

Add note.md: architecture & operations guide for onboarding

Browse files

Documents the live data flow (Power Automate + Office Script -> OneDrive Excel -> app HTTP GET), the DATA_URL secret, key files/functions, deploy procedure, gotchas, and a self-maintenance section.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. note.md +190 -0
note.md ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # note.md β€” Architecture & Operations (read this first)
2
+
3
+ > **Purpose:** Onboard a new AI/dev to this project fast. Keep it current β€” see
4
+ > **[Β§9 Maintaining this file](#9-maintaining-this-file)**. Last updated: **2026-05-23**.
5
+
6
+ ---
7
+
8
+ ## 1. What this app is
9
+
10
+ **Auto Form Generator** (Thai: *ระบบΰΈͺΰΈ£ΰΉ‰ΰΈ²ΰΈ‡ΰΈ«ΰΈ™ΰΈ±ΰΈ‡ΰΈͺΰΈ·ΰΈ­ΰΈ£ΰΈ²ΰΈŠΰΈΰΈ²ΰΈ£ΰΈ­ΰΈ±ΰΈ•ΰΉ‚ΰΈ™ΰΈ‘ΰΈ±ΰΈ•ΰΈ΄*) β€” a Streamlit web
11
+ app that generates 5 standard Thai procurement letters (Word `.docx`) from templates.
12
+ The user types a **PO number**, the app looks it up in an Excel sheet and auto-fills
13
+ vendor name, job title, etc., then renders a downloadable document.
14
+
15
+ - **Live deployment:** Hugging Face Space **`Menixis/auto-form`** (private).
16
+ URL: https://huggingface.co/spaces/Menixis/auto-form
17
+ - **SDK:** Docker (see `Dockerfile`), Streamlit on **port 7860** (HF convention).
18
+ - **Theme:** "Forest Night" dark theme, no sidebar, inline data-source bar.
19
+
20
+ > ⚠️ The user sometimes edits `streamlit_app.py` **directly in the HF web editor**, so
21
+ > the Space can diverge from this local clone / GitHub. **Always `git fetch origin` and
22
+ > compare before pushing. Never force-push.**
23
+
24
+ ---
25
+
26
+ ## 2. The data-flow architecture (the important part)
27
+
28
+ This is the part that took the most effort to get right. Read it carefully.
29
+
30
+ ```
31
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” Power Automate flow β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
32
+ β”‚ Power BI β”‚ ── Office Script ─▢ β”‚ Excel file on the user's β”‚
33
+ β”‚ "Data For β”‚ (cloud, on PC-off)β”‚ PERSONAL OneDrive β”‚
34
+ β”‚ Usage" β”‚ β”‚ (pttplc-my.sharepoint…) β”‚
35
+ β”‚ model β”‚ β”‚ shared "Anyone w/ link" β”‚
36
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
37
+ β”‚ HTTP GET (anonymous link
38
+ β”‚ + &download=1 + browser UA)
39
+ β–Ό
40
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
41
+ β”‚ HF Space Menixis/auto-formβ”‚
42
+ β”‚ load_live_data(DATA_URL) β”‚
43
+ β”‚ β†’ auto-clean β†’ lookup β”‚
44
+ β”‚ cached ~1h, sample fallbckβ”‚
45
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
46
+ ```
47
+
48
+ **Division of labor:**
49
+ - **User side is automated:** a **Power Automate flow** refreshes the Power BI dataset
50
+ and runs an **Office Script (Excel script)** that pulls the latest data into the Excel
51
+ file on OneDrive. This runs in the Microsoft cloud on a schedule, so it happens **even
52
+ with the user's PC off** β€” no manual save step.
53
+ - **App** fetches that file live on each load. No upload step, no database, no dataset,
54
+ no Microsoft auth token, no GitHub Action, no IT involvement β€” and it works **with the
55
+ user's PC turned off** (the file lives in the cloud).
56
+
57
+ **Why this design (the hard constraints that drove it):** the data had to be
58
+ PC-off-accessible, free, with no admin/IT involvement and no fragile periodic re-login.
59
+ Those constraints ruled out service principals, Power Automate premium, scheduled
60
+ uploads, etc. The breakthrough: **PTT blocks anonymous share links on team SharePoint
61
+ sites but allows them on personal OneDrive** (`pttplc-my.sharepoint.com`). So the app
62
+ reads a personal-OneDrive anonymous link directly. Tradeoff the user accepted: anyone
63
+ with that obscure link can read the file.
64
+
65
+ **Data source upstream:** Power BI semantic model **"Data For Usage"**
66
+ (workspace `461e9fbe-d85e-4070-bf3b-526fbb127b4c`, dataset
67
+ `6fffb9e8-1be2-4091-874b-4aa8f04ea1c4`), table **"Data for Auto Form"**, lookup key
68
+ column **`poNo`**.
69
+
70
+ ---
71
+
72
+ ## 3. Configuration β€” the one secret that matters
73
+
74
+ | Where | Name | Value | Purpose |
75
+ |-------|------|-------|---------|
76
+ | HF Space β†’ Settings β†’ Variables and secrets | **`DATA_URL`** | the personal-OneDrive "Anyone with the link" URL to the Excel | Live data source. If unset/unreachable, app falls back to `sample_po_data.xlsx`. |
77
+
78
+ The code (`streamlit_app.py`) reads it via `os.environ.get("DATA_URL")`, auto-appends
79
+ `&download=1`, and sends a browser `User-Agent` (a plain GET 403s without it).
80
+
81
+ > The actual `DATA_URL` value is **not stored in this repo** β€” it lives only in the HF
82
+ > Space secret. Don't paste it into committed files.
83
+
84
+ **Legacy secrets that are now UNUSED** (safe to delete from the Space): `HF_TOKEN`.
85
+ Also unused: the `procurement-tools/ptt-auto-form-data` HF dataset and the
86
+ `menixis-data-read` HF token. These are leftovers from an earlier "upload to a private
87
+ HF dataset" approach that was replaced by the OneDrive-GET approach.
88
+
89
+ ---
90
+
91
+ ## 4. Key files
92
+
93
+ | File | Role |
94
+ |------|------|
95
+ | `streamlit_app.py` | **Main entry point.** Web UI + data loading + PO lookup. ~1127 lines. |
96
+ | `form_logic.py` | Shared doc-generation logic. Exports `generate_form`, `FORM_TYPES`, `DOC_TYPE_OPTIONS`, `DISPLAY_FIELDS`. |
97
+ | `auto_form_generator.py` | Desktop (tkinter) version β€” not used by the web app. |
98
+ | `excel_lookup.py` | Older standalone Excel lookup helper β€” the web app uses `lookup_po_in_bytes()` inside `streamlit_app.py`, not this. |
99
+ | `templates/form1-5.docx` | The 5 Word templates that get filled in. |
100
+ | `sample_po_data.xlsx` | Demo data + fallback when `DATA_URL` is unreachable. Demo POs: `4500987654 / 4500987655 / 4500987656`. |
101
+ | `Dockerfile` | HF Spaces container (python:3.11-slim, runs Streamlit on 7860). |
102
+ | `requirements.txt` | Deps. NOTE: lists `huggingface_hub` (now unused/legacy). `requests` is used by `load_live_data` and comes in transitively via Streamlit. |
103
+ | `README.md` | Has the HF YAML frontmatter (title/sdk/app_port) β€” **do not delete the frontmatter**, HF needs it. |
104
+ | `.streamlit/config.toml` | Theme + toolbar config. |
105
+
106
+ ---
107
+
108
+ ## 5. Key code (in `streamlit_app.py`)
109
+
110
+ Two functions carry the data layer. Both are robust against the **raw Power BI Excel
111
+ export format** (a banner row like "Data returned for …", a blank row, then headers
112
+ wrapped as `Data for Auto Form[poNo]`).
113
+
114
+ - **`_strip_col_wrapper(name)`** β€” turns `Data for Auto Form[poNo]` β†’ `poNo`.
115
+ - **`lookup_po_in_bytes(po_number, file_bytes, sheet_name=None, po_column="poNo")`** β€”
116
+ scans the first 15 rows to auto-detect the header row (so banner/blank rows don't
117
+ break it), strips the wrappers, matches the PO (handles floats like `4.14e9` β†’ int
118
+ string), and returns `(result_dict, available_cols)`. Returns `(None, cols)` if PO not
119
+ found, or `{"__error__": ...}` if the `poNo` column is missing.
120
+ - **`load_live_data(url)`** β€” `@st.cache_data(ttl=3600)`; GETs the OneDrive link,
121
+ validates the response starts with `PK` (a real `.xlsx` zip β€” catches login-redirect
122
+ HTML), returns `(bytes, fetched_at)`.
123
+
124
+ **Data-source selection** (~line 904): if the user uploaded a file this session, use it;
125
+ else `load_live_data(DATA_URL)`; else fall back to `sample_po_data.xlsx`. The upload
126
+ path is preserved as an override.
127
+
128
+ ---
129
+
130
+ ## 6. Deploying changes
131
+
132
+ The Space deploys on push to `main`.
133
+
134
+ ```powershell
135
+ cd "C:\Users\640097\Documents\Applications\auto-form-generator"
136
+ git fetch origin # ALWAYS β€” the Space may have been edited in the web UI
137
+ git log --oneline origin/main -3 # compare; reconcile before pushing
138
+ # ... make edits, commit ...
139
+ git push origin main
140
+ ```
141
+
142
+ **HF git auth:** there are no ambient credentials. The working pattern is: mint a
143
+ **temporary** fine-grained *write* token in the HF web UI, push using
144
+ `https://Menixis:<TOKEN>@huggingface.co/spaces/Menixis/auto-form`, then **revoke the
145
+ token** immediately after.
146
+
147
+ > ⚠️ **Do not leave a token embedded in the git remote.** (As of 2026-05-23 the local
148
+ > `.git/config` remote URL still had a token baked in β€” that should be scrubbed; see
149
+ > Β§8.) Use a clean remote: `git remote set-url origin https://huggingface.co/spaces/Menixis/auto-form`.
150
+
151
+ Pushing to `main` is gated by Claude Code's permission classifier and needs a
152
+ `PowerShell(git -C "<repo>" push*)` allow rule in `~/.claude/settings.json` (already
153
+ added for this repo path).
154
+
155
+ ---
156
+
157
+ ## 7. Gotchas (things that already bit us)
158
+
159
+ 1. **Don't keep this repo inside OneDrive-synced `Documents`.** Mid-session, OneDrive
160
+ dehydrated/removed the working folder and broke git. The folder is currently pinned
161
+ with `attrib +P -U`, but the real fix is to **move it out of OneDrive** (e.g. `C:\dev\`).
162
+ 2. **Team-site SharePoint links don't work headless** β€” they redirect to login. Only the
163
+ **personal OneDrive** anonymous link works, and only with a browser `User-Agent`.
164
+ 3. **The HF Space code can diverge from this clone** (web-UI edits). Always fetch first.
165
+ 4. **`requirements.txt` lists `huggingface_hub`** but nothing imports it anymore β€” it's a
166
+ legacy dep. Harmless; remove only if doing a deliberate cleanup.
167
+
168
+ ---
169
+
170
+ ## 8. Known cleanup items (optional, none urgent)
171
+
172
+ - [ ] Scrub the embedded HF token from `.git/config` remote URL (see Β§6).
173
+ - [ ] Delete unused HF assets: `HF_TOKEN` Space secret, `menixis-data-read` token,
174
+ `procurement-tools/ptt-auto-form-data` dataset.
175
+ - [ ] Move this repo out of OneDrive-synced `Documents`.
176
+ - [ ] (Optional) drop `huggingface_hub` from `requirements.txt`.
177
+
178
+ ---
179
+
180
+ ## 9. Maintaining this file
181
+
182
+ Update `note.md` whenever any of these change, and bump the "Last updated" date at the top:
183
+
184
+ - The data-flow architecture or the data source (Β§2) β€” e.g. if the OneDrive approach is
185
+ ever replaced.
186
+ - The `DATA_URL` secret or any other config/secret (Β§3).
187
+ - Key files added/removed/renamed, or the data-layer functions (Β§4–§5).
188
+ - The deploy target Space or the push procedure (Β§6).
189
+ - Any new gotcha worth warning the next person about (Β§7), or when a cleanup item in Β§8
190
+ is done (check it off / remove it).