Chat Agent: full Agent Skills library (SKILL.md anatomy)
Browse filesRebuilds the flat guardrail-string Skills feature into Anthropic's real
Agent Skills format: name + description + instructions (always injected
into the prompt), plus references/assets (indexed into the same
knowledge base the agentic search tool already searches), scripts
(stored/exported for documentation only — never executed, see
SKILLS-SECURITY.md), and evals (test cases graded by substring match or
an LLM judge). Skills export/import as real, portable SKILL.md files.
Also fixes a resilience gap found while testing live in the browser: an
indexing failure (e.g. the embeddings API being down) was crashing the
whole skill save with a 500 and losing the admin's text edits. The
skill's own fields now commit first, indexing runs in its own
try/except, and a failed index attempt rolls back cleanly so the next
save retries instead of getting stuck thinking it's already indexed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- SKILLS-SECURITY.md +58 -0
- backend/app/chat_agent_chat.py +74 -6
- backend/app/chat_agent_common.py +240 -11
- backend/app/chat_agent_ingest.py +89 -2
- backend/app/chat_agent_routes.py +167 -6
- backend/app/database.py +57 -2
- backend/app/models.py +46 -2
- frontend/src/components/settings/ChatAgentPanel.jsx +3 -137
- frontend/src/components/settings/ChatAgentSkillsTab.jsx +724 -0
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Chat Agent Skills — security notes
|
| 2 |
+
|
| 3 |
+
The Chat Agent's Skills feature follows Anthropic's Agent Skills anatomy: each skill is a
|
| 4 |
+
SKILL.md-format file with `name` + `description` + `instructions` (the "brain," always injected
|
| 5 |
+
into the agent's system prompt), plus optional `references/` (domain knowledge), `assets/`
|
| 6 |
+
(output templates), `scripts/` (mechanical helpers), and `evals/` (test cases). This document
|
| 7 |
+
covers the two things an admin authoring skills should understand before relying on them:
|
| 8 |
+
**scripts are never executed**, and **references/assets are visitor-searchable**.
|
| 9 |
+
|
| 10 |
+
## Scripts are documentation only — never executed
|
| 11 |
+
|
| 12 |
+
You can author, store, export, and import script content in a skill's Scripts section. **The
|
| 13 |
+
agent never runs this code, under any circumstances.** This is a deliberate design decision, not
|
| 14 |
+
a missing feature, for the following reasons:
|
| 15 |
+
|
| 16 |
+
- **Shared blast radius.** The Chat Agent runs in the same process and container as the rest of
|
| 17 |
+
this CRM — the same SQLite database with real customer/lead data, the same `OPENAI_API_KEY`,
|
| 18 |
+
`GOOGLE_CLIENT_SECRET`, and session-signing secret. There is no separation between "the chat
|
| 19 |
+
agent's sandbox" and "everything else this app can touch."
|
| 20 |
+
- **No execution isolation on this hosting.** A safe code-execution feature needs an isolated,
|
| 21 |
+
ephemeral, no-network/no-filesystem/no-secret-access runtime per invocation. Standard Hugging
|
| 22 |
+
Face Spaces hosting (or a single Docker container generally) doesn't provide that — there's no
|
| 23 |
+
Docker-in-Docker or on-demand sandboxed compute available to spin up per script run.
|
| 24 |
+
- **Prompt injection changes the threat model.** The chat agent is driven by messages from
|
| 25 |
+
anonymous website visitors. Even if every script *body* is admin-authored and reviewed, a
|
| 26 |
+
visitor's message can influence what the agent decides to do with a tool at runtime. A
|
| 27 |
+
developer-supervised local coding agent choosing to run a script it just wrote is a different
|
| 28 |
+
threat model from a public-facing chat endpoint where an untrusted third party is one of the
|
| 29 |
+
inputs steering the conversation. Live script execution reachable from that surface is not a
|
| 30 |
+
risk this app takes on.
|
| 31 |
+
- **A README can't retroactively secure an unsafe architecture.** If real script execution is
|
| 32 |
+
ever wanted, it needs to be built as its own isolated execution service (ephemeral sandbox,
|
| 33 |
+
no ambient secrets, strict output size/time limits, admin-only trigger) — not bolted onto the
|
| 34 |
+
existing shared process.
|
| 35 |
+
|
| 36 |
+
**If you need the agent to actually *do* something** (call an API, run a calculation, look
|
| 37 |
+
something up), the safe path already exists: it's the same pattern as the built-in
|
| 38 |
+
`search_knowledge_base` tool. Ask for a new, narrow, purpose-built tool with its own explicit
|
| 39 |
+
inputs/outputs — reviewed and wired in like any other backend change — rather than arbitrary
|
| 40 |
+
script execution.
|
| 41 |
+
|
| 42 |
+
## References and assets are visitor-searchable
|
| 43 |
+
|
| 44 |
+
Unlike scripts, a skill's **references** and **assets** *are* indexed into the same knowledge
|
| 45 |
+
base used for your Google Drive and website content, so the agentic `search_knowledge_base` tool
|
| 46 |
+
can retrieve and quote them for a visitor when relevant. Treat anything you put in a
|
| 47 |
+
reference/asset with the same review standard you'd apply to a page on your public website or a
|
| 48 |
+
document you've shared to the Drive folder the agent indexes — don't put anything there you
|
| 49 |
+
wouldn't want a visitor to potentially see quoted back to them.
|
| 50 |
+
|
| 51 |
+
## Summary
|
| 52 |
+
|
| 53 |
+
| Skill section | Injected into every reply? | Visitor-searchable via the KB? | Executed? |
|
| 54 |
+
|-----------------|:---------------------------:|:-------------------------------:|:--------------:|
|
| 55 |
+
| Instructions | Yes, always | No | N/A (a prompt) |
|
| 56 |
+
| References | No | Yes | N/A |
|
| 57 |
+
| Assets | No | Yes | N/A |
|
| 58 |
+
| Scripts | No | No | **Never** |
|
|
@@ -75,6 +75,16 @@ SEARCH_TOOL_SCHEMA = {
|
|
| 75 |
_client: Optional[OpenAI] = None
|
| 76 |
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
def _openai_client() -> OpenAI:
|
| 79 |
global _client
|
| 80 |
if _client is None:
|
|
@@ -173,13 +183,14 @@ def build_system_prompt(
|
|
| 173 |
website_urls_seen: List[str] = []
|
| 174 |
for i, c in enumerate(retrieved_chunks, start=1):
|
| 175 |
title = c.get("title") or c.get("source_ref")
|
| 176 |
-
|
|
|
|
| 177 |
url = c.get("source_ref") or ""
|
| 178 |
-
parts.append(f"[{i}]
|
| 179 |
if url and url not in website_urls_seen:
|
| 180 |
website_urls_seen.append(url)
|
| 181 |
else:
|
| 182 |
-
parts.append(f"[{i}]
|
| 183 |
|
| 184 |
if website_urls_seen:
|
| 185 |
parts += [
|
|
@@ -282,14 +293,71 @@ def _format_tool_results(chunks: List[Dict[str, Any]]) -> str:
|
|
| 282 |
parts: List[str] = []
|
| 283 |
for i, c in enumerate(chunks, start=1):
|
| 284 |
title = c.get("title") or c.get("source_ref")
|
| 285 |
-
|
| 286 |
-
|
|
|
|
| 287 |
else:
|
| 288 |
-
label = f'
|
| 289 |
parts.append(f"[{i}] {label}\n{(c.get('text') or '')[:1400]}")
|
| 290 |
return "\n\n".join(parts)
|
| 291 |
|
| 292 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 293 |
def stream_agentic_reply_openai(
|
| 294 |
system_prompt: str,
|
| 295 |
history: List[Dict[str, str]],
|
|
|
|
| 75 |
_client: Optional[OpenAI] = None
|
| 76 |
|
| 77 |
|
| 78 |
+
def _source_label(source_type: Optional[str], title: str) -> str:
|
| 79 |
+
if source_type == "website":
|
| 80 |
+
return f'Website page "{title}"'
|
| 81 |
+
if source_type == "skill_reference":
|
| 82 |
+
return f'Skill reference "{title}"'
|
| 83 |
+
if source_type == "skill_asset":
|
| 84 |
+
return f'Skill asset/template "{title}"'
|
| 85 |
+
return f'Internal document "{title}"'
|
| 86 |
+
|
| 87 |
+
|
| 88 |
def _openai_client() -> OpenAI:
|
| 89 |
global _client
|
| 90 |
if _client is None:
|
|
|
|
| 183 |
website_urls_seen: List[str] = []
|
| 184 |
for i, c in enumerate(retrieved_chunks, start=1):
|
| 185 |
title = c.get("title") or c.get("source_ref")
|
| 186 |
+
source_type = c.get("source_type")
|
| 187 |
+
if source_type == "website":
|
| 188 |
url = c.get("source_ref") or ""
|
| 189 |
+
parts.append(f"[{i}] {_source_label(source_type, title)} ({url})\n{c.get('text', '')[:1400]}")
|
| 190 |
if url and url not in website_urls_seen:
|
| 191 |
website_urls_seen.append(url)
|
| 192 |
else:
|
| 193 |
+
parts.append(f"[{i}] {_source_label(source_type, title)} (no public URL)\n{c.get('text', '')[:1400]}")
|
| 194 |
|
| 195 |
if website_urls_seen:
|
| 196 |
parts += [
|
|
|
|
| 293 |
parts: List[str] = []
|
| 294 |
for i, c in enumerate(chunks, start=1):
|
| 295 |
title = c.get("title") or c.get("source_ref")
|
| 296 |
+
source_type = c.get("source_type")
|
| 297 |
+
if source_type == "website":
|
| 298 |
+
label = f'{_source_label(source_type, title)} ({c.get("source_ref")})'
|
| 299 |
else:
|
| 300 |
+
label = f'{_source_label(source_type, title)} (no public URL)'
|
| 301 |
parts.append(f"[{i}] {label}\n{(c.get('text') or '')[:1400]}")
|
| 302 |
return "\n\n".join(parts)
|
| 303 |
|
| 304 |
|
| 305 |
+
def llm_judge(prompt: str, response: str, assertion_value: str) -> Dict[str, str]:
|
| 306 |
+
"""Cheap LLM-graded eval assertion: does `response` satisfy the natural-language rubric in
|
| 307 |
+
`assertion_value`, given the original `prompt`? Same lightweight-call pattern as
|
| 308 |
+
extract_visitor_name above — not part of the live chat path, only used by the admin 'Run
|
| 309 |
+
evals' action on a skill.
|
| 310 |
+
"""
|
| 311 |
+
try:
|
| 312 |
+
resp = _openai_client().chat.completions.create(
|
| 313 |
+
model=CHAT_MODEL_OPENAI,
|
| 314 |
+
messages=[
|
| 315 |
+
{
|
| 316 |
+
"role": "system",
|
| 317 |
+
"content": (
|
| 318 |
+
"You are grading whether a chat assistant's response satisfies a rubric. "
|
| 319 |
+
"Reply with exactly PASS or FAIL on the first line, then one short "
|
| 320 |
+
"sentence explaining why."
|
| 321 |
+
),
|
| 322 |
+
},
|
| 323 |
+
{
|
| 324 |
+
"role": "user",
|
| 325 |
+
"content": (
|
| 326 |
+
f"Visitor asked: {prompt}\n\nAssistant replied: {response}\n\n"
|
| 327 |
+
f"Rubric: {assertion_value}"
|
| 328 |
+
),
|
| 329 |
+
},
|
| 330 |
+
],
|
| 331 |
+
temperature=0,
|
| 332 |
+
max_tokens=100,
|
| 333 |
+
)
|
| 334 |
+
text = (resp.choices[0].message.content or "").strip()
|
| 335 |
+
first_line, _, rest = text.partition("\n")
|
| 336 |
+
passed = first_line.strip().upper().startswith("PASS")
|
| 337 |
+
detail = rest.strip() or first_line.strip()
|
| 338 |
+
return {"status": "pass" if passed else "fail", "detail": detail[:500]}
|
| 339 |
+
except Exception as e:
|
| 340 |
+
logger.warning("llm_judge grading failed: %s", e)
|
| 341 |
+
return {"status": "error", "detail": str(e)[:500]}
|
| 342 |
+
|
| 343 |
+
|
| 344 |
+
def run_eval_assertion(prompt: str, response: str, assertion_type: str, assertion_value: str) -> Dict[str, str]:
|
| 345 |
+
"""Grade one eval case's response against its assertion. must_contain/must_not_contain are
|
| 346 |
+
plain case-insensitive substring checks; llm_judge delegates to the grading call above."""
|
| 347 |
+
at = (assertion_type or "").strip()
|
| 348 |
+
av = assertion_value or ""
|
| 349 |
+
resp_lower = (response or "").lower()
|
| 350 |
+
if at == "must_contain":
|
| 351 |
+
passed = av.lower() in resp_lower
|
| 352 |
+
return {"status": "pass" if passed else "fail", "detail": f'Looked for "{av}" in the response.'}
|
| 353 |
+
if at == "must_not_contain":
|
| 354 |
+
passed = av.lower() not in resp_lower
|
| 355 |
+
return {"status": "pass" if passed else "fail", "detail": f'Checked the response did not contain "{av}".'}
|
| 356 |
+
if at == "llm_judge":
|
| 357 |
+
return llm_judge(prompt, response, av)
|
| 358 |
+
return {"status": "error", "detail": f"Unknown assertion_type: {at}"}
|
| 359 |
+
|
| 360 |
+
|
| 361 |
def stream_agentic_reply_openai(
|
| 362 |
system_prompt: str,
|
| 363 |
history: List[Dict[str, str]],
|
|
@@ -4,6 +4,7 @@ ingestion modules, the chat engine, and the routes module — kept separate to a
|
|
| 4 |
|
| 5 |
from __future__ import annotations
|
| 6 |
|
|
|
|
| 7 |
import uuid
|
| 8 |
from typing import Any, List, Optional
|
| 9 |
|
|
@@ -12,10 +13,22 @@ from sqlalchemy.orm import Session
|
|
| 12 |
|
| 13 |
from .database import ChatAgentConfig, ChatAgentSkill
|
| 14 |
|
| 15 |
-
DEFAULT_SKILLS: List[
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
]
|
| 20 |
|
| 21 |
|
|
@@ -51,11 +64,14 @@ def ensure_default_skills(db: Session, tenant_id: int) -> None:
|
|
| 51 |
has_any = db.query(ChatAgentSkill).filter(ChatAgentSkill.tenant_id == tenant_id).first()
|
| 52 |
if has_any:
|
| 53 |
return
|
| 54 |
-
for i,
|
| 55 |
db.add(
|
| 56 |
ChatAgentSkill(
|
| 57 |
tenant_id=tenant_id,
|
| 58 |
-
text=
|
|
|
|
|
|
|
|
|
|
| 59 |
is_default=1,
|
| 60 |
is_active=1,
|
| 61 |
sort_order=i,
|
|
@@ -65,7 +81,9 @@ def ensure_default_skills(db: Session, tenant_id: int) -> None:
|
|
| 65 |
|
| 66 |
|
| 67 |
def compile_skills_md(db: Session, tenant_id: int) -> str:
|
| 68 |
-
"""Compile active skills into a 'Skills.md'-style block
|
|
|
|
|
|
|
| 69 |
rows = (
|
| 70 |
db.query(ChatAgentSkill)
|
| 71 |
.filter(ChatAgentSkill.tenant_id == tenant_id, ChatAgentSkill.is_active == 1)
|
|
@@ -76,10 +94,221 @@ def compile_skills_md(db: Session, tenant_id: int) -> str:
|
|
| 76 |
return ""
|
| 77 |
lines = ["# Skills.md — Agent behavior rules (non-negotiable)", ""]
|
| 78 |
for r in rows:
|
| 79 |
-
|
| 80 |
-
if
|
| 81 |
-
|
| 82 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
|
| 85 |
def config_to_dict(row: Optional[ChatAgentConfig]) -> dict:
|
|
|
|
| 4 |
|
| 5 |
from __future__ import annotations
|
| 6 |
|
| 7 |
+
import re
|
| 8 |
import uuid
|
| 9 |
from typing import Any, List, Optional
|
| 10 |
|
|
|
|
| 13 |
|
| 14 |
from .database import ChatAgentConfig, ChatAgentSkill
|
| 15 |
|
| 16 |
+
DEFAULT_SKILLS: List[dict] = [
|
| 17 |
+
{
|
| 18 |
+
"name": "no-internal-pricing",
|
| 19 |
+
"description": "Keep pricing conversations limited to what's already public.",
|
| 20 |
+
"instructions": "Don't share internal pricing details unless they are already published on the website.",
|
| 21 |
+
},
|
| 22 |
+
{
|
| 23 |
+
"name": "no-client-names",
|
| 24 |
+
"description": "Protect customer confidentiality unless a public case study already names them.",
|
| 25 |
+
"instructions": "Don't share existing client/customer names or details unless the website already has a public case study naming that client.",
|
| 26 |
+
},
|
| 27 |
+
{
|
| 28 |
+
"name": "no-sensitive-info",
|
| 29 |
+
"description": "Refuse to reveal sensitive, private, or internal-only information to visitors.",
|
| 30 |
+
"instructions": "Never reveal sensitive or private information (personal data, credentials, internal-only documents) to website visitors — always err on the side of caution.",
|
| 31 |
+
},
|
| 32 |
]
|
| 33 |
|
| 34 |
|
|
|
|
| 64 |
has_any = db.query(ChatAgentSkill).filter(ChatAgentSkill.tenant_id == tenant_id).first()
|
| 65 |
if has_any:
|
| 66 |
return
|
| 67 |
+
for i, skill in enumerate(DEFAULT_SKILLS):
|
| 68 |
db.add(
|
| 69 |
ChatAgentSkill(
|
| 70 |
tenant_id=tenant_id,
|
| 71 |
+
text=skill["instructions"],
|
| 72 |
+
name=skill["name"],
|
| 73 |
+
description=skill["description"],
|
| 74 |
+
instructions=skill["instructions"],
|
| 75 |
is_default=1,
|
| 76 |
is_active=1,
|
| 77 |
sort_order=i,
|
|
|
|
| 81 |
|
| 82 |
|
| 83 |
def compile_skills_md(db: Session, tenant_id: int) -> str:
|
| 84 |
+
"""Compile all active skills' instructions into a merged 'Skills.md'-style block that's
|
| 85 |
+
always injected into the live chat system prompt. Each skill also has its own standalone,
|
| 86 |
+
exportable SKILL.md rendered by compile_skill_md() below."""
|
| 87 |
rows = (
|
| 88 |
db.query(ChatAgentSkill)
|
| 89 |
.filter(ChatAgentSkill.tenant_id == tenant_id, ChatAgentSkill.is_active == 1)
|
|
|
|
| 94 |
return ""
|
| 95 |
lines = ["# Skills.md — Agent behavior rules (non-negotiable)", ""]
|
| 96 |
for r in rows:
|
| 97 |
+
instructions = safe_str(r.instructions) or safe_str(r.text)
|
| 98 |
+
if not instructions:
|
| 99 |
+
continue
|
| 100 |
+
name = safe_str(r.name)
|
| 101 |
+
if name:
|
| 102 |
+
lines.append(f"## {name}")
|
| 103 |
+
description = safe_str(r.description)
|
| 104 |
+
if description:
|
| 105 |
+
lines.append(f"_{description}_")
|
| 106 |
+
lines.append("")
|
| 107 |
+
lines.append(instructions)
|
| 108 |
+
lines.append("")
|
| 109 |
+
return "\n".join(lines).rstrip()
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
# ---- SKILL.md compile / parse (Anthropic Agent Skills anatomy) ----
|
| 113 |
+
#
|
| 114 |
+
# Each skill is stored as structured columns (name, description, instructions, references_json,
|
| 115 |
+
# assets_json, scripts_json, evals_json) but is always rendered to / parsed from a single portable
|
| 116 |
+
# SKILL.md text file — YAML frontmatter (name + description) followed by the markdown instructions
|
| 117 |
+
# ("the brain"), then optional ## References / ## Assets / ## Scripts / ## Evals sections. This
|
| 118 |
+
# lets an admin export a skill, hand-edit or share it, and import it back losslessly.
|
| 119 |
+
|
| 120 |
+
_FRONTMATTER_RE = re.compile(r"\A---\s*\n(.*?)\n---\s*\n?", re.DOTALL)
|
| 121 |
+
_H3_RE = re.compile(r"^### (.+?)\s*$", re.MULTILINE)
|
| 122 |
+
_SECTION_RE = re.compile(r"^## (References|Assets|Scripts|Evals)\s*$", re.MULTILINE)
|
| 123 |
+
_EVAL_ITEM_RE = re.compile(
|
| 124 |
+
r"^- \*\*Prompt:\*\* (.*?)\n\s*\*\*Assertion \(([^)]*)\):\*\* (.*?)(?=\n- \*\*Prompt|\Z)",
|
| 125 |
+
re.DOTALL,
|
| 126 |
+
)
|
| 127 |
+
|
| 128 |
+
|
| 129 |
+
def _frontmatter_value(block: str, key: str) -> str:
|
| 130 |
+
m = re.search(rf"^{key}:\s*(.*)$", block, re.MULTILINE)
|
| 131 |
+
if not m:
|
| 132 |
+
return ""
|
| 133 |
+
val = m.group(1).strip()
|
| 134 |
+
if len(val) >= 2 and val[0] == val[-1] and val[0] in ("'", '"'):
|
| 135 |
+
val = val[1:-1]
|
| 136 |
+
return val
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
def _strip_code_fence(text: str) -> str:
|
| 140 |
+
text = text.strip()
|
| 141 |
+
m = re.match(r"^```[a-zA-Z0-9_+-]*\n(.*?)\n```$", text, re.DOTALL)
|
| 142 |
+
if m:
|
| 143 |
+
return m.group(1)
|
| 144 |
+
return text
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
def compile_skill_md(skill: ChatAgentSkill) -> str:
|
| 148 |
+
"""Render one skill as a standalone, portable SKILL.md file."""
|
| 149 |
+
name = safe_str(skill.name) or f"skill-{skill.id}"
|
| 150 |
+
description = safe_str(skill.description)
|
| 151 |
+
instructions = safe_str(skill.instructions) or safe_str(skill.text)
|
| 152 |
+
|
| 153 |
+
lines = ["---", f"name: {name}", f"description: {description}", "---", "", instructions, ""]
|
| 154 |
+
|
| 155 |
+
references = skill.references_json or []
|
| 156 |
+
if references:
|
| 157 |
+
lines.append("## References")
|
| 158 |
+
lines.append("")
|
| 159 |
+
for ref in references:
|
| 160 |
+
title = safe_str(ref.get("title")) if isinstance(ref, dict) else ""
|
| 161 |
+
content = safe_str(ref.get("content")) if isinstance(ref, dict) else ""
|
| 162 |
+
lines.append(f"### {title or 'Untitled reference'}")
|
| 163 |
+
lines.append("")
|
| 164 |
+
lines.append(content)
|
| 165 |
+
lines.append("")
|
| 166 |
+
|
| 167 |
+
assets = skill.assets_json or []
|
| 168 |
+
if assets:
|
| 169 |
+
lines.append("## Assets")
|
| 170 |
+
lines.append("")
|
| 171 |
+
for asset in assets:
|
| 172 |
+
aname = safe_str(asset.get("name")) if isinstance(asset, dict) else ""
|
| 173 |
+
content = safe_str(asset.get("content")) if isinstance(asset, dict) else ""
|
| 174 |
+
lines.append(f"### {aname or 'Untitled asset'}")
|
| 175 |
+
lines.append("")
|
| 176 |
+
lines.append("```")
|
| 177 |
+
lines.append(content)
|
| 178 |
+
lines.append("```")
|
| 179 |
+
lines.append("")
|
| 180 |
+
|
| 181 |
+
scripts = skill.scripts_json or []
|
| 182 |
+
if scripts:
|
| 183 |
+
lines.append("## Scripts")
|
| 184 |
+
lines.append("")
|
| 185 |
+
lines.append("_Stored for documentation and portability only — never executed by the agent._")
|
| 186 |
+
lines.append("")
|
| 187 |
+
for script in scripts:
|
| 188 |
+
sname = safe_str(script.get("name")) if isinstance(script, dict) else ""
|
| 189 |
+
slang = safe_str(script.get("language")) if isinstance(script, dict) else ""
|
| 190 |
+
sdesc = safe_str(script.get("description")) if isinstance(script, dict) else ""
|
| 191 |
+
scontent = safe_str(script.get("content")) if isinstance(script, dict) else ""
|
| 192 |
+
header = f"### {sname or 'Untitled script'}"
|
| 193 |
+
if slang:
|
| 194 |
+
header += f" ({slang})"
|
| 195 |
+
lines.append(header)
|
| 196 |
+
lines.append("")
|
| 197 |
+
if sdesc:
|
| 198 |
+
lines.append(sdesc)
|
| 199 |
+
lines.append("")
|
| 200 |
+
lines.append(f"```{slang}")
|
| 201 |
+
lines.append(scontent)
|
| 202 |
+
lines.append("```")
|
| 203 |
+
lines.append("")
|
| 204 |
+
|
| 205 |
+
evals = skill.evals_json or []
|
| 206 |
+
if evals:
|
| 207 |
+
lines.append("## Evals")
|
| 208 |
+
lines.append("")
|
| 209 |
+
for ev in evals:
|
| 210 |
+
prompt = safe_str(ev.get("prompt")) if isinstance(ev, dict) else ""
|
| 211 |
+
atype = safe_str(ev.get("assertion_type")) if isinstance(ev, dict) else ""
|
| 212 |
+
aval = safe_str(ev.get("assertion_value")) if isinstance(ev, dict) else ""
|
| 213 |
+
lines.append(f"- **Prompt:** {prompt}")
|
| 214 |
+
lines.append(f" **Assertion ({atype}):** {aval}")
|
| 215 |
+
lines.append("")
|
| 216 |
+
|
| 217 |
+
return "\n".join(lines).rstrip() + "\n"
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
def _parse_h3_items(section_body: str, key: str) -> list:
|
| 221 |
+
items = []
|
| 222 |
+
parts = _H3_RE.split(section_body)
|
| 223 |
+
for i in range(1, len(parts), 2):
|
| 224 |
+
title = parts[i].strip()
|
| 225 |
+
body = (parts[i + 1] if i + 1 < len(parts) else "").strip()
|
| 226 |
+
if key == "name":
|
| 227 |
+
body = _strip_code_fence(body)
|
| 228 |
+
items.append({key: title, "content": body})
|
| 229 |
+
return items
|
| 230 |
+
|
| 231 |
+
|
| 232 |
+
def _parse_scripts_section(section_body: str) -> list:
|
| 233 |
+
scripts = []
|
| 234 |
+
parts = _H3_RE.split(section_body)
|
| 235 |
+
for i in range(1, len(parts), 2):
|
| 236 |
+
header = parts[i].strip()
|
| 237 |
+
body = (parts[i + 1] if i + 1 < len(parts) else "").strip()
|
| 238 |
+
m = re.match(r"^(.*?)\s*\(([^)]+)\)\s*$", header)
|
| 239 |
+
sname, slang = (m.group(1).strip(), m.group(2).strip()) if m else (header, "")
|
| 240 |
+
fence_m = re.search(r"```([a-zA-Z0-9_+-]*)\n(.*?)\n```", body, re.DOTALL)
|
| 241 |
+
if fence_m:
|
| 242 |
+
slang = slang or fence_m.group(1)
|
| 243 |
+
scontent = fence_m.group(2)
|
| 244 |
+
sdesc = body[: fence_m.start()].strip()
|
| 245 |
+
else:
|
| 246 |
+
scontent = body
|
| 247 |
+
sdesc = ""
|
| 248 |
+
scripts.append({"name": sname, "description": sdesc, "language": slang, "content": scontent})
|
| 249 |
+
return scripts
|
| 250 |
+
|
| 251 |
+
|
| 252 |
+
def _parse_evals_section(section_body: str) -> list:
|
| 253 |
+
evals = []
|
| 254 |
+
for m in _EVAL_ITEM_RE.finditer(section_body.strip() + "\n"):
|
| 255 |
+
evals.append(
|
| 256 |
+
{
|
| 257 |
+
"prompt": m.group(1).strip(),
|
| 258 |
+
"assertion_type": m.group(2).strip(),
|
| 259 |
+
"assertion_value": m.group(3).strip(),
|
| 260 |
+
}
|
| 261 |
+
)
|
| 262 |
+
return evals
|
| 263 |
+
|
| 264 |
+
|
| 265 |
+
def parse_skill_md(content: str) -> dict:
|
| 266 |
+
"""Parse a SKILL.md file back into a dict of fields, matching compile_skill_md()'s format.
|
| 267 |
+
Name/description/instructions parse reliably from any valid frontmatter; References/Assets/
|
| 268 |
+
Scripts/Evals sections parse best-effort when they match the exported layout — a hand-edited
|
| 269 |
+
file that omits or reformats a section simply yields an empty list for it, not an error.
|
| 270 |
+
"""
|
| 271 |
+
content = (content or "").replace("\r\n", "\n").strip("\n")
|
| 272 |
+
name = ""
|
| 273 |
+
description = ""
|
| 274 |
+
body = content
|
| 275 |
+
|
| 276 |
+
fm_match = _FRONTMATTER_RE.match(content + "\n")
|
| 277 |
+
if fm_match:
|
| 278 |
+
block = fm_match.group(1)
|
| 279 |
+
name = _frontmatter_value(block, "name")
|
| 280 |
+
description = _frontmatter_value(block, "description")
|
| 281 |
+
body = content[fm_match.end():]
|
| 282 |
+
|
| 283 |
+
parts = _SECTION_RE.split(body)
|
| 284 |
+
instructions = parts[0].strip()
|
| 285 |
+
|
| 286 |
+
references: list = []
|
| 287 |
+
assets: list = []
|
| 288 |
+
scripts: list = []
|
| 289 |
+
evals: list = []
|
| 290 |
+
|
| 291 |
+
for i in range(1, len(parts), 2):
|
| 292 |
+
section_name = parts[i]
|
| 293 |
+
section_body = parts[i + 1] if i + 1 < len(parts) else ""
|
| 294 |
+
if section_name == "References":
|
| 295 |
+
references = _parse_h3_items(section_body, "title")
|
| 296 |
+
elif section_name == "Assets":
|
| 297 |
+
assets = _parse_h3_items(section_body, "name")
|
| 298 |
+
elif section_name == "Scripts":
|
| 299 |
+
scripts = _parse_scripts_section(section_body)
|
| 300 |
+
elif section_name == "Evals":
|
| 301 |
+
evals = _parse_evals_section(section_body)
|
| 302 |
+
|
| 303 |
+
return {
|
| 304 |
+
"name": name,
|
| 305 |
+
"description": description,
|
| 306 |
+
"instructions": instructions,
|
| 307 |
+
"references": references,
|
| 308 |
+
"assets": assets,
|
| 309 |
+
"scripts": scripts,
|
| 310 |
+
"evals": evals,
|
| 311 |
+
}
|
| 312 |
|
| 313 |
|
| 314 |
def config_to_dict(row: Optional[ChatAgentConfig]) -> dict:
|
|
@@ -1,8 +1,10 @@
|
|
| 1 |
"""
|
| 2 |
Shared ingestion pipeline for the Chat Agent knowledge base: markdown chunking, OpenAI embeddings,
|
| 3 |
content-hash diffing (skip unchanged Drive files/pages on re-index), and the reindex_drive /
|
| 4 |
-
reindex_website orchestration functions that write ChatAgentDocument/
|
| 5 |
-
|
|
|
|
|
|
|
| 6 |
"""
|
| 7 |
|
| 8 |
from __future__ import annotations
|
|
@@ -316,3 +318,88 @@ def reindex_website(tenant_id: int) -> Dict[str, Any]:
|
|
| 316 |
return {"ok": False, "error": str(e)}
|
| 317 |
finally:
|
| 318 |
db.close()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
"""
|
| 2 |
Shared ingestion pipeline for the Chat Agent knowledge base: markdown chunking, OpenAI embeddings,
|
| 3 |
content-hash diffing (skip unchanged Drive files/pages on re-index), and the reindex_drive /
|
| 4 |
+
reindex_website / reindex_skill_content orchestration functions that write ChatAgentDocument/
|
| 5 |
+
ChatAgentChunk rows. Drive/website reindexing runs as background jobs and logs to
|
| 6 |
+
ChatAgentIndexLog for admin visibility; skill reference/asset indexing runs inline on skill save
|
| 7 |
+
(small enough not to need a background job) reusing the same upsert/embed/prune machinery.
|
| 8 |
"""
|
| 9 |
|
| 10 |
from __future__ import annotations
|
|
|
|
| 318 |
return {"ok": False, "error": str(e)}
|
| 319 |
finally:
|
| 320 |
db.close()
|
| 321 |
+
|
| 322 |
+
|
| 323 |
+
def reindex_skill_content(db, tenant_id: int, skill) -> Dict[str, Any]:
|
| 324 |
+
"""Index a skill's references/assets into the same searchable knowledge base used for
|
| 325 |
+
Drive/website content, so the agentic search_knowledge_base tool can surface them on demand.
|
| 326 |
+
Reuses the caller's db session (called inline from the skill save route, not a background
|
| 327 |
+
job) and the existing upsert/embed/prune machinery above.
|
| 328 |
+
"""
|
| 329 |
+
processed = 0
|
| 330 |
+
seen_refs: set = set()
|
| 331 |
+
|
| 332 |
+
for idx, ref in enumerate(skill.references_json or []):
|
| 333 |
+
if not isinstance(ref, dict):
|
| 334 |
+
continue
|
| 335 |
+
content = (ref.get("content") or "").strip()
|
| 336 |
+
if not content:
|
| 337 |
+
continue
|
| 338 |
+
source_ref = f"skill:{skill.id}:reference:{idx}"
|
| 339 |
+
seen_refs.add(source_ref)
|
| 340 |
+
title = (ref.get("title") or f"Reference {idx + 1}").strip()
|
| 341 |
+
if _upsert_document_and_chunks(
|
| 342 |
+
db,
|
| 343 |
+
tenant_id=tenant_id,
|
| 344 |
+
source_type="skill_reference",
|
| 345 |
+
source_ref=source_ref,
|
| 346 |
+
title=f"{skill.name or skill.id}: {title}",
|
| 347 |
+
markdown=content,
|
| 348 |
+
):
|
| 349 |
+
processed += 1
|
| 350 |
+
|
| 351 |
+
for idx, asset in enumerate(skill.assets_json or []):
|
| 352 |
+
if not isinstance(asset, dict):
|
| 353 |
+
continue
|
| 354 |
+
content = (asset.get("content") or "").strip()
|
| 355 |
+
if not content:
|
| 356 |
+
continue
|
| 357 |
+
source_ref = f"skill:{skill.id}:asset:{idx}"
|
| 358 |
+
seen_refs.add(source_ref)
|
| 359 |
+
title = (asset.get("name") or f"Asset {idx + 1}").strip()
|
| 360 |
+
if _upsert_document_and_chunks(
|
| 361 |
+
db,
|
| 362 |
+
tenant_id=tenant_id,
|
| 363 |
+
source_type="skill_asset",
|
| 364 |
+
source_ref=source_ref,
|
| 365 |
+
title=f"{skill.name or skill.id}: {title}",
|
| 366 |
+
markdown=content,
|
| 367 |
+
):
|
| 368 |
+
processed += 1
|
| 369 |
+
|
| 370 |
+
# Scope the prune to THIS skill's source_ref prefix — a plain source_type filter would wipe
|
| 371 |
+
# every other skill's reference/asset content too, since they all share "skill_reference"/
|
| 372 |
+
# "skill_asset" as source_type.
|
| 373 |
+
stale = (
|
| 374 |
+
db.query(ChatAgentDocument)
|
| 375 |
+
.filter(
|
| 376 |
+
ChatAgentDocument.tenant_id == tenant_id,
|
| 377 |
+
ChatAgentDocument.source_type.in_(["skill_reference", "skill_asset"]),
|
| 378 |
+
ChatAgentDocument.source_ref.like(f"skill:{skill.id}:%"),
|
| 379 |
+
~ChatAgentDocument.source_ref.in_(seen_refs),
|
| 380 |
+
)
|
| 381 |
+
.all()
|
| 382 |
+
)
|
| 383 |
+
for d in stale:
|
| 384 |
+
db.query(ChatAgentChunk).filter(ChatAgentChunk.document_id == d.id).delete()
|
| 385 |
+
db.delete(d)
|
| 386 |
+
|
| 387 |
+
return {"ok": True, "processed": processed, "pruned": len(stale)}
|
| 388 |
+
|
| 389 |
+
|
| 390 |
+
def delete_skill_content(db, tenant_id: int, skill_id: int) -> int:
|
| 391 |
+
"""Remove all indexed reference/asset content for a skill. Called when the skill itself is
|
| 392 |
+
deleted, since ChatAgentDocument has no FK cascade to ChatAgentSkill."""
|
| 393 |
+
docs = (
|
| 394 |
+
db.query(ChatAgentDocument)
|
| 395 |
+
.filter(
|
| 396 |
+
ChatAgentDocument.tenant_id == tenant_id,
|
| 397 |
+
ChatAgentDocument.source_type.in_(["skill_reference", "skill_asset"]),
|
| 398 |
+
ChatAgentDocument.source_ref.like(f"skill:{skill_id}:%"),
|
| 399 |
+
)
|
| 400 |
+
.all()
|
| 401 |
+
)
|
| 402 |
+
for d in docs:
|
| 403 |
+
db.query(ChatAgentChunk).filter(ChatAgentChunk.document_id == d.id).delete()
|
| 404 |
+
db.delete(d)
|
| 405 |
+
return len(docs)
|
|
@@ -18,17 +18,26 @@ from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request
|
|
| 18 |
from fastapi.responses import JSONResponse, PlainTextResponse, Response, StreamingResponse
|
| 19 |
from sqlalchemy import func
|
| 20 |
from sqlalchemy.orm import Session
|
|
|
|
| 21 |
|
| 22 |
from .chat_agent_chat import (
|
| 23 |
build_agentic_system_prompt,
|
| 24 |
build_system_prompt,
|
| 25 |
extract_email,
|
| 26 |
extract_visitor_name,
|
|
|
|
| 27 |
stream_agentic_reply_openai,
|
| 28 |
stream_reply_gemini,
|
| 29 |
)
|
| 30 |
-
from .chat_agent_common import
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
from .chat_agent_retrieval import retrieve_relevant_chunks
|
| 33 |
from .crm_lead_contacts import link_lead_to_contact
|
| 34 |
from .database import (
|
|
@@ -48,6 +57,7 @@ from .models import (
|
|
| 48 |
ChatAgentReindexRequest,
|
| 49 |
ChatAgentSessionRequest,
|
| 50 |
ChatAgentSkillCreateRequest,
|
|
|
|
| 51 |
ChatAgentSkillPatchRequest,
|
| 52 |
)
|
| 53 |
from .oauth_utils import public_request_base
|
|
@@ -85,7 +95,13 @@ def _parse_drive_folder_id(url_or_id: str) -> str:
|
|
| 85 |
def _skill_to_dict(r: ChatAgentSkill) -> dict:
|
| 86 |
return {
|
| 87 |
"id": r.id,
|
| 88 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
"is_default": bool(r.is_default),
|
| 90 |
"is_active": bool(r.is_active),
|
| 91 |
"sort_order": r.sort_order,
|
|
@@ -105,6 +121,20 @@ def _conversation_to_dict(r: ChatAgentConversation) -> dict:
|
|
| 105 |
}
|
| 106 |
|
| 107 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 108 |
def _embed_snippet(origin: str, public_id: str) -> str:
|
| 109 |
if not origin:
|
| 110 |
return ""
|
|
@@ -175,9 +205,17 @@ def create_skill(body: ChatAgentSkillCreateRequest, tc: TenantContext = Depends(
|
|
| 175 |
.scalar()
|
| 176 |
or 0
|
| 177 |
)
|
|
|
|
| 178 |
row = ChatAgentSkill(
|
| 179 |
tenant_id=tc.tenant_id,
|
| 180 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
is_default=0,
|
| 182 |
is_active=1,
|
| 183 |
sort_order=max_order + 1,
|
|
@@ -185,6 +223,7 @@ def create_skill(body: ChatAgentSkillCreateRequest, tc: TenantContext = Depends(
|
|
| 185 |
tc.db.add(row)
|
| 186 |
db_commit_with_retry(tc.db)
|
| 187 |
tc.db.refresh(row)
|
|
|
|
| 188 |
return _skill_to_dict(row)
|
| 189 |
|
| 190 |
|
|
@@ -197,14 +236,32 @@ def patch_skill(skill_id: int, body: ChatAgentSkillPatchRequest, tc: TenantConte
|
|
| 197 |
)
|
| 198 |
if not row:
|
| 199 |
raise HTTPException(status_code=404, detail="Skill not found")
|
| 200 |
-
if body.
|
| 201 |
-
row.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
if body.is_active is not None:
|
| 203 |
row.is_active = 1 if body.is_active else 0
|
| 204 |
if body.sort_order is not None:
|
| 205 |
row.sort_order = body.sort_order
|
| 206 |
row.updated_at = datetime.utcnow()
|
| 207 |
db_commit_with_retry(tc.db)
|
|
|
|
|
|
|
| 208 |
return _skill_to_dict(row)
|
| 209 |
|
| 210 |
|
|
@@ -217,11 +274,115 @@ def delete_skill(skill_id: int, tc: TenantContext = Depends(require_tenant_admin
|
|
| 217 |
)
|
| 218 |
if not row:
|
| 219 |
raise HTTPException(status_code=404, detail="Skill not found")
|
|
|
|
| 220 |
tc.db.delete(row)
|
| 221 |
db_commit_with_retry(tc.db)
|
| 222 |
return {"ok": True}
|
| 223 |
|
| 224 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
# ---- Admin: reindex + status ----
|
| 226 |
|
| 227 |
|
|
|
|
| 18 |
from fastapi.responses import JSONResponse, PlainTextResponse, Response, StreamingResponse
|
| 19 |
from sqlalchemy import func
|
| 20 |
from sqlalchemy.orm import Session
|
| 21 |
+
from sqlalchemy.orm.attributes import flag_modified
|
| 22 |
|
| 23 |
from .chat_agent_chat import (
|
| 24 |
build_agentic_system_prompt,
|
| 25 |
build_system_prompt,
|
| 26 |
extract_email,
|
| 27 |
extract_visitor_name,
|
| 28 |
+
run_eval_assertion,
|
| 29 |
stream_agentic_reply_openai,
|
| 30 |
stream_reply_gemini,
|
| 31 |
)
|
| 32 |
+
from .chat_agent_common import (
|
| 33 |
+
compile_skill_md,
|
| 34 |
+
compile_skills_md,
|
| 35 |
+
config_to_dict,
|
| 36 |
+
get_or_create_config,
|
| 37 |
+
parse_skill_md,
|
| 38 |
+
safe_str,
|
| 39 |
+
)
|
| 40 |
+
from .chat_agent_ingest import delete_skill_content, reindex_drive, reindex_skill_content, reindex_website
|
| 41 |
from .chat_agent_retrieval import retrieve_relevant_chunks
|
| 42 |
from .crm_lead_contacts import link_lead_to_contact
|
| 43 |
from .database import (
|
|
|
|
| 57 |
ChatAgentReindexRequest,
|
| 58 |
ChatAgentSessionRequest,
|
| 59 |
ChatAgentSkillCreateRequest,
|
| 60 |
+
ChatAgentSkillImportRequest,
|
| 61 |
ChatAgentSkillPatchRequest,
|
| 62 |
)
|
| 63 |
from .oauth_utils import public_request_base
|
|
|
|
| 95 |
def _skill_to_dict(r: ChatAgentSkill) -> dict:
|
| 96 |
return {
|
| 97 |
"id": r.id,
|
| 98 |
+
"name": r.name or "",
|
| 99 |
+
"description": r.description or "",
|
| 100 |
+
"instructions": r.instructions or r.text or "",
|
| 101 |
+
"references": r.references_json or [],
|
| 102 |
+
"assets": r.assets_json or [],
|
| 103 |
+
"scripts": r.scripts_json or [],
|
| 104 |
+
"evals": r.evals_json or [],
|
| 105 |
"is_default": bool(r.is_default),
|
| 106 |
"is_active": bool(r.is_active),
|
| 107 |
"sort_order": r.sort_order,
|
|
|
|
| 121 |
}
|
| 122 |
|
| 123 |
|
| 124 |
+
def _safe_reindex_skill_content(db: Session, tenant_id: int, skill: ChatAgentSkill) -> None:
|
| 125 |
+
"""Index a skill's references/assets, but never let an indexing failure (e.g. the embeddings
|
| 126 |
+
API being down) fail the save of the skill's own name/description/instructions — those are
|
| 127 |
+
committed by the caller before this runs. On failure, roll back just the partial
|
| 128 |
+
document/chunk writes so a later save retries cleanly instead of leaving a document marked
|
| 129 |
+
up-to-date with zero chunks."""
|
| 130 |
+
try:
|
| 131 |
+
reindex_skill_content(db, tenant_id, skill)
|
| 132 |
+
db_commit_with_retry(db)
|
| 133 |
+
except Exception as e:
|
| 134 |
+
logger.warning("Skill content reindex failed for skill %s: %s", skill.id, e)
|
| 135 |
+
db.rollback()
|
| 136 |
+
|
| 137 |
+
|
| 138 |
def _embed_snippet(origin: str, public_id: str) -> str:
|
| 139 |
if not origin:
|
| 140 |
return ""
|
|
|
|
| 205 |
.scalar()
|
| 206 |
or 0
|
| 207 |
)
|
| 208 |
+
instructions = body.instructions.strip()
|
| 209 |
row = ChatAgentSkill(
|
| 210 |
tenant_id=tc.tenant_id,
|
| 211 |
+
name=body.name.strip(),
|
| 212 |
+
description=body.description.strip(),
|
| 213 |
+
instructions=instructions,
|
| 214 |
+
text=instructions,
|
| 215 |
+
references_json=[r.model_dump() for r in body.references],
|
| 216 |
+
assets_json=[a.model_dump() for a in body.assets],
|
| 217 |
+
scripts_json=[s.model_dump() for s in body.scripts],
|
| 218 |
+
evals_json=[e.model_dump() for e in body.evals],
|
| 219 |
is_default=0,
|
| 220 |
is_active=1,
|
| 221 |
sort_order=max_order + 1,
|
|
|
|
| 223 |
tc.db.add(row)
|
| 224 |
db_commit_with_retry(tc.db)
|
| 225 |
tc.db.refresh(row)
|
| 226 |
+
_safe_reindex_skill_content(tc.db, tc.tenant_id, row)
|
| 227 |
return _skill_to_dict(row)
|
| 228 |
|
| 229 |
|
|
|
|
| 236 |
)
|
| 237 |
if not row:
|
| 238 |
raise HTTPException(status_code=404, detail="Skill not found")
|
| 239 |
+
if body.name is not None:
|
| 240 |
+
row.name = body.name.strip()
|
| 241 |
+
if body.description is not None:
|
| 242 |
+
row.description = body.description.strip()
|
| 243 |
+
if body.instructions is not None:
|
| 244 |
+
row.instructions = body.instructions.strip()
|
| 245 |
+
row.text = row.instructions
|
| 246 |
+
content_changed = False
|
| 247 |
+
if body.references is not None:
|
| 248 |
+
row.references_json = [r.model_dump() for r in body.references]
|
| 249 |
+
content_changed = True
|
| 250 |
+
if body.assets is not None:
|
| 251 |
+
row.assets_json = [a.model_dump() for a in body.assets]
|
| 252 |
+
content_changed = True
|
| 253 |
+
if body.scripts is not None:
|
| 254 |
+
row.scripts_json = [s.model_dump() for s in body.scripts]
|
| 255 |
+
if body.evals is not None:
|
| 256 |
+
row.evals_json = [e.model_dump() for e in body.evals]
|
| 257 |
if body.is_active is not None:
|
| 258 |
row.is_active = 1 if body.is_active else 0
|
| 259 |
if body.sort_order is not None:
|
| 260 |
row.sort_order = body.sort_order
|
| 261 |
row.updated_at = datetime.utcnow()
|
| 262 |
db_commit_with_retry(tc.db)
|
| 263 |
+
if content_changed:
|
| 264 |
+
_safe_reindex_skill_content(tc.db, tc.tenant_id, row)
|
| 265 |
return _skill_to_dict(row)
|
| 266 |
|
| 267 |
|
|
|
|
| 274 |
)
|
| 275 |
if not row:
|
| 276 |
raise HTTPException(status_code=404, detail="Skill not found")
|
| 277 |
+
delete_skill_content(tc.db, tc.tenant_id, row.id)
|
| 278 |
tc.db.delete(row)
|
| 279 |
db_commit_with_retry(tc.db)
|
| 280 |
return {"ok": True}
|
| 281 |
|
| 282 |
|
| 283 |
+
@router.get("/api/chat-agent/skills/{skill_id}/export")
|
| 284 |
+
def export_skill(skill_id: int, tc: TenantContext = Depends(require_tenant_admin)):
|
| 285 |
+
row = (
|
| 286 |
+
tc.db.query(ChatAgentSkill)
|
| 287 |
+
.filter(ChatAgentSkill.tenant_id == tc.tenant_id, ChatAgentSkill.id == skill_id)
|
| 288 |
+
.first()
|
| 289 |
+
)
|
| 290 |
+
if not row:
|
| 291 |
+
raise HTTPException(status_code=404, detail="Skill not found")
|
| 292 |
+
md = compile_skill_md(row)
|
| 293 |
+
filename = re.sub(r"[^a-zA-Z0-9_-]+", "-", (row.name or f"skill-{row.id}")).strip("-") or f"skill-{row.id}"
|
| 294 |
+
return PlainTextResponse(
|
| 295 |
+
md,
|
| 296 |
+
media_type="text/markdown",
|
| 297 |
+
headers={"Content-Disposition": f'attachment; filename="{filename}.SKILL.md"'},
|
| 298 |
+
)
|
| 299 |
+
|
| 300 |
+
|
| 301 |
+
@router.post("/api/chat-agent/skills/import")
|
| 302 |
+
def import_skill(body: ChatAgentSkillImportRequest, tc: TenantContext = Depends(require_tenant_admin)):
|
| 303 |
+
parsed = parse_skill_md(body.content)
|
| 304 |
+
if not parsed["instructions"]:
|
| 305 |
+
raise HTTPException(status_code=400, detail="Couldn't find any instructions in that SKILL.md content")
|
| 306 |
+
max_order = (
|
| 307 |
+
tc.db.query(func.max(ChatAgentSkill.sort_order))
|
| 308 |
+
.filter(ChatAgentSkill.tenant_id == tc.tenant_id)
|
| 309 |
+
.scalar()
|
| 310 |
+
or 0
|
| 311 |
+
)
|
| 312 |
+
row = ChatAgentSkill(
|
| 313 |
+
tenant_id=tc.tenant_id,
|
| 314 |
+
name=(parsed["name"] or "imported-skill").strip(),
|
| 315 |
+
description=parsed["description"].strip(),
|
| 316 |
+
instructions=parsed["instructions"].strip(),
|
| 317 |
+
text=parsed["instructions"].strip(),
|
| 318 |
+
references_json=parsed["references"],
|
| 319 |
+
assets_json=parsed["assets"],
|
| 320 |
+
scripts_json=parsed["scripts"],
|
| 321 |
+
evals_json=parsed["evals"],
|
| 322 |
+
is_default=0,
|
| 323 |
+
is_active=1,
|
| 324 |
+
sort_order=max_order + 1,
|
| 325 |
+
)
|
| 326 |
+
tc.db.add(row)
|
| 327 |
+
db_commit_with_retry(tc.db)
|
| 328 |
+
tc.db.refresh(row)
|
| 329 |
+
_safe_reindex_skill_content(tc.db, tc.tenant_id, row)
|
| 330 |
+
return _skill_to_dict(row)
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
@router.post("/api/chat-agent/skills/{skill_id}/evals/run")
|
| 334 |
+
def run_skill_evals(skill_id: int, tc: TenantContext = Depends(require_tenant_admin)):
|
| 335 |
+
row = (
|
| 336 |
+
tc.db.query(ChatAgentSkill)
|
| 337 |
+
.filter(ChatAgentSkill.tenant_id == tc.tenant_id, ChatAgentSkill.id == skill_id)
|
| 338 |
+
.first()
|
| 339 |
+
)
|
| 340 |
+
if not row:
|
| 341 |
+
raise HTTPException(status_code=404, detail="Skill not found")
|
| 342 |
+
evals = list(row.evals_json or [])
|
| 343 |
+
if not evals:
|
| 344 |
+
return {"skill": _skill_to_dict(row), "results": []}
|
| 345 |
+
|
| 346 |
+
skills_md = compile_skills_md(tc.db, tc.tenant_id)
|
| 347 |
+
results = []
|
| 348 |
+
for ev in evals[:10]:
|
| 349 |
+
prompt = (ev.get("prompt") or "").strip()
|
| 350 |
+
if not prompt:
|
| 351 |
+
continue
|
| 352 |
+
atype = ev.get("assertion_type") or "must_contain"
|
| 353 |
+
aval = ev.get("assertion_value") or ""
|
| 354 |
+
# has_visitor_info=True + already_asked_for_contact=True skip the lead-capture ask, so the
|
| 355 |
+
# eval run tests the actual answer to the prompt rather than a contact-info request.
|
| 356 |
+
system_prompt = build_agentic_system_prompt(
|
| 357 |
+
skills_md=skills_md,
|
| 358 |
+
greeting_message="",
|
| 359 |
+
has_visitor_info=True,
|
| 360 |
+
already_asked_for_contact=True,
|
| 361 |
+
)
|
| 362 |
+
try:
|
| 363 |
+
response_text = "".join(
|
| 364 |
+
stream_agentic_reply_openai(
|
| 365 |
+
system_prompt, [{"role": "user", "content": prompt}], tc.tenant_id, tc.db
|
| 366 |
+
)
|
| 367 |
+
).strip()
|
| 368 |
+
except Exception as e:
|
| 369 |
+
logger.warning("Eval run failed to generate a response for skill %s: %s", skill_id, e)
|
| 370 |
+
response_text = ""
|
| 371 |
+
graded = run_eval_assertion(prompt, response_text, atype, aval)
|
| 372 |
+
ev["last_run_status"] = graded["status"]
|
| 373 |
+
ev["last_run_detail"] = (
|
| 374 |
+
f"{graded['detail']} — Response: {response_text[:300]}" if response_text else graded["detail"]
|
| 375 |
+
)
|
| 376 |
+
ev["last_run_at"] = datetime.utcnow().isoformat()
|
| 377 |
+
results.append({"prompt": prompt, "response": response_text, **graded})
|
| 378 |
+
|
| 379 |
+
row.evals_json = evals
|
| 380 |
+
flag_modified(row, "evals_json")
|
| 381 |
+
row.updated_at = datetime.utcnow()
|
| 382 |
+
db_commit_with_retry(tc.db)
|
| 383 |
+
return {"skill": _skill_to_dict(row), "results": results}
|
| 384 |
+
|
| 385 |
+
|
| 386 |
# ---- Admin: reindex + status ----
|
| 387 |
|
| 388 |
|
|
@@ -439,13 +439,28 @@ class ChatAgentConfig(Base):
|
|
| 439 |
|
| 440 |
|
| 441 |
class ChatAgentSkill(Base):
|
| 442 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 443 |
|
| 444 |
__tablename__ = "chat_agent_skills"
|
| 445 |
|
| 446 |
id = Column(Integer, primary_key=True, index=True)
|
| 447 |
tenant_id = Column(Integer, ForeignKey("tenants.id"), nullable=False, index=True)
|
| 448 |
-
text = Column(Text, nullable=False, default="")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 449 |
is_default = Column(Integer, default=0)
|
| 450 |
is_active = Column(Integer, default=1)
|
| 451 |
sort_order = Column(Integer, default=0)
|
|
@@ -756,6 +771,46 @@ def run_migrations(connection_engine):
|
|
| 756 |
if "embed_height" not in ccols:
|
| 757 |
conn.execute(text("ALTER TABLE cta_forms ADD COLUMN embed_height TEXT DEFAULT '480'"))
|
| 758 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 759 |
|
| 760 |
# Create tables then migrate legacy SQLite schemas
|
| 761 |
Base.metadata.create_all(bind=engine)
|
|
|
|
| 439 |
|
| 440 |
|
| 441 |
class ChatAgentSkill(Base):
|
| 442 |
+
"""
|
| 443 |
+
Admin-editable skill, following Anthropic's Agent Skills anatomy (SKILL.md: YAML
|
| 444 |
+
frontmatter name+description + markdown instructions, plus references/assets/scripts/evals).
|
| 445 |
+
`instructions` (the "brain") is always compiled into the chat system prompt; references and
|
| 446 |
+
assets are indexed into the same searchable knowledge base as Drive/website content so the
|
| 447 |
+
agentic search_knowledge_base tool can surface them on demand. `scripts_json` is stored/
|
| 448 |
+
exportable for documentation and portability ONLY — never executed by the agent (see
|
| 449 |
+
SKILLS-SECURITY.md for why).
|
| 450 |
+
"""
|
| 451 |
|
| 452 |
__tablename__ = "chat_agent_skills"
|
| 453 |
|
| 454 |
id = Column(Integer, primary_key=True, index=True)
|
| 455 |
tenant_id = Column(Integer, ForeignKey("tenants.id"), nullable=False, index=True)
|
| 456 |
+
text = Column(Text, nullable=False, default="") # legacy; superseded by `instructions` below
|
| 457 |
+
name = Column(String, default="")
|
| 458 |
+
description = Column(Text, default="")
|
| 459 |
+
instructions = Column(Text, default="")
|
| 460 |
+
references_json = Column(JSON, nullable=True) # [{title, content}]
|
| 461 |
+
assets_json = Column(JSON, nullable=True) # [{name, content}]
|
| 462 |
+
scripts_json = Column(JSON, nullable=True) # [{name, description, language, content}] — doc only
|
| 463 |
+
evals_json = Column(JSON, nullable=True) # [{prompt, assertion_type, assertion_value, last_run_*}]
|
| 464 |
is_default = Column(Integer, default=0)
|
| 465 |
is_active = Column(Integer, default=1)
|
| 466 |
sort_order = Column(Integer, default=0)
|
|
|
|
| 771 |
if "embed_height" not in ccols:
|
| 772 |
conn.execute(text("ALTER TABLE cta_forms ADD COLUMN embed_height TEXT DEFAULT '480'"))
|
| 773 |
|
| 774 |
+
insp = inspect(connection_engine)
|
| 775 |
+
if insp.has_table("chat_agent_skills"):
|
| 776 |
+
skcols = [c["name"] for c in insp.get_columns("chat_agent_skills")]
|
| 777 |
+
if "name" not in skcols:
|
| 778 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN name TEXT DEFAULT ''"))
|
| 779 |
+
if "description" not in skcols:
|
| 780 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN description TEXT"))
|
| 781 |
+
if "instructions" not in skcols:
|
| 782 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN instructions TEXT"))
|
| 783 |
+
if "references_json" not in skcols:
|
| 784 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN references_json TEXT"))
|
| 785 |
+
if "assets_json" not in skcols:
|
| 786 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN assets_json TEXT"))
|
| 787 |
+
if "scripts_json" not in skcols:
|
| 788 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN scripts_json TEXT"))
|
| 789 |
+
if "evals_json" not in skcols:
|
| 790 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN evals_json TEXT"))
|
| 791 |
+
if "sort_order" not in skcols:
|
| 792 |
+
conn.execute(text("ALTER TABLE chat_agent_skills ADD COLUMN sort_order INTEGER DEFAULT 0"))
|
| 793 |
+
# Backfill legacy rows (created before `instructions` existed) so every skill has a
|
| 794 |
+
# real name/description/instructions triple, matching the SKILL.md anatomy.
|
| 795 |
+
conn.execute(
|
| 796 |
+
text(
|
| 797 |
+
"UPDATE chat_agent_skills SET instructions = text "
|
| 798 |
+
"WHERE (instructions IS NULL OR instructions = '') AND text IS NOT NULL"
|
| 799 |
+
)
|
| 800 |
+
)
|
| 801 |
+
conn.execute(
|
| 802 |
+
text(
|
| 803 |
+
"UPDATE chat_agent_skills SET name = 'skill-' || id "
|
| 804 |
+
"WHERE name IS NULL OR name = ''"
|
| 805 |
+
)
|
| 806 |
+
)
|
| 807 |
+
conn.execute(
|
| 808 |
+
text(
|
| 809 |
+
"UPDATE chat_agent_skills SET description = 'Legacy skill — edit to add a name and description' "
|
| 810 |
+
"WHERE description IS NULL OR description = ''"
|
| 811 |
+
)
|
| 812 |
+
)
|
| 813 |
+
|
| 814 |
|
| 815 |
# Create tables then migrate legacy SQLite schemas
|
| 816 |
Base.metadata.create_all(bind=engine)
|
|
@@ -354,16 +354,60 @@ class ChatAgentConfigPatch(BaseModel):
|
|
| 354 |
reindex_interval_hours: Optional[int] = Field(None, ge=1, le=168)
|
| 355 |
|
| 356 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 357 |
class ChatAgentSkillCreateRequest(BaseModel):
|
| 358 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 359 |
|
| 360 |
|
| 361 |
class ChatAgentSkillPatchRequest(BaseModel):
|
| 362 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 363 |
is_active: Optional[bool] = None
|
| 364 |
sort_order: Optional[int] = None
|
| 365 |
|
| 366 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 367 |
class ChatAgentReindexRequest(BaseModel):
|
| 368 |
source: Literal["drive", "website", "all"] = "all"
|
| 369 |
|
|
|
|
| 354 |
reindex_interval_hours: Optional[int] = Field(None, ge=1, le=168)
|
| 355 |
|
| 356 |
|
| 357 |
+
class ChatAgentSkillReferenceItem(BaseModel):
|
| 358 |
+
title: str = Field("", max_length=300)
|
| 359 |
+
content: str = Field("", max_length=20000)
|
| 360 |
+
|
| 361 |
+
|
| 362 |
+
class ChatAgentSkillAssetItem(BaseModel):
|
| 363 |
+
name: str = Field("", max_length=300)
|
| 364 |
+
content: str = Field("", max_length=20000)
|
| 365 |
+
|
| 366 |
+
|
| 367 |
+
class ChatAgentSkillScriptItem(BaseModel):
|
| 368 |
+
# Stored/exported for documentation only — the agent never executes this content. See
|
| 369 |
+
# SKILLS-SECURITY.md.
|
| 370 |
+
name: str = Field("", max_length=300)
|
| 371 |
+
description: str = Field("", max_length=2000)
|
| 372 |
+
language: str = Field("", max_length=40)
|
| 373 |
+
content: str = Field("", max_length=20000)
|
| 374 |
+
|
| 375 |
+
|
| 376 |
+
class ChatAgentSkillEvalItem(BaseModel):
|
| 377 |
+
prompt: str = Field("", max_length=2000)
|
| 378 |
+
assertion_type: Literal["must_contain", "must_not_contain", "llm_judge"] = "must_contain"
|
| 379 |
+
assertion_value: str = Field("", max_length=2000)
|
| 380 |
+
last_run_status: Optional[str] = None
|
| 381 |
+
last_run_detail: Optional[str] = None
|
| 382 |
+
last_run_at: Optional[str] = None
|
| 383 |
+
|
| 384 |
+
|
| 385 |
class ChatAgentSkillCreateRequest(BaseModel):
|
| 386 |
+
name: str = Field(..., min_length=1, max_length=200)
|
| 387 |
+
description: str = Field("", max_length=1000)
|
| 388 |
+
instructions: str = Field(..., min_length=1, max_length=8000)
|
| 389 |
+
references: List[ChatAgentSkillReferenceItem] = Field(default_factory=list, max_length=20)
|
| 390 |
+
assets: List[ChatAgentSkillAssetItem] = Field(default_factory=list, max_length=20)
|
| 391 |
+
scripts: List[ChatAgentSkillScriptItem] = Field(default_factory=list, max_length=20)
|
| 392 |
+
evals: List[ChatAgentSkillEvalItem] = Field(default_factory=list, max_length=10)
|
| 393 |
|
| 394 |
|
| 395 |
class ChatAgentSkillPatchRequest(BaseModel):
|
| 396 |
+
name: Optional[str] = Field(None, min_length=1, max_length=200)
|
| 397 |
+
description: Optional[str] = Field(None, max_length=1000)
|
| 398 |
+
instructions: Optional[str] = Field(None, min_length=1, max_length=8000)
|
| 399 |
+
references: Optional[List[ChatAgentSkillReferenceItem]] = Field(None, max_length=20)
|
| 400 |
+
assets: Optional[List[ChatAgentSkillAssetItem]] = Field(None, max_length=20)
|
| 401 |
+
scripts: Optional[List[ChatAgentSkillScriptItem]] = Field(None, max_length=20)
|
| 402 |
+
evals: Optional[List[ChatAgentSkillEvalItem]] = Field(None, max_length=10)
|
| 403 |
is_active: Optional[bool] = None
|
| 404 |
sort_order: Optional[int] = None
|
| 405 |
|
| 406 |
|
| 407 |
+
class ChatAgentSkillImportRequest(BaseModel):
|
| 408 |
+
content: str = Field(..., min_length=1, max_length=100000)
|
| 409 |
+
|
| 410 |
+
|
| 411 |
class ChatAgentReindexRequest(BaseModel):
|
| 412 |
source: Literal["drive", "website", "all"] = "all"
|
| 413 |
|
|
@@ -7,10 +7,8 @@ import {
|
|
| 7 |
Loader2,
|
| 8 |
Mail,
|
| 9 |
MessageSquare,
|
| 10 |
-
Plus,
|
| 11 |
RefreshCw,
|
| 12 |
Send,
|
| 13 |
-
Trash2,
|
| 14 |
} from 'lucide-react';
|
| 15 |
import AppShell from '@/components/layout/AppShell';
|
| 16 |
import { Button } from '@/components/ui/button';
|
|
@@ -24,6 +22,7 @@ import {
|
|
| 24 |
SelectTrigger,
|
| 25 |
SelectValue,
|
| 26 |
} from '@/components/ui/select';
|
|
|
|
| 27 |
import { apiFetch } from '@/lib/api';
|
| 28 |
import { cn } from '@/lib/utils';
|
| 29 |
|
|
@@ -325,11 +324,6 @@ export default function ChatAgentPanel({ onBack }) {
|
|
| 325 |
const [error, setError] = useState('');
|
| 326 |
const [savingConfig, setSavingConfig] = useState(false);
|
| 327 |
|
| 328 |
-
const [skills, setSkills] = useState([]);
|
| 329 |
-
const [skillsLoading, setSkillsLoading] = useState(true);
|
| 330 |
-
const [newSkillText, setNewSkillText] = useState('');
|
| 331 |
-
const [addingSkill, setAddingSkill] = useState(false);
|
| 332 |
-
|
| 333 |
const [status, setStatus] = useState(null);
|
| 334 |
const [statusLoading, setStatusLoading] = useState(true);
|
| 335 |
const [reindexBusy, setReindexBusy] = useState('');
|
|
@@ -361,19 +355,6 @@ export default function ChatAgentPanel({ onBack }) {
|
|
| 361 |
}
|
| 362 |
}, []);
|
| 363 |
|
| 364 |
-
const loadSkills = useCallback(async () => {
|
| 365 |
-
setSkillsLoading(true);
|
| 366 |
-
try {
|
| 367 |
-
const r = await apiFetch('/api/chat-agent/skills');
|
| 368 |
-
const data = r.ok ? await r.json() : { skills: [] };
|
| 369 |
-
setSkills(data.skills || []);
|
| 370 |
-
} catch {
|
| 371 |
-
setSkills([]);
|
| 372 |
-
} finally {
|
| 373 |
-
setSkillsLoading(false);
|
| 374 |
-
}
|
| 375 |
-
}, []);
|
| 376 |
-
|
| 377 |
const loadStatus = useCallback(async () => {
|
| 378 |
setStatusLoading(true);
|
| 379 |
try {
|
|
@@ -402,10 +383,9 @@ export default function ChatAgentPanel({ onBack }) {
|
|
| 402 |
|
| 403 |
useEffect(() => {
|
| 404 |
loadConfig();
|
| 405 |
-
loadSkills();
|
| 406 |
loadStatus();
|
| 407 |
loadConversations();
|
| 408 |
-
}, [loadConfig,
|
| 409 |
|
| 410 |
// Reflect Drive OAuth callback redirect (?chat_agent_drive=connected / ?chat_agent_drive_error=...)
|
| 411 |
useEffect(() => {
|
|
@@ -485,56 +465,6 @@ export default function ChatAgentPanel({ onBack }) {
|
|
| 485 |
}
|
| 486 |
};
|
| 487 |
|
| 488 |
-
const addSkill = async () => {
|
| 489 |
-
const text = newSkillText.trim();
|
| 490 |
-
if (!text) return;
|
| 491 |
-
setAddingSkill(true);
|
| 492 |
-
try {
|
| 493 |
-
const res = await apiFetch('/api/chat-agent/skills', {
|
| 494 |
-
method: 'POST',
|
| 495 |
-
headers: { 'Content-Type': 'application/json' },
|
| 496 |
-
body: JSON.stringify({ text }),
|
| 497 |
-
});
|
| 498 |
-
const data = await res.json();
|
| 499 |
-
if (!res.ok) throw new Error(data.detail || 'Could not add skill');
|
| 500 |
-
setSkills((prev) => [...prev, data]);
|
| 501 |
-
setNewSkillText('');
|
| 502 |
-
} catch (e) {
|
| 503 |
-
setError(e.message || 'Could not add skill');
|
| 504 |
-
} finally {
|
| 505 |
-
setAddingSkill(false);
|
| 506 |
-
}
|
| 507 |
-
};
|
| 508 |
-
|
| 509 |
-
const toggleSkillActive = async (skill) => {
|
| 510 |
-
try {
|
| 511 |
-
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}`, {
|
| 512 |
-
method: 'PATCH',
|
| 513 |
-
headers: { 'Content-Type': 'application/json' },
|
| 514 |
-
body: JSON.stringify({ is_active: !skill.is_active }),
|
| 515 |
-
});
|
| 516 |
-
const data = await res.json();
|
| 517 |
-
if (!res.ok) throw new Error(data.detail || 'Update failed');
|
| 518 |
-
setSkills((prev) => prev.map((s) => (s.id === skill.id ? data : s)));
|
| 519 |
-
} catch (e) {
|
| 520 |
-
setError(e.message || 'Update failed');
|
| 521 |
-
}
|
| 522 |
-
};
|
| 523 |
-
|
| 524 |
-
const deleteSkill = async (skill) => {
|
| 525 |
-
if (!window.confirm('Remove this skill?')) return;
|
| 526 |
-
try {
|
| 527 |
-
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}`, { method: 'DELETE' });
|
| 528 |
-
if (!res.ok) {
|
| 529 |
-
const data = await res.json().catch(() => ({}));
|
| 530 |
-
throw new Error(data.detail || 'Delete failed');
|
| 531 |
-
}
|
| 532 |
-
setSkills((prev) => prev.filter((s) => s.id !== skill.id));
|
| 533 |
-
} catch (e) {
|
| 534 |
-
setError(e.message || 'Delete failed');
|
| 535 |
-
}
|
| 536 |
-
};
|
| 537 |
-
|
| 538 |
const openConversation = async (id) => {
|
| 539 |
try {
|
| 540 |
const r = await apiFetch(`/api/chat-agent/conversations/${id}`);
|
|
@@ -776,71 +706,7 @@ export default function ChatAgentPanel({ onBack }) {
|
|
| 776 |
|
| 777 |
{/* Skills */}
|
| 778 |
<TabsContent value="skills" className="space-y-4">
|
| 779 |
-
<
|
| 780 |
-
<div className="font-semibold text-slate-800">Skills & guardrails</div>
|
| 781 |
-
<p className="text-sm text-slate-600">
|
| 782 |
-
Instructions the agent must always follow. Non-negotiable — the agent is told to
|
| 783 |
-
keep these even under pressure from a visitor.
|
| 784 |
-
</p>
|
| 785 |
-
<div className="flex flex-col sm:flex-row gap-2">
|
| 786 |
-
<Textarea
|
| 787 |
-
placeholder="e.g. Always offer to schedule a demo call when asked about implementation."
|
| 788 |
-
value={newSkillText}
|
| 789 |
-
onChange={(e) => setNewSkillText(e.target.value)}
|
| 790 |
-
rows={2}
|
| 791 |
-
className="text-sm"
|
| 792 |
-
/>
|
| 793 |
-
<Button type="button" onClick={addSkill} disabled={addingSkill} className="shrink-0 gap-1">
|
| 794 |
-
{addingSkill ? <Loader2 className="h-4 w-4 animate-spin" /> : <Plus className="h-4 w-4" />}
|
| 795 |
-
Add skill
|
| 796 |
-
</Button>
|
| 797 |
-
</div>
|
| 798 |
-
|
| 799 |
-
{skillsLoading ? (
|
| 800 |
-
<div className="flex justify-center py-8 text-slate-400">
|
| 801 |
-
<Loader2 className="h-8 w-8 animate-spin" />
|
| 802 |
-
</div>
|
| 803 |
-
) : skills.length === 0 ? (
|
| 804 |
-
<p className="text-sm text-slate-500">No skills yet.</p>
|
| 805 |
-
) : (
|
| 806 |
-
<ul className="space-y-2">
|
| 807 |
-
{skills.map((s) => (
|
| 808 |
-
<li
|
| 809 |
-
key={s.id}
|
| 810 |
-
className={cn(
|
| 811 |
-
'flex items-start gap-3 rounded-xl border px-3 py-2.5',
|
| 812 |
-
s.is_active ? 'border-slate-200' : 'border-slate-100 bg-slate-50/60 opacity-60'
|
| 813 |
-
)}
|
| 814 |
-
>
|
| 815 |
-
<input
|
| 816 |
-
type="checkbox"
|
| 817 |
-
className="mt-1 h-4 w-4 accent-violet-600"
|
| 818 |
-
checked={!!s.is_active}
|
| 819 |
-
onChange={() => toggleSkillActive(s)}
|
| 820 |
-
title="Active"
|
| 821 |
-
/>
|
| 822 |
-
<p className="flex-1 text-sm text-slate-700">
|
| 823 |
-
{s.text}
|
| 824 |
-
{s.is_default ? (
|
| 825 |
-
<span className="ml-2 inline-flex rounded-full bg-violet-100 px-1.5 py-0.5 text-[10px] font-medium text-violet-800">
|
| 826 |
-
default
|
| 827 |
-
</span>
|
| 828 |
-
) : null}
|
| 829 |
-
</p>
|
| 830 |
-
<Button
|
| 831 |
-
type="button"
|
| 832 |
-
variant="ghost"
|
| 833 |
-
size="icon"
|
| 834 |
-
className="h-8 w-8 text-red-500 shrink-0"
|
| 835 |
-
onClick={() => deleteSkill(s)}
|
| 836 |
-
>
|
| 837 |
-
<Trash2 className="h-4 w-4" />
|
| 838 |
-
</Button>
|
| 839 |
-
</li>
|
| 840 |
-
))}
|
| 841 |
-
</ul>
|
| 842 |
-
)}
|
| 843 |
-
</section>
|
| 844 |
</TabsContent>
|
| 845 |
|
| 846 |
{/* Test Chat */}
|
|
|
|
| 7 |
Loader2,
|
| 8 |
Mail,
|
| 9 |
MessageSquare,
|
|
|
|
| 10 |
RefreshCw,
|
| 11 |
Send,
|
|
|
|
| 12 |
} from 'lucide-react';
|
| 13 |
import AppShell from '@/components/layout/AppShell';
|
| 14 |
import { Button } from '@/components/ui/button';
|
|
|
|
| 22 |
SelectTrigger,
|
| 23 |
SelectValue,
|
| 24 |
} from '@/components/ui/select';
|
| 25 |
+
import ChatAgentSkillsTab from '@/components/settings/ChatAgentSkillsTab';
|
| 26 |
import { apiFetch } from '@/lib/api';
|
| 27 |
import { cn } from '@/lib/utils';
|
| 28 |
|
|
|
|
| 324 |
const [error, setError] = useState('');
|
| 325 |
const [savingConfig, setSavingConfig] = useState(false);
|
| 326 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 327 |
const [status, setStatus] = useState(null);
|
| 328 |
const [statusLoading, setStatusLoading] = useState(true);
|
| 329 |
const [reindexBusy, setReindexBusy] = useState('');
|
|
|
|
| 355 |
}
|
| 356 |
}, []);
|
| 357 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 358 |
const loadStatus = useCallback(async () => {
|
| 359 |
setStatusLoading(true);
|
| 360 |
try {
|
|
|
|
| 383 |
|
| 384 |
useEffect(() => {
|
| 385 |
loadConfig();
|
|
|
|
| 386 |
loadStatus();
|
| 387 |
loadConversations();
|
| 388 |
+
}, [loadConfig, loadStatus, loadConversations]);
|
| 389 |
|
| 390 |
// Reflect Drive OAuth callback redirect (?chat_agent_drive=connected / ?chat_agent_drive_error=...)
|
| 391 |
useEffect(() => {
|
|
|
|
| 465 |
}
|
| 466 |
};
|
| 467 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 468 |
const openConversation = async (id) => {
|
| 469 |
try {
|
| 470 |
const r = await apiFetch(`/api/chat-agent/conversations/${id}`);
|
|
|
|
| 706 |
|
| 707 |
{/* Skills */}
|
| 708 |
<TabsContent value="skills" className="space-y-4">
|
| 709 |
+
<ChatAgentSkillsTab />
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 710 |
</TabsContent>
|
| 711 |
|
| 712 |
{/* Test Chat */}
|
|
@@ -0,0 +1,724 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import React, { useCallback, useEffect, useRef, useState } from 'react';
|
| 2 |
+
import {
|
| 3 |
+
AlertTriangle,
|
| 4 |
+
CheckCircle2,
|
| 5 |
+
ChevronDown,
|
| 6 |
+
ChevronRight,
|
| 7 |
+
Download,
|
| 8 |
+
FlaskConical,
|
| 9 |
+
Loader2,
|
| 10 |
+
Plus,
|
| 11 |
+
Trash2,
|
| 12 |
+
Upload,
|
| 13 |
+
XCircle,
|
| 14 |
+
} from 'lucide-react';
|
| 15 |
+
import { Button } from '@/components/ui/button';
|
| 16 |
+
import { Input } from '@/components/ui/input';
|
| 17 |
+
import { Textarea } from '@/components/ui/textarea';
|
| 18 |
+
import { apiFetch } from '@/lib/api';
|
| 19 |
+
import { cn } from '@/lib/utils';
|
| 20 |
+
|
| 21 |
+
const AUTOSAVE_MS = 600;
|
| 22 |
+
|
| 23 |
+
const ASSERTION_LABELS = {
|
| 24 |
+
must_contain: 'Response must contain',
|
| 25 |
+
must_not_contain: 'Response must NOT contain',
|
| 26 |
+
llm_judge: 'LLM judge rubric',
|
| 27 |
+
};
|
| 28 |
+
|
| 29 |
+
function draftFromSkill(s) {
|
| 30 |
+
return {
|
| 31 |
+
name: s.name || '',
|
| 32 |
+
description: s.description || '',
|
| 33 |
+
instructions: s.instructions || '',
|
| 34 |
+
references: s.references || [],
|
| 35 |
+
assets: s.assets || [],
|
| 36 |
+
scripts: s.scripts || [],
|
| 37 |
+
evals: s.evals || [],
|
| 38 |
+
};
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
function draftSnapshot(d) {
|
| 42 |
+
return JSON.stringify(d);
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
function emptyDraft() {
|
| 46 |
+
return { name: '', description: '', instructions: '', references: [], assets: [], scripts: [], evals: [] };
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
/** One collapsible References/Assets/Scripts/Evals section shared shell. */
|
| 50 |
+
function Section({ title, icon, count, open, onToggle, notice, children }) {
|
| 51 |
+
return (
|
| 52 |
+
<div className="rounded-xl border border-slate-100">
|
| 53 |
+
<button
|
| 54 |
+
type="button"
|
| 55 |
+
onClick={onToggle}
|
| 56 |
+
className="flex w-full items-center gap-2 px-3 py-2 text-left text-sm font-medium text-slate-700 hover:bg-slate-50 rounded-xl"
|
| 57 |
+
>
|
| 58 |
+
{open ? <ChevronDown className="h-3.5 w-3.5 shrink-0" /> : <ChevronRight className="h-3.5 w-3.5 shrink-0" />}
|
| 59 |
+
{icon}
|
| 60 |
+
{title}
|
| 61 |
+
<span className="text-xs font-normal text-slate-400">{count}</span>
|
| 62 |
+
</button>
|
| 63 |
+
{notice ? (
|
| 64 |
+
<div className="flex items-start gap-1.5 px-3 pb-1 text-xs text-amber-700">
|
| 65 |
+
<AlertTriangle className="h-3.5 w-3.5 shrink-0 mt-0.5" />
|
| 66 |
+
<span>{notice}</span>
|
| 67 |
+
</div>
|
| 68 |
+
) : null}
|
| 69 |
+
{open ? <div className="space-y-2 px-3 pb-3">{children}</div> : null}
|
| 70 |
+
</div>
|
| 71 |
+
);
|
| 72 |
+
}
|
| 73 |
+
|
| 74 |
+
function SkillCard({ skill, onSaved, onDeleted, onError, defaultExpanded }) {
|
| 75 |
+
const [draft, setDraft] = useState(() => draftFromSkill(skill));
|
| 76 |
+
const [saveState, setSaveState] = useState('idle');
|
| 77 |
+
const [sectionsOpen, setSectionsOpen] = useState({
|
| 78 |
+
references: !!defaultExpanded,
|
| 79 |
+
assets: false,
|
| 80 |
+
scripts: false,
|
| 81 |
+
evals: false,
|
| 82 |
+
});
|
| 83 |
+
const [deleteBusy, setDeleteBusy] = useState(false);
|
| 84 |
+
const [evalRunBusy, setEvalRunBusy] = useState(false);
|
| 85 |
+
const [evalResults, setEvalResults] = useState(null);
|
| 86 |
+
|
| 87 |
+
const skipAutosaveRef = useRef(true);
|
| 88 |
+
const saveTimerRef = useRef(null);
|
| 89 |
+
const lastSyncedSnapshotRef = useRef(draftSnapshot(draftFromSkill(skill)));
|
| 90 |
+
const savedFadeTimerRef = useRef(null);
|
| 91 |
+
|
| 92 |
+
// Resync when the skill changes underneath us (e.g. an evals/run response overwrote
|
| 93 |
+
// last_run_status), but not when it's just an echo of what we ourselves just saved.
|
| 94 |
+
useEffect(() => {
|
| 95 |
+
const next = draftFromSkill(skill);
|
| 96 |
+
const snap = draftSnapshot(next);
|
| 97 |
+
if (snap === lastSyncedSnapshotRef.current) return;
|
| 98 |
+
skipAutosaveRef.current = true;
|
| 99 |
+
lastSyncedSnapshotRef.current = snap;
|
| 100 |
+
setDraft(next);
|
| 101 |
+
}, [skill]);
|
| 102 |
+
|
| 103 |
+
useEffect(() => {
|
| 104 |
+
if (skipAutosaveRef.current) {
|
| 105 |
+
skipAutosaveRef.current = false;
|
| 106 |
+
return;
|
| 107 |
+
}
|
| 108 |
+
const snapshot = draftSnapshot(draft);
|
| 109 |
+
if (snapshot === lastSyncedSnapshotRef.current) return;
|
| 110 |
+
if (!draft.name.trim() || !draft.instructions.trim()) return;
|
| 111 |
+
|
| 112 |
+
if (saveTimerRef.current) clearTimeout(saveTimerRef.current);
|
| 113 |
+
saveTimerRef.current = setTimeout(async () => {
|
| 114 |
+
if (draftSnapshot(draft) !== snapshot) return;
|
| 115 |
+
setSaveState('saving');
|
| 116 |
+
try {
|
| 117 |
+
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}`, {
|
| 118 |
+
method: 'PATCH',
|
| 119 |
+
headers: { 'Content-Type': 'application/json' },
|
| 120 |
+
body: JSON.stringify({
|
| 121 |
+
name: draft.name.trim(),
|
| 122 |
+
description: draft.description,
|
| 123 |
+
instructions: draft.instructions.trim(),
|
| 124 |
+
references: draft.references,
|
| 125 |
+
assets: draft.assets,
|
| 126 |
+
scripts: draft.scripts,
|
| 127 |
+
evals: draft.evals,
|
| 128 |
+
}),
|
| 129 |
+
});
|
| 130 |
+
const data = await res.json();
|
| 131 |
+
if (!res.ok) throw new Error(data.detail || 'Save failed');
|
| 132 |
+
lastSyncedSnapshotRef.current = draftSnapshot(draftFromSkill(data));
|
| 133 |
+
onSaved(data);
|
| 134 |
+
if (savedFadeTimerRef.current) clearTimeout(savedFadeTimerRef.current);
|
| 135 |
+
setSaveState('saved');
|
| 136 |
+
savedFadeTimerRef.current = window.setTimeout(() => setSaveState('idle'), 2000);
|
| 137 |
+
} catch (e) {
|
| 138 |
+
onError(e.message || 'Save failed');
|
| 139 |
+
setSaveState('error');
|
| 140 |
+
}
|
| 141 |
+
}, AUTOSAVE_MS);
|
| 142 |
+
return () => {
|
| 143 |
+
if (saveTimerRef.current) clearTimeout(saveTimerRef.current);
|
| 144 |
+
};
|
| 145 |
+
// eslint-disable-next-line react-hooks/exhaustive-deps
|
| 146 |
+
}, [draft]);
|
| 147 |
+
|
| 148 |
+
const toggleSection = (key) => setSectionsOpen((s) => ({ ...s, [key]: !s[key] }));
|
| 149 |
+
|
| 150 |
+
const updateListItem = (key, idx, patch) => {
|
| 151 |
+
setDraft((d) => ({ ...d, [key]: d[key].map((item, i) => (i === idx ? { ...item, ...patch } : item)) }));
|
| 152 |
+
};
|
| 153 |
+
const addListItem = (key, item) => setDraft((d) => ({ ...d, [key]: [...d[key], item] }));
|
| 154 |
+
const removeListItem = (key, idx) => setDraft((d) => ({ ...d, [key]: d[key].filter((_, i) => i !== idx) }));
|
| 155 |
+
|
| 156 |
+
const toggleActive = async () => {
|
| 157 |
+
try {
|
| 158 |
+
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}`, {
|
| 159 |
+
method: 'PATCH',
|
| 160 |
+
headers: { 'Content-Type': 'application/json' },
|
| 161 |
+
body: JSON.stringify({ is_active: !skill.is_active }),
|
| 162 |
+
});
|
| 163 |
+
const data = await res.json();
|
| 164 |
+
if (!res.ok) throw new Error(data.detail || 'Update failed');
|
| 165 |
+
onSaved(data);
|
| 166 |
+
} catch (e) {
|
| 167 |
+
onError(e.message || 'Update failed');
|
| 168 |
+
}
|
| 169 |
+
};
|
| 170 |
+
|
| 171 |
+
const handleDelete = async () => {
|
| 172 |
+
if (
|
| 173 |
+
!window.confirm(
|
| 174 |
+
`Delete skill "${skill.name || 'untitled'}"? This also removes its indexed references/assets from the knowledge base.`
|
| 175 |
+
)
|
| 176 |
+
)
|
| 177 |
+
return;
|
| 178 |
+
setDeleteBusy(true);
|
| 179 |
+
try {
|
| 180 |
+
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}`, { method: 'DELETE' });
|
| 181 |
+
if (!res.ok) {
|
| 182 |
+
const data = await res.json().catch(() => ({}));
|
| 183 |
+
throw new Error(data.detail || 'Delete failed');
|
| 184 |
+
}
|
| 185 |
+
onDeleted(skill.id);
|
| 186 |
+
} catch (e) {
|
| 187 |
+
onError(e.message || 'Delete failed');
|
| 188 |
+
setDeleteBusy(false);
|
| 189 |
+
}
|
| 190 |
+
};
|
| 191 |
+
|
| 192 |
+
const handleExport = () => {
|
| 193 |
+
window.open(`/api/chat-agent/skills/${skill.id}/export`, '_blank');
|
| 194 |
+
};
|
| 195 |
+
|
| 196 |
+
const runEvals = async () => {
|
| 197 |
+
setEvalRunBusy(true);
|
| 198 |
+
setEvalResults(null);
|
| 199 |
+
try {
|
| 200 |
+
const res = await apiFetch(`/api/chat-agent/skills/${skill.id}/evals/run`, { method: 'POST' });
|
| 201 |
+
const data = await res.json();
|
| 202 |
+
if (!res.ok) throw new Error(data.detail || 'Eval run failed');
|
| 203 |
+
onSaved(data.skill);
|
| 204 |
+
setEvalResults(data.results);
|
| 205 |
+
} catch (e) {
|
| 206 |
+
onError(e.message || 'Eval run failed');
|
| 207 |
+
} finally {
|
| 208 |
+
setEvalRunBusy(false);
|
| 209 |
+
}
|
| 210 |
+
};
|
| 211 |
+
|
| 212 |
+
return (
|
| 213 |
+
<div
|
| 214 |
+
className={cn(
|
| 215 |
+
'rounded-2xl border bg-white p-4 shadow-sm space-y-3',
|
| 216 |
+
skill.is_active ? 'border-slate-200' : 'border-slate-100 bg-slate-50/60 opacity-70'
|
| 217 |
+
)}
|
| 218 |
+
>
|
| 219 |
+
<div className="flex items-start gap-3">
|
| 220 |
+
<input
|
| 221 |
+
type="checkbox"
|
| 222 |
+
className="mt-2.5 h-4 w-4 accent-violet-600 shrink-0"
|
| 223 |
+
checked={!!skill.is_active}
|
| 224 |
+
onChange={toggleActive}
|
| 225 |
+
title="Active"
|
| 226 |
+
/>
|
| 227 |
+
<div className="flex-1 space-y-2 min-w-0">
|
| 228 |
+
<div className="flex flex-wrap items-center gap-2">
|
| 229 |
+
<Input
|
| 230 |
+
value={draft.name}
|
| 231 |
+
onChange={(e) => setDraft((d) => ({ ...d, name: e.target.value }))}
|
| 232 |
+
placeholder="skill-name"
|
| 233 |
+
className="h-8 max-w-[220px] font-medium"
|
| 234 |
+
/>
|
| 235 |
+
{skill.is_default ? (
|
| 236 |
+
<span className="inline-flex rounded-full bg-violet-100 px-1.5 py-0.5 text-[10px] font-medium text-violet-800">
|
| 237 |
+
default
|
| 238 |
+
</span>
|
| 239 |
+
) : null}
|
| 240 |
+
<span className="text-xs text-slate-400">
|
| 241 |
+
{saveState === 'saving' ? 'Saving…' : saveState === 'saved' ? 'Saved' : saveState === 'error' ? 'Save failed' : ''}
|
| 242 |
+
</span>
|
| 243 |
+
<div className="ml-auto flex items-center gap-1">
|
| 244 |
+
<Button type="button" variant="ghost" size="sm" className="h-7 gap-1 text-xs" onClick={handleExport}>
|
| 245 |
+
<Download className="h-3.5 w-3.5" />
|
| 246 |
+
Export
|
| 247 |
+
</Button>
|
| 248 |
+
<Button
|
| 249 |
+
type="button"
|
| 250 |
+
variant="ghost"
|
| 251 |
+
size="icon"
|
| 252 |
+
className="h-7 w-7 text-red-500"
|
| 253 |
+
disabled={deleteBusy}
|
| 254 |
+
onClick={handleDelete}
|
| 255 |
+
>
|
| 256 |
+
{deleteBusy ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <Trash2 className="h-3.5 w-3.5" />}
|
| 257 |
+
</Button>
|
| 258 |
+
</div>
|
| 259 |
+
</div>
|
| 260 |
+
<Input
|
| 261 |
+
value={draft.description}
|
| 262 |
+
onChange={(e) => setDraft((d) => ({ ...d, description: e.target.value }))}
|
| 263 |
+
placeholder="One-line description (when the agent should use this skill)"
|
| 264 |
+
className="h-8 text-sm"
|
| 265 |
+
/>
|
| 266 |
+
<Textarea
|
| 267 |
+
value={draft.instructions}
|
| 268 |
+
onChange={(e) => setDraft((d) => ({ ...d, instructions: e.target.value }))}
|
| 269 |
+
placeholder="Instructions — the brain of this skill. Always injected into the agent's system prompt."
|
| 270 |
+
rows={3}
|
| 271 |
+
className="text-sm"
|
| 272 |
+
/>
|
| 273 |
+
</div>
|
| 274 |
+
</div>
|
| 275 |
+
|
| 276 |
+
<div className="space-y-2 pl-7">
|
| 277 |
+
<Section
|
| 278 |
+
title="References"
|
| 279 |
+
icon={null}
|
| 280 |
+
count={draft.references.length}
|
| 281 |
+
open={sectionsOpen.references}
|
| 282 |
+
onToggle={() => toggleSection('references')}
|
| 283 |
+
>
|
| 284 |
+
<p className="text-xs text-slate-500">
|
| 285 |
+
Domain knowledge the agent can search for on demand (indexed into the knowledge base).
|
| 286 |
+
</p>
|
| 287 |
+
{draft.references.map((r, idx) => (
|
| 288 |
+
<div key={idx} className="space-y-1 rounded-lg border border-slate-100 p-2">
|
| 289 |
+
<div className="flex items-center gap-2">
|
| 290 |
+
<Input
|
| 291 |
+
value={r.title}
|
| 292 |
+
onChange={(e) => updateListItem('references', idx, { title: e.target.value })}
|
| 293 |
+
placeholder="Title"
|
| 294 |
+
className="h-7 text-xs"
|
| 295 |
+
/>
|
| 296 |
+
<Button
|
| 297 |
+
type="button"
|
| 298 |
+
variant="ghost"
|
| 299 |
+
size="icon"
|
| 300 |
+
className="h-7 w-7 text-red-500 shrink-0"
|
| 301 |
+
onClick={() => removeListItem('references', idx)}
|
| 302 |
+
>
|
| 303 |
+
<Trash2 className="h-3.5 w-3.5" />
|
| 304 |
+
</Button>
|
| 305 |
+
</div>
|
| 306 |
+
<Textarea
|
| 307 |
+
value={r.content}
|
| 308 |
+
onChange={(e) => updateListItem('references', idx, { content: e.target.value })}
|
| 309 |
+
placeholder="Reference content..."
|
| 310 |
+
rows={2}
|
| 311 |
+
className="text-xs"
|
| 312 |
+
/>
|
| 313 |
+
</div>
|
| 314 |
+
))}
|
| 315 |
+
<Button
|
| 316 |
+
type="button"
|
| 317 |
+
variant="outline"
|
| 318 |
+
size="sm"
|
| 319 |
+
className="h-7 gap-1 text-xs"
|
| 320 |
+
onClick={() => addListItem('references', { title: '', content: '' })}
|
| 321 |
+
>
|
| 322 |
+
<Plus className="h-3.5 w-3.5" />
|
| 323 |
+
Add reference
|
| 324 |
+
</Button>
|
| 325 |
+
</Section>
|
| 326 |
+
|
| 327 |
+
<Section
|
| 328 |
+
title="Assets"
|
| 329 |
+
icon={null}
|
| 330 |
+
count={draft.assets.length}
|
| 331 |
+
open={sectionsOpen.assets}
|
| 332 |
+
onToggle={() => toggleSection('assets')}
|
| 333 |
+
>
|
| 334 |
+
<p className="text-xs text-slate-500">
|
| 335 |
+
Output templates/snippets the agent can search for and reuse when answering.
|
| 336 |
+
</p>
|
| 337 |
+
{draft.assets.map((a, idx) => (
|
| 338 |
+
<div key={idx} className="space-y-1 rounded-lg border border-slate-100 p-2">
|
| 339 |
+
<div className="flex items-center gap-2">
|
| 340 |
+
<Input
|
| 341 |
+
value={a.name}
|
| 342 |
+
onChange={(e) => updateListItem('assets', idx, { name: e.target.value })}
|
| 343 |
+
placeholder="Name"
|
| 344 |
+
className="h-7 text-xs"
|
| 345 |
+
/>
|
| 346 |
+
<Button
|
| 347 |
+
type="button"
|
| 348 |
+
variant="ghost"
|
| 349 |
+
size="icon"
|
| 350 |
+
className="h-7 w-7 text-red-500 shrink-0"
|
| 351 |
+
onClick={() => removeListItem('assets', idx)}
|
| 352 |
+
>
|
| 353 |
+
<Trash2 className="h-3.5 w-3.5" />
|
| 354 |
+
</Button>
|
| 355 |
+
</div>
|
| 356 |
+
<Textarea
|
| 357 |
+
value={a.content}
|
| 358 |
+
onChange={(e) => updateListItem('assets', idx, { content: e.target.value })}
|
| 359 |
+
placeholder="Asset content..."
|
| 360 |
+
rows={2}
|
| 361 |
+
className="text-xs"
|
| 362 |
+
/>
|
| 363 |
+
</div>
|
| 364 |
+
))}
|
| 365 |
+
<Button
|
| 366 |
+
type="button"
|
| 367 |
+
variant="outline"
|
| 368 |
+
size="sm"
|
| 369 |
+
className="h-7 gap-1 text-xs"
|
| 370 |
+
onClick={() => addListItem('assets', { name: '', content: '' })}
|
| 371 |
+
>
|
| 372 |
+
<Plus className="h-3.5 w-3.5" />
|
| 373 |
+
Add asset
|
| 374 |
+
</Button>
|
| 375 |
+
</Section>
|
| 376 |
+
|
| 377 |
+
<Section
|
| 378 |
+
title="Scripts"
|
| 379 |
+
icon={null}
|
| 380 |
+
count={draft.scripts.length}
|
| 381 |
+
open={sectionsOpen.scripts}
|
| 382 |
+
onToggle={() => toggleSection('scripts')}
|
| 383 |
+
notice="Stored for documentation/portability only — never executed by the agent. See SKILLS-SECURITY.md."
|
| 384 |
+
>
|
| 385 |
+
{draft.scripts.map((s, idx) => (
|
| 386 |
+
<div key={idx} className="space-y-1 rounded-lg border border-slate-100 p-2">
|
| 387 |
+
<div className="flex items-center gap-2">
|
| 388 |
+
<Input
|
| 389 |
+
value={s.name}
|
| 390 |
+
onChange={(e) => updateListItem('scripts', idx, { name: e.target.value })}
|
| 391 |
+
placeholder="Name"
|
| 392 |
+
className="h-7 text-xs"
|
| 393 |
+
/>
|
| 394 |
+
<Input
|
| 395 |
+
value={s.language}
|
| 396 |
+
onChange={(e) => updateListItem('scripts', idx, { language: e.target.value })}
|
| 397 |
+
placeholder="language"
|
| 398 |
+
className="h-7 w-28 text-xs"
|
| 399 |
+
/>
|
| 400 |
+
<Button
|
| 401 |
+
type="button"
|
| 402 |
+
variant="ghost"
|
| 403 |
+
size="icon"
|
| 404 |
+
className="h-7 w-7 text-red-500 shrink-0"
|
| 405 |
+
onClick={() => removeListItem('scripts', idx)}
|
| 406 |
+
>
|
| 407 |
+
<Trash2 className="h-3.5 w-3.5" />
|
| 408 |
+
</Button>
|
| 409 |
+
</div>
|
| 410 |
+
<Input
|
| 411 |
+
value={s.description}
|
| 412 |
+
onChange={(e) => updateListItem('scripts', idx, { description: e.target.value })}
|
| 413 |
+
placeholder="What this script is for"
|
| 414 |
+
className="h-7 text-xs"
|
| 415 |
+
/>
|
| 416 |
+
<Textarea
|
| 417 |
+
value={s.content}
|
| 418 |
+
onChange={(e) => updateListItem('scripts', idx, { content: e.target.value })}
|
| 419 |
+
placeholder="Script content (not executed)..."
|
| 420 |
+
rows={3}
|
| 421 |
+
className="text-xs font-mono"
|
| 422 |
+
/>
|
| 423 |
+
</div>
|
| 424 |
+
))}
|
| 425 |
+
<Button
|
| 426 |
+
type="button"
|
| 427 |
+
variant="outline"
|
| 428 |
+
size="sm"
|
| 429 |
+
className="h-7 gap-1 text-xs"
|
| 430 |
+
onClick={() => addListItem('scripts', { name: '', description: '', language: '', content: '' })}
|
| 431 |
+
>
|
| 432 |
+
<Plus className="h-3.5 w-3.5" />
|
| 433 |
+
Add script
|
| 434 |
+
</Button>
|
| 435 |
+
</Section>
|
| 436 |
+
|
| 437 |
+
<Section
|
| 438 |
+
title="Evals"
|
| 439 |
+
icon={<FlaskConical className="h-3.5 w-3.5 shrink-0" />}
|
| 440 |
+
count={draft.evals.length}
|
| 441 |
+
open={sectionsOpen.evals}
|
| 442 |
+
onToggle={() => toggleSection('evals')}
|
| 443 |
+
>
|
| 444 |
+
<p className="text-xs text-slate-500">Test cases — a prompt plus an assertion the agent's reply must satisfy.</p>
|
| 445 |
+
{draft.evals.map((ev, idx) => {
|
| 446 |
+
const result = evalResults?.[idx];
|
| 447 |
+
return (
|
| 448 |
+
<div key={idx} className="space-y-1 rounded-lg border border-slate-100 p-2">
|
| 449 |
+
<div className="flex items-center gap-2">
|
| 450 |
+
<Input
|
| 451 |
+
value={ev.prompt}
|
| 452 |
+
onChange={(e) => updateListItem('evals', idx, { prompt: e.target.value })}
|
| 453 |
+
placeholder="Prompt to send the agent"
|
| 454 |
+
className="h-7 text-xs"
|
| 455 |
+
/>
|
| 456 |
+
<Button
|
| 457 |
+
type="button"
|
| 458 |
+
variant="ghost"
|
| 459 |
+
size="icon"
|
| 460 |
+
className="h-7 w-7 text-red-500 shrink-0"
|
| 461 |
+
onClick={() => removeListItem('evals', idx)}
|
| 462 |
+
>
|
| 463 |
+
<Trash2 className="h-3.5 w-3.5" />
|
| 464 |
+
</Button>
|
| 465 |
+
</div>
|
| 466 |
+
<div className="flex items-center gap-2">
|
| 467 |
+
<select
|
| 468 |
+
value={ev.assertion_type}
|
| 469 |
+
onChange={(e) => updateListItem('evals', idx, { assertion_type: e.target.value })}
|
| 470 |
+
className="h-7 rounded-md border border-slate-200 bg-white text-xs px-1.5"
|
| 471 |
+
>
|
| 472 |
+
{Object.entries(ASSERTION_LABELS).map(([v, label]) => (
|
| 473 |
+
<option key={v} value={v}>
|
| 474 |
+
{label}
|
| 475 |
+
</option>
|
| 476 |
+
))}
|
| 477 |
+
</select>
|
| 478 |
+
<Input
|
| 479 |
+
value={ev.assertion_value}
|
| 480 |
+
onChange={(e) => updateListItem('evals', idx, { assertion_value: e.target.value })}
|
| 481 |
+
placeholder={
|
| 482 |
+
ev.assertion_type === 'llm_judge' ? 'Rubric describing a passing answer' : 'Text to check for'
|
| 483 |
+
}
|
| 484 |
+
className="h-7 text-xs flex-1"
|
| 485 |
+
/>
|
| 486 |
+
</div>
|
| 487 |
+
{ev.last_run_status ? (
|
| 488 |
+
<div
|
| 489 |
+
className={cn(
|
| 490 |
+
'flex items-start gap-1.5 rounded-md px-2 py-1 text-xs',
|
| 491 |
+
ev.last_run_status === 'pass'
|
| 492 |
+
? 'bg-emerald-50 text-emerald-800'
|
| 493 |
+
: ev.last_run_status === 'fail'
|
| 494 |
+
? 'bg-red-50 text-red-700'
|
| 495 |
+
: 'bg-slate-100 text-slate-600'
|
| 496 |
+
)}
|
| 497 |
+
>
|
| 498 |
+
{ev.last_run_status === 'pass' ? (
|
| 499 |
+
<CheckCircle2 className="h-3.5 w-3.5 shrink-0 mt-px" />
|
| 500 |
+
) : (
|
| 501 |
+
<XCircle className="h-3.5 w-3.5 shrink-0 mt-px" />
|
| 502 |
+
)}
|
| 503 |
+
<span>
|
| 504 |
+
{ev.last_run_status.toUpperCase()} · {ev.last_run_detail || ''}
|
| 505 |
+
{ev.last_run_at ? ` · ${new Date(ev.last_run_at).toLocaleString()}` : ''}
|
| 506 |
+
</span>
|
| 507 |
+
</div>
|
| 508 |
+
) : null}
|
| 509 |
+
{result ? (
|
| 510 |
+
<p className="text-[11px] text-slate-400 truncate" title={result.response}>
|
| 511 |
+
Response: {result.response || '(empty)'}
|
| 512 |
+
</p>
|
| 513 |
+
) : null}
|
| 514 |
+
</div>
|
| 515 |
+
);
|
| 516 |
+
})}
|
| 517 |
+
<div className="flex flex-wrap items-center gap-2">
|
| 518 |
+
<Button
|
| 519 |
+
type="button"
|
| 520 |
+
variant="outline"
|
| 521 |
+
size="sm"
|
| 522 |
+
className="h-7 gap-1 text-xs"
|
| 523 |
+
onClick={() =>
|
| 524 |
+
addListItem('evals', { prompt: '', assertion_type: 'must_contain', assertion_value: '' })
|
| 525 |
+
}
|
| 526 |
+
>
|
| 527 |
+
<Plus className="h-3.5 w-3.5" />
|
| 528 |
+
Add eval
|
| 529 |
+
</Button>
|
| 530 |
+
{draft.evals.length > 0 ? (
|
| 531 |
+
<Button
|
| 532 |
+
type="button"
|
| 533 |
+
variant="outline"
|
| 534 |
+
size="sm"
|
| 535 |
+
className="h-7 gap-1 text-xs"
|
| 536 |
+
disabled={evalRunBusy}
|
| 537 |
+
onClick={runEvals}
|
| 538 |
+
>
|
| 539 |
+
{evalRunBusy ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <FlaskConical className="h-3.5 w-3.5" />}
|
| 540 |
+
Run evals
|
| 541 |
+
</Button>
|
| 542 |
+
) : null}
|
| 543 |
+
</div>
|
| 544 |
+
</Section>
|
| 545 |
+
</div>
|
| 546 |
+
</div>
|
| 547 |
+
);
|
| 548 |
+
}
|
| 549 |
+
|
| 550 |
+
export default function ChatAgentSkillsTab() {
|
| 551 |
+
const [skills, setSkills] = useState([]);
|
| 552 |
+
const [loading, setLoading] = useState(true);
|
| 553 |
+
const [error, setError] = useState('');
|
| 554 |
+
const [creating, setCreating] = useState(false);
|
| 555 |
+
const [lastCreatedId, setLastCreatedId] = useState(null);
|
| 556 |
+
|
| 557 |
+
const [importOpen, setImportOpen] = useState(false);
|
| 558 |
+
const [importText, setImportText] = useState('');
|
| 559 |
+
const [importBusy, setImportBusy] = useState(false);
|
| 560 |
+
const fileInputRef = useRef(null);
|
| 561 |
+
|
| 562 |
+
const loadSkills = useCallback(async () => {
|
| 563 |
+
setLoading(true);
|
| 564 |
+
try {
|
| 565 |
+
const r = await apiFetch('/api/chat-agent/skills');
|
| 566 |
+
const data = r.ok ? await r.json() : { skills: [] };
|
| 567 |
+
setSkills(data.skills || []);
|
| 568 |
+
} catch {
|
| 569 |
+
setSkills([]);
|
| 570 |
+
} finally {
|
| 571 |
+
setLoading(false);
|
| 572 |
+
}
|
| 573 |
+
}, []);
|
| 574 |
+
|
| 575 |
+
useEffect(() => {
|
| 576 |
+
loadSkills();
|
| 577 |
+
}, [loadSkills]);
|
| 578 |
+
|
| 579 |
+
const handleSaved = (updated) => {
|
| 580 |
+
setSkills((prev) => prev.map((s) => (s.id === updated.id ? updated : s)));
|
| 581 |
+
};
|
| 582 |
+
const handleDeleted = (id) => {
|
| 583 |
+
setSkills((prev) => prev.filter((s) => s.id !== id));
|
| 584 |
+
};
|
| 585 |
+
|
| 586 |
+
const addSkill = async () => {
|
| 587 |
+
setCreating(true);
|
| 588 |
+
setError('');
|
| 589 |
+
try {
|
| 590 |
+
const res = await apiFetch('/api/chat-agent/skills', {
|
| 591 |
+
method: 'POST',
|
| 592 |
+
headers: { 'Content-Type': 'application/json' },
|
| 593 |
+
body: JSON.stringify({
|
| 594 |
+
name: `new-skill-${skills.length + 1}`,
|
| 595 |
+
description: '',
|
| 596 |
+
instructions: 'Describe what the agent should do or avoid doing.',
|
| 597 |
+
}),
|
| 598 |
+
});
|
| 599 |
+
const data = await res.json();
|
| 600 |
+
if (!res.ok) throw new Error(data.detail || 'Could not add skill');
|
| 601 |
+
setSkills((prev) => [...prev, data]);
|
| 602 |
+
setLastCreatedId(data.id);
|
| 603 |
+
} catch (e) {
|
| 604 |
+
setError(e.message || 'Could not add skill');
|
| 605 |
+
} finally {
|
| 606 |
+
setCreating(false);
|
| 607 |
+
}
|
| 608 |
+
};
|
| 609 |
+
|
| 610 |
+
const runImport = async () => {
|
| 611 |
+
const content = importText.trim();
|
| 612 |
+
if (!content) return;
|
| 613 |
+
setImportBusy(true);
|
| 614 |
+
setError('');
|
| 615 |
+
try {
|
| 616 |
+
const res = await apiFetch('/api/chat-agent/skills/import', {
|
| 617 |
+
method: 'POST',
|
| 618 |
+
headers: { 'Content-Type': 'application/json' },
|
| 619 |
+
body: JSON.stringify({ content }),
|
| 620 |
+
});
|
| 621 |
+
const data = await res.json();
|
| 622 |
+
if (!res.ok) throw new Error(data.detail || 'Import failed');
|
| 623 |
+
setSkills((prev) => [...prev, data]);
|
| 624 |
+
setLastCreatedId(data.id);
|
| 625 |
+
setImportText('');
|
| 626 |
+
setImportOpen(false);
|
| 627 |
+
} catch (e) {
|
| 628 |
+
setError(e.message || 'Import failed');
|
| 629 |
+
} finally {
|
| 630 |
+
setImportBusy(false);
|
| 631 |
+
}
|
| 632 |
+
};
|
| 633 |
+
|
| 634 |
+
const handleFilePicked = (e) => {
|
| 635 |
+
const file = e.target.files?.[0];
|
| 636 |
+
if (!file) return;
|
| 637 |
+
const reader = new FileReader();
|
| 638 |
+
reader.onload = () => setImportText(String(reader.result || ''));
|
| 639 |
+
reader.readAsText(file);
|
| 640 |
+
e.target.value = '';
|
| 641 |
+
};
|
| 642 |
+
|
| 643 |
+
return (
|
| 644 |
+
<section className="rounded-2xl border border-slate-200 bg-white p-6 shadow-sm space-y-4">
|
| 645 |
+
<div className="flex flex-wrap items-center justify-between gap-2">
|
| 646 |
+
<div>
|
| 647 |
+
<div className="font-semibold text-slate-800">Skills</div>
|
| 648 |
+
<p className="text-sm text-slate-600">
|
| 649 |
+
Each skill is a real, exportable SKILL.md: name + description + instructions (always
|
| 650 |
+
applied), plus optional references/assets the agent can search for, scripts (documentation
|
| 651 |
+
only), and evals (test cases).
|
| 652 |
+
</p>
|
| 653 |
+
</div>
|
| 654 |
+
<div className="flex items-center gap-2">
|
| 655 |
+
<Button type="button" variant="outline" size="sm" className="gap-1.5" onClick={() => setImportOpen((v) => !v)}>
|
| 656 |
+
<Upload className="h-3.5 w-3.5" />
|
| 657 |
+
Import SKILL.md
|
| 658 |
+
</Button>
|
| 659 |
+
<Button type="button" size="sm" className="gap-1.5" onClick={addSkill} disabled={creating}>
|
| 660 |
+
{creating ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <Plus className="h-3.5 w-3.5" />}
|
| 661 |
+
Add skill
|
| 662 |
+
</Button>
|
| 663 |
+
</div>
|
| 664 |
+
</div>
|
| 665 |
+
|
| 666 |
+
{importOpen ? (
|
| 667 |
+
<div className="space-y-2 rounded-xl border border-slate-100 bg-slate-50/60 p-3">
|
| 668 |
+
<div className="flex items-center gap-2">
|
| 669 |
+
<input
|
| 670 |
+
ref={fileInputRef}
|
| 671 |
+
type="file"
|
| 672 |
+
accept=".md,text/markdown,text/plain"
|
| 673 |
+
className="hidden"
|
| 674 |
+
onChange={handleFilePicked}
|
| 675 |
+
/>
|
| 676 |
+
<Button type="button" variant="outline" size="sm" onClick={() => fileInputRef.current?.click()}>
|
| 677 |
+
Choose file...
|
| 678 |
+
</Button>
|
| 679 |
+
<span className="text-xs text-slate-500">or paste SKILL.md content below</span>
|
| 680 |
+
</div>
|
| 681 |
+
<Textarea
|
| 682 |
+
value={importText}
|
| 683 |
+
onChange={(e) => setImportText(e.target.value)}
|
| 684 |
+
placeholder={'---\nname: my-skill\ndescription: ...\n---\n\nInstructions...'}
|
| 685 |
+
rows={6}
|
| 686 |
+
className="text-xs font-mono"
|
| 687 |
+
/>
|
| 688 |
+
<div className="flex justify-end gap-2">
|
| 689 |
+
<Button type="button" variant="ghost" size="sm" onClick={() => setImportOpen(false)}>
|
| 690 |
+
Cancel
|
| 691 |
+
</Button>
|
| 692 |
+
<Button type="button" size="sm" disabled={importBusy || !importText.trim()} onClick={runImport}>
|
| 693 |
+
{importBusy ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : null}
|
| 694 |
+
Import
|
| 695 |
+
</Button>
|
| 696 |
+
</div>
|
| 697 |
+
</div>
|
| 698 |
+
) : null}
|
| 699 |
+
|
| 700 |
+
{error ? <p className="text-sm text-red-600 rounded-lg border border-red-200 bg-red-50 px-3 py-2">{error}</p> : null}
|
| 701 |
+
|
| 702 |
+
{loading ? (
|
| 703 |
+
<div className="flex justify-center py-8 text-slate-400">
|
| 704 |
+
<Loader2 className="h-8 w-8 animate-spin" />
|
| 705 |
+
</div>
|
| 706 |
+
) : skills.length === 0 ? (
|
| 707 |
+
<p className="text-sm text-slate-500">No skills yet.</p>
|
| 708 |
+
) : (
|
| 709 |
+
<div className="space-y-3">
|
| 710 |
+
{skills.map((s) => (
|
| 711 |
+
<SkillCard
|
| 712 |
+
key={s.id}
|
| 713 |
+
skill={s}
|
| 714 |
+
onSaved={handleSaved}
|
| 715 |
+
onDeleted={handleDeleted}
|
| 716 |
+
onError={setError}
|
| 717 |
+
defaultExpanded={s.id === lastCreatedId}
|
| 718 |
+
/>
|
| 719 |
+
))}
|
| 720 |
+
</div>
|
| 721 |
+
)}
|
| 722 |
+
</section>
|
| 723 |
+
);
|
| 724 |
+
}
|