debug-log
Claude Opus 4.6
feat: Insight Repository โ€” GitLab ์ดˆ๊ธฐํ™” + HF ์ž๋™ ๋ฐฐํฌ
501bc8c
|
Raw History Blame Contribute Delete
6.45 kB

Task Management

  • [/] Research Data Sources
    • Analyze research_report.json structure
    • Analyze baymard_unified_database_v2.json structure
  • Draft Product Requirements Document (PRD)
    • Define User Personas and Goals
    • detailed Feature Specifications (Search, RAG, Image Analysis)
    • Define Technical Architecture (High-level)
    • Define Metrics and Success Criteria
  • Research File Linking
    • Explore research_pages directory naming convention
    • Create script to match JSON entries with file paths
    • Update research_report.json with file paths
  • Update File Structure
    • Update script for new paths and relative linking
    • Run script to fix research_report.json
  • Verify Match Quality
    • Create script to check similarity between JSON title and file path
    • Report ambiguous matches to user
  • Handle Unmatched Items
    • Mark missing file paths as "ํŒŒ์ผ ์—†์Œ"
  • [/] Review PRD with User
    • Review PRD with User
    • Update PRD with finalized data structure (file_path)
    • Request final PRD approval

Chatbot PoC Development (Metadata Search)

  • Setup Development Environment
    • Create virtual environment and install dependencies (langchain, google-genai)
    • Verify Gemini API Key (gemini-3-flash-preview)
  • Data Loader Implementation
    • Implement data_loader.py for both JSON sources
  • Search Logic Implementation
    • Implement keyword extraction & source routing with LLM
    • Implement filtering logic for JSON data
    • Implement answer generation with Gemini 3 Flash
  • Interface Implementation
    • Build CLI for testing
    • Verify specific used cases (Review, Internal Only, etc.)

Web App Development (Streamlit)

  • Setup Web Environment
    • Update requirements.txt with streamlit
    • Install new dependencies
  • Implement Web Interface
    • Create app.py using Streamlit chat elements
    • Integrate SearchEngine with Streamlit session state
  • Verification
    • Run Streamlit app and verify chat flow

Feature Enhancements

  • Add Conversation Context
    • Update search_engine.py to handle chat history
    • Update app.py to pass session history
    • Verify context-aware follow-up questions

RAG Integration Phase

  • Data Preparation: Assign Unique IDs
    • Create script to assign IDs to JSON and sync with MD files
    • Run ID assignment script
  • Vector DB Setup
    • Install search dependencies
    • Implement search indexing for separate collections (Internal/Baymard)
  • Hybrid Search Implementation
    • Update search_engine.py to combine Metadata + Vector Search
    • Verify RAG results
    • Enhance Baymard Indexing (PDF Extraction)

Multimodal Support

  • Implement File Upload in Web App

    • Add st.file_uploader to app.py
    • Update search_engine.py to handle image input
  • Verification

  • Test image analysis flow

  • Deployment

  • Prepare for Hugging Face Space

    • Update .env with HF Token
    • Copy Data to Project Directory (Containerization)
    • Update Path Logic in Code (Relative Paths)
    • Create Dockerfile
    • Create deploy_to_hf.py script
  • Review PRD with User

Slackbot Integration

  • Slack App Setup
    • Create Setup Guide
    • Configure Scopes & Tokens
  • Implement Slackbot Logic
    • Add slack_bolt dependency
    • Create slack_bot.py
    • Handle File Uploads (Images/Videos)
    • Handle DMs and Thread Replies
    • Implement Thread Context Awareness
      • Update slack_bot.py to fetch thread history
      • Convert Slack history to SearchEngine format
  • Deployment Updates
    • Update Dockerfile for dual execution (or separate)
    • Separate Web App and Slackbot Deployments
    • Migrate Data to HF Dataset (Fix Storage Limit)
      • Upload data to HF Dataset
      • Update Dockerfile to download data on startup
      • Create Dockerfile.web and deploy_web.py
      • Create Dockerfile.slack and deploy_slack.py
      • Deploy to separate Spaces
    • Fix Slack Formatting Issues (Markdown to mrkdwn)
      • Update search_engine for strict formatting instructions
      • Implement deterministic Regex converter
      • Redeploy Slackbot (Clean Install)

Search & Context Improvements

  • Improve Search Quality
    • Add thefuzz for fuzzy matching logic
    • Refactor filter_internal_data to use fuzzy scoring
  • Enhance Context Awareness
    • Update keyword_extraction_prompt to inherit context from history
    • Verify multi-turn conversation flow