Spaces:
Running
Running
| # Technischer Implementierungsplan: Entdecken-Feed 2.0 (v3 – Final) | |
| > **Übergabe-Dokument für den Programmierer.** Jede Sektion enthält exakte Spezifikationen, Datenstrukturen und Code-Skelette. Stand: Juli 2026. | |
| --- | |
| ## 0. Hosting-Constraint: HuggingFace Spaces (Free Tier) | |
| > [!CAUTION] | |
| > **HuggingFace Spaces Free Tier schläft nach ~15 Minuten Inaktivität ein!** Ein `setInterval`-Cron-Job (alle 6h) feuert NIEMALS zuverlässig. Alle zeitgesteuerten Hintergrund-Prozesse sind auf Free Spaces nutzlos. | |
| ### Lösung: On-Demand-Generierung statt Cron | |
| ``` | |
| User öffnet Entdecken-Tab | |
| │ | |
| ▼ | |
| Frontend: GET /api/news → Backend antwortet mit { articles, lastRun } | |
| │ | |
| ▼ | |
| Frontend prüft: (Date.now() - lastRun) > 6 Stunden? | |
| │ | |
| ┌────┴────┐ | |
| │ NEIN │ JA | |
| │ ▼ | |
| │ Frontend: POST /api/news/generate | |
| │ (zeigt Skeleton-Loader) | |
| │ │ | |
| │ ▼ | |
| │ Backend generiert im Hintergrund | |
| │ Frontend pollt GET /api/news alle 8s | |
| │ │ | |
| │ ▼ | |
| │ Neue Artikel erscheinen → Loader verschwindet | |
| │ │ | |
| └─────────┘ | |
| │ | |
| ▼ | |
| Artikel werden angezeigt | |
| ``` | |
| **Code-Konsequenzen:** | |
| - `setInterval()` in `initNewsService()` wird **entfernt** | |
| - `GET /api/news` liefert ab sofort: `res.json({ articles: data.articles, lastRun: data.lastRun })` | |
| - Frontend übernimmt die Trigger-Logik | |
| --- | |
| ## 1. Modell-Auswahl (Stand: Juli 2026) | |
| > [!WARNING] | |
| > **KI-Modelle veralten schnell!** Die folgenden Modellnamen wurden am 06.07.2026 recherchiert. Vor der Implementierung bitte auf [ai.google.dev/gemini-api/docs/models](https://ai.google.dev/gemini-api/docs/models) verifizieren. | |
| ### Gemini-Modelle (Free Tier via Google AI Studio) | |
| | Modell | API-Name | Free Tier | Stärken | Einsatz im Plan | | |
| |---|---|---|---|---| | |
| | **Gemini 3.5 Flash** | `gemini-3.5-flash` | ✅ Ja (~1500 RPD) | Neuestes Flash-Modell (Mai 2026), top bei Coding & Agentic Tasks | **Primär** für alle Calls | | |
| | **Gemini 3.1 Flash-Lite** | `gemini-3.1-flash-lite` | ✅ Ja | Ultra-günstig, schnell | **Fallback** bei Rate-Limits | | |
| | **Gemini 2.5 Flash** | `gemini-2.5-flash` | ✅ Ja | Bewährt, stabil | **Letzter Fallback** | | |
| | ~~Gemini 3.5 Pro~~ | ~~`gemini-3.5-pro`~~ | ❌ Nur Paid | Frontier-Qualität | Nicht nutzbar (kein Free Tier) | | |
| ### Modell-Kaskade (Fallback-Strategie) | |
| ```javascript | |
| // EXAKTE Modell-Kaskade für den Backend-Code: | |
| const MODEL_CASCADE = [ | |
| 'gemini-3.5-flash', // Primär: Neuestes & bestes Free-Modell (Mai 2026) | |
| 'gemini-3.1-flash-lite', // Fallback 1: Leichtgewichtig, kaum Rate-Limits | |
| 'gemini-2.5-flash', // Fallback 2: Bewährt, stabil | |
| ]; | |
| ``` | |
| > [!IMPORTANT] | |
| > **Der aktuelle Code nutzt `gemini-3.5-flash` als Primär-Modell** (Zeile 193 in news-service.js). Das ist korrekt und aktuell. `gemini-2.5-flash` ist der letzte Fallback. **NICHT** `gemini-2.0` oder andere veraltete Modelle verwenden! | |
| ### Alternative Free-Tier-Anbieter (für spätere Erweiterung) | |
| | Anbieter | Modelle | Free Tier | Sinnvoll für | | |
| |---|---|---|---| | |
| | **Groq** | Llama-basiert, GPT-OSS | ✅ ~30 RPM, ~14.400 RPD | Ultra-schnelle Inference (z.B. für Stufe 2 Extraction) | | |
| | **OpenRouter** | 50+ Modelle aggregiert | ✅ Begrenzte Free Credits | Multi-Modell-Fallback | | |
| | **Mistral** | Mistral Small, Codestral | ✅ Free Tier | Europäische Alternative | | |
| > Für den MVP fokussieren wir uns auf **Gemini Free Tier** – das reicht vollkommen aus. Die anderen Anbieter sind optionale Erweiterungen. | |
| --- | |
| ## 2. Tavily-Credit-Strategie | |
| ### Ist-Zustand | |
| | Feature | Calls/Event | Frequenz | Tavily-Credits/Tag | | |
| |---|---|---|---| | |
| | Discover Feed | 3 (3 Topics × 1) | alle 6h (4×/Tag) | **12** (basic = 1 Credit) | | |
| | Deep Research | 8 Calls pro Session | User-triggered | **16** (advanced = 2 Credits) | | |
| | **Gesamt-Baseline** | — | — | **~12 + 16 pro DR** | | |
| ### Soll-Zustand: Zero-Tavily-Discover + Power Deep Research | |
| | Feature | Tavily-Credits | Strategie | | |
| |---|---|---| | |
| | **Discover Feed** | **0** | Google News RSS + Jina Reader + Wikipedia (alles kostenlos) | | |
| | **Deep Research** | bis zu 16 pro Session | Unverändert, alle 1000 Credits/Monat stehen zur Verfügung | | |
| **Resultat:** ~62 Deep Research Sessions/Monat (statt ~40) | |
| --- | |
| ## 3. Free-Tier-Budget (alles $0) | |
| | Service | Free Tier | Unser Verbrauch/Tag | Headroom | | |
| |---|---|---|---| | |
| | **Gemini 3.5 Flash** | ~1500 RPD, ~1M TPD | ~13-52 Requests | ✅ 96%+ frei | | |
| | **Tavily** | 1000 Credits/Monat | **0 für Discover** | ✅ 100% für Deep Research | | |
| | **Jina Reader** (`r.jina.ai`) | 20 RPM ohne Key | 6-18 Calls/Zyklus | ✅ Kostenlos | | |
| | **Google Trends RSS** | Unbegrenzt | 1-4 Calls/Tag | ✅ Kostenlos | | |
| | **Google News RSS** | Unbegrenzt | 1-20 Calls/Tag | ✅ Kostenlos | | |
| | **Wikipedia API** | Unbegrenzt (mit UA) | 2-12 Calls/Tag | ✅ Kostenlos | | |
| --- | |
| ## 4. Architektur-Übersicht | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ ON-DEMAND TRIGGER (User öffnet Entdecken-Tab) │ | |
| │ Bedingung: lastRun > 6h oder manueller Refresh │ | |
| │ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 0: TREND-DETEKTION (kostenlos) │ │ | |
| │ │ • Google Trends RSS (DE) → 10-20 Trending-Topics │ │ | |
| │ │ • Google News RSS Headlines → Top-5 News-Themen │ │ | |
| │ │ • User-Topics → mit Datums-Enrichment │ │ | |
| │ │ → Merge + Deduplizierung → 8-12 Final-Topics │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 1: DUPLIKAT-VORFILTER │ │ | |
| │ │ Jaccard-Similarity gegen existierende Artikel-Titel │ │ | |
| │ │ + Vergleich mit data.lastTrends │ │ | |
| │ │ → Nur NEUE/VERÄNDERTE Topics weiter → max 6 │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 2: QUERY-DEKOMPOSITION (1× Gemini 3.5 Flash) │ │ | |
| │ │ Batch-Call: ALLE Topics → je 3 Sub-Queries │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 3: WEB-RECHERCHE (kostenlos) │ │ | |
| │ │ Pro Sub-Query: │ │ | |
| │ │ • Google News RSS Search → Headlines + Links │ │ | |
| │ │ Pro Topic (Top 3 URLs): │ │ | |
| │ │ ★ Jina Reader (r.jina.ai) → VOLLER Artikeltext ★ │ │ | |
| │ │ • Wikipedia API → Kontext + Bilder │ │ | |
| │ │ → URL-Deduplizierung gegen data.usedUrls │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 4: STUFE 1 – Grounded Generation │ │ | |
| │ │ (1× Gemini 3.5 Flash pro Topic) │ │ | |
| │ │ Input: VOLLE Artikeltexte + Wiki-Kontext + Datum │ │ | |
| │ │ → Umfassender Artikel (800-1200 Wörter) │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 5: STUFE 2 – Structured Extraction │ │ | |
| │ │ (1× Gemini 3.5 Flash pro Topic, temp=0.1) │ │ | |
| │ │ → JSON: { title, snippet, content, category, sources } │ │ | |
| │ │ + Title-Similarity-Check gegen existierende Artikel │ │ | |
| │ └─────────────────────────┬─────────────────────────────────┘ │ | |
| │ │ │ | |
| │ ▼ │ | |
| │ ┌───────────────────────────────────────────────────────────┐ │ | |
| │ │ Phase 6: SPEICHERN + CLEANUP │ │ | |
| │ │ • data.articles.unshift(article) │ │ | |
| │ │ • data.usedUrls updaten (48h TTL) │ │ | |
| │ │ • data.lastTrends updaten │ │ | |
| │ │ • Max 50 Artikel behalten │ │ | |
| │ │ • saveData() │ │ | |
| │ └───────────────────────────────────────────────────────────┘ │ | |
| │ │ | |
| │ Kosten pro Zyklus: $0.00 (Gemini Free + Jina Free + RSS/Wiki) │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| --- | |
| ## 5. Datenstruktur (news.json – erweitert) | |
| ```jsonc | |
| { | |
| "topics": ["KI Tools", "Abitur Vorbereitung"], | |
| "articles": [ | |
| { | |
| "id": "1720281600000_a3f2x", | |
| "topic": "KI Tools", | |
| "category": "Technologie", // Von Gemini klassifiziert | |
| "title": "Neues KI-Tool revolutioniert...", | |
| "snippet": "OpenAI hat heute...", // Max 150 Zeichen | |
| "content": "## Vollständiger Artikel...", // Markdown, 800-1200 Wörter | |
| "sources": [ | |
| { "title": "heise.de", "url": "https://...", "snippet": "..." } | |
| ], | |
| "imageUrl": "https://...", | |
| "timestamp": 1720281600000, | |
| "isTrending": true, | |
| "trendSource": "google_trends" // "google_trends"|"google_news"|"user_topic" | |
| } | |
| ], | |
| "lastRun": 1720281600000, | |
| "lastTrends": ["Hitzewelle", "EM 2026"], // NEU | |
| "usedUrls": { // NEU: URL-Dedup mit 48h TTL | |
| "https://heise.de/article/123": 1720281600000 | |
| } | |
| } | |
| ``` | |
| --- | |
| ## 6. Detaillierte Phase-Spezifikationen | |
| ### Phase 0: Trend-Detektion (kostenlos) | |
| ```javascript | |
| async function detectTrends() { | |
| const trends = []; | |
| // === Quelle 1: Google Trends RSS (DE) – KOSTENLOS === | |
| try { | |
| const url = 'https://trends.google.de/trending/rss?geo=DE'; | |
| const res = await fetch(url); | |
| const xml = await res.text(); | |
| const $ = cheerio.load(xml, { xmlMode: true }); | |
| $('item').each((i, el) => { | |
| if (i >= 10) return; | |
| const title = $(el).find('title').text().trim(); | |
| const traffic = $(el).find('ht\\:approx_traffic, approx_traffic').text(); | |
| const newsItems = []; | |
| $(el).find('ht\\:news_item_title, news_item_title').each((_, newsEl) => { | |
| newsItems.push($(newsEl).text()); | |
| }); | |
| trends.push({ | |
| topic: title, traffic, relatedHeadlines: newsItems.slice(0, 3), | |
| source: 'google_trends' | |
| }); | |
| }); | |
| } catch (e) { | |
| console.error('[TrendDetector] Google Trends RSS failed:', e.message); | |
| } | |
| // === Quelle 2: Google News RSS Top-Headlines (DE) – KOSTENLOS === | |
| try { | |
| const url = 'https://news.google.com/rss?hl=de&gl=DE&ceid=DE:de'; | |
| const res = await fetch(url); | |
| const xml = await res.text(); | |
| const $ = cheerio.load(xml, { xmlMode: true }); | |
| $('item').each((i, el) => { | |
| if (i >= 5) return; | |
| const title = $(el).find('title').text().trim(); | |
| const cleanTitle = title.replace(/\s*-\s*[^-]+$/, '').trim(); | |
| if (!trends.some(t => jaccardSimilarity(t.topic, cleanTitle) > 0.5)) { | |
| trends.push({ | |
| topic: cleanTitle, traffic: 'headline', relatedHeadlines: [], | |
| source: 'google_news' | |
| }); | |
| } | |
| }); | |
| } catch (e) { | |
| console.error('[TrendDetector] Google News RSS failed:', e.message); | |
| } | |
| return trends; | |
| } | |
| ``` | |
| ### User-Topics Enrichment | |
| ```javascript | |
| function enrichUserTopics(userTopics) { | |
| const now = new Date(); | |
| const monthYear = now.toLocaleDateString('de-DE', { month: 'long', year: 'numeric' }); | |
| return userTopics.map(topic => ({ | |
| topic: topic, | |
| searchQuery: `${topic} Neuigkeiten ${monthYear}`, | |
| traffic: 'user_topic', relatedHeadlines: [], source: 'user_topic' | |
| })); | |
| } | |
| ``` | |
| ### Phase 1: Duplikat-Vorfilter | |
| ```javascript | |
| function jaccardSimilarity(str1, str2) { | |
| const normalize = s => s.toLowerCase() | |
| .replace(/[^\wäöüß]/g, ' ').split(/\s+/).filter(w => w.length > 2); | |
| const set1 = new Set(normalize(str1)); | |
| const set2 = new Set(normalize(str2)); | |
| if (set1.size === 0 || set2.size === 0) return 0; | |
| const intersection = new Set([...set1].filter(x => set2.has(x))); | |
| const union = new Set([...set1, ...set2]); | |
| return intersection.size / union.size; | |
| } | |
| function filterDuplicateTopics(newTopics, existingArticles, lastTrends) { | |
| return newTopics.filter(trend => { | |
| const isRepeatTrend = lastTrends.some(lt => jaccardSimilarity(trend.topic, lt) > 0.7); | |
| const recentArticles = existingArticles | |
| .filter(a => Date.now() - a.timestamp < 24 * 60 * 60 * 1000); | |
| const hasSimilarArticle = recentArticles | |
| .some(a => jaccardSimilarity(trend.topic, a.title) > 0.5); | |
| if (isRepeatTrend && hasSimilarArticle) { | |
| console.log(`[DupFilter] Skipping "${trend.topic}" – already covered`); | |
| return false; | |
| } | |
| return true; | |
| }); | |
| } | |
| ``` | |
| ### Phase 2: Query-Dekomposition (1 Gemini-Call für ALLE Topics) | |
| ```javascript | |
| async function decomposeTopics(topics, apiKey) { | |
| const topicsList = topics.map((t, i) => | |
| `${i+1}. "${t.topic}" (Quelle: ${t.source})` | |
| ).join('\n'); | |
| const prompt = `Du bist ein Recherche-Assistent. Heute ist der ${new Date().toLocaleDateString('de-DE', { weekday: 'long', day: 'numeric', month: 'long', year: 'numeric' })}. | |
| Für jedes Thema: Generiere exakt 3 präzise, unterschiedliche Suchphrasen auf Deutsch, die verschiedene Aspekte abdecken. Die Phrasen sollen AKTUELLE Informationen finden. | |
| THEMEN: | |
| ${topicsList} | |
| Antworte EXAKT als JSON: | |
| { | |
| "decomposed": [ | |
| { | |
| "originalTopic": "Thema", | |
| "queries": ["Phrase 1", "Phrase 2", "Phrase 3"], | |
| "category": "Technologie|Wissenschaft|Politik|Wirtschaft|Bildung|Gesellschaft|Sport|Kultur|Gesundheit" | |
| } | |
| ] | |
| }`; | |
| return JSON.parse(await callGemini(prompt, apiKey, { | |
| temperature: 0.3, responseMimeType: 'application/json' | |
| })); | |
| } | |
| ``` | |
| ### Phase 3: Web-Recherche + ★ Jina Reader Content-Enrichment ★ | |
| > [!TIP] | |
| > **Das ist der Game-Changer gegenüber dem alten Plan.** Statt nur RSS-Snippets (1 Satz) bekommt Gemini den **vollen Artikeltext** über Jina Reader – kostenlos, 20 RPM ohne API-Key. | |
| ```javascript | |
| async function researchTopic(decomposedTopic) { | |
| const allResults = []; | |
| const allImages = []; | |
| // === SCHRITT 1: Google News RSS Search (kostenlos, pro Sub-Query) === | |
| for (const query of decomposedTopic.queries) { | |
| try { | |
| const url = `https://news.google.com/rss/search?q=${encodeURIComponent(query)}&hl=de&gl=DE&ceid=DE:de`; | |
| const res = await fetch(url); | |
| const xml = await res.text(); | |
| const $ = cheerio.load(xml, { xmlMode: true }); | |
| $('item').each((i, el) => { | |
| if (i >= 4) return; // 4 pro Query × 3 Queries = max 12 URLs | |
| const title = $(el).find('title').text(); | |
| const link = $(el).find('link').text(); | |
| const pubDate = $(el).find('pubDate').text(); | |
| const description = $(el).find('description').text().replace(/<[^>]+>/g, ''); | |
| if (!allResults.some(r => r.url === link) && !isUrlUsed(link)) { | |
| allResults.push({ | |
| title: title.replace(/\s*-\s*[^-]+$/, ''), | |
| url: link, | |
| snippet: description, | |
| fullContent: null, // Wird in Schritt 2 befüllt | |
| pubDate, query | |
| }); | |
| } | |
| }); | |
| } catch (e) { | |
| console.warn(`[Research] RSS failed for "${query}":`, e.message); | |
| } | |
| await new Promise(r => setTimeout(r, 500)); | |
| } | |
| // === SCHRITT 2: ★ Jina Reader – Volle Artikeltexte scrapen ★ (kostenlos) === | |
| // Nur die Top-3 relevantesten URLs scrapen (Rate Limit schonen) | |
| const urlsToScrape = allResults.slice(0, 3); | |
| for (const result of urlsToScrape) { | |
| try { | |
| const jinaUrl = `https://r.jina.ai/${result.url}`; | |
| const jinaRes = await fetch(jinaUrl, { | |
| headers: { | |
| 'Accept': 'text/plain', | |
| // Optional: 'Authorization': 'Bearer jina_xxx' für 500 RPM | |
| } | |
| }); | |
| if (jinaRes.ok) { | |
| const fullText = await jinaRes.text(); | |
| // Limitiere auf ~2000 Zeichen pro Quelle (Gemini Context schonen) | |
| result.fullContent = fullText.substring(0, 2000); | |
| console.log(`[Jina] Scraped ${result.url} (${fullText.length} chars)`); | |
| } | |
| } catch (e) { | |
| console.warn(`[Jina] Failed to scrape ${result.url}:`, e.message); | |
| } | |
| // Rate Limit: Max 20 RPM ohne Key → 3 Sekunden Pause | |
| await new Promise(r => setTimeout(r, 3000)); | |
| } | |
| // === SCHRITT 3: Wikipedia für Kontext + Bilder (kostenlos) === | |
| try { | |
| const wikiHeaders = { 'User-Agent': 'SchoolMindApp/1.0 (contact@schoolmind.ai)' }; | |
| const searchUrl = `https://de.wikipedia.org/w/api.php?action=query&list=search&srsearch=${encodeURIComponent(decomposedTopic.originalTopic)}&format=json&utf8=&srlimit=2`; | |
| const searchRes = await fetch(searchUrl, { headers: wikiHeaders }); | |
| const searchData = await searchRes.json(); | |
| for (const result of (searchData.query?.search || []).slice(0, 2)) { | |
| const extractUrl = `https://de.wikipedia.org/w/api.php?action=query&titles=${encodeURIComponent(result.title)}&prop=extracts|pageimages&exintro=true&explaintext=true&format=json&pithumbsize=800`; | |
| const extractRes = await fetch(extractUrl, { headers: wikiHeaders }); | |
| const extractData = await extractRes.json(); | |
| const page = Object.values(extractData.query?.pages || {})[0]; | |
| if (page?.extract) { | |
| allResults.push({ | |
| title: `Wikipedia: ${result.title}`, | |
| url: `https://de.wikipedia.org/wiki/${encodeURIComponent(result.title)}`, | |
| snippet: page.extract.substring(0, 500), | |
| fullContent: page.extract.substring(0, 2000), | |
| pubDate: null, query: 'wikipedia_context' | |
| }); | |
| } | |
| if (page?.thumbnail?.source) allImages.push(page.thumbnail.source); | |
| } | |
| } catch (e) { | |
| console.warn('[Research] Wikipedia failed:', e.message); | |
| } | |
| return { results: allResults, images: allImages }; | |
| } | |
| ``` | |
| ### URL-Dedup-Helfer | |
| ```javascript | |
| function isUrlUsed(url) { | |
| if (!data.usedUrls) return false; | |
| const entry = data.usedUrls[url]; | |
| if (!entry) return false; | |
| if (Date.now() - entry > 48 * 60 * 60 * 1000) { delete data.usedUrls[url]; return false; } | |
| return true; | |
| } | |
| function markUrlsAsUsed(results) { | |
| if (!data.usedUrls) data.usedUrls = {}; | |
| const now = Date.now(); | |
| for (const r of results) data.usedUrls[r.url] = now; | |
| // Cleanup alte Einträge (> 48h) | |
| for (const [url, ts] of Object.entries(data.usedUrls)) { | |
| if (now - ts > 48 * 60 * 60 * 1000) delete data.usedUrls[url]; | |
| } | |
| } | |
| ``` | |
| ### Phase 4: Stufe 1 – Grounded Generation (1× Gemini 3.5 Flash pro Topic) | |
| ```javascript | |
| async function stage1_groundedGeneration(topic, category, searchResults, images, apiKey) { | |
| const today = new Date().toLocaleDateString('de-DE', { | |
| weekday: 'long', day: 'numeric', month: 'long', year: 'numeric' | |
| }); | |
| // Baue Quellen-Text mit VOLLEN Artikeltexten (dank Jina Reader!) | |
| const sourcesText = searchResults.map((s, i) => { | |
| const dateInfo = s.pubDate | |
| ? ` (${new Date(s.pubDate).toLocaleDateString('de-DE')})` : ''; | |
| const content = s.fullContent | |
| ? `\nVOLLER TEXT:\n${s.fullContent}` | |
| : `\nSnippet: ${s.snippet}`; | |
| return `[${i+1}] ${s.title}${dateInfo}\nURL: ${s.url}${content}`; | |
| }).join('\n\n---\n\n'); | |
| const imagesText = images.length > 0 | |
| ? `\n\nVERFÜGBARE BILDER:\n${images.map(url => `- ${url}`).join('\n')}` | |
| : ''; | |
| const prompt = `DATUM HEUTE: ${today} | |
| KATEGORIE: ${category} | |
| Du bist ein preisgekrönter Wissenschaftsjournalist für ein anspruchsvolles Schüler-/Studenten-Magazin. | |
| THEMA: "${topic}" | |
| QUELLEN (${searchResults.length} Stück, teilweise mit VOLLEM Artikeltext): | |
| ${sourcesText} | |
| ${imagesText} | |
| AUFGABE: Schreibe einen UMFASSENDEN, TIEFGRÜNDIGEN Artikel (800-1200 Wörter). | |
| QUALITÄTSSTANDARDS: | |
| 1. **Zusammenfassung zuerst:** Beginne mit 2-3 Sätzen, die das Wichtigste auf den Punkt bringen. | |
| 2. **Tiefe:** Erkläre nicht nur WAS passiert, sondern WARUM es wichtig ist und welche Konsequenzen es hat. | |
| 3. **Struktur:** 3-5 Sektionen mit ### Markdown-Überschriften. | |
| 4. **Quellen:** Verweise IMMER mit [1], [2] etc. auf die konkreten Quellen. | |
| 5. **Aktualität:** NEUERE Quellen > ältere. Widersprüche benennen. | |
| 6. **Bilder:** 1-2 Bilder aus der Liste passend einbinden: . NICHT am Textanfang. | |
| 7. **Diagramme:** Wo sinnvoll \`\`\`mermaid Diagramme nutzen. KEINE Sonderzeichen in Labels. | |
| 8. **Verständlichkeit:** Fachbegriffe erklären. Zielgruppe: 16-25-Jährige. | |
| Wenn das Thema NICHT nachrichtenwürdig ist, antworte EXAKT mit "SKIP". | |
| Antworte NUR mit dem Markdown-Artikel oder "SKIP".`; | |
| return await callGemini(prompt, apiKey, { temperature: 0.7 }); | |
| } | |
| ``` | |
| ### Phase 5: Stufe 2 – Structured Extraction (1× Gemini 3.5 Flash, temp=0.1) | |
| ```javascript | |
| async function stage2_structuredExtraction(rawArticle, searchResults, topic, category, images, apiKey) { | |
| const prompt = `Extrahiere aus folgendem Artikel-Text ein JSON-Objekt. | |
| ARTIKEL: | |
| ${rawArticle} | |
| QUELLEN: | |
| ${searchResults.map((s, i) => `[${i+1}] "${s.title}" | ${s.url}`).join('\n')} | |
| BILDER: ${images.join(', ') || 'Keine'} | |
| TOPIC: "${topic}" | |
| KATEGORIE-VORSCHLAG: "${category}" | |
| Antworte EXAKT als JSON: | |
| { | |
| "title": "Fesselnde Überschrift (max 80 Zeichen, KEIN Markdown)", | |
| "snippet": "1-2 Sätze für den Feed (max 150 Zeichen)", | |
| "content": "Vollständiger Artikel in Markdown (übernehme den Input, korrigiere Formatierung)", | |
| "category": "Technologie|Wissenschaft|Politik|Wirtschaft|Bildung|Gesellschaft|Sport|Kultur|Gesundheit", | |
| "imageUrl": "Beste Bild-URL oder null", | |
| "sources": [{ "title": "domain.de", "url": "https://...", "snippet": "Kurz" }] | |
| }`; | |
| return JSON.parse(await callGemini(prompt, apiKey, { | |
| temperature: 0.1, responseMimeType: 'application/json' | |
| })); | |
| } | |
| ``` | |
| ### Zentrale Gemini-Helfer-Funktion mit Modell-Kaskade | |
| ```javascript | |
| async function callGemini(prompt, apiKey, options = {}) { | |
| const { temperature = 0.7, responseMimeType = null } = options; | |
| const body = { | |
| contents: [{ parts: [{ text: prompt }] }], | |
| generationConfig: { temperature } | |
| }; | |
| if (responseMimeType) body.generationConfig.responseMimeType = responseMimeType; | |
| // ★ AKTUELLE Modell-Kaskade (Juli 2026) ★ | |
| const models = [ | |
| 'gemini-3.5-flash', // Primär: Neuestes Free-Modell (Mai 2026) | |
| 'gemini-3.1-flash-lite', // Fallback 1: Leicht, selten Rate-Limited | |
| 'gemini-2.5-flash', // Fallback 2: Bewährt | |
| ]; | |
| for (const model of models) { | |
| try { | |
| const url = `https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent?key=${apiKey}`; | |
| const response = await fetch(url, { | |
| method: 'POST', | |
| headers: { 'Content-Type': 'application/json' }, | |
| body: JSON.stringify(body) | |
| }); | |
| if (!response.ok) { | |
| const err = await response.text(); | |
| throw new Error(`${model}: ${response.status} – ${err.substring(0, 200)}`); | |
| } | |
| const json = await response.json(); | |
| const text = json.candidates?.[0]?.content?.parts?.map(p => p.text || '').join('') || ''; | |
| console.log(`[Gemini] ✅ ${model} succeeded`); | |
| return text.trim(); | |
| } catch (e) { | |
| console.error(`[Gemini] ❌ ${model} failed:`, e.message); | |
| } | |
| } | |
| throw new Error('All Gemini models in cascade failed'); | |
| } | |
| ``` | |
| --- | |
| ## 7. Haupt-Orchestrierung (`runGenerationCycle`) | |
| ```javascript | |
| async function runGenerationCycle(options = {}) { | |
| const { dryRun = false, singleTopic = null } = options; | |
| const apiKey = process.env.GEMINI_API_KEY; | |
| if (!apiKey) return { status: 'skipped', reason: 'no_api_key' }; | |
| const log = []; | |
| // Phase 0 | |
| let trendTopics = singleTopic | |
| ? [{ topic: singleTopic, traffic: 'manual', relatedHeadlines: [], source: 'manual_test' }] | |
| : [...await detectTrends(), ...enrichUserTopics(data.topics)]; | |
| log.push(`Phase 0: ${trendTopics.length} topics detected`); | |
| if (dryRun) { | |
| trendTopics.forEach(t => log.push(` → "${t.topic}" (${t.source})`)); | |
| return { status: 'dry_run', log, topics: trendTopics }; | |
| } | |
| // Phase 1 | |
| const filtered = filterDuplicateTopics(trendTopics, data.articles, data.lastTrends || []); | |
| const topicsToProcess = filtered.slice(0, 6); | |
| log.push(`Phase 1: ${topicsToProcess.length} new topics after dedup`); | |
| if (topicsToProcess.length === 0) { | |
| data.lastRun = Date.now(); saveData(); | |
| return { status: 'no_new_topics', log }; | |
| } | |
| // Phase 2 | |
| let decomposed; | |
| try { | |
| decomposed = await decomposeTopics(topicsToProcess, apiKey); | |
| } catch (e) { | |
| decomposed = { decomposed: topicsToProcess.map(t => ({ | |
| originalTopic: t.topic, queries: [t.searchQuery || t.topic], category: 'Allgemein' | |
| }))}; | |
| } | |
| // Phases 3-5 | |
| let newArticles = 0; | |
| for (const item of (decomposed.decomposed || [])) { | |
| try { | |
| const researchData = await researchTopic(item); | |
| log.push(`"${item.originalTopic}": ${researchData.results.length} sources`); | |
| if (researchData.results.length === 0) { log.push(` → SKIPPED (no sources)`); continue; } | |
| const rawArticle = await stage1_groundedGeneration( | |
| item.originalTopic, item.category, researchData.results, researchData.images, apiKey | |
| ); | |
| if (!rawArticle || rawArticle.startsWith('SKIP')) { log.push(` → SKIPPED by AI`); continue; } | |
| const structured = await stage2_structuredExtraction( | |
| rawArticle, researchData.results, item.originalTopic, item.category, researchData.images, apiKey | |
| ); | |
| if (data.articles.some(a => jaccardSimilarity(a.title, structured.title) > 0.7)) { | |
| log.push(` → DUPLICATE title, skipping`); continue; | |
| } | |
| const trendInfo = topicsToProcess.find(t => t.topic === item.originalTopic); | |
| data.articles.unshift({ | |
| id: Date.now().toString() + '_' + Math.random().toString(36).substr(2, 5), | |
| topic: item.originalTopic, | |
| category: structured.category || item.category || 'Allgemein', | |
| title: structured.title, | |
| snippet: structured.snippet, | |
| content: structured.content, | |
| sources: structured.sources || [], | |
| imageUrl: structured.imageUrl || researchData.images[0] || null, | |
| timestamp: Date.now(), | |
| isTrending: trendInfo?.source === 'google_trends', | |
| trendSource: trendInfo?.source || 'unknown' | |
| }); | |
| markUrlsAsUsed(researchData.results); | |
| newArticles++; | |
| log.push(` → ✅ "${structured.title}"`); | |
| await new Promise(r => setTimeout(r, 3000)); | |
| } catch (e) { | |
| log.push(` → ❌ ERROR: ${e.message}`); | |
| } | |
| } | |
| // Phase 6 | |
| data.lastTrends = topicsToProcess.map(t => t.topic); | |
| data.lastRun = Date.now(); | |
| if (data.articles.length > 50) data.articles = data.articles.slice(0, 50); | |
| saveData(); | |
| return { status: 'complete', newArticles, log }; | |
| } | |
| ``` | |
| --- | |
| ## 8. API-Endpunkte | |
| ```javascript | |
| // BESTEHEND (unverändert): | |
| // GET /api/news, GET /api/topics, POST /api/topics, | |
| // DELETE /api/topics/:t, DELETE /api/news/:id | |
| // GEÄNDERT: | |
| app.get('/api/news', (req, res) => { | |
| res.json({ articles: data.articles, lastRun: data.lastRun }); // ← lastRun hinzugefügt! | |
| }); | |
| app.post('/api/news/generate', express.json(), async (req, res) => { | |
| const { dryRun, singleTopic } = req.body || {}; | |
| res.json({ status: 'started' }); // Sofort antworten | |
| runGenerationCycle({ dryRun, singleTopic }); // Async im Hintergrund | |
| }); | |
| // NEU: | |
| app.get('/api/news/trends', async (req, res) => { | |
| const trends = await detectTrends(); | |
| res.json(trends); | |
| }); | |
| app.get('/api/news/debug', (req, res) => { | |
| res.json({ | |
| lastRun: data.lastRun, | |
| lastRunAgo: `${Math.round((Date.now() - data.lastRun) / 3600000)}h ago`, | |
| lastTrends: data.lastTrends || [], | |
| articleCount: data.articles.length, | |
| usedUrlCount: Object.keys(data.usedUrls || {}).length, | |
| topicsCount: data.topics.length | |
| }); | |
| }); | |
| ``` | |
| --- | |
| ## 9. Frontend-Anpassungen ([news-view.js](file:///c:/Antigravity/APP/src/views/news/news-view.js)) | |
| ### 9.1 On-Demand-Trigger in `fetchData()` | |
| ```javascript | |
| async fetchData() { | |
| try { | |
| this.isLoading = true; | |
| this.render(); | |
| const newsRes = await fetch(getApiUrl('/api/news')); | |
| if (newsRes.ok) { | |
| const data = await newsRes.json(); | |
| this.articles = data.articles; // ← NEU: articles aus Objekt | |
| this.lastRun = data.lastRun; // ← NEU: lastRun speichern | |
| // Extrahiere Kategorien aus Artikeln für die Tabs | |
| this.categories = [...new Set(this.articles.map(a => a.category).filter(Boolean))]; | |
| } | |
| const topicsRes = await fetch(getApiUrl('/api/topics')); | |
| if (topicsRes.ok) this.topics = await topicsRes.json(); | |
| // ★ ON-DEMAND TRIGGER ★ | |
| const sixHours = 6 * 60 * 60 * 1000; | |
| if (!this.lastRun || (Date.now() - this.lastRun) > sixHours) { | |
| console.log('[NewsView] Triggering generation (lastRun > 6h)...'); | |
| this.triggerGeneration(); | |
| } | |
| } catch (e) { | |
| console.error('Error fetching news:', e); | |
| } finally { | |
| this.isLoading = false; | |
| this.render(); | |
| } | |
| } | |
| ``` | |
| ### 9.2 Topic-Tabs aus Artikel-Kategorien | |
| ```javascript | |
| // Zeile 70 – VORHER: | |
| // ['Alle', ...Array.from(new Set(['Allgemeine News', ...this.topics]))] | |
| // NACHHER: | |
| const tabs = ['Alle', ...(this.categories || []), 'Meine Themen']; | |
| ``` | |
| ### 9.3 Trending-Badge | |
| ```javascript | |
| // Zeile 234 – VORHER: | |
| // ${index === 0 ? '<div class="news-tag highlight-tag">Neu</div>' : ''} | |
| // NACHHER: | |
| ${article.isTrending | |
| ? '<div class="news-tag highlight-tag">🔥 Trending</div>' | |
| : (Date.now() - article.timestamp < 6*60*60*1000 | |
| ? '<div class="news-tag">Neu</div>' : '')} | |
| ``` | |
| ### 9.4 Filter-Logik | |
| ```javascript | |
| // Zeile 210: | |
| if (this.activeTopic === 'Alle') { | |
| filteredArticles = this.articles; | |
| } else if (this.activeTopic === 'Meine Themen') { | |
| filteredArticles = this.articles.filter(a => a.trendSource === 'user_topic'); | |
| } else { | |
| filteredArticles = this.articles.filter(a => a.category === this.activeTopic); | |
| } | |
| ``` | |
| --- | |
| ## 10. Dateien-Änderungen Zusammenfassung | |
| ### [MODIFY] [news-service.js](file:///c:/Antigravity/APP/server/news-service.js) | |
| - **Entfernt:** `searchWeb()`, `generateArticleWithGemini()`, `setInterval()`-Cron | |
| - **Neu:** `detectTrends()`, `enrichUserTopics()`, `jaccardSimilarity()`, `filterDuplicateTopics()`, `decomposeTopics()`, `researchTopic()` (mit Jina Reader!), `stage1_groundedGeneration()`, `stage2_structuredExtraction()`, `callGemini()` (mit Modell-Kaskade), `isUrlUsed()`, `markUrlsAsUsed()` | |
| - **Geändert:** `runGenerationCycle()` (mit `dryRun`, `singleTopic`), `GET /api/news` (+ lastRun), neue Endpoints `/api/news/trends`, `/api/news/debug` | |
| - **Modelle:** `gemini-3.5-flash` → `gemini-3.1-flash-lite` → `gemini-2.5-flash` | |
| - **Tavily komplett entfernt** aus diesem Service | |
| ### [MODIFY] [news-view.js](file:///c:/Antigravity/APP/src/views/news/news-view.js) | |
| - `fetchData()`: Parst `{ articles, lastRun }` statt Array, On-Demand-Trigger | |
| - Topic-Tabs: Dynamisch aus Artikel-Kategorien + "Meine Themen" | |
| - Trending-Badge (🔥) für Trend-Artikel | |
| - Filter-Logik für Kategorien | |
| ### [MODIFY] [index.html](file:///c:/Antigravity/APP/index.html) | |
| - Versionsnummer hochzählen | |
| --- | |
| ## 11. Test-Strategie | |
| ### Test 1: Dry-Run (0 API-Kosten) | |
| ```bash | |
| curl -X POST http://localhost:5173/api/news/generate \ | |
| -H "Content-Type: application/json" -d '{"dryRun": true}' | |
| ``` | |
| → Zeigt erkannte Trends, keine Gemini-Calls | |
| ### Test 2: Single-Topic (2 Gemini-Calls + 3 Jina-Calls) | |
| ```bash | |
| curl -X POST http://localhost:5173/api/news/generate \ | |
| -H "Content-Type: application/json" -d '{"singleTopic": "Künstliche Intelligenz 2026"}' | |
| ``` | |
| → Minimaler Verbrauch, voller Pipeline-Test | |
| ### Test 3: Debug (0 Kosten) | |
| ```bash | |
| curl http://localhost:5173/api/news/debug | |
| ``` | |
| ### Test 4: Trends anschauen (0 Kosten) | |
| ```bash | |
| curl http://localhost:5173/api/news/trends | |
| ``` | |
| ### Test 5: Duplikat-Check | |
| ```bash | |
| # Gleichen Single-Topic nochmal – sollte als Duplikat gefiltert werden | |
| curl -X POST http://localhost:5173/api/news/generate \ | |
| -H "Content-Type: application/json" -d '{"singleTopic": "Künstliche Intelligenz 2026"}' | |
| ``` | |
| --- | |
| ## 12. Qualitätsvergleich: Vorher vs. Nachher | |
| | Aspekt | Vorher | Nachher | | |
| |---|---|---| | |
| | **Quell-Tiefe** | Tavily Snippet (~200 Wörter) | Jina Reader Volltext (~2000 Wörter) | | |
| | **Quellen-Anzahl** | 5 pro Topic | 8-12 pro Topic (3 Sub-Queries × 4) | | |
| | **Trend-Erkennung** | ❌ Keine | ✅ Google Trends + News RSS | | |
| | **Duplikat-Schutz** | ❌ Keiner | ✅ Jaccard + URL-Dedup | | |
| | **Artikel-Länge** | ~200-300 Wörter | ~800-1200 Wörter | | |
| | **Kosten** | 12 Tavily-Credits/Tag | $0.00 | | |
| | **Modelle** | `gemini-3.5-flash` (nur 1) | Kaskade: `3.5-flash` → `3.1-flash-lite` → `2.5-flash` | | |