Spaces:
Sleeping
Sleeping
Commit History
-
fix(multimodal): remove fake image support, model cannot decode base64
Testing revealed the model does NOT actually see image pixels when base64 is embedded in text. It only guesses from context/filename. Same image gives inconsistent answers; different images give same answers. This is hallucination, not real vision. Real multimodal requires ProcessFile streaming RPC (WIZ protocol) which cannot be implemented in a pure HTTP proxy. Image inputs now return a note asking user to describe in text.ad06fde -
feat(multimodal): auto-compress large images for reliable processing
- Add _compress_b64_if_needed(): resize to 256px + JPEG q60 if >50KB - Verified with real images: 210KB anime, 37KB signature, 510KB photo - All correctly identified by model after compression3b13ffe -
feat(multimodal): working image support via base64 text embedding
Images sent as image_url (base64) are embedded directly into the prompt text. Gemini's multimodal model processes the base64 data and attempts to describe the image content. This bypasses the ProcessFile streaming RPC requirement (which needs WIZ protocol) by leveraging the model's ability to parse base64 image data from text input. Limitations: - Works best with small images (<100KB) - Model accuracy may vary vs native image channel - URL-based images still need fetch + convert to base64f7f1672 -
feat: add tool_choice / toolConfig support for all API paths
LiangLliu- OpenAI Chat & Responses: handle tool_choice parameter ("none" / "auto" / "required" / {function: {name}}) - Google Native API: handle toolConfig.functionCallingConfig.mode (NONE / AUTO / ANY) and allowedFunctionNames - tool_choice=none skips tool prompt injection and tool call parsing - tool_choice=none enables real streaming even when tools are declared - tool_choice=required adds constraint instruction to prompt8840d2c -
feat: full Google Native API support for Gemini CLI compatibility
LiangLliuAdd complete support for Google's generateContent/streamGenerateContent endpoints, enabling seamless integration with Gemini CLI and the @google/genai SDK. Key changes: 1. Real streaming (server.py) - Stream via generate_stream() with per-delta SSE chunks - Empty delta filtering (SDK rejects part.text === "") - Separate final chunk with finishReason: "STOP" + usageMetadata - Tool requests fall back to non-streaming to avoid truncated blocks 2. Function calling via prompt engineering (tools.py) - build_tool_prompt(): natural language tool instructions that avoid Gemini Web's prompt-injection detection - google_contents_to_prompt(): converts systemInstruction, tools, functionCall/functionResponse parts into prompt string + images - parse_google_function_calls(): extracts function calls from model output (3 formats: backtick blocks, bare blocks, raw JSON) 3. SDK protocol compliance - SSE format matches SDK regex /^\s*data: (.*)(?:\n\n)/ - func…3f5152e
-
feat: multimodal framework + image upload (WIP)
- Add multimodal.py: ProcessFile upload protocol implementation - Update tools.py: parse image_url from messages, return (prompt, images) - Update server.py: _upload_images() processes images before generation - Update gemini.py: file_refs parameter for StreamGenerate with images Note: ProcessFile requires cookie auth and correct proto format. Anonymous multimodal not yet working (error code 7). Text-only + streaming + tool calling all stable.510909e -
fix: code review issues (P1-P3)
- Cookie file caching with mtime check (no disk read per request) - Streaming retry logic (same as non-streaming) - Reusable httpx.Client and ssl context (no per-request allocation) - Shared _extract_texts_from_line() eliminates parsing duplication - _start_sse() helper eliminates 4x SSE header repetition - _usage() helper for token estimation - _parse_body() with proper 400 on invalid JSON - ValueError guard on @think= parameter - Single-pass regex in parse_tool_calls - Fixed __main__.py httpx detection (was always showing 'fallback') - Removed unused base64 import from gemini.pyd27caee -
feat: modularize + true streaming + multimodal support
- Split into package: gemini_web2api/{config,models,gemini,tools,server} - True SSE streaming via httpx (falls back to urllib if not installed) - Multimodal: image_url support in messages (base64 or URL) - pyproject.toml + requirements.txt for pip install - Entry: python -m gemini_web2api or gemini-web2api CLI85919ae