Spaces:
Runtime error
Runtime error
[Delta-02] Drill C retry/backoff 또는 graceful fallback 동작 확인용 코드 제거
Browse files- app/embedding_utils.py +2 -12
- app/main.py +2 -12
- docs/04-weekend2/deployment-checklist-lab.md +10 -2
app/embedding_utils.py
CHANGED
|
@@ -3,19 +3,10 @@ import nltk
|
|
| 3 |
import numpy as np
|
| 4 |
from openai import OpenAI, RateLimitError
|
| 5 |
from sentence_transformers import CrossEncoder
|
| 6 |
-
from tenacity import
|
| 7 |
-
retry,
|
| 8 |
-
wait_exponential,
|
| 9 |
-
stop_after_attempt,
|
| 10 |
-
retry_if_exception,
|
| 11 |
-
before_sleep_log,
|
| 12 |
-
)
|
| 13 |
-
import logging
|
| 14 |
from app.config import OPENAI_API_KEY, OPENAI_EMBEDDING_MODEL
|
| 15 |
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
client = OpenAI(api_key=OPENAI_API_KEY, timeout=1.0)
|
| 19 |
|
| 20 |
# Cross-encoder 리랭커 (첫 호출 시 모델 다운로드)
|
| 21 |
reranker = CrossEncoder("BAAI/bge-reranker-v2-m3")
|
|
@@ -61,7 +52,6 @@ def _is_rate_limit_retryable(e: Exception) -> bool:
|
|
| 61 |
retry=retry_if_exception(_is_rate_limit_retryable),
|
| 62 |
wait=wait_exponential(multiplier=1, min=2, max=30),
|
| 63 |
stop=stop_after_attempt(2),
|
| 64 |
-
before_sleep=before_sleep_log(logger, logging.WARNING),
|
| 65 |
)
|
| 66 |
def _openai_embed(text: str) -> List[float]:
|
| 67 |
response = client.embeddings.create(
|
|
|
|
| 3 |
import numpy as np
|
| 4 |
from openai import OpenAI, RateLimitError
|
| 5 |
from sentence_transformers import CrossEncoder
|
| 6 |
+
from tenacity import retry, wait_exponential, stop_after_attempt, retry_if_exception
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
from app.config import OPENAI_API_KEY, OPENAI_EMBEDDING_MODEL
|
| 8 |
|
| 9 |
+
client = OpenAI(api_key=OPENAI_API_KEY)
|
|
|
|
|
|
|
| 10 |
|
| 11 |
# Cross-encoder 리랭커 (첫 호출 시 모델 다운로드)
|
| 12 |
reranker = CrossEncoder("BAAI/bge-reranker-v2-m3")
|
|
|
|
| 52 |
retry=retry_if_exception(_is_rate_limit_retryable),
|
| 53 |
wait=wait_exponential(multiplier=1, min=2, max=30),
|
| 54 |
stop=stop_after_attempt(2),
|
|
|
|
| 55 |
)
|
| 56 |
def _openai_embed(text: str) -> List[float]:
|
| 57 |
response = client.embeddings.create(
|
app/main.py
CHANGED
|
@@ -26,23 +26,14 @@ from .config import (
|
|
| 26 |
|
| 27 |
import pdfplumber
|
| 28 |
from openai import OpenAI, RateLimitError, AuthenticationError
|
| 29 |
-
from tenacity import
|
| 30 |
-
retry,
|
| 31 |
-
wait_exponential,
|
| 32 |
-
stop_after_attempt,
|
| 33 |
-
retry_if_exception,
|
| 34 |
-
before_sleep_log,
|
| 35 |
-
)
|
| 36 |
-
import logging
|
| 37 |
import hashlib
|
| 38 |
import time
|
| 39 |
from concurrent.futures import ThreadPoolExecutor
|
| 40 |
|
| 41 |
-
logger = logging.getLogger(__name__)
|
| 42 |
-
|
| 43 |
app = FastAPI(title="DocuFlow AI Agent")
|
| 44 |
|
| 45 |
-
client = OpenAI(api_key=OPENAI_API_KEY
|
| 46 |
|
| 47 |
|
| 48 |
@app.get("/health")
|
|
@@ -62,7 +53,6 @@ def _is_rate_limit_retryable(e: Exception) -> bool:
|
|
| 62 |
retry=retry_if_exception(_is_rate_limit_retryable),
|
| 63 |
wait=wait_exponential(multiplier=1, min=2, max=30),
|
| 64 |
stop=stop_after_attempt(2),
|
| 65 |
-
before_sleep=before_sleep_log(logger, logging.WARNING),
|
| 66 |
)
|
| 67 |
def _call_llm(messages: list, model: str) -> str:
|
| 68 |
response = client.chat.completions.create(model=model, messages=messages)
|
|
|
|
| 26 |
|
| 27 |
import pdfplumber
|
| 28 |
from openai import OpenAI, RateLimitError, AuthenticationError
|
| 29 |
+
from tenacity import retry, wait_exponential, stop_after_attempt, retry_if_exception
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
import hashlib
|
| 31 |
import time
|
| 32 |
from concurrent.futures import ThreadPoolExecutor
|
| 33 |
|
|
|
|
|
|
|
| 34 |
app = FastAPI(title="DocuFlow AI Agent")
|
| 35 |
|
| 36 |
+
client = OpenAI(api_key=OPENAI_API_KEY)
|
| 37 |
|
| 38 |
|
| 39 |
@app.get("/health")
|
|
|
|
| 53 |
retry=retry_if_exception(_is_rate_limit_retryable),
|
| 54 |
wait=wait_exponential(multiplier=1, min=2, max=30),
|
| 55 |
stop=stop_after_attempt(2),
|
|
|
|
| 56 |
)
|
| 57 |
def _call_llm(messages: list, model: str) -> str:
|
| 58 |
response = client.chat.completions.create(model=model, messages=messages)
|
docs/04-weekend2/deployment-checklist-lab.md
CHANGED
|
@@ -119,9 +119,17 @@
|
|
| 119 |
{"detail":"Request timed out."}
|
| 120 |
- 복구 시간(분): 3분
|
| 121 |
|
| 122 |
-
- [
|
| 123 |
- 관찰 내용:
|
| 124 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
|
| 126 |
---
|
| 127 |
|
|
|
|
| 119 |
{"detail":"Request timed out."}
|
| 120 |
- 복구 시간(분): 3분
|
| 121 |
|
| 122 |
+
- [x] retry/backoff 또는 graceful fallback 동작 확인
|
| 123 |
- 관찰 내용:
|
| 124 |
+
- 변경한 것들 - 1) timeout=1.0 — OpenAI 클라이언트에 1초 타임아웃을 설정해서 거의 확실히 타임아웃이 발생하게 했습니다. - 2) before_sleep_log — tenacity가 재시도하기 직전에 서버 로그에 이런 메시지가 남습니다:
|
| 125 |
+
`WARNING - Retrying _call_llm in 2.0 seconds as it raised <예외>.` - 3) import logging + logger — 위 로그를 출력하기 위한 설정.
|
| 126 |
+
|
| 127 |
+
- 결과:
|
| 128 |
+
- API Timeout Error 발생하면서 재시도 없이 바로 실패함.
|
| 129 |
+
- rate limit 또는 timeout 재현: timeout=1.0으로 강제 타임아웃 재현 성공
|
| 130 |
+
- retry/backoff 동작 확인: 타임아웃은 retry 조건에 포함되지 않아 즉시 실패. rate limit만 재시도 대상임을 확인
|
| 131 |
+
- 사용자에게 "Request timed out." 메시지가 500으로 그대로 노출됨
|
| 132 |
+
- 복구 시간(분): 4분
|
| 133 |
|
| 134 |
---
|
| 135 |
|