llama-gemma

Running

llama-gemma / patch_auth_test32.py

Upload folder using huggingface_hub

cdda9a5 verified 15 days ago

1.11 kB

with open("gateway.py", "r") as f:
    code = f.read()

# Let's restore the fact that the user's authorization header might need to be passed down.
# Oh! Wait! What if we just bypass authentication entirely for the internal request?
# We CAN'T if `llama-swap` is reading the system `API_KEY` and enforcing it.
# AND we can't unset `API_KEY` in `start.sh` because then `gateway.py` won't know the `API_KEY` to authenticate users!
# The easiest solution:
# In `gateway.py`, we must send `Authorization: Bearer {API_KEY}` to `llama-swap`.
# Let's verify `API_KEY` is not empty.
# If `API_KEY` IS empty, then `llama-swap` shouldn't require one.
# So `llama_auth_headers()` doing `return {"Authorization": f"Bearer {API_KEY}"} if API_KEY else {}` should work!
# BUT WAIT.
# Why did `llama_auth_headers()` FAIL to authenticate to `llama-swap` earlier?
# Maybe `llama-swap` expects the header to be `X-API-Key`?
# Let's check `llama.cpp` server API key header. It supports `Authorization: Bearer <key>`.
# Does `llama-swap` support `Authorization: Bearer <key>`? YES!
# Let's manually test `llama-swap` if we can run it.

	with open("gateway.py", "r") as f:
	code = f.read()

	# Let's restore the fact that the user's authorization header might need to be passed down.
	# Oh! Wait! What if we just bypass authentication entirely for the internal request?
	# We CAN'T if `llama-swap` is reading the system `API_KEY` and enforcing it.
	# AND we can't unset `API_KEY` in `start.sh` because then `gateway.py` won't know the `API_KEY` to authenticate users!
	# The easiest solution:
	# In `gateway.py`, we must send `Authorization: Bearer {API_KEY}` to `llama-swap`.
	# Let's verify `API_KEY` is not empty.
	# If `API_KEY` IS empty, then `llama-swap` shouldn't require one.
	# So `llama_auth_headers()` doing `return {"Authorization": f"Bearer {API_KEY}"} if API_KEY else {}` should work!
	# BUT WAIT.
	# Why did `llama_auth_headers()` FAIL to authenticate to `llama-swap` earlier?
	# Maybe `llama-swap` expects the header to be `X-API-Key`?
	# Let's check `llama.cpp` server API key header. It supports `Authorization: Bearer <key>`.
	# Does `llama-swap` support `Authorization: Bearer <key>`? YES!
	# Let's manually test `llama-swap` if we can run it.