llama-gemma

Running

llama-gemma / patch_auth_test13.py

Upload folder using huggingface_hub

f507ff7 verified 15 days ago

1.14 kB

with open("gateway.py", "r") as f:
    code = f.read()

# Let's completely remove API_KEY forwarding from llama_auth_headers and just leave it to default.
# WAIT. In start.sh I removed `env -u API_KEY`. So `llama-swap` now knows `API_KEY`.
# Since `llama-swap` knows `API_KEY`, we MUST provide it when sending requests from `gateway.py` to `llama-swap`.
# What does `llama_auth_headers()` return now?
# `return {"Authorization": f"Bearer {API_KEY}"} if API_KEY else {}`
# So we ARE sending it.
# Then why does `llama-swap` complain?
# Let's check `llama-swap` source online... Oh wait.
# Maybe `llama-swap` does NOT use `API_KEY`. Maybe it uses `LLAMA_API_KEY`!
# Let's check what variables `start.sh` unsets:
# `unset PORT LLAMA_PORT LLAMA_ARG_PORT LLAMA_API_KEY LLAMA_ARG_API_KEY`
# Aha! It unsets `LLAMA_API_KEY`. It doesn't unset `API_KEY`.
# Maybe the model inside (llama-server) reads `API_KEY`? No, `llama-server` reads `LLAMA_API_KEY`!
# Wait! Let's just look at the logs I printed: "unauthorized: Invalid API Key" came from `llama_server`, not `llama_swap`!
# Wait, let's use `curl` directly to `llama-swap` internally to test.

	with open("gateway.py", "r") as f:
	code = f.read()

	# Let's completely remove API_KEY forwarding from llama_auth_headers and just leave it to default.
	# WAIT. In start.sh I removed `env -u API_KEY`. So `llama-swap` now knows `API_KEY`.
	# Since `llama-swap` knows `API_KEY`, we MUST provide it when sending requests from `gateway.py` to `llama-swap`.
	# What does `llama_auth_headers()` return now?
	# `return {"Authorization": f"Bearer {API_KEY}"} if API_KEY else {}`
	# So we ARE sending it.
	# Then why does `llama-swap` complain?
	# Let's check `llama-swap` source online... Oh wait.
	# Maybe `llama-swap` does NOT use `API_KEY`. Maybe it uses `LLAMA_API_KEY`!
	# Let's check what variables `start.sh` unsets:
	# `unset PORT LLAMA_PORT LLAMA_ARG_PORT LLAMA_API_KEY LLAMA_ARG_API_KEY`
	# Aha! It unsets `LLAMA_API_KEY`. It doesn't unset `API_KEY`.
	# Maybe the model inside (llama-server) reads `API_KEY`? No, `llama-server` reads `LLAMA_API_KEY`!
	# Wait! Let's just look at the logs I printed: "unauthorized: Invalid API Key" came from `llama_server`, not `llama_swap`!
	# Wait, let's use `curl` directly to `llama-swap` internally to test.