llama-gemma

Running

llama-gemma / patch_auth_test12.py

Upload folder using huggingface_hub

4a442b0 verified 16 days ago

916 Bytes

with open("gateway.py", "r") as f:
    code = f.read()

# Instead of passing API_KEY to llama-server, what if we just strip it from the environment entirely before running uvicorn too? 
# Then llama_server doesn't need API_KEY. But wait, `gateway.py` relies on `API_KEY` to authenticate incoming requests!
# So we DO need it. 
# BUT we want `llama-server` to NOT require authentication!
# In `start.sh`, we used `env -u API_KEY /usr/local/bin/llama-swap ...`. That means `llama-swap` is run WITHOUT `API_KEY`.
# If `llama-swap` is run without `API_KEY`, it shouldn't require authentication!
# Oh, wait. `llama-swap` isn't `llama-server`. `llama-swap` is a proxy.
# Does `llama-swap` pass its environment variables to `llama-server` when it spawns it?
# Let's check `llama-swap` repository. 
# Regardless, if we set `LLAMA_API_KEY` explicitly when starting `llama-swap` and pass it from `gateway.py`, it should work!

	with open("gateway.py", "r") as f:
	code = f.read()

	# Instead of passing API_KEY to llama-server, what if we just strip it from the environment entirely before running uvicorn too?
	# Then llama_server doesn't need API_KEY. But wait, `gateway.py` relies on `API_KEY` to authenticate incoming requests!
	# So we DO need it.
	# BUT we want `llama-server` to NOT require authentication!
	# In `start.sh`, we used `env -u API_KEY /usr/local/bin/llama-swap ...`. That means `llama-swap` is run WITHOUT `API_KEY`.
	# If `llama-swap` is run without `API_KEY`, it shouldn't require authentication!
	# Oh, wait. `llama-swap` isn't `llama-server`. `llama-swap` is a proxy.
	# Does `llama-swap` pass its environment variables to `llama-server` when it spawns it?
	# Let's check `llama-swap` repository.
	# Regardless, if we set `LLAMA_API_KEY` explicitly when starting `llama-swap` and pass it from `gateway.py`, it should work!