llama-gemma

Running

llama-gemma / patch_auth_test22.py

Upload folder using huggingface_hub

40449b2 verified 15 days ago

943 Bytes

import re

with open("gateway.py", "r") as f:
    code = f.read()

# Is there any other place where the API_KEY might be required?
# Oh! The client requests are to `http://127.0.0.1:8080/v1/chat/completions`.
# Maybe `llama-swap` passes the request directly to the model, AND the model is being launched with an api key somehow?
# Wait! In `llama-swap.yaml`, `llama-server` is started. Does `llama-server` use an environment variable `API_KEY` or `LLAMA_API_KEY`?
# In `llama.cpp`'s `llama-server`, the environment variable is `LLAMA_API_KEY`!
# Let's check `start.sh`.
# `unset PORT LLAMA_PORT LLAMA_ARG_PORT LLAMA_API_KEY LLAMA_ARG_API_KEY`
# It UNSETS `LLAMA_API_KEY`!
# So if it unsets it, why would `llama-server` require it?
# Wait, I didn't verify that `LLAMA_API_KEY` is completely unset. It IS in `start.sh`.
# BUT maybe there is `API_KEY` being picked up by `llama-swap`?
# Let's see what `llama-swap`'s `--help` says about API key.

	import re

	with open("gateway.py", "r") as f:
	code = f.read()

	# Is there any other place where the API_KEY might be required?
	# Oh! The client requests are to `http://127.0.0.1:8080/v1/chat/completions`.
	# Maybe `llama-swap` passes the request directly to the model, AND the model is being launched with an api key somehow?
	# Wait! In `llama-swap.yaml`, `llama-server` is started. Does `llama-server` use an environment variable `API_KEY` or `LLAMA_API_KEY`?
	# In `llama.cpp`'s `llama-server`, the environment variable is `LLAMA_API_KEY`!
	# Let's check `start.sh`.
	# `unset PORT LLAMA_PORT LLAMA_ARG_PORT LLAMA_API_KEY LLAMA_ARG_API_KEY`
	# It UNSETS `LLAMA_API_KEY`!
	# So if it unsets it, why would `llama-server` require it?
	# Wait, I didn't verify that `LLAMA_API_KEY` is completely unset. It IS in `start.sh`.
	# BUT maybe there is `API_KEY` being picked up by `llama-swap`?
	# Let's see what `llama-swap`'s `--help` says about API key.