Spaces:
Running
Running
| with open("gateway.py", "r") as f: | |
| code = f.read() | |
| # The problem is that llama-server is picking up API_KEY from the environment because it was exported by Hugging Face Spaces! | |
| # And because `start.sh` doesn't `unset API_KEY`, `llama-server` gets it and expects it. | |
| # BUT `gateway.py` isn't properly getting `API_KEY`? Or it IS getting it, but sending it as `Bearer {API_KEY}` and maybe `llama-server` expects it differently? | |
| # No, `llama-server` expects `Authorization: Bearer <key>`. | |
| # Wait! Let's just unset `API_KEY` in `start.sh`! | |
| # Oh, in `patch_auth_key2.py` I did `env -u API_KEY /usr/local/bin/llama-swap`. | |
| # Let's see what happens if I do `env -u API_KEY` for llama-swap. | |
| # Ah! I realized `gateway.py` sets `INFERENCE_API_KEY = os.getenv("INFERENCE_API_KEY", API_KEY).strip()`. | |
| # If `start.sh` has `unset API_KEY` before it runs `gateway.py`, then `gateway.py` won't know the `API_KEY`! | |
| # So `start.sh` should NOT `unset API_KEY`. | |
| # Instead, we should pass `-u API_KEY` to `llama-swap`. | |
| # Let's restore the `env -u API_KEY` to `start.sh`! | |
| # BUT wait! If I do `env -u API_KEY`, `llama-swap` is run without `API_KEY`. | |
| # Then `llama-swap` spawns `llama-server`. Does it spawn it with `env -u API_KEY`? Yes, because it inherits the environment of `llama-swap`! | |
| # Then `llama-server` will not have `API_KEY` set, and therefore won't require an API key! | |