Download docs/plugins/migration-guide.mdx from shimen/yapapaway: direct link, hf CLI and curl.
- Browser
- Download file 14.7 kB
-
https://huggingface.co/spaces/shimen/yapapaway/resolve/main/docs/plugins/migration-guide.mdx
- Command line
-
hf download hf://spaces/shimen/yapapaway/docs/plugins/migration-guide.mdx
-
curl -L -o migration-guide.mdx https://huggingface.co/spaces/shimen/yapapaway/resolve/main/docs/plugins/migration-guide.mdx
14.7 kB
| --- | |
| title: "Plugin Migration Guide" | |
| description: "How to migrate your Bifrost plugins from v1.3.x to v1.4.x" | |
| icon: "arrow-up-right-dots" | |
| --- | |
| ## Overview | |
| Bifrost v1.4.x introduces a new plugin interface for HTTP transport layer interception. This guide helps you migrate existing plugins from the v1.3.x `TransportInterceptor` pattern to the v1.4.x `HTTPTransportPreHook` and `HTTPTransportPostHook` pattern. | |
| <Note> | |
| If your plugin doesn't use `TransportInterceptor`, no migration is needed. The `PreLLMHook`, `PostLLMHook`, `Init`, `GetName`, and `Cleanup` functions remain unchanged. | |
| </Note> | |
| ## What Changed? | |
| The HTTP transport interception mechanism changed from a simple function that receives and returns headers/body to a dual-hook pattern that works with both native `.so` plugins and WASM plugins. | |
| ### Key Differences | |
| | Aspect | v1.3.x (TransportInterceptor) | v1.4.x+ (Pre/Post Hooks) | | |
| |--------|-------------------------------|--------------------------| | |
| | Signature | `TransportInterceptor(ctx, url, headers, body)` | `HTTPTransportPreHook(ctx, req)` + `HTTPTransportPostHook(ctx, req, resp)` | | |
| | Return type | `(headers, body, error)` | Pre: `(*HTTPResponse, error)`, Post: `error` | | |
| | Request type | Separate `headers map`, `body map` | Unified `*HTTPRequest` struct | | |
| | Response access | Not available | Post-hook receives `*HTTPResponse` | | |
| | Modification | Return modified maps | Modify `req`/`resp` in-place | | |
| | Short-circuit | Return error | Return `*HTTPResponse` | | |
| | WASM support | No | Yes | | |
| | Context | Limited `BifrostContext` | Full `*BifrostContext` with `SetValue`/`Value` | | |
| ### Why the Change? | |
| The new dual-hook pattern provides: | |
| 1. **WASM plugin support** - Serializable types work across WASM boundary | |
| 2. **Response interception** - Post-hook can modify responses before returning to client | |
| 3. **Simpler API** - No middleware wrapper, direct function call | |
| 4. **Better testability** - No fasthttp dependency in plugin tests | |
| 5. **Full context access** - BifrostContext available for sharing data between hooks | |
| 6. **Custom response short-circuits** - Return a full response to short-circuit | |
| ## Migration Steps | |
| ### Step 1: Update Imports | |
| Remove the `fasthttp` import if present: | |
| ```go | |
| import ( | |
| "fmt" | |
| "github.com/maximhq/bifrost/core/schemas" | |
| // Remove: "github.com/valyala/fasthttp" | |
| ) | |
| ``` | |
| ### Step 2: Replace the Function | |
| **Before (v1.3.x):** | |
| ```go | |
| // TransportInterceptor modifies raw HTTP headers and body | |
| func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) { | |
| // Add custom header | |
| headers["X-Custom-Header"] = "value" | |
| // Modify body | |
| body["custom_field"] = "custom_value" | |
| return headers, body, nil | |
| } | |
| ``` | |
| **After (v1.4.x+):** | |
| ```go | |
| // HTTPTransportPreHook intercepts requests BEFORE they enter Bifrost core | |
| // Modify req in-place. Return (*HTTPResponse, nil) to short-circuit. | |
| func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) { | |
| // Add custom header (in-place modification) | |
| req.Headers["x-custom-header"] = "value" | |
| // Modify body (in-place modification) | |
| var body map[string]any | |
| sonic.Unmarshal(req.Body, &body) | |
| body["custom_field"] = "custom_value" | |
| req.Body, _ = sonic.Marshal(body) | |
| // Store values in context for use in post-hook | |
| ctx.SetValue(schemas.BifrostContextKey("my-plugin-key"), "my-value") | |
| // Return nil to continue, or return &HTTPResponse{} to short-circuit | |
| return nil, nil | |
| } | |
| // HTTPTransportPostHook intercepts responses AFTER they exit Bifrost core | |
| // Modify resp in-place. Called in reverse order of pre-hooks. | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Add response header | |
| resp.Headers["x-processed-by"] = "my-plugin" | |
| // Read values set in pre-hook | |
| if val := ctx.Value(schemas.BifrostContextKey("my-plugin-key")); val != nil { | |
| fmt.Println("Context value:", val) | |
| } | |
| // Return nil to continue, or return error to short-circuit | |
| return nil | |
| } | |
| ``` | |
| ### Step 3: Update Body Modification Logic | |
| In v1.3.x, you received the body as a `map[string]any`. In v1.4.x, you work with `req.Body` bytes: | |
| **Before (v1.3.x):** | |
| ```go | |
| func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) { | |
| // Direct map access | |
| body["model"] = "gpt-4" | |
| return headers, body, nil | |
| } | |
| ``` | |
| **After (v1.4.x+):** | |
| ```go | |
| import "github.com/bytedance/sonic" | |
| func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) { | |
| // Parse body | |
| var body map[string]any | |
| if err := sonic.Unmarshal(req.Body, &body); err == nil { | |
| // Modify body | |
| body["model"] = "gpt-4" | |
| // Update req.Body in-place | |
| req.Body, _ = sonic.Marshal(body) | |
| } | |
| return nil, nil | |
| } | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Modify response body if needed | |
| var respBody map[string]any | |
| if err := sonic.Unmarshal(resp.Body, &respBody); err == nil { | |
| respBody["plugin_processed"] = true | |
| resp.Body, _ = sonic.Marshal(respBody) | |
| } | |
| return nil | |
| } | |
| ``` | |
| ## Common Migration Patterns | |
| ### Adding Headers | |
| **v1.3.x:** | |
| ```go | |
| headers["authorization"] = "Bearer " + token | |
| return headers, body, nil | |
| ``` | |
| **v1.4.x+:** | |
| ```go | |
| // In HTTPTransportPreHook - modify request headers | |
| req.Headers["authorization"] = "Bearer " + token | |
| return nil, nil | |
| // In HTTPTransportPostHook - modify response headers | |
| resp.Headers["x-request-id"] = requestID | |
| return nil | |
| ``` | |
| ### Reading Headers | |
| **v1.3.x:** | |
| ```go | |
| apiKey := headers["X-API-Key"] | |
| ``` | |
| **v1.4.x+:** | |
| ```go | |
| // Use case-insensitive helper for reading (recommended) | |
| apiKey := req.CaseInsensitiveHeaderLookup("X-API-Key") | |
| // Or direct map access (case-sensitive) | |
| apiKey := req.Headers["x-api-key"] | |
| ``` | |
| ### Conditional Processing | |
| **v1.3.x:** | |
| ```go | |
| func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) { | |
| if headers["x-skip-processing"] == "true" { | |
| return headers, body, nil | |
| } | |
| // Process... | |
| return headers, body, nil | |
| } | |
| ``` | |
| **v1.4.x+:** | |
| ```go | |
| func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) { | |
| if req.CaseInsensitiveHeaderLookup("x-skip-processing") == "true" { | |
| return nil, nil // Continue without modification | |
| } | |
| // Process... | |
| return nil, nil | |
| } | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Post-hook always runs unless pre-hook short-circuited | |
| return nil | |
| } | |
| ``` | |
| ### Error Handling / Short-Circuit | |
| **v1.3.x:** | |
| ```go | |
| func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) { | |
| if headers["x-api-key"] == "" { | |
| return nil, nil, fmt.Errorf("missing API key") | |
| } | |
| return headers, body, nil | |
| } | |
| ``` | |
| **v1.4.x+:** | |
| ```go | |
| func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) { | |
| if req.CaseInsensitiveHeaderLookup("x-api-key") == "" { | |
| // Return a custom response to short-circuit | |
| return &schemas.HTTPResponse{ | |
| StatusCode: 401, | |
| Headers: map[string]string{"Content-Type": "application/json"}, | |
| Body: []byte(`{"error": "missing API key"}`), | |
| }, nil | |
| } | |
| return nil, nil | |
| } | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Not called if pre-hook short-circuited | |
| return nil | |
| } | |
| ``` | |
| ### Accessing Request Method and Path | |
| **v1.3.x:** | |
| ```go | |
| // url parameter contained the full URL | |
| func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) { | |
| // Limited access to URL | |
| return headers, body, nil | |
| } | |
| ``` | |
| **v1.4.x+:** | |
| ```go | |
| func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) { | |
| // Full access to request properties | |
| method := req.Method // "GET", "POST", etc. | |
| path := req.Path // "/v1/chat/completions" | |
| query := req.Query // map[string]string of query params | |
| pathParams := req.PathParams // map[string]string of path variables (e.g., {model}) | |
| return nil, nil | |
| } | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Access both request and response | |
| statusCode := resp.StatusCode | |
| responseHeaders := resp.Headers | |
| responseBody := resp.Body | |
| _ = statusCode // Use variables... | |
| _ = responseHeaders | |
| _ = responseBody | |
| return nil | |
| } | |
| ``` | |
| ## Testing Your Migration | |
| 1. **Build your updated plugin:** | |
| ```bash | |
| go build -buildmode=plugin -o my-plugin.so main.go | |
| ``` | |
| 2. **Update Bifrost to v1.4.x:** | |
| ```bash | |
| go get github.com/maximhq/bifrost/core@v1.4.0 | |
| ``` | |
| 3. **Test with a simple request:** | |
| ```bash | |
| curl -X POST http://localhost:8080/v1/chat/completions \ | |
| -H "Content-Type: application/json" \ | |
| -d '{"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}' | |
| ``` | |
| 4. **Verify logs show both hooks being called:** | |
| ``` | |
| HTTPTransportPreHook called | |
| PreLLMHook called | |
| PostLLMHook called | |
| HTTPTransportPostHook called | |
| ``` | |
| ## Troubleshooting | |
| ### Plugin fails to load after migration | |
| **Error:** `plugin: symbol TransportInterceptor not found` | |
| This error occurs if Bifrost v1.4.x is looking for the old function. Make sure: | |
| 1. You've updated to `HTTPTransportPreHook` and `HTTPTransportPostHook` | |
| 2. The function signatures match exactly: | |
| - `func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error)` | |
| - `func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error` | |
| 3. You've rebuilt the plugin with the correct core version | |
| ### Body modification not working | |
| Make sure you're assigning back to `req.Body` in the pre-hook: | |
| ```go | |
| // Wrong - body changes lost | |
| var body map[string]any | |
| sonic.Unmarshal(req.Body, &body) | |
| body["model"] = "gpt-4" | |
| // Missing: req.Body = ... | |
| // Correct - body changes applied | |
| var body map[string]any | |
| sonic.Unmarshal(req.Body, &body) | |
| body["model"] = "gpt-4" | |
| req.Body, _ = sonic.Marshal(body) // Assign back! | |
| ``` | |
| ### Response modification not working | |
| Make sure you're modifying `resp` in the post-hook: | |
| ```go | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Modify response headers | |
| resp.Headers["x-custom-header"] = "value" | |
| // Modify response body | |
| var body map[string]any | |
| sonic.Unmarshal(resp.Body, &body) | |
| body["extra_field"] = "value" | |
| resp.Body, _ = sonic.Marshal(body) | |
| return nil | |
| } | |
| ``` | |
| ### Headers not being set | |
| Make sure you're modifying `req.Headers` or `resp.Headers` directly: | |
| ```go | |
| // Set request header in pre-hook | |
| req.Headers["x-custom-header"] = "value" | |
| // Set response header in post-hook | |
| resp.Headers["x-custom-header"] = "value" | |
| // Read headers using case-insensitive helper | |
| value := req.CaseInsensitiveHeaderLookup("X-Custom-Header") | |
| ``` | |
| ### Context values not available in post-hook | |
| Make sure you're using the correct context key type: | |
| ```go | |
| // In pre-hook - set value | |
| ctx.SetValue(schemas.BifrostContextKey("my-key"), "my-value") | |
| // In post-hook - read value | |
| if val := ctx.Value(schemas.BifrostContextKey("my-key")); val != nil { | |
| // Use val | |
| } | |
| ``` | |
| ## Streaming Chunk Hook (v1.4.x) | |
| Bifrost v1.4.x introduces a new hook for intercepting streaming response chunks: | |
| ### HTTPTransportStreamChunkHook | |
| This hook is called for each chunk during streaming responses, allowing plugins to modify or filter chunks before they're sent to the client. | |
| ```go | |
| // HTTPTransportStreamChunkHook intercepts streaming chunks BEFORE they're written to the client. | |
| // Modify chunk data or return nil to skip the chunk entirely. | |
| // Only called for streaming responses when using HTTP transport (bifrost-http). | |
| func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) { | |
| // chunk is a typed struct containing one of: | |
| // - BifrostTextCompletionResponse (text completion streaming) | |
| // - BifrostChatResponse (chat completion streaming) | |
| // - BifrostResponsesStreamResponse (responses API streaming) | |
| // - BifrostSpeechStreamResponse (speech synthesis streaming) | |
| // - BifrostTranscriptionStreamResponse (transcription streaming) | |
| // - BifrostImageGenerationStreamResponse (image generation streaming) | |
| // - BifrostError (error during streaming) | |
| // Return chunk unchanged to pass through | |
| return chunk, nil | |
| // Return nil to skip/filter this chunk | |
| // return nil, nil | |
| // Return modified chunk | |
| // modifiedChunk := &schemas.BifrostStreamChunk{BifrostChatResponse: ...} | |
| // return modifiedChunk, nil | |
| } | |
| ``` | |
| **Key differences from `HTTPTransportPostHook`:** | |
| | Aspect | HTTPTransportPostHook | HTTPTransportStreamChunkHook | | |
| |--------|----------------------|------------------------------| | |
| | When called | After complete response | Per-chunk during streaming | | |
| | Input | Full HTTPResponse | `*BifrostStreamChunk` (typed struct) | | |
| | Can modify | Full response | Individual chunk struct | | |
| | Can skip | N/A | Return nil to skip chunk | | |
| <Note> | |
| `HTTPTransportPostHook` is **not called** for streaming responses. Use `HTTPTransportStreamChunkHook` instead to intercept streaming data. | |
| </Note> | |
| ### Migration for Existing Plugins | |
| If your plugin implements `HTTPTransportPostHook` and you want to also handle streaming responses, add the new hook: | |
| ```go | |
| // Existing hook for non-streaming responses | |
| func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error { | |
| // Handle complete responses | |
| return nil | |
| } | |
| // NEW: Add this for streaming responses | |
| func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) { | |
| // Handle streaming chunks (typed struct, not raw bytes) | |
| // Return chunk unchanged if no modification needed | |
| return chunk, nil | |
| } | |
| ``` | |
| ## Need Help? | |
| - **Discord Community**: [Join our Discord](https://discord.gg/exN5KAydbU) | |
| - **GitHub Issues**: [Report bugs or request features](https://github.com/maximhq/bifrost/issues) | |
| - **Writing Plugins Guide**: [Full plugin documentation](./writing-plugin) | |