File size: 10,237 Bytes
8baf122 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 | ---
title: JSON Parser
description: A simple Bifrost plugin that handles partial JSON chunks in streaming responses by making them valid JSON objects.
icon: "code-branch"
---
## Overview
When using AI providers that stream JSON responses, the individual chunks often contain incomplete JSON that cannot be parsed directly. This plugin automatically detects and fixes partial JSON chunks by adding the necessary closing braces, brackets, and quotes to make them valid JSON.
## Features
- **Automatic JSON Completion**: Detects partial JSON and adds missing closing characters
- **Streaming Only**: Processes only streaming responses (non-streaming responses are ignored)
- **Flexible Usage Modes**: Supports two usage types for different deployment scenarios
- **Safe Fallback**: Returns original content if JSON cannot be fixed
- **Memory Leak Prevention**: Automatic cleanup of stale accumulated content with configurable intervals
- **Zero Dependencies**: Only depends on Go's standard library
## Usage
### Usage Types
The plugin supports two usage types:
1. **AllRequests**: Processes all streaming responses automatically
2. **PerRequest**: Processes only when explicitly enabled via request context
```go
package main
import (
"time"
"github.com/maximhq/bifrost/core"
"github.com/maximhq/bifrost/core/schemas"
"github.com/maximhq/bifrost/plugins/jsonparser"
)
func main() {
// Create the JSON parser plugin for all requests
jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
Usage: jsonparser.AllRequests,
CleanupInterval: 2 * time.Minute, // Cleanup every 2 minutes
MaxAge: 10 * time.Minute, // Remove entries older than 10 minutes
})
// Initialize Bifrost with the plugin
client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
Account: &MyAccount{},
LLMPlugins: []schemas.LLMPlugin{
jsonPlugin,
},
})
if err != nil {
panic(err)
}
// Use the client normally - JSON parsing happens automatically
// in the PostLLMHook for all streaming responses
}
```
### PerRequest Mode
```go
package main
import (
"context"
"time"
"github.com/maximhq/bifrost/core"
"github.com/maximhq/bifrost/core/schemas"
"github.com/maximhq/bifrost/plugins/jsonparser"
)
func main() {
// Create the JSON parser plugin for per-request control
jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
Usage: jsonparser.PerRequest,
CleanupInterval: 2 * time.Minute, // Cleanup every 2 minutes
MaxAge: 10 * time.Minute, // Remove entries older than 10 minutes
})
// Initialize Bifrost with the plugin
client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
Account: &MyAccount{},
LLMPlugins: []schemas.LLMPlugin{
jsonPlugin,
},
})
if err != nil {
panic(err)
}
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)
// Enable JSON parsing for specific requests
stream, bifrostErr := client.ChatCompletionStreamRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), request)
if bifrostErr != nil {
// handle error
}
for chunk := range stream {
_ = chunk // handle each streaming chunk
}
}
```
### Configuration
```go
// Custom cleanup configuration
plugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
Usage: jsonparser.AllRequests,
CleanupInterval: 2 * time.Minute, // Cleanup every 2 minutes
MaxAge: 10 * time.Minute, // Remove entries older than 10 minutes
})
```
#### Default Values
- **CleanupInterval**: 5 minutes (how often to run cleanup)
- **MaxAge**: 30 minutes (how old entries can be before cleanup)
- **Usage**: Must be specified (AllRequests or PerRequest)
### Context Key for PerRequest Mode
When using `PerRequest` mode, the plugin checks for the context key `jsonparser.EnableStreamingJSONParser` with a boolean value:
- `true`: Enable JSON parsing for this request
- `false`: Disable JSON parsing for this request
- Key not present: Disable JSON parsing for this request
**Example:**
```go
import (
"context"
"github.com/maximhq/bifrost/plugins/jsonparser"
)
// Enable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)
// Disable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, false)
// No context key - JSON parsing disabled (default behavior)
ctx := context.Background()
```
## How It Works
The plugin implements an optimized `parsePartialJSON` function with the following steps:
1. **Usage Check**: Determines if processing should occur based on usage type and context
2. **Validates Input**: First tries to parse the string as valid JSON
3. **Character Analysis**: If invalid, processes the string character-by-character to track:
- String boundaries (inside/outside quotes)
- Escape sequences
- Opening/closing braces and brackets
4. **Auto-Completion**: Adds missing closing characters in the correct order
5. **Validation**: Verifies the completed JSON is valid
6. **Fallback**: Returns original content if completion fails
### Memory Management
The plugin automatically manages memory by:
1. **Accumulating Content**: Stores partial JSON chunks with timestamps for each request
2. **Periodic Cleanup**: Runs a background goroutine that removes stale entries based on `MaxAge`
3. **Request Completion**: Automatically clears accumulated content when requests complete successfully
4. **Configurable Intervals**: Allows customization of cleanup frequency and retention periods
### Real-Life Streaming Example
Here's a practical example showing how the JSON parser plugin fixes broken JSON chunks in streaming responses:
```go
package main
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/maximhq/bifrost/core"
"github.com/maximhq/bifrost/core/schemas"
"github.com/maximhq/bifrost/plugins/jsonparser"
)
func main() {
// Create JSON parser plugin
jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
Usage: jsonparser.AllRequests,
CleanupInterval: 2 * time.Minute,
MaxAge: 10 * time.Minute,
})
// Initialize Bifrost with the plugin
client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
Account: &MyAccount{},
LLMPlugins: []schemas.LLMPlugin{jsonPlugin},
})
if err != nil {
panic(err)
}
defer client.Shutdown()
// Request structured JSON response
request := &schemas.BifrostChatRequest{
Provider: schemas.OpenAI,
Model: "gpt-4o-mini",
Input: []schemas.ChatMessage{
{
Role: schemas.ChatMessageRoleUser,
Content: schemas.ChatMessageContent{
ContentStr: bifrost.Ptr("Return user profile as JSON: {\"name\": \"John Doe\", \"email\": \"john@example.com\"}"),
},
},
},
}
// Stream the response
stream, bifrostErr := client.ChatCompletionStreamRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), request)
if bifrostErr != nil {
panic(bifrostErr)
}
fmt.Println("Streaming JSON response:")
for chunk := range stream {
if chunk.BifrostChatResponse != nil && len(chunk.BifrostChatResponse.Choices) > 0 {
choice := chunk.BifrostChatResponse.Choices[0]
if choice.ChatStreamResponseChoice != nil && choice.ChatStreamResponseChoice.Delta != nil {
content := *choice.ChatStreamResponseChoice.Delta.Content
fmt.Printf("Chunk: %s\n", content)
// With JSON parser, you can parse each chunk immediately
var jsonData map[string]interface{}
if err := json.Unmarshal([]byte(content), &jsonData); err == nil {
fmt.Printf("β
Valid JSON parsed successfully\n")
} else {
fmt.Printf("β Invalid JSON: %v\n", err)
}
}
}
}
}
```
**Without JSON Parser** (raw streaming chunks):
```
Chunk 1: `{` β Invalid JSON
Chunk 2: `{"name"` β Invalid JSON
Chunk 3: `{"name": "John"` β Invalid JSON
Chunk 4: `{"name": "John Doe"` β Invalid JSON
```
**With JSON Parser** (processed chunks):
```
Chunk 1: `{}` β
Valid JSON
Chunk 2: `{"name": ""}` β
Valid JSON
Chunk 3: `{"name": "John"}` β
Valid JSON
Chunk 4: `{"name": "John Doe"}` β
Valid JSON
```
### Use Cases
- **Function Calling**: Stream tool call arguments as valid JSON throughout the response
- **Structured Data**: Stream complex JSON objects (user profiles, product catalogs) progressively
- **Real-time Parsing**: Enable client-side JSON parsing at each streaming step without waiting for completion
- **API Integration**: Forward streaming JSON to downstream services that expect valid JSON
- **Live Updates**: Update UI components with valid JSON data as it streams in
### Example Transformations
| Input | Output |
|-------|--------|
| `{"name": "John"` | `{"name": "John"}` |
| `["apple", "banana"` | `["apple", "banana"]` |
| `{"user": {"name": "John"` | `{"user": {"name": "John"}}` |
| `{"message": "Hello\nWorld"` | `{"message": "Hello\nWorld"}` |
| `""` (empty string) | `{}` |
| `" "` (whitespace only) | `{}` |
## Testing
Run the test suite:
```bash
cd plugins/jsonparser
go test -v
```
The tests cover:
- Plugin interface compliance
- Both usage types (AllRequests and PerRequest)
- Context-based enabling/disabling
- Streaming responses only (non-streaming responses are ignored)
- Various JSON completion scenarios
- Edge cases and error conditions
- Memory cleanup functionality with real and simulated requests
- Configuration options and default values |