File size: 10,237 Bytes
8baf122
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
---
title: JSON Parser
description: A simple Bifrost plugin that handles partial JSON chunks in streaming responses by making them valid JSON objects.
icon: "code-branch"
---

## Overview

When using AI providers that stream JSON responses, the individual chunks often contain incomplete JSON that cannot be parsed directly. This plugin automatically detects and fixes partial JSON chunks by adding the necessary closing braces, brackets, and quotes to make them valid JSON.

## Features

- **Automatic JSON Completion**: Detects partial JSON and adds missing closing characters
- **Streaming Only**: Processes only streaming responses (non-streaming responses are ignored)
- **Flexible Usage Modes**: Supports two usage types for different deployment scenarios
- **Safe Fallback**: Returns original content if JSON cannot be fixed
- **Memory Leak Prevention**: Automatic cleanup of stale accumulated content with configurable intervals
- **Zero Dependencies**: Only depends on Go's standard library

## Usage

### Usage Types

The plugin supports two usage types:

1. **AllRequests**: Processes all streaming responses automatically
2. **PerRequest**: Processes only when explicitly enabled via request context


```go
package main

import (
    "time"
    "github.com/maximhq/bifrost/core"
    "github.com/maximhq/bifrost/core/schemas"
    "github.com/maximhq/bifrost/plugins/jsonparser"
)

func main() {
    // Create the JSON parser plugin for all requests
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.AllRequests,
        CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
        MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
    })
    
    // Initialize Bifrost with the plugin
    client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{
            jsonPlugin,
        },
    })
    
    if err != nil {
        panic(err)
    }
    
    // Use the client normally - JSON parsing happens automatically
    // in the PostLLMHook for all streaming responses
}
```

### PerRequest Mode

```go
package main

import (
    "context"
    "time"
    "github.com/maximhq/bifrost/core"
    "github.com/maximhq/bifrost/core/schemas"
    "github.com/maximhq/bifrost/plugins/jsonparser"
)

func main() {
    // Create the JSON parser plugin for per-request control
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.PerRequest,
        CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
        MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
    })
    
    // Initialize Bifrost with the plugin
    client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{
            jsonPlugin,
        },
    })
    
    if err != nil {
        panic(err)
    }

    ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)
    
    // Enable JSON parsing for specific requests
    stream, bifrostErr := client.ChatCompletionStreamRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), request)
    if bifrostErr != nil {
            // handle error
    }
    for chunk := range stream {
        _ = chunk // handle each streaming chunk
    }
}
```

### Configuration

```go
// Custom cleanup configuration
plugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
    Usage:           jsonparser.AllRequests,
    CleanupInterval: 2 * time.Minute,  // Cleanup every 2 minutes
    MaxAge:          10 * time.Minute,  // Remove entries older than 10 minutes
})
```

#### Default Values

- **CleanupInterval**: 5 minutes (how often to run cleanup)
- **MaxAge**: 30 minutes (how old entries can be before cleanup)
- **Usage**: Must be specified (AllRequests or PerRequest)

### Context Key for PerRequest Mode

When using `PerRequest` mode, the plugin checks for the context key `jsonparser.EnableStreamingJSONParser` with a boolean value:

- `true`: Enable JSON parsing for this request
- `false`: Disable JSON parsing for this request
- Key not present: Disable JSON parsing for this request

**Example:**

```go
import (
    "context"

    "github.com/maximhq/bifrost/plugins/jsonparser"
)

// Enable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, true)

// Disable JSON parsing for this request
ctx := context.WithValue(context.Background(), jsonparser.EnableStreamingJSONParser, false)

// No context key - JSON parsing disabled (default behavior)
ctx := context.Background()
```

## How It Works

The plugin implements an optimized `parsePartialJSON` function with the following steps:

1. **Usage Check**: Determines if processing should occur based on usage type and context
2. **Validates Input**: First tries to parse the string as valid JSON
3. **Character Analysis**: If invalid, processes the string character-by-character to track:
   - String boundaries (inside/outside quotes)
   - Escape sequences
   - Opening/closing braces and brackets
4. **Auto-Completion**: Adds missing closing characters in the correct order
5. **Validation**: Verifies the completed JSON is valid
6. **Fallback**: Returns original content if completion fails

### Memory Management

The plugin automatically manages memory by:

1. **Accumulating Content**: Stores partial JSON chunks with timestamps for each request
2. **Periodic Cleanup**: Runs a background goroutine that removes stale entries based on `MaxAge`
3. **Request Completion**: Automatically clears accumulated content when requests complete successfully
4. **Configurable Intervals**: Allows customization of cleanup frequency and retention periods

### Real-Life Streaming Example

Here's a practical example showing how the JSON parser plugin fixes broken JSON chunks in streaming responses:

```go
package main

import (
    "context"
    "encoding/json"
    "fmt"
    "time"
    "github.com/maximhq/bifrost/core"
    "github.com/maximhq/bifrost/core/schemas"
    "github.com/maximhq/bifrost/plugins/jsonparser"
)

func main() {
    // Create JSON parser plugin
    jsonPlugin := jsonparser.NewJsonParserPlugin(jsonparser.PluginConfig{
        Usage:           jsonparser.AllRequests,
        CleanupInterval: 2 * time.Minute,
        MaxAge:          10 * time.Minute,
    })
    
    // Initialize Bifrost with the plugin
    client, err := bifrost.Init(context.Background(), schemas.BifrostConfig{
        Account: &MyAccount{},
        LLMPlugins: []schemas.LLMPlugin{jsonPlugin},
    })
    if err != nil {
        panic(err)
    }
    defer client.Shutdown()

    // Request structured JSON response  
    request := &schemas.BifrostChatRequest{
        Provider: schemas.OpenAI,
        Model:    "gpt-4o-mini",
        Input: []schemas.ChatMessage{
            {
                Role: schemas.ChatMessageRoleUser,
                Content: schemas.ChatMessageContent{
                    ContentStr: bifrost.Ptr("Return user profile as JSON: {\"name\": \"John Doe\", \"email\": \"john@example.com\"}"),
                },
            },
        },
    }

    // Stream the response
    stream, bifrostErr := client.ChatCompletionStreamRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), request)
    if bifrostErr != nil {
        panic(bifrostErr)
    }

    fmt.Println("Streaming JSON response:")
    for chunk := range stream {
        if chunk.BifrostChatResponse != nil && len(chunk.BifrostChatResponse.Choices) > 0 {
            choice := chunk.BifrostChatResponse.Choices[0]
            if choice.ChatStreamResponseChoice != nil && choice.ChatStreamResponseChoice.Delta != nil {
                content := *choice.ChatStreamResponseChoice.Delta.Content
                fmt.Printf("Chunk: %s\n", content)

                // With JSON parser, you can parse each chunk immediately
                var jsonData map[string]interface{}
                if err := json.Unmarshal([]byte(content), &jsonData); err == nil {
                    fmt.Printf("βœ… Valid JSON parsed successfully\n")
                } else {
                    fmt.Printf("❌ Invalid JSON: %v\n", err)
                }
            }
        }
    }
}
```

**Without JSON Parser** (raw streaming chunks):
```
Chunk 1: `{`                    ❌ Invalid JSON
Chunk 2: `{"name"`              ❌ Invalid JSON  
Chunk 3: `{"name": "John"`      ❌ Invalid JSON
Chunk 4: `{"name": "John Doe"`  ❌ Invalid JSON
```

**With JSON Parser** (processed chunks):
```
Chunk 1: `{}`                               βœ… Valid JSON
Chunk 2: `{"name": ""}`                     βœ… Valid JSON
Chunk 3: `{"name": "John"}`                 βœ… Valid JSON  
Chunk 4: `{"name": "John Doe"}`             βœ… Valid JSON
```

### Use Cases

- **Function Calling**: Stream tool call arguments as valid JSON throughout the response
- **Structured Data**: Stream complex JSON objects (user profiles, product catalogs) progressively
- **Real-time Parsing**: Enable client-side JSON parsing at each streaming step without waiting for completion
- **API Integration**: Forward streaming JSON to downstream services that expect valid JSON
- **Live Updates**: Update UI components with valid JSON data as it streams in

### Example Transformations

| Input | Output |
|-------|--------|
| `{"name": "John"` | `{"name": "John"}` |
| `["apple", "banana"` | `["apple", "banana"]` |
| `{"user": {"name": "John"` | `{"user": {"name": "John"}}` |
| `{"message": "Hello\nWorld"` | `{"message": "Hello\nWorld"}` |
| `""` (empty string) | `{}` |
| `"   "` (whitespace only) | `{}` |

## Testing

Run the test suite:

```bash
cd plugins/jsonparser
go test -v
```

The tests cover:
- Plugin interface compliance
- Both usage types (AllRequests and PerRequest)
- Context-based enabling/disabling
- Streaming responses only (non-streaming responses are ignored)
- Various JSON completion scenarios
- Edge cases and error conditions
- Memory cleanup functionality with real and simulated requests
- Configuration options and default values