* support anthropic PDF beta
* upstream merge, remove commented out console log line
* Fixing type errors.
the anthropic API does not yet include a "DocumentBlock" for
support PDFs, so an extended type has been added to the endpoint.
* changed document processor to async (matching image processor)
* use the beta api types rather than custom extension
* rudimentary tool testing
* interim commit (tool re-passing, file handling)
* remove merge error
* tidy up, isolate beta classes to utils
* anthropic tool calling support.
* improve handling of directlyAnswer tool
* fix streaming
* slight tidy up to tools flow handling
* fix: dont pass tools in final generation, instead deduce tools from tool results
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Add support for Amazon Nova (#1629)
Tarek Becker+1: nsarrazin
Unverified signature
d965f78
* Add support for Amazon Nova
* fix linting issue
* fix: replace model.id check with `isNova` flag and remove debug logs
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
fix: disable caching on OpenAI-compatible endpoint type
* fix(openai): systemRoleSupported model configuration for openai endpoints
* feat(openai): max_completion_tokens now used over max_tokens for chat_completeions endpoint.
* fix: lint
* feat(docs): add o1 example
* fix: make parameter default to false and fix type checks
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Anthropic PDF Beta Support (#1571)
evalstate+1: nsarrazin
Unverified signature
63c93cf
* support anthropic PDF beta
* upstream merge, remove commented out console log line
* Fixing type errors.
the anthropic API does not yet include a "DocumentBlock" for
support PDFs, so an extended type has been added to the endpoint.
* changed document processor to async (matching image processor)
* use the beta api types rather than custom extension
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
fix: remove accessToken min length for llama.cpp endpoint
* support temperature between 0 and 2
* added parsing of presence_penalty from model parameters. OAI endpoint now correctly uses (frequency penalty=parameters.repetition penalty) and (presence penalty=parameters.presence_penalty) for both chat and completion paths.
* added parsing of presence_penalty from model parameters. OAI endpoint now correctly uses (frequency penalty=parameters.repetition penalty) and (presence penalty=parameters.presence_penalty) for both chat and completion paths.
* feat: paste long context as a file
* fix: anim sources
* fix: animations
* feat: actually inject plaintext files in the prompt
* filter out files that are plain text
* feat: use custom MIME type for clipboard content
* feat: add better UI affordance for pasting
* fix: cleanup animations
[feat] Add support for endpoint web sources (#1544)
goupilew+1: nsarrazin
Unverified signature
f6846e2
* [feat] Add support for endpoint web sources
* chores: lint
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
fix(endpoint): fallback to hf token for openai endpoint type
* feat(tools): Basic tool support for OpenAI models
* feat(tools): Basic tool support for OpenAI models
* fix: tools with document input (not image)
* fix: double yield of streaming tokens
* fix: boolean parameter processing error
took me a while to find this one lol
* feat: throw error if using tools in `completions` mode
* feat: fix image processing in tools mode
* fix: remove correlationKey and clean up types
---------
Co-authored-by: Jacob Gladish <jacob_gladish@comcast.com>
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Add support for model version in Vertex AI (#1496)
* Add support for model version in Vertex AI
* Update src/lib/server/endpoints/google/endpointVertex.ts
Co-authored-by: goupilew <arthurgoupil@gmail.com>
* fix: optional chaining on extraBody
---------
Co-authored-by: goupilew <arthurgoupil@gmail.com>
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Fix: Cohere Endpoint - Add Support for Tools (#1490)
dvirmor
Unverified signature
e8978b1
* fix(endpointCohere): Added force_single_step param to chatStream call
* refactor(endpointCohere): change force_single_step parameter to forceSingleStep for linting
- Updated the documentation with the latest links from Google AI Studio
- Fixed the incorrect documentation on `safetyThreshold`
- Fixed the `safetySettings` to exclude unspecified category, as it is
not considered as a valid input category in Google's API
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
feat: add support for multimodal in Vertex (#1338)
goupilew
Unverified signature
a0199fc
* feat: add support for multimodal in Vertex
* Nit changes and remove tools if multimodal
* revert model name change
* Fix tools/multimodal condition
* chores(lint): fix formatting
---------
Co-authored-by: Thomas <thomas.poc@gmail.com>
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Add support for Anthropic models via AWS Bedrock (#1413)
ABarLT+1: nsarrazin
Unverified signature
b6274e8
* Add support for Anthropic models via AWS Bedrock
* deps
* Fixed type errors
* Temporary fix for continue button showing up on Claude
* Fix continue button issue by setting the last message token's special to true
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Dynamic credential resolution for AWS endpoints (#1419)
Mike Nason+1: nsarrazin
Unverified signature
48ab25e
* Dynamic credential resolution for AWS endpoints
If AWS credentials are provided, they will continue to be used. With this
change, they become optional parameters and the AWS SDK will attempt to
use the default resolution chain if they are not provided.
* cleanup deps
* restore optional aws region in endpointAwsParametersSchema
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
fix(openai): do not throw "Generation failed" when finish_reason="length" (#1429)
Refactor logger error messages for consistency. (#1257)
Denzel Mendez
Unverified signature
fa0afa9
Fix: tools in endpointVertex now optional (#1234)
SebSchroen
Unverified signature
3806b68
The googleSearchRetrieval leads to very odd behavior when chatting with Gemini, making the argument optional and removing it for certain use cases fixes the issue
Add sampling parameter support to cloudflare (#1233)
Logging was switched to Pino. For error logging, Pino wants the object first, and
then the string message. For example see this thread: https://github.com/pinojs/pino/issues/673
Send `ollama pull` request if model is not available locally (#1227)
* Initial work on function calling
* wip
* Add websearch as a tool
* lint
* smol
* Move tools to their own files
* directly answer check
* Add text2img tool
* group tool calls together
* show retry even if no message
* fix trailing urls
* Add image popup
* format
* wip
* clean-up
* better loading indicator
* text colors
* extra example to summarize
* switch default model
* Add fetchUrl tool
* wip
* Add latest gradio
* fix types
* version bump sharp
* Basic tools menu
* working menu
* fix menu positioning
* deps fix
* add deps
* cleanup
* more cleanup
* cleanup
* package cleanups
* moar cleanup
* ui update
* Update ChatMessage.svelte
* upgrade gradio dep
* lint
* refactor and pass results to cohere
* feat: code interpreter tool
* fix: add e2b dependency
* feat: working TGI endpoint with tool results
* feat: allow image model to expand to 90dvw
* fix: dont block on title gen
* bump sharp
* misc cleanup
* fix url fetcher, adjust tool typing
*…
Generic Multimodal Support Fixed (#1147)
Liam Dyer+2: nsarrazin, Mishig
Unverified signature
b4690c8
* feat: multimodal anthropic support
* docs: add claude haiku and multimodal support
* feat: uploaded file detection and image conversion
* fix deps with sharp
* fix resvg deps?
* fix: image conversion, retry with files
* feat: generic image processing and size target
* docs: multimodal review comments
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* docs: multimodal review comments
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* feat: review comment resolution
* fix: type error on image params
* feat: add multimodal for vertex ai anthropic
* style: uploadFile timeout number
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* fix: TGI endpoints expecting multimodal
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
Revert "Generic Multimodal Support" (#1146)
Mishig
Unverified signature
2a40544
Revert "Generic Multimodal Support (#1021)"
This reverts commit 57f89346cc6d953246d0f1705e342e2e66fce386.
Generic Multimodal Support (#1021)
Liam Dyer+2: nsarrazin, Mishig
Unverified signature
f493225
* feat: multimodal anthropic support
* docs: add claude haiku and multimodal support
* feat: uploaded file detection and image conversion
* fix deps with sharp
* fix resvg deps?
* fix: image conversion, retry with files
* feat: generic image processing and size target
* docs: multimodal review comments
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* docs: multimodal review comments
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* feat: review comment resolution
* fix: type error on image params
* feat: add multimodal for vertex ai anthropic
* style: uploadFile timeout number
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
Extend endpointOai.ts to allow usage of extra sampling parameters (#1032)
* Extend endpointOai.ts to allow usage of extra sampling parameters when calling vllm as an OpenAI compatible
* refactor : prettier endpointOai.ts
* Fix: Corrected type imports in endpointOai.ts
* Simplifies code a bit and adds `extraBody` to open ai endpooint
* Update zod schema to allow any type in extraBody
---------
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
evalstate +1: nsarrazin Unverified signature* support anthropic PDF beta * upstream merge, remove commented out console log line * Fixing type errors. the anthropic API does not yet include a "DocumentBlock" for support PDFs, so an extended type has been added to the endpoint. * changed document processor to async (matching image processor) * use the beta api types rather than custom extension * rudimentary tool testing * interim commit (tool re-passing, file handling) * remove merge error * tidy up, isolate beta classes to utils * anthropic tool calling support. * improve handling of directlyAnswer tool * fix streaming * slight tidy up to tools flow handling * fix: dont pass tools in final generation, instead deduce tools from tool results --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>1ac3a3c
