Commit History
-
Move vars to dynamic, add metrics (#1085)
Unverified signature* Add healthcheck route * Add prom client with basic metrics * wip: serve metrics on a different port * exclude server from tsconfig * latest * refacto(all): use class & singleton * refacto(all): use class & singleton * refacto(all): add request logs * Make all the env vars dynamic (#1092) * Add truncate to task model (#1090) * retry text area height (#1091) * Make all the env vars dynamic * only check for db at runtime * remove secret from build step * improve dockerfile * Wrap db in singleton * add .env.local to dockerignore * changes to dockerfile * lint * aborted generations * move collections build check * Use a single dockerfile * lint * fixes * lint * don't return db during building * remove dev --------- Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * refacto(all): update default metrics port * fix recursion error * add version check * revert vite preview * Move request …98b1c51
-
Use pino for logs (#1086)
Unverified signature* Use pino for logs * Use logger for info * Add errorId to the logsddbe7d2
-
Kill app on tokenizer load fail (#1044)
Unverified signature4af31ae
-
Add logs to tokenizer fetch (#1030)
Unverified signaturecc79662
-
Add langserve endpoint (#1009)
Antonio Ramos +1: antonioraUnverified signature* Add support for langserve endpoints * Add support for langserve endpoints * Fix linting * Fix linting issues * Fix issue import --------- Co-authored-by: antoniora <antonio.ramos@adyen.com>f12455d
-
Move default template so it doesn't override tokenizer (#987)
Unverified signature9437108 -
Set ChatML as default chat prompt template (#985)
Unverified signature619722b
-
Add support for cohere endpoints (#976)
Unverified signature* Add support for cohere endpoints * readme updatea99cca3 -
Use jinja template for chat formatting (#730) (#744)
Unverified signature* Use jinja template for chat formatting * Add support for transformers js chat template * update to latest transformers version * Make sure to `add_generation_prompt` * unindent0819256
-
Implement Cloudflare Workers AI endpoint (#907) (#972)
Unverified signature* Implement Cloudflare Workers AI endpoint (#907) * Renamed to Cloudflare Workers AI in docs * Add note about sampling parameters * clean up env examplecb000d3
-
Google Vertex API support (#950)
Unverified signature* Added Google Vertex API support Google Application credentials can be set via GOOGLE_APPLICATION_CREDENTIALS = clientid.json * Updated the readme file * Updated the package-lock file * Updated the package-lock file Fixed minor issues * Moved "@google-cloud/vertexai" to optional dependencies * Updated for lint checker3ebd3a8
-
Count system prompt tokens (#850)
Mishig +1: nsarrazin Unverified signature* Count sysmte prompt tokens * show error only once * fix * fix css * simplify * fix * simplify * Revert "simplify" This reverts commit 61c0a2281f9795c4a4351d9ad473993a3c01c59f. * `model.tokenizer` config & fix reactivity issues * rm gated tokenizer * use `truncate` --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>50d8483
-
Anthropic Endpoint Support (#923)
Liam DyerUnverified signaturefeat: anthropic endpoint5a31018
-
Add gemma model to prod config (#854)
Unverified signature* Add `google/gemma-7b-it` to production config * Add gemma prompt format to PROMPTS.md * Make sampling parameters optional5f78b90 -
Models page (#849)
Unverified signature* add models page * add logo * models discussion * add default hint * Update +page.svelte * mishig review * fix typecheck --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>25c844d
-
Make lorax openai endpoint working (#755)
Unverified signaturemake base model for lorax inference working Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>10d7a5a
-
[SmallModel] Nit update the comment (#726)
MishigUnverified signature* [SmallModel] Nit update the comment * even more precise77adc4d
-
Add embedding models configurable, from both transformers.js and TEI (#646)
Unverified signature* Add embedding models configurable, from both Xenova and TEI * fix lint and format * Fix bug in sentenceSimilarity * Batches for TEI using /info route * Fix web search disapear when finish searching * Fix lint and format * Add more options for better embedding model usage * Fixing CR issues * Fix websearch disapear in later PR * Fix lint * Fix more minor code CR * Valiadate embeddingModelName field in model config * Add embeddingModel into shared conversation * Fix lint and format * Add default embedding model, and more readme explanation * Fix minor embedding model readme detailed * Update settings.json * Update README.md Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Update README.md Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Apply suggestions from code review Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Resolved more issues * lint * Fix more issues …3a01622
-
Use JSON5 for parsing all config vars (#671)
Unverified signaturef09c5f3
-
Add feature to unlist a model (#625)
Unverified signature41f29a4
-
Standardize HF_ACCESS_TOKEN -> HF_TOKEN (#610)
Unverified signature* Standardize HF_ACCESS_TOKEN -> HF_TOKEN * Replace HF_ACCESS_TOKEN by HF_TOKEN in .env * Add legacy support for HF_ACCESS_TOKEN --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>3cbea34
-
Add ollama endpoint support (#569)
Unverified signature* Add ollama endpoint support * replace if by switch * Add Ollama example in docs14f0244
-
Add support for tgi multimodal models (#531)
Unverified signature* wip: add support for tgi multimodal models * wip work on passing images to prompt * working idefics config! * rm allowed conv feature * lint * Add image resizing * fix ssr * add upload button * add delete button * misc formatting * lint * server file size check * optimistic update of images * retry with images * fix websearch button * lint * better error handling & max one image at a time * replace test image by blank one * disable loading on page change * Fix sharing of images * fix comments * Update filedropzone (#544) * Update src/lib/buildPrompt.ts Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * small tweaks * Fix merge conflicts * lint * wildcard image mime type * fix lint and comment * added comments * added comment about file size * Readme update --------- Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> Co-authored-by: Victor Mustar <victor.mustar@g…0e5c445
-
fix: Rename sagemaker endpoint to aws endpoint
Unverified signatureb5041c1 -
Modular backends & support for openAI & AWS endpoints (#541)
Unverified signature* Fix the response Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Should use /completions Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Use async generator Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Use openai npm Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Fix generateFromDefaultEndpoint Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Fix last char become undefined Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Better support for system prompt Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Updates Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Revert Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Update README Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * Default system prompt Signed-off-by: Hung-Han (Henry) Chen <chenhungh@gmail.com> * remove sk- Signed-off-by: Hung-Han …9db8ced
-
Switch chat model back to mistral, use zephyr for small tasks (#515)
Unverified signature* Switch chat model back to mistral, use zephyr for small tasks * typo * fix tests9187ced
-
Update websearch prompting & summary prompting (#503)
Unverified signature* Use the task model for websearch query generation * summary with examples in chat format * Update to websearch prompting - Removed web search template - Added examples in `generateQuery` - Passed previous questions when generating the web search query - Added previous questions to the bottom of the context window in `buildPrompt` when using websearch * wait for model if needed * rm console.log * get rid of regex * added a few time related examples * Update src/lib/server/websearch/generateQuery.ts Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * reformat yesterday * test * fix whitespace * remove extra whitespace for mistral * removed mistral extra whitespace * Update src/lib/server/summarize.ts Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * Update src/lib/server/summarize.ts Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * add an emoji if none is found in the first three characters ----…1e5090f
-
Pass stop sequences properly to TGI (#499)
Unverified signature* Pass stop sequences properly to TGI * fix typesa89fb7b
-
move some tasks to small model (#479)
Unverified signaturebf15962
-
Revert "Improve error message when the .env MODELS is not well formatted (#451)"
This reverts commit 8ce8b634e9687f00ad31d572dc047113e5b9f0d0.7ddda31 -
Improve error message when the .env MODELS is not well formatted (#451)
Unverified signature* feat: Improve error handling and parsing of MODELS environment variable * Add more verbose parsing error * Lint * improve message * lint * refactor error handling and default values in models * improve * format --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>8ce8b63
-
[Websearch] update (#427)
Mishig +2: julien-c, victor Unverified signature* Fix reuqest body * update webSearchQueryPromptTemplate * update generate google query parser * Add today's date to google search query creator * crawl top stories if exts; remove answer_box & knowledgeGraph * Create paragraph chunks from top articles * flattened paragprah chunks * update status texts * add gradio client * call gradio app for RAG * Web scrape only "p, li, span" els * add MAX_N_CHUNKS * gradio result typing * parse only <p> elements * rm dev change * update typing WebSearch * buld RAG prompt * Rm dev change * change websearch context msg from user to assisntat type * use hosted gradio app * fix lint * prompt engineering * more prompt engineering * MAX_N_PAGES_SCRAPE = 10 * better error msg * more prompt engineering * revert websearch prompt to previous * rm `top_stories` from websearch as the results are not good * Stop using gradio client, use regular fetch * chore * Rm websearch…ebac87f
-
🐛 Fix - Remove the min length validation on preprompt
Should fix the docker container7533ab7
-
Support custom system prompts from the user (#399)
Unverified signature* Support custom system prompts from the user * linter * types & lintcd6894d -
Make all prompt templates configurable (#400)
Unverified signature* Make all prompt templates configurable This PR enables all prompts to be configurable using handlebar templates as described in #382. For backward compatibility the old hardcoded templates are reimplemented as the default templates. The old template parameters such as `preprompt`, `userMessageToken`, `userMessageEndToken`, `assistantMessageToken`, `assistantMessageEndToken` are now considered legacy. They still work as they are exposed as variables to the default template. However, new prompt configurations should not use these. And it is recommended that the legacy support is eventually removed. As an example, this is how the default chat prompt template is implemented: ``` {{preprompt}} {{#each messages}} {{#ifUser}}{{@root.userMessageToken}}{{content}}{{@root.userMessageEndToken}}{{/ifUser}} {{#ifAssistant}}{{@root.assistantMessageToken}}{{content}}{{@root.assistantMessageEndToken}}{{/ifAssistant}} {{/each}} {{assistantMessageToken}} ``` In addition, …447c0ca
-
Add Sagemaker support (#401)
Unverified signature* work on sagemaker support * fix sagemaker integration * remove unnecessary deps * fix default endpoint * remove unneeded deps, fixed types * Use conditional validation for endpoints This was needed because the discriminated union couldn't handle the legacy case where `host` is undefined. * add note in readme about aws sagemaker2e6d1bb
-
allow different user and assistant end-token (#375)
Unverified signature* allow different user and assistant end-token For models like Llama2, the EndToken is not the same for a userMessage and an assistantMessage. This implements `userMessageEndToken` and `assistantMessageEndToken` which overwrites the messageEndToken behavior. This PR also allows empty strings as userMessageToken and assistantMessageToken and makes this the default. This adds additional flexibility, which is required in the case of Llama2 where the first userMessage is effectively different because of the system message. Note that because `userMessageEndToken` and `assistantMessageToken` are nearly always concatenated, it is almost redundant to have both. The exception is `generateQuery` for websearch which have several consecutive user messages. * Make model branding customizable based on env var (#345) * rm open assistant branding * Update SettingsModal.svelte * make settings work with a dynamic list of models * fixed types --------- Co-authored-…f209301
-
Add ability to define custom model/dataset URLs (#347)
Unverified signature* Add ability to define custom model/dataset URLs * lint --------- Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>ce2231f
-
Disable models (#187)
Adrien Denat +1: Eliott C.Unverified signature* add ability for a model to be disabled * fallback to default model if user has invalid/disabled model in settings * set conversations to read-only if conv model is disabled * add better disabled state on chat input * fix inverted if statement * remove bigcode from .env * fix not updating settings Co-authored-by: Eliott C. <coyotte508@gmail.com> * refactor solution to use an OLD_MODELS env var instead * OLD_MODELS needs to be in .env but can be empty * fix typing issue + typo * refactor models validation * old models displayName is optional * set read only prop on user message not assistant messages * remove console.log * ♻️ Refacto isReadOnly * 🩹 Fluff: use 410 http code --------- Co-authored-by: Eliott C. <coyotte508@gmail.com>b7b2c8c
-
✨ Add model id to be able to switch models while keeping conversations valid (#181)
Eliott CUnverified signaturecf7ac8d
-
make models description dynamic (#169)
Adrien Denat +1: Eliott CUnverified signatureCo-authored-by: Eliott C <coyotte508@gmail.com>12c3a5a -
🐛 Stop bigcode prompts (#170)
Eliott CUnverified signaturead6275a -
Models selector (#164)
Adrien Denat +3: victor, julien-c, coyotte508 Unverified signatureCo-authored-by: Victor Mustar <victor.mustar@gmail.com> Co-authored-by: Julien Chaumond <julien@huggingface.co> Co-authored-by: coyotte508 <coyotte508@gmail.com>2606dde
-
♻️ Simplify setup, prepare for multi-models (#156)
Eliott CUnverified signature7764421