* refactor: remove tokenizer-related functionality and dependencies
- Removed tokenizer dependencies from package.json.
- Deleted TokensCounter component and its usages across the application.
- Updated model configurations to exclude tokenizer properties.
- Refactored model processing logic to support only OpenAI-compatible endpoints.
- Adjusted API responses to omit tokenizer information.
- Cleaned up related utility functions and imports.
* Remove web search functionality and related components
- Deleted endpoints for various web search APIs (serpApi, serpStack, serper, webLocal, youApi).
- Removed generateQuery function and its usage in search.
- Eliminated web search related types and interfaces from the codebase.
- Updated Assistant and Conversation types to remove references to web search and embedding models.
- Cleaned up related utility functions and message updates for web search.
- Adjusted API routes and components to reflect the removal of web search features.
- Updated…
feat(test): add support for testing svelte components in isolation (#1692)
* feat(test): add support for testing svelte components in isolation
* refactor: use `vitest-browser-svelte` for tests, fix markdownRenderer tests
* test: remove todos on tests that pass
* fix: deps & tests
* feat(config): wip, make config controllable from DB, public side for now
* fix: build
* feat(config): add config manager env var toggle
* wip: make backend config use config manager
* fix: typechecks
* fix: circular import
* refactor: init hook sveltekit
* fix: disable config manager during tests
* refactor: use enum for semaphore, move delete from auto after 60s to separate `deleteAt` field
* feat: config manager checks for updates on requests
* fix: config script exit
* fix: remove top-level awaits
* fix: script await
* feat: filter config keys based on public environment variables for more security
* fix: change import to type for public environment variables
* feat: implement admin token management and CLI login functionality
- Added `ADMIN_CLI_LOGIN` and `ADMIN_TOKEN` to the environment configuration.
- Introduced `AdminTokenManager` for handling admin sessions and token validation.
- Updated user session handling to include admin status.
- Enhanced API routes to utilize admin checks based on the new token management.
- Modified frontend components to reflect admin status correctly.
- Added a new endpoint for validating admin tokens.
* fix: remove token after validation
* feat(migrations): clean up empty conversations from db
* fix(tests): better test coverage, fix bug with sessions
* feat: better migration
* fix: stop test server after running tests
* fix: migration code
* fix: refactor to take into account large size of session collection
* Add healthcheck route
* Add prom client with basic metrics
* wip: serve metrics on a different port
* exclude server from tsconfig
* latest
* refacto(all): use class & singleton
* refacto(all): use class & singleton
* refacto(all): add request logs
* Make all the env vars dynamic (#1092)
* Add truncate to task model (#1090)
* retry text area height (#1091)
* Make all the env vars dynamic
* only check for db at runtime
* remove secret from build step
* improve dockerfile
* Wrap db in singleton
* add .env.local to dockerignore
* changes to dockerfile
* lint
* aborted generations
* move collections build check
* Use a single dockerfile
* lint
* fixes
* lint
* don't return db during building
* remove dev
---------
Co-authored-by: Victor Muštar <victor.mustar@gmail.com>
* refacto(all): update default metrics port
* fix recursion error
* add version check
* revert vite preview
* Move request logs to debug level and add an env var to filter by log level in d…
* Add limits on messages, conversations, assistants and messages/minute
* Add max message length limit
* remove rate limits from public config
* add `RATE_LIMITS` to secrets
* Add `MESSAGES_BEFORE_LOGIN` to secrets
* replace `RATE_LIMITS` by `USAGE_LIMITS`
* replace `RateLimits` by `usageLimits` and only get nEvents if needed
* rename schema too
* replace \r\n by \n