diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 0000000000000000000000000000000000000000..de43641f6737cdcb95128d4f7e98bacc145adb8c --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,571 @@ +# How to contribute + +We would love to accept your patches and contributions to this project. This +document includes: + +- **[Before you begin](#before-you-begin):** Essential steps to take before + becoming a Gemini CLI contributor. +- **[Code contribution process](#code-contribution-process):** How to contribute + code to Gemini CLI. +- **[Development setup and workflow](#development-setup-and-workflow):** How to + set up your development environment and workflow. +- **[Documentation contribution process](#documentation-contribution-process):** + How to contribute documentation to Gemini CLI. + +We're looking forward to seeing your contributions! + +## Before you begin + +### Sign our Contributor License Agreement + +Contributions to this project must be accompanied by a +[Contributor License Agreement](https://cla.developers.google.com/about) (CLA). +You (or your employer) retain the copyright to your contribution; this simply +gives us permission to use and redistribute your contributions as part of the +project. + +If you or your current employer have already signed the Google CLA (even if it +was for a different project), you probably don't need to do it again. + +Visit to see your current agreements or to +sign a new one. + +### Review our Community Guidelines + +This project follows +[Google's Open Source Community Guidelines](https://opensource.google/conduct/). + +## Code contribution process + +### Get started + +The process for contributing code is as follows: + +1. **Find an issue** that you want to work on. If an issue is tagged as + `πŸ”’Maintainers only`, this means it is reserved for project maintainers. We + will not accept pull requests related to these issues. In the near future, + we will explicitly mark issues looking for contributions using the + `help-wanted` label. If you believe an issue is a good candidate for + community contribution, please leave a comment on the issue. A maintainer + will review it and apply the `help-wanted` label if appropriate. Only + maintainers should attempt to add the `help-wanted` label to an issue. +2. **Fork the repository** and create a new branch. +3. **Make your changes** in the `packages/` directory. +4. **Ensure all checks pass** by running `npm run preflight`. +5. **Open a pull request** with your changes. + +### Code reviews + +All submissions, including submissions by project members, require review. We +use [GitHub pull requests](https://docs.github.com/articles/about-pull-requests) +for this purpose. + +To assist with the review process, we provide an automated review tool that +helps detect common anti-patterns, testing issues, and other best practices that +are easy to miss. + +#### Using the automated review tool + +You can run the review tool in two ways: + +1. **Using the helper script (Recommended):** We provide a script that + automatically handles checking out the PR into a separate worktree, + installing dependencies, building the project, and launching the review + tool. + + ```bash + ./scripts/review.sh [model] + ``` + + **Warning:** If you run `scripts/review.sh`, you must have first verified + that the code for the PR being reviewed is safe to run and does not contain + data exfiltration attacks. + + **Authors are strongly encouraged to run this script on their own PRs** + immediately after creation. This allows you to catch and fix simple issues + locally before a maintainer performs a full review. + + **Note on Models:** By default, the script uses the latest Pro model + (`gemini-3.1-pro-preview`). If you do not have enough Pro quota, you can run + it with the latest Flash model instead: + `./scripts/review.sh gemini-3-flash-preview`. + +2. **Manually from within Gemini CLI:** If you already have the PR checked out + and built, you can run the tool directly from the CLI prompt: + + ```text + /review-frontend + ``` + +Replace `` with your pull request number. Reviewers should use this +tool to augment, not replace, their manual review process. + +### Self-assigning and unassigning issues + +To assign an issue to yourself, simply add a comment with the text `/assign`. To +unassign yourself from an issue, add a comment with the text `/unassign`. + +The comment must contain only that text and nothing else. These commands will +assign or unassign the issue as requested, provided the conditions are met +(e.g., an issue must be unassigned to be assigned). + +Please note that you can have a maximum of 3 issues assigned to you at any given +time and that only +[issues labeled "help wanted"](https://github.com/google-gemini/gemini-cli/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22help%20wanted%22) +may be self-assigned. + +### Pull request guidelines + +To help us review and merge your PRs quickly, please follow these guidelines. +PRs that do not meet these standards may be closed. + +#### 1. Link to an existing issue + +All PRs should be linked to an existing issue in our tracker. This ensures that +every change has been discussed and is aligned with the project's goals before +any code is written. + +- **For bug fixes:** The PR should be linked to the bug report issue. +- **For features:** The PR should be linked to the feature request or proposal + issue that has been approved by a maintainer. + +If an issue for your change doesn't exist, we will automatically close your PR +along with a comment reminding you to associate the PR with an issue. The ideal +workflow starts with an issue that has been reviewed and approved by a +maintainer. Please **open the issue first** and wait for feedback before you +start coding. + +#### 2. Keep it small and focused + +We favor small, atomic PRs that address a single issue or add a single, +self-contained feature. + +- **Do:** Create a PR that fixes one specific bug or adds one specific feature. +- **Don't:** Bundle multiple unrelated changes (e.g., a bug fix, a new feature, + and a refactor) into a single PR. + +Large changes should be broken down into a series of smaller, logical PRs that +can be reviewed and merged independently. + +#### 3. Use draft PRs for work in progress + +If you'd like to get early feedback on your work, please use GitHub's **Draft +Pull Request** feature. This signals to the maintainers that the PR is not yet +ready for a formal review but is open for discussion and initial feedback. + +#### 4. Ensure all checks pass + +Before submitting your PR, ensure that all automated checks are passing by +running `npm run preflight`. This command runs all tests, linting, and other +style checks. + +#### 5. Update documentation + +If your PR introduces a user-facing change (e.g., a new command, a modified +flag, or a change in behavior), you must also update the relevant documentation +in the `/docs` directory. + +See more about writing documentation: +[Documentation contribution process](#documentation-contribution-process). + +#### 6. Write clear commit messages and a good PR description + +Your PR should have a clear, descriptive title and a detailed description of the +changes. Follow the [Conventional Commits](https://www.conventionalcommits.org/) +standard for your commit messages. + +- **Good PR title:** `feat(cli): Add --json flag to 'config get' command` +- **Bad PR title:** `Made some changes` + +In the PR description, explain the "why" behind your changes and link to the +relevant issue (e.g., `Fixes #123`). + +### Forking + +If you are forking the repository you will be able to run the Build, Test and +Integration test workflows. However in order to make the integration tests run +you'll need to add a +[GitHub Repository Secret](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) +with a value of `GEMINI_API_KEY` and set that to a valid API key that you have +available. Your key and secret are private to your repo; no one without access +can see your key and you cannot see any secrets related to this repo. + +Additionally you will need to click on the `Actions` tab and enable workflows +for your repository, you'll find it's the large blue button in the center of the +screen. + +### Development setup and workflow + +This section guides contributors on how to build, modify, and understand the +development setup of this project. + +### Setting up the development environment + +**Prerequisites:** + +1. **Node.js**: + - **Development:** Please use Node.js `~20.19.0`. This specific version is + required due to an upstream development dependency issue. You can use a + tool like [nvm](https://github.com/nvm-sh/nvm) to manage Node.js versions. + - **Production:** For running the CLI in a production environment, any + version of Node.js `>=20` is acceptable. +2. **Git** + +### Build process + +To clone the repository: + +```bash +git clone https://github.com/google-gemini/gemini-cli.git # Or your fork's URL +cd gemini-cli +``` + +To install dependencies defined in `package.json` as well as root dependencies: + +```bash +npm install +``` + +To build the entire project (all packages): + +```bash +npm run build +``` + +This command typically compiles TypeScript to JavaScript, bundles assets, and +prepares the packages for execution. Refer to `scripts/build.js` and +`package.json` scripts for more details on what happens during the build. + +### Enabling sandboxing + +[Sandboxing](#sandboxing) is highly recommended and requires, at a minimum, +setting `GEMINI_SANDBOX=true` in your `~/.env` and ensuring a sandboxing +provider (e.g. `macOS Seatbelt`, `docker`, or `podman`) is available. See +[Sandboxing](#sandboxing) for details. + +To build both the `gemini` CLI utility and the sandbox container, run +`build:all` from the root directory: + +```bash +npm run build:all +``` + +To skip building the sandbox container, you can use `npm run build` instead. + +### Running the CLI + +To start the Gemini CLI from the source code (after building), run the following +command from the root directory: + +```bash +npm start +``` + +If you'd like to run the source build outside of the gemini-cli folder, you can +utilize `npm link path/to/gemini-cli/packages/cli` (see: +[docs](https://docs.npmjs.com/cli/v9/commands/npm-link)) or +`alias gemini="node path/to/gemini-cli/packages/cli"` to run with `gemini` + +### Running tests + +This project contains two types of tests: unit tests and integration tests. + +#### Unit tests + +To execute the unit test suite for the project: + +```bash +npm run test +``` + +This will run tests located in the `packages/core` and `packages/cli` +directories. Ensure tests pass before submitting any changes. For a more +comprehensive check, it is recommended to run `npm run preflight`. + +#### Integration tests + +The integration tests are designed to validate the end-to-end functionality of +the Gemini CLI. They are not run as part of the default `npm run test` command. + +To run the integration tests, use the following command: + +```bash +npm run test:e2e +``` + +For more detailed information on the integration testing framework, please see +the +[Integration Tests documentation](https://geminicli.com/docs/integration-tests). + +### Linting and preflight checks + +To ensure code quality and formatting consistency, run the preflight check: + +```bash +npm run preflight +``` + +This command will run ESLint, Prettier, all tests, and other checks as defined +in the project's `package.json`. + +_ProTip_ + +after cloning create a git precommit hook file to ensure your commits are always +clean. + +```bash +echo " +# Run npm build and check for errors +if ! npm run preflight; then + echo "npm build failed. Commit aborted." + exit 1 +fi +" > .git/hooks/pre-commit && chmod +x .git/hooks/pre-commit +``` + +#### Formatting + +To separately format the code in this project, run the following command from +the root directory: + +```bash +npm run format +``` + +This command uses Prettier to format the code according to the project's style +guidelines. + +#### Linting + +To separately lint the code in this project, run the following command from the +root directory: + +```bash +npm run lint +``` + +### Coding conventions + +- Please adhere to the coding style, patterns, and conventions used throughout + the existing codebase. +- Consult + [GEMINI.md](https://github.com/google-gemini/gemini-cli/blob/main/GEMINI.md) + (typically found in the project root) for specific instructions related to + AI-assisted development, including conventions for React, comments, and Git + usage. +- **Imports:** Pay special attention to import paths. The project uses ESLint to + enforce restrictions on relative imports between packages. + +### Debugging + +#### VS Code + +0. Run the CLI to interactively debug in VS Code with `F5` +1. Start the CLI in debug mode from the root directory: + ```bash + npm run debug + ``` + This command runs `node --inspect-brk dist/gemini.js` within the + `packages/cli` directory, pausing execution until a debugger attaches. You + can then open `chrome://inspect` in your Chrome browser to connect to the + debugger. +2. In VS Code, use the "Attach" launch configuration (found in + `.vscode/launch.json`). + +Alternatively, you can use the "Launch Program" configuration in VS Code if you +prefer to launch the currently open file directly, but 'F5' is generally +recommended. + +To hit a breakpoint inside the sandbox container run: + +```bash +DEBUG=1 gemini +``` + +**Note:** If you have `DEBUG=true` in a project's `.env` file, it won't affect +gemini-cli due to automatic exclusion. Use `.gemini/.env` files for gemini-cli +specific debug settings. + +### React DevTools + +To debug the CLI's React-based UI, you can use React DevTools. + +1. **Start the Gemini CLI in development mode:** + + ```bash + DEV=true npm start + ``` + +2. **Install and run React DevTools version 6 (which matches the CLI's + `react-devtools-core`):** + + You can either install it globally: + + ```bash + npm install -g react-devtools@6 + react-devtools + ``` + + Or run it directly using npx: + + ```bash + npx react-devtools@6 + ``` + + Your running CLI application should then connect to React DevTools. + ![](/docs/assets/connected_devtools.png) + +### Sandboxing + +#### macOS Seatbelt + +On macOS, `gemini` uses Seatbelt (`sandbox-exec`) under a `permissive-open` +profile (see `packages/cli/src/utils/sandbox-macos-permissive-open.sb`) that +denies operations by default, confining writes to the project folder while +allowing broad file reads and outbound network traffic ("open") by default. You +can switch to a `strict-open` profile (see +`packages/cli/src/utils/sandbox-macos-strict-open.sb`) that restricts both reads +and writes to the working directory while allowing outbound network traffic by +setting `SEATBELT_PROFILE=strict-open` in your environment or `.env` file. +Available built-in profiles are `permissive-{open,proxied}`, +`restrictive-{open,proxied}`, and `strict-{open,proxied}` (see below for proxied +networking). You can also switch to a custom profile +`SEATBELT_PROFILE=` if you also create a file +`.gemini/sandbox-macos-.sb` under your project settings directory +`.gemini`. + +#### Container-based sandboxing (all platforms) + +For stronger container-based sandboxing on macOS or other platforms, you can set +`GEMINI_SANDBOX=true|docker|podman|` in your environment or `.env` +file. The specified command (or if `true` then either `docker` or `podman`) must +be installed on the host machine. Once enabled, `npm run build:all` will build a +minimal container ("sandbox") image and `npm start` will launch inside a fresh +instance of that container. The first build can take 20-30s (mostly due to +downloading of the base image) but after that both build and start overhead +should be minimal. Default builds (`npm run build`) will not rebuild the +sandbox. + +Container-based sandboxing mounts the project directory (and system temp +directory) with read-write access and is started/stopped/removed automatically +as you start/stop Gemini CLI. Files created within the sandbox should be +automatically mapped to your user/group on host machine. You can easily specify +additional mounts, ports, or environment variables by setting +`SANDBOX_{MOUNTS,PORTS,ENV}` as needed. You can also fully customize the sandbox +for your projects by creating the files `.gemini/sandbox.Dockerfile` and/or +`.gemini/sandbox.bashrc` under your project settings directory (`.gemini`) and +running `gemini` with `BUILD_SANDBOX=1` to trigger building of your custom +sandbox. + +#### Proxied networking + +All sandboxing methods, including macOS Seatbelt using `*-proxied` profiles, +support restricting outbound network traffic through a custom proxy server that +can be specified as `GEMINI_SANDBOX_PROXY_COMMAND=`, where `` +must start a proxy server that listens on `:::8877` for relevant requests. See +`docs/examples/proxy-script.md` for a minimal proxy that only allows `HTTPS` +connections to `example.com:443` (e.g. `curl https://example.com`) and declines +all other requests. The proxy is started and stopped automatically alongside the +sandbox. + +### Manual publish + +We publish an artifact for each commit to our internal registry. But if you need +to manually cut a local build, then run the following commands: + +``` +npm run clean +npm install +npm run auth +npm run prerelease:dev +npm publish --workspaces +``` + +## Documentation contribution process + +Our documentation must be kept up-to-date with our code contributions. We want +our documentation to be clear, concise, and helpful to our users. We value: + +- **Clarity:** Use simple and direct language. Avoid jargon where possible. +- **Accuracy:** Ensure all information is correct and up-to-date. +- **Completeness:** Cover all aspects of a feature or topic. +- **Examples:** Provide practical examples to help users understand how to use + Gemini CLI. + +### Getting started + +The process for contributing to the documentation is similar to contributing +code. + +1. **Fork the repository** and create a new branch. +2. **Make your changes** in the `/docs` directory. +3. **Preview your changes locally** in Markdown rendering. +4. **Lint and format your changes.** Our preflight check includes linting and + formatting for documentation files. + ```bash + npm run preflight + ``` +5. **Open a pull request** with your changes. + +### Documentation structure + +Our documentation is organized using +[sidebar.json](https://github.com/google-gemini/gemini-cli/blob/main/docs/sidebar.json) +as the table of contents. When adding new documentation: + +1. Create your markdown file **in the appropriate directory** under `/docs`. +2. Add an entry to `sidebar.json` in the relevant section. +3. Ensure all internal links use relative paths and point to existing files. + +### Style guide + +We follow the +[Google Developer Documentation Style Guide](https://developers.google.com/style). +Please refer to it for guidance on writing style, tone, and formatting. + +#### Key style points + +- Use sentence case for headings. +- Write in second person ("you") when addressing the reader. +- Use present tense. +- Keep paragraphs short and focused. +- Use code blocks with appropriate language tags for syntax highlighting. +- Include practical examples whenever possible. + +### Linting and formatting + +We use `prettier` to enforce a consistent style across our documentation. The +`npm run preflight` command will check for any linting issues. + +You can also run the linter and formatter separately: + +- `npm run lint` - Check for linting issues +- `npm run format` - Auto-format markdown files +- `npm run lint:fix` - Auto-fix linting issues where possible + +Please make sure your contributions are free of linting errors before submitting +a pull request. + +### Before you submit + +Before submitting your documentation pull request, please: + +1. Run `npm run preflight` to ensure all checks pass. +2. Review your changes for clarity and accuracy. +3. Check that all links work correctly. +4. Ensure any code examples are tested and functional. +5. Sign the + [Contributor License Agreement (CLA)](https://cla.developers.google.com/) if + you haven't already. + +### Need help? + +If you have questions about contributing documentation: + +- Check our [FAQ](https://geminicli.com/docs/resources/faq). +- Review existing documentation for examples. +- Open [an issue](https://github.com/google-gemini/gemini-cli/issues) to discuss + your proposed changes. +- Reach out to the maintainers. + +We appreciate your contributions to making Gemini CLI documentation better! diff --git a/docs/behavioral-evals.md b/docs/behavioral-evals.md new file mode 100644 index 0000000000000000000000000000000000000000..823be20bf9372dce9ecb28aaf4739de6b701b409 --- /dev/null +++ b/docs/behavioral-evals.md @@ -0,0 +1,185 @@ +# Behavioral Evaluations & EDK Guide + +This guide introduces the **Eval Development Kit (EDK)** and details how to +write, validate, run, and report on **behavioral evaluations** in the Gemini CLI +codebase. + +--- + +## Overview + +Behavioral evaluations are automated tests designed to assert on the +**behavior** of the Gemini CLI agent (e.g., verifying which tools are called, +checking call ordering, or avoiding destructive commands) rather than checking +the final prose output. + +Evaluating agent behavior is critical because: + +1. Model responses are non-deterministic, making exact prose matching highly + fragile. +2. We must ensure the model utilizes the most efficient tools (e.g., batching + files via `read_many_files` instead of sequential `read_file` calls). +3. We must enforce safety boundaries (e.g., preventing execution of raw shell + commands when safe alternatives exist). + +All behavioral evaluations are stored under the `evals/` directory. + +--- + +## EDK Developer Commands + +The EDK provides CLI tools under `scripts/` to help contributors audit, check, +and monitor evals. + +### 1. `npm run eval:inventory` + +Scans all eval files under `evals/`, statically parses them, and provides a +structured overview of what exists in the repository. + +- **Usage:** + ```bash + npm run eval:inventory + ``` +- **JSON Output:** For CI integration or inventory indexing, generate a + machine-readable JSON report: + ```bash + npm run eval:inventory -- --json + ``` +- **Custom Root:** Run against another directory or repository: + ```bash + npm run eval:inventory -- --root /path/to/other/repo + ``` + +--- + +### 2. `npm run eval:validate` + +A lint-like checker that validates eval source files against standard structural +guidelines and best practices. + +- **Usage:** + ```bash + npm run eval:validate + ``` +- **Custom Scopes:** Validate a specific file: + ```bash + npm run eval:validate -- evals/my-test.eval.ts + ``` + +#### Validation Rules & Severities + +| Rule ID | Severity | Description | +| :------------------- | :---------- | :--------------------------------------------------------------------------------------------------------------------- | +| `file-naming` | **Error** | File must match `*.eval.ts` or `*.eval.tsx` naming conventions. | +| `valid-policy` | **Error** | Policy must be one of `ALWAYS_PASSES`, `USUALLY_PASSES`, or `USUALLY_FAILS`. | +| `suite-metadata` | **Error** | Both `suiteName` and `suiteType` must be present as static string literals. | +| `prompt-presence` | **Error** | Every eval case must have a non-empty `prompt` string. | +| `case-name-static` | **Error** | The case name must be a static string literal, not computed dynamically. | +| `invalid-tool-refs` | **Error** | All tools referenced in assertions must match known built-in or legacy tools. | +| `positive-assertion` | **Error** | Evaluation cases must assert on at least one tool call (e.g., check `waitForToolCall` has been invoked). | +| `workspace-setup` | **Error** | Workspace behaviors (like file-system edits/reads) must set up a `files` object. | +| `new-evals-policy` | **Warning** | New evals must not use `ALWAYS_PASSES` policy initially (they should be promoted after nightly data proves stability). | + +Warnings (`new-evals-policy`) will be logged with `⚠` and will **not** cause +the CLI process to exit with status `1`. Errors (`βœ—`) will block CI builds and +return exit status `1`. + +--- + +### 3. `npm run eval:report` + +Aggregates local vitest `report.json` artifacts, maps them against inventory +policies, and summarizes the pass rates per model. + +- **Usage:** + ```bash + npm run eval:report + ``` + By default, it scans `evals/logs/` recursively for `report.json` files. +- **Specifying Directory:** + ```bash + npm run eval:report -- /path/to/logs + ``` +- **JSON Output:** + ```bash + npm run eval:report -- --json + ``` + +--- + +## Contributor Workflow + +When writing a new behavioral evaluation, adhere to this workflow to ensure +high-quality, non-flaky test runs. + +### Step-by-Step Guide + +1. **Identify the Target Behavior**: Determine which tool calls need + verification (e.g., `web_fetch` must be called). +2. **Author the Eval File**: Create your file under `evals/.eval.ts` + naming it properly. +3. **Configure Workspace Files**: If the eval reads or edits files, define them + inside the `files` metadata field. +4. **Assert Behavior, Not Prose**: Ensure the `assert` block checks tool + interactions using `rig.waitForToolCall` or similar. Do not check final + prose. +5. **Run Locally**: + ```bash + RUN_EVALS=true npx vitest run evals/my-test.eval.ts + ``` +6. **Deflake**: Run your eval at least 3 times locally to verify it does not + fail due to model variance. +7. **Run Validation**: Run `npm run eval:validate` to ensure no linting errors + are present. + +### Acceptance Criteria Checklist + +- [ ] **Naming**: File ends with `.eval.ts` or `.eval.tsx`. +- [ ] **Policy**: New evals start as `USUALLY_PASSES`. +- [ ] **Metadata**: Static `suiteName` and `suiteType` (e.g. `'behavioral'`) are + specified. +- [ ] **Assertions**: Uses `rig.waitForToolCall` or asserts tool arguments + explicitly. +- [ ] **Clean workspace**: Does not write to files outside `rig.testDir`. + +### Common Anti-Patterns to Avoid + +- **Restricting core tools**: Never override `settings.tools.core` to limit + tools. Evals must run against the default toolset. +- **Checking model prose**: Avoid `expect(result).toContain('something')` since + model wording is non-deterministic. +- **Integration-only testing**: Evals that only write files without checking + realistic model prompts are integration tests and belong under + `integration-tests/`. + +--- + +## CI & Dashboard Integration + +You can easily automate behavioral evaluations or compile dashboard data using +EDK's JSON reporters. + +### CI Validation Block + +Add a step in your PR checks or GitHub workflows to automatically lint new evals +and block pull requests containing validation errors: + +```yaml +- name: Run Eval Validator + run: npm run eval:validate +``` + +### Publishing to a Dashboard + +To record nightly performance metrics across multiple models: + +1. Configure your workflow to run evaluations with the JSON reporter: + ```bash + cross-env GEMINI_MODEL=gemini-2.5-pro npx vitest run --config evals/vitest.config.ts --reporter=json --outputFile="evals/logs/eval-logs-gemini-2.5-pro/report.json" + ``` +2. Aggregate all test runs using the reporting tool: + ```bash + npm run eval:report -- evals/logs --json > aggregated_report.json + ``` +3. Upload `aggregated_report.json` to your dashboard storage backend to + visualize pass rates over time. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000000000000000000000000000000000000..d5fefce47c41816a5b7e8757789404e7e9a11a17 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,139 @@ +# Gemini CLI documentation + +Gemini CLI brings the power of Gemini models directly into your terminal. Use it +to understand code, automate tasks, and build workflows with your local project +context. + +## Install + +```bash +npm install -g @google/gemini-cli +``` + +## Get started + +Jump in to Gemini CLI. + +- **[Quickstart](./get-started/index.md):** Your first session with Gemini CLI. +- **[Installation](./get-started/installation.mdx):** How to install Gemini CLI + on your system. +- **[Authentication](./get-started/authentication.mdx):** Setup instructions for + personal and enterprise accounts. +- **[CLI cheatsheet](./cli/cli-reference.md):** A quick reference for common + commands and options. +- **[Gemini 3 on Gemini CLI](./get-started/gemini-3.md):** Learn about Gemini 3 + support in Gemini CLI. + +## Use Gemini CLI + +User-focused guides and tutorials for daily development workflows. + +- **[File management](./cli/tutorials/file-management.md):** How to work with + local files and directories. +- **[Get started with Agent skills](./cli/tutorials/skills-getting-started.md):** + Getting started with specialized expertise. +- **[Manage context and memory](./cli/tutorials/memory-management.md):** + Managing persistent instructions and facts. +- **[Execute shell commands](./cli/tutorials/shell-commands.md):** Executing + system commands safely. +- **[Manage sessions and history](./cli/tutorials/session-management.md):** + Resuming, managing, and rewinding conversations. +- **[Plan tasks with todos](./cli/tutorials/task-planning.md):** Using todos for + complex workflows. +- **[Web search and fetch](./cli/tutorials/web-tools.md):** Searching and + fetching content from the web. +- **[Set up an MCP server](./cli/tutorials/mcp-setup.md):** Set up an MCP + server. +- **[Automate tasks](./cli/tutorials/automation.md):** Automate tasks. + +## Features + +Technical documentation for each capability of Gemini CLI. + +- **[Extensions](./extensions/index.md):** Extend Gemini CLI with new tools and + capabilities. +- **[Agent Skills](./cli/skills.md):** Use specialized agents for specific + tasks. +- **[Checkpointing](./cli/checkpointing.md):** Automatic session snapshots. +- **[Headless mode](./cli/headless.md):** Programmatic and scripting interface. +- **[Hooks](./hooks/index.md):** Customize Gemini CLI behavior with scripts. +- **[IDE integration](./ide-integration/index.md):** Integrate Gemini CLI with + your favorite IDE. +- **[MCP servers](./tools/mcp-server.md):** Connect to and use remote agents. +- **[Model routing](./cli/model-routing.md):** Automatic fallback resilience. +- **[Model selection](./cli/model.md):** Choose the best model for your needs. +- **[Plan mode πŸ”¬](./cli/plan-mode.md):** Use a safe, read-only mode for + planning complex changes. +- **[Subagents πŸ”¬](./core/subagents.md):** Using specialized agents for specific + tasks. +- **[Remote subagents πŸ”¬](./core/remote-agents.md):** Connecting to and using + remote agents. +- **[Rewind](./cli/rewind.md):** Rewind and replay sessions. +- **[Sandboxing](./cli/sandbox.md):** Isolate tool execution. +- **[Settings](./cli/settings.md):** Full configuration reference. +- **[Telemetry](./cli/telemetry.md):** Usage and performance metric details. +- **[Token caching](./cli/token-caching.md):** Performance optimization. + +## Configuration + +Settings and customization options for Gemini CLI. + +- **[Custom commands](./cli/custom-commands.md):** Personalized shortcuts. +- **[Enterprise configuration](./cli/enterprise.md):** Professional environment + controls. +- **[Ignore files (.geminiignore)](./cli/gemini-ignore.md):** Exclusion pattern + reference. +- **[Model configuration](./cli/generation-settings.md):** Fine-tune generation + parameters like temperature and thinking budget. +- **[Project context (GEMINI.md)](./cli/gemini-md.md):** Technical hierarchy of + context files. +- **[System prompt override](./cli/system-prompt.md):** Instruction replacement + logic. +- **[Themes](./cli/themes.md):** UI personalization technical guide. +- **[Trusted folders](./cli/trusted-folders.md):** Security permission logic. + +## Reference + +Deep technical documentation and API specifications. + +- **[Command reference](./reference/commands.md):** Detailed slash command + guide. +- **[Configuration reference](./reference/configuration.md):** Settings and + environment variables. +- **[Keyboard shortcuts](./reference/keyboard-shortcuts.md):** Productivity + tips. +- **[Memory import processor](./reference/memport.md):** How Gemini CLI + processes memory from various sources. +- **[Policy engine](./reference/policy-engine.md):** Fine-grained execution + control. +- **[Tools reference](./reference/tools.md):** Information on how tools are + defined, registered, and used. + +## Resources + +Support, release history, and legal information. + +- **[FAQ](./resources/faq.md):** Answers to frequently asked questions. +- **[Quota and pricing](./resources/quota-and-pricing.md):** Limits and billing + details. +- **[Terms and privacy](./resources/tos-privacy.md):** Official notices and + terms. +- **[Troubleshooting](./resources/troubleshooting.md):** Common issues and + solutions. +- **[Uninstall](./resources/uninstall.md):** How to uninstall Gemini CLI. + +## Development + +- **[Contribution guide](/docs/contributing):** How to contribute to Gemini CLI. +- **[Integration testing](./integration-tests.md):** Running integration tests. +- **[Issue and PR automation](./issue-and-pr-automation.md):** Automation for + issues and pull requests. +- **[Local development](./local-development.md):** Setting up a local + development environment. +- **[NPM package structure](./npm.md):** The structure of the NPM packages. + +## Releases + +- **[Release notes](./changelogs/index.md):** Release notes for all versions. +- **[Stable release](./changelogs/latest.md):** The latest stable release. +- **[Preview release](./changelogs/preview.md):** The latest preview release. diff --git a/docs/integration-tests.md b/docs/integration-tests.md new file mode 100644 index 0000000000000000000000000000000000000000..06ac3a347f2600d8581711fe1de906d038e7c7e1 --- /dev/null +++ b/docs/integration-tests.md @@ -0,0 +1,293 @@ +# Integration tests + +This document provides information about the integration testing framework used +in this project. + +## Overview + +The integration tests are designed to validate the end-to-end functionality of +Gemini CLI. They execute the built binary in a controlled environment and verify +that it behaves as expected when interacting with the file system. + +These tests are located in the `integration-tests` directory and are run using a +custom test runner. + +## Building the tests + +Prior to running any integration tests, you need to create a release bundle that +you want to actually test: + +```bash +npm run bundle +``` + +You must re-run this command after making any changes to the CLI source code, +but not after making changes to tests. + +## Running the tests + +The integration tests are not run as part of the default `npm run test` command. +They must be run explicitly using the `npm run test:integration:all` script. + +The integration tests can also be run using the following shortcut: + +```bash +npm run test:e2e +``` + +## Running a specific set of tests + +To run a subset of test files, you can use +`npm run ....` where <integration +test command> is either `test:e2e` or `test:integration*` and `` +is any of the `.test.js` files in the `integration-tests/` directory. For +example, the following command runs `list_directory.test.js` and +`write_file.test.js`: + +```bash +npm run test:e2e list_directory write_file +``` + +### Running a single test by name + +To run a single test by its name, use the `--test-name-pattern` flag: + +```bash +npm run test:e2e -- --test-name-pattern "reads a file" +``` + +### Regenerating model responses + +Some integration tests use faked out model responses, which may need to be +regenerated from time to time as the implementations change. + +To regenerate these golden files, set the REGENERATE_MODEL_GOLDENS environment +variable to "true" when running the tests, for example: + +**WARNING**: If running locally you should review these updated responses for +any information about yourself or your system that gemini may have included in +these responses. + +```bash +REGENERATE_MODEL_GOLDENS="true" npm run test:e2e +``` + +**WARNING**: Make sure you run **await rig.cleanup()** at the end of your test, +else the golden files will not be updated. + +### Deflaking a test + +Before adding a **new** integration test, you should test it at least 5 times +with the deflake script or workflow to make sure that it is not flaky. + +### Deflake script + +```bash +npm run deflake -- --runs=5 --command="npm run test:e2e -- -- --test-name-pattern ''" +``` + +#### Deflake workflow + +```bash +gh workflow run deflake.yml --ref -f test_name_pattern="" +``` + +### Running all tests + +To run the entire suite of integration tests, use the following command: + +```bash +npm run test:integration:all +``` + +### Sandbox matrix + +The `all` command will run tests for `no sandboxing`, `docker` and `podman`. +Each individual type can be run using the following commands: + +```bash +npm run test:integration:sandbox:none +``` + +```bash +npm run test:integration:sandbox:docker +``` + +```bash +npm run test:integration:sandbox:podman +``` + +## Memory regression tests + +Memory regression tests are designed to detect heap growth and leaks across key +CLI scenarios. They are located in the `memory-tests` directory. + +These tests are distinct from standard integration tests because they measure +memory usage and compare it against committed baselines. + +### Running memory tests + +Memory tests are not run as part of the default `npm run test` or +`npm run test:e2e` commands. They are run nightly in CI but can be run manually: + +```bash +npm run test:memory +``` + +### Updating baselines + +If you intentionally change behavior that affects memory usage, you may need to +update the baselines. Set the `UPDATE_MEMORY_BASELINES` environment variable to +`true`: + +```bash +UPDATE_MEMORY_BASELINES=true npm run test:memory +``` + +This will run the tests, take median snapshots, and overwrite +`memory-tests/baselines.json`. You should review the changes and commit the +updated baseline file. + +### How it works + +The harness (`MemoryTestHarness` in `packages/test-utils`): + +- Forces garbage collection multiple times to reduce noise. +- Takes median snapshots to filter spikes. +- Compares against baselines with a 10% tolerance. +- Can analyze sustained leaks across 3 snapshots using `analyzeSnapshots()`. + +## Performance regression tests + +Performance regression tests are designed to detect wall-clock time, CPU usage, +and event loop delay regressions across key CLI scenarios. They are located in +the `perf-tests` directory. + +These tests are distinct from standard integration tests because they measure +performance metrics and compare it against committed baselines. + +### Running performance tests + +Performance tests are not run as part of the default `npm run test` or +`npm run test:e2e` commands. They are run nightly in CI but can be run manually: + +```bash +npm run test:perf +``` + +### Updating baselines + +If you intentionally change behavior that affects performance, you may need to +update the baselines. Set the `UPDATE_PERF_BASELINES` environment variable to +`true`: + +```bash +UPDATE_PERF_BASELINES=true npm run test:perf +``` + +This will run the tests multiple times (with warmup), apply IQR outlier +filtering, and overwrite `perf-tests/baselines.json`. You should review the +changes and commit the updated baseline file. + +### How it works + +The harness (`PerfTestHarness` in `packages/test-utils`): + +- Measures wall-clock time using `performance.now()`. +- Measures CPU usage using `process.cpuUsage()`. +- Monitors event loop delay using `perf_hooks.monitorEventLoopDelay()`. +- Applies IQR (Interquartile Range) filtering to remove outlier samples. +- Compares against baselines with a 15% tolerance. + +## Diagnostics + +The integration test runner provides several options for diagnostics to help +track down test failures. + +### Keeping test output + +You can preserve the temporary files created during a test run for inspection. +This is useful for debugging issues with file system operations. + +To keep the test output set the `KEEP_OUTPUT` environment variable to `true`. + +```bash +KEEP_OUTPUT=true npm run test:integration:sandbox:none +``` + +When output is kept, the test runner will print the path to the unique directory +for the test run. + +### Verbose output + +For more detailed debugging, set the `VERBOSE` environment variable to `true`. + +```bash +VERBOSE=true npm run test:integration:sandbox:none +``` + +When using `VERBOSE=true` and `KEEP_OUTPUT=true` in the same command, the output +is streamed to the console and also saved to a log file within the test's +temporary directory. + +The verbose output is formatted to clearly identify the source of the logs: + +``` +--- TEST: : --- +... output from the gemini command ... +--- END TEST: : --- +``` + +## Linting and formatting + +To ensure code quality and consistency, the integration test files are linted as +part of the main build process. You can also manually run the linter and +auto-fixer. + +### Running the linter + +To check for linting errors, run the following command: + +```bash +npm run lint +``` + +You can include the `:fix` flag in the command to automatically fix any fixable +linting errors: + +```bash +npm run lint:fix +``` + +## Directory structure + +The integration tests create a unique directory for each test run inside the +`.integration-tests` directory. Within this directory, a subdirectory is created +for each test file, and within that, a subdirectory is created for each +individual test case. + +This structure makes it easy to locate the artifacts for a specific test run, +file, or case. + +``` +.integration-tests/ +└── / + └── .test.js/ + └── / + β”œβ”€β”€ output.log + └── ...other test artifacts... +``` + +## Continuous integration + +To ensure the integration tests are always run, a GitHub Actions workflow is +defined in `.github/workflows/chained_e2e.yml`. This workflow automatically runs +the integrations tests for pull requests against the `main` branch, or when a +pull request is added to a merge queue. + +The workflow runs the tests in different sandboxing environments to ensure +Gemini CLI is tested across each: + +- `sandbox:none`: Runs the tests without any sandboxing. +- `sandbox:docker`: Runs the tests in a Docker container. +- `sandbox:podman`: Runs the tests in a Podman container. diff --git a/docs/issue-and-pr-automation.md b/docs/issue-and-pr-automation.md new file mode 100644 index 0000000000000000000000000000000000000000..cdcfbb75c78ccf903dba8bdd1b5aff4198b03594 --- /dev/null +++ b/docs/issue-and-pr-automation.md @@ -0,0 +1,203 @@ +# Automation and triage processes + +This document provides a detailed overview of the automated processes we use to +manage and triage issues and pull requests. Our goal is to provide prompt +feedback and ensure that contributions are reviewed and integrated efficiently. +Understanding this automation will help you as a contributor know what to expect +and how to best interact with our repository bots. + +## Guiding principle: Issues and pull requests + +First and foremost, almost every Pull Request (PR) should be linked to a +corresponding Issue. The issue describes the "what" and the "why" (the bug or +feature), while the PR is the "how" (the implementation). This separation helps +us track work, prioritize features, and maintain clear historical context. Our +automation is built around this principle. + + +> [!NOTE] +> Issues tagged as "πŸ”’Maintainers only" are reserved for project +> maintainers. We will not accept pull requests related to these issues. + +--- + +## Detailed automation workflows + +Here is a breakdown of the specific automation workflows that run in our +repository. + +### 1. When you open an issue: `Automated Issue Triage` + +This is the first bot you will interact with when you create an issue. Its job +is to perform an initial analysis and apply the correct labels. + +- **Workflow File**: `.github/workflows/gemini-automated-issue-triage.yml` +- **When it runs**: Immediately after an issue is created or reopened. +- **What it does**: + - It uses a Gemini model to analyze the issue's title and body against a + detailed set of guidelines. + - **Applies one `area/*` label**: Categorizes the issue into a functional area + of the project (for example, `area/ux`, `area/models`, `area/platform`). + - **Applies one `kind/*` label**: Identifies the type of issue (for example, + `kind/bug`, `kind/enhancement`, `kind/question`). + - **Applies one `priority/*` label**: Assigns a priority from P0 (critical) to + P3 (low) based on the described impact. + - **May apply `status/need-information`**: If the issue lacks critical details + (like logs or reproduction steps), it will be flagged for more information. + - **May apply `status/need-retesting`**: If the issue references a CLI version + that is more than six versions old, it will be flagged for retesting on a + current version. +- **What you should do**: + - Fill out the issue template as completely as possible. The more detail you + provide, the more accurate the triage will be. + - If the `status/need-information` label is added, provide the requested + details in a comment. + +### 2. When you open a pull request: `Continuous Integration (CI)` + +This workflow ensures that all changes meet our quality standards before they +can be merged. + +- **Workflow File**: `.github/workflows/ci.yml` +- **When it runs**: On every push to a pull request. +- **What it does**: + - **Lint**: Checks that your code adheres to our project's formatting and + style rules. + - **Test**: Runs our full suite of automated tests across macOS, Windows, and + Linux, and on multiple Node.js versions. This is the most time-consuming + part of the CI process. + - **Post Coverage Comment**: After all tests have successfully passed, a bot + will post a comment on your PR. This comment provides a summary of how well + your changes are covered by tests. +- **What you should do**: + - Ensure all CI checks pass. A green checkmark βœ… will appear next to your + commit when everything is successful. + - If a check fails (a red "X" ❌), click the "Details" link next to the failed + check to view the logs, identify the problem, and push a fix. + +### 3. Ongoing triage for pull requests: `PR Auditing and Label Sync` + +This workflow runs periodically to ensure all open PRs are correctly linked to +issues and have consistent labels. + +- **Workflow File**: `.github/workflows/gemini-scheduled-pr-triage.yml` +- **When it runs**: Every 15 minutes on all open pull requests. +- **What it does**: + - **Checks for a linked issue**: The bot scans your PR description for a + keyword that links it to an issue (for example, `Fixes #123`, + `Closes #456`). + - **Adds `status/need-issue`**: If no linked issue is found, the bot will add + the `status/need-issue` label to your PR. This is a clear signal that an + issue needs to be created and linked. + - **Synchronizes labels**: If an issue _is_ linked, the bot ensures the PR's + labels perfectly match the issue's labels. It will add any missing labels + and remove any that don't belong, and it will remove the `status/need-issue` + label if it was present. +- **What you should do**: + - **Always link your PR to an issue.** This is the most important step. Add a + line like `Resolves #` to your PR description. + - This will ensure your PR is correctly categorized and moves through the + review process smoothly. + +### 4. Ongoing triage for issues: `Scheduled Issue Triage` + +This is a fallback workflow to ensure that no issue gets missed by the triage +process. + +- **Workflow File**: `.github/workflows/gemini-scheduled-issue-triage.yml` +- **When it runs**: Every hour on all open issues. +- **What it does**: + - It actively seeks out issues that either have no labels at all or still have + the `status/need-triage` label. + - It then triggers the same powerful Gemini-based analysis as the initial + triage bot to apply the correct labels. +- **What you should do**: + - You typically don't need to do anything. This workflow is a safety net to + ensure every issue is eventually categorized, even if the initial triage + fails. + +### 5. Automatic unassignment of inactive contributors: `Unassign Inactive Issue Assignees` + +To keep the list of open `help wanted` issues accessible to all contributors, +this workflow automatically removes **external contributors** who have not +opened a linked pull request within **7 days** of being assigned. Maintainers, +org members, and repo collaborators with write access or above are always exempt +and will never be auto-unassigned. + +- **Workflow File**: `.github/workflows/unassign-inactive-assignees.yml` +- **When it runs**: Every day at 09:00 UTC, and can be triggered manually with + an optional `dry_run` mode. +- **What it does**: + 1. Finds every open issue labeled `help wanted` that has at least one + assignee. + 2. Identifies privileged users (team members, repo collaborators with write+ + access, maintainers) and skips them entirely. + 3. For each remaining (external) assignee it reads the issue's timeline to + determine: + - The exact date they were assigned (using `assigned` timeline events). + - Whether they have opened a PR that is already linked/cross-referenced to + the issue. + 4. Each cross-referenced PR is fetched to verify it is **ready for review**: + open and non-draft, or already merged. Draft PRs do not count. + 5. If an assignee has been assigned for **more than 7 days** and no qualifying + PR is found, they are automatically unassigned and a comment is posted + explaining the reason and how to re-claim the issue. + 6. Assignees who have a non-draft, open or merged PR linked to the issue are + **never** unassigned by this workflow. +- **What you should do**: + - **Open a real PR, not a draft**: Within 7 days of being assigned, open a PR + that is ready for review and include `Fixes #` in the + description. Draft PRs do not satisfy the requirement and will not prevent + auto-unassignment. + - **Re-assign if unassigned by mistake**: Comment `/assign` on the issue to + assign yourself again. + - **Unassign yourself** if you can no longer work on the issue by commenting + `/unassign`, so other contributors can pick it up right away. + +### 6. Automatically label PRs by size: `PR Size Labeler` + +To help maintainers estimate review effort and keep the PR history clean, this +workflow automatically tags every pull request with a size label representing +the total volume of line changes. + +- **Workflow File**: `.github/workflows/pr-size-labeler.yml` +- **When it runs**: Immediately after a pull request is created, synchronized + (new commits pushed), or reopened. It can also be triggered manually via + `workflow_dispatch` with a PR number. +- **What it does**: + - **Calculates total changes**: Summarizes additions and deletions across all + changed files in a single consolidated API request. + - **Applies standard size labels**: + - `size/XS`: < 10 lines changed + - `size/S`: 10-49 lines changed + - `size/M`: 50-249 lines changed + - `size/L`: 250-999 lines changed + - `size/XL`: >= 1000 lines changed + - **Updates size tag atomically**: Adds the new correct size label and removes + any obsolete size labels in one atomic step. + - **Updates/Posts PR size info comment**: Instead of spamming a new comment on + every commit push, it updates the existing size labeler status comment + inline to keep the PR conversation timeline perfectly neat and clean. +- **What you should do**: + - You do not need to take any actions. The workflow runs automatically and + updates the label and comment seamlessly as you push new updates. + +### 7. Release automation + +This workflow handles the process of packaging and publishing new versions of +Gemini CLI. + +- **Workflow File**: `.github/workflows/release-manual.yml` +- **When it runs**: On a daily schedule for "nightly" releases, and manually for + official patch/minor releases. +- **What it does**: + - Automatically builds the project, bumps the version numbers, and publishes + the packages to npm. + - Creates a corresponding release on GitHub with generated release notes. +- **What you should do**: + - As a contributor, you don't need to do anything for this process. You can be + confident that once your PR is merged into the `main` branch, your changes + will be included in the very next nightly release. + +We hope this detailed overview is helpful. If you have any questions about our +automation or processes, don't hesitate to ask! diff --git a/docs/local-development.md b/docs/local-development.md new file mode 100644 index 0000000000000000000000000000000000000000..e6f862044dc01da6b0b6b86007d5400c11208658 --- /dev/null +++ b/docs/local-development.md @@ -0,0 +1,182 @@ +# Local development guide + +This guide provides instructions for setting up and using local development +features for Gemini CLI. + +## Tracing + +Gemini CLI uses OpenTelemetry (OTel) to record traces that help you debug agent +behavior. Traces instrument key events like model calls, tool scheduler +operations, and tool calls. + +Traces provide deep visibility into agent behavior and help you debug complex +issues. They are captured automatically when you enable telemetry. + +### View traces + +You can view traces using Genkit Developer UI, Jaeger, or Google Cloud. + +#### Use Genkit + +Genkit provides a web-based UI for viewing traces and other telemetry data. + +1. **Start the Genkit telemetry server:** + + Run the following command to start the Genkit server: + + ```bash + npm run telemetry -- --target=genkit + ``` + + The script will output the URL for the Genkit Developer UI. For example: + `Genkit Developer UI: http://localhost:4000` + +2. **Run Gemini CLI:** + + In a separate terminal, run your Gemini CLI command: + + ```bash + gemini + ``` + +3. **View the traces:** + + Open the Genkit Developer UI URL in your browser and navigate to the + **Traces** tab to view the traces. + +#### Use Jaeger + +You can view traces in the Jaeger UI for local development. + +1. **Start the telemetry collector:** + + Run the following command in your terminal to download and start Jaeger and + an OTel collector: + + ```bash + npm run telemetry -- --target=local + ``` + + This command configures your workspace for local telemetry and provides a + link to the Jaeger UI (usually `http://localhost:16686`). + + - **Collector logs:** `~/.gemini/tmp//otel/collector.log` + +2. **Run Gemini CLI:** + + In a separate terminal, run your Gemini CLI command: + + ```bash + gemini + ``` + +3. **View the traces:** + + After running your command, open the Jaeger UI link in your browser to view + the traces. + +#### Use Google Cloud + +You can use an OpenTelemetry collector to forward telemetry data to Google Cloud +Trace for custom processing or routing. + + +> [!WARNING] +> Ensure you complete the +> [Google Cloud telemetry prerequisites](./cli/telemetry.md#prerequisites) +> (Project ID, authentication, IAM roles, and APIs) before using this method. + +1. **Configure `.gemini/settings.json`:** + + ```json + { + "telemetry": { + "enabled": true, + "target": "gcp", + "useCollector": true + } + } + ``` + +2. **Start the telemetry collector:** + + Run the following command to start a local OTel collector that forwards to + Google Cloud: + + ```bash + npm run telemetry -- --target=gcp + ``` + + The script outputs links to view traces, metrics, and logs in the Google + Cloud Console. + + - **Collector logs:** `~/.gemini/tmp//otel/collector-gcp.log` + +3. **Run Gemini CLI:** + + In a separate terminal, run your Gemini CLI command: + + ```bash + gemini + ``` + +4. **View logs, metrics, and traces:** + + After sending prompts, view your data in the Google Cloud Console. See the + [telemetry documentation](./cli/telemetry.md#view-google-cloud-telemetry) + for links to Logs, Metrics, and Trace explorers. + +For more detailed information on telemetry, see the +[telemetry documentation](./cli/telemetry.md). + +### Instrument code with traces + +You can add traces to your own code for more detailed instrumentation. + +Adding traces helps you debug and understand the flow of execution. Use the +`runInDevTraceSpan` function to wrap any section of code in a trace span. + +Here is a basic example: + +```typescript +import { runInDevTraceSpan } from '@google/gemini-cli-core'; +import { GeminiCliOperation } from '@google/gemini-cli-core/lib/telemetry/constants.js'; + +await runInDevTraceSpan( + { + operation: GeminiCliOperation.ToolCall, + attributes: { + [GEN_AI_AGENT_NAME]: 'gemini-cli', + }, + }, + async ({ metadata }) => { + // metadata allows you to record the input and output of the + // operation as well as other attributes. + metadata.input = { key: 'value' }; + // Set custom attributes. + metadata.attributes['custom.attribute'] = 'custom.value'; + + // Your code to be traced goes here. + try { + const output = await somethingRisky(); + metadata.output = output; + return output; + } catch (e) { + metadata.error = e; + throw e; + } + }, +); +``` + +In this example: + +- `operation`: The operation type of the span, represented by the + `GeminiCliOperation` enum. +- `metadata.input`: (Optional) An object containing the input data for the + traced operation. +- `metadata.output`: (Optional) An object containing the output data from the + traced operation. +- `metadata.attributes`: (Optional) A record of custom attributes to add to the + span. +- `metadata.error`: (Optional) An error object to record if the operation fails. diff --git a/docs/npm.md b/docs/npm.md new file mode 100644 index 0000000000000000000000000000000000000000..3ceab3c5e717d9c15123eccfd1682839339c53d1 --- /dev/null +++ b/docs/npm.md @@ -0,0 +1,62 @@ +# Package overview + +This monorepo contains two main packages: `@google/gemini-cli` and +`@google/gemini-cli-core`. + +## `@google/gemini-cli` + +This is the main package for Gemini CLI. It is responsible for the user +interface, command parsing, and all other user-facing functionality. + +When this package is published, it is bundled into a single executable file. +This bundle includes all of the package's dependencies, including +`@google/gemini-cli-core`. This means that whether a user installs the package +with `npm install -g @google/gemini-cli` or runs it directly with +`npx @google/gemini-cli`, they are using this single, self-contained executable. + +## `@google/gemini-cli-core` + +This package contains the core logic for interacting with the Gemini API. It is +responsible for making API requests, handling authentication, and managing the +local cache. + +This package is not bundled. When it is published, it is published as a standard +Node.js package with its own dependencies. This allows it to be used as a +standalone package in other projects, if needed. All transpiled js code in the +`dist` folder is included in the package. + +## NPM workspaces + +This project uses +[NPM Workspaces](https://docs.npmjs.com/cli/v10/using-npm/workspaces) to manage +the packages within this monorepo. This simplifies development by allowing us to +manage dependencies and run scripts across multiple packages from the root of +the project. + +### How it works + +The root `package.json` file defines the workspaces for this project: + +```json +{ + "workspaces": ["packages/*"] +} +``` + +This tells NPM that any folder inside the `packages` directory is a separate +package that should be managed as part of the workspace. + +### Benefits of workspaces + +- **Simplified dependency management**: Running `npm install` from the root of + the project will install all dependencies for all packages in the workspace + and link them together. This means you don't need to run `npm install` in each + package's directory. +- **Automatic linking**: Packages within the workspace can depend on each other. + When you run `npm install`, NPM will automatically create symlinks between the + packages. This means that when you make changes to one package, the changes + are immediately available to other packages that depend on it. +- **Simplified script execution**: You can run scripts in any package from the + root of the project using the `--workspace` flag. For example, to run the + `build` script in the `cli` package, you can run + `npm run build --workspace @google/gemini-cli`. diff --git a/docs/redirects.json b/docs/redirects.json new file mode 100644 index 0000000000000000000000000000000000000000..db2dae4333c22501a2f29b3fd4531ba585db9b83 --- /dev/null +++ b/docs/redirects.json @@ -0,0 +1,21 @@ +{ + "/docs/architecture": "/docs/cli/index", + "/docs/cli/commands": "/docs/reference/commands", + "/docs/cli": "/docs", + "/docs/cli/index": "/docs", + "/docs/cli/keyboard-shortcuts": "/docs/reference/keyboard-shortcuts", + "/docs/cli/uninstall": "/docs/resources/uninstall", + "/docs/core/concepts": "/docs", + "/docs/core/memport": "/docs/reference/memport", + "/docs/core/policy-engine": "/docs/reference/policy-engine", + "/docs/core/tools-api": "/docs/reference/tools", + "/docs/reference/tools-api": "/docs/reference/tools", + "/docs/faq": "/docs/resources/faq", + "/docs/get-started/configuration": "/docs/reference/configuration", + "/docs/get-started/configuration-v1": "/docs/reference/configuration", + "/docs/get-started/examples": "/docs/get-started/index", + "/docs/index": "/docs", + "/docs/quota-and-pricing": "/docs/resources/quota-and-pricing", + "/docs/tos-privacy": "/docs/resources/tos-privacy", + "/docs/troubleshooting": "/docs/resources/troubleshooting" +} diff --git a/docs/release-confidence.md b/docs/release-confidence.md new file mode 100644 index 0000000000000000000000000000000000000000..7b6bd06249049075c3ddc7f0a7026f9c77ec8431 --- /dev/null +++ b/docs/release-confidence.md @@ -0,0 +1,168 @@ +# Release confidence strategy + +This document outlines the strategy for gaining confidence in every release of +Gemini CLI. It serves as a checklist and quality gate for release manager to +ensure we are shipping a high-quality product. + +## The goal + +To answer the question, "Is this release _truly_ ready for our users?" with a +high degree of confidence, based on a holistic evaluation of automated signals, +manual verification, and data. + +## Level 1: Automated gates (must pass) + +These are the baseline requirements. If any of these fail, the release is a +no-go. + +### 1. CI/CD health + +All workflows in `.github/workflows/ci.yml` must pass on the `main` branch (for +nightly) or the release branch (for preview/stable). + +- **Platforms:** Tests must pass on **Linux and macOS**. + +- **Checks:** + - **Linting:** No linting errors (ESLint, Prettier, etc.). + - **Typechecking:** No TypeScript errors. + - **Unit Tests:** All unit tests in `packages/core` and `packages/cli` must + pass. + - **Build:** The project must build and bundle successfully. + +### 2. End-to-end (E2E) tests + +All workflows in `.github/workflows/chained_e2e.yml` must pass. + +- **Platforms:** **Linux, macOS and Windows**. +- **Sandboxing:** Tests must pass with both `sandbox:none` and `sandbox:docker` + on Linux. + +### 3. Post-deployment smoke tests + +After a release is published to npm, the `smoke-test.yml` workflow runs. This +must pass to confirm the package is installable and the binary is executable. + +- **Command:** `npx -y @google/gemini-cli@ --version` must return the + correct version without error. +- **Platform:** Currently runs on `ubuntu-latest`. + +## Level 2: Manual verification and dogfooding + +Automated tests cannot catch everything, especially UX issues. + +### 1. Dogfooding via `preview` tag + +The weekly release cadence promotes code from `main` -> `nightly` -> `preview` +-> `stable`. + +- **Requirement:** The `preview` release must be used by maintainers for at + least **one week** before being promoted to `stable`. +- **Action:** Maintainers should install the preview version locally: + ```bash + npm install -g @google/gemini-cli@preview + ``` +- **Goal:** To catch regressions and UX issues in day-to-day usage before they + reach the broad user base. + +### 2. Critical user journey (CUJ) checklist + +Before promoting a `preview` release to `stable`, a release manager must +manually run through this checklist. + +- **Setup:** + + - [ ] Uninstall any existing global version: + `npm uninstall -g @google/gemini-cli` + - [ ] Clear npx cache (optional but recommended): `npm cache clean --force` + - [ ] Install the preview version: `npm install -g @google/gemini-cli@preview` + - [ ] Verify version: `gemini --version` + +- **Authentication:** + + - [ ] In interactive mode run `/auth` and verify all sign in flows work: + - [ ] Sign in with Google + - [ ] API Key + - [ ] Vertex AI + +- **Basic prompting:** + + - [ ] Run `gemini "Tell me a joke"` and verify a sensible response. + - [ ] Run in interactive mode: `gemini`. Ask a follow-up question to test + context. + +- **Piped input:** + + - [ ] Run `echo "Summarize this" | gemini` and verify it processes stdin. + +- **Context management:** + + - [ ] In interactive mode, use `@file` to add a local file to context. Ask a + question about it. + +- **Settings:** + + - [ ] In interactive mode run `/settings` and make modifications + - [ ] Validate that setting is changed + +- **Function calling:** + - [ ] In interactive mode, ask gemini to "create a file named hello.md with + the content 'hello world'" and verify the file is created correctly. + +If any of these CUJs fail, the release is a no-go until a patch is applied to +the `preview` channel. + +### 3. Pre-Launch bug bash (tier 1 and 2 launches) + +For high-impact releases, an organized bug bash is required to ensure a higher +level of quality and to catch issues across a wider range of environments and +use cases. + +**Definition of tiers:** + +- **Tier 1:** Industry-Moving News πŸš€ +- **Tier 2:** Important News for Our Users πŸ“£ +- **Tier 3:** Relevant, but Not Life-Changing πŸ’‘ +- **Tier 4:** Bug Fixes βš’οΈ + +**Requirement:** + +A bug bash must be scheduled at least **72 hours in advance** of any Tier 1 or +Tier 2 launch. + +**Rule of thumb:** + +A bug bash should be considered for any release that involves: + +- A blog post +- Coordinated social media announcements +- Media relations or press outreach +- A "Turbo" launch event + +## Level 3: Telemetry and data review + +### Dashboard health + +- [ ] Go to `go/gemini-cli-dash`. +- [ ] Navigate to the "Tool Call" tab. +- [ ] Validate that there are no spikes in errors for the release you would like + to promote. + +### Model evaluation + +- [ ] Navigate to `go/gemini-cli-offline-evals-dash`. +- [ ] Make sure that the release you want to promote's recurring run is within + average eval runs. + +## The "go/no-go" decision + +Before triggering the `Release: Promote` workflow to move `preview` to `stable`: + +1. [ ] **Level 1:** CI and E2E workflows are green for the commit corresponding + to the current `preview` tag. +2. [ ] **Level 2:** The `preview` version has been out for one week, and the + CUJ checklist has been completed successfully by a release manager. No + blocking issues have been reported. +3. [ ] **Level 3:** Dashboard Health and Model Evaluation checks have been + completed and show no regressions. + +If all checks pass, proceed with the promotion. diff --git a/docs/releases.md b/docs/releases.md new file mode 100644 index 0000000000000000000000000000000000000000..70a9f069ce2f4fafb331d88f37e9e2f33985037a --- /dev/null +++ b/docs/releases.md @@ -0,0 +1,550 @@ +# Gemini CLI releases + + +> [!IMPORTANT] +> **Coordinate with the Release Manager:** The release manager is responsible for coordinating patches and releases. Please update them before performing any of the release actions described in this document. + +## `dev` vs `prod` environment + +Our release flows support both `dev` and `prod` environments. + +The `dev` environment pushes to a private GitHub-hosted NPM repository, with the +package names beginning with `@google-gemini/**` instead of `@google/**`. + +The `prod` environment pushes to the public global NPM registry via Wombat +Dressing Room, which is Google's system for managing NPM packages in the +`@google/**` namespace. The packages are all named `@google/**`. + +More information can be found about these systems in the +[NPM Package Overview](npm.md) + +### Package scopes + +| Package | `prod` (Wombat Dressing Room) | `dev` (GitHub Private NPM Repo) | +| ---------- | ----------------------------- | ----------------------------------------- | +| CLI | @google/gemini-cli | @google-gemini/gemini-cli | +| Core | @google/gemini-cli-core | @google-gemini/gemini-cli-core A2A Server | +| A2A Server | @google/gemini-cli-a2a-server | @google-gemini/gemini-cli-a2a-server | + +## Release cadence and tags + +We will follow https://semver.org/ as closely as possible but will call out when +or if we have to deviate from it. Our weekly releases will be minor version +increments and any bug or hotfixes between releases will go out as patch +versions on the most recent release. + +Each Tuesday ~20:00 UTC new Stable and Preview releases will be cut. The +promotion flow is: + +- Code is committed to main and pushed each night to nightly +- After no more than 1 week on main, code is promoted to the `preview` channel +- After 1 week the most recent `preview` channel is promoted to `stable` channel +- Patch fixes will be produced against both `preview` and `stable` as needed, + with the final 'patch' version number incrementing each time. + +### Preview + +These releases will not have been fully vetted and may contain regressions or +other outstanding issues. Help us test and install with `preview` tag. + +```bash +npm install -g @google/gemini-cli@preview +``` + +### Stable + +This will be the full promotion of last week's release + any bug fixes and +validations. Use `latest` tag. + +```bash +npm install -g @google/gemini-cli@latest +``` + +### Nightly + +- New releases will be published each day at UTC 00:00. This will be all changes + from the main branch as represented at time of release. It should be assumed + there are pending validations and issues. Use `nightly` tag. + +```bash +npm install -g @google/gemini-cli@nightly +``` + +## Weekly release promotion + +Each Tuesday, the on-call engineer will trigger the "Promote Release" workflow. +This single action automates the entire weekly release process: + +1. **Promotes preview to stable:** The workflow identifies the latest `preview` + release and promotes it to `stable`. This becomes the new `latest` version + on npm. +2. **Promotes nightly to preview:** The latest `nightly` release is then + promoted to become the new `preview` version. +3. **Prepares for next nightly:** A pull request is automatically created and + merged to bump the version in `main` in preparation for the next nightly + release. + +This process ensures a consistent and reliable release cadence with minimal +manual intervention. + +### Source of truth for versioning + +To ensure the highest reliability, the release promotion process uses the **NPM +registry as the single source of truth** for determining the current version of +each release channel (`stable`, `preview`, and `nightly`). + +1. **Fetch from NPM:** The workflow begins by querying NPM's `dist-tags` + (`latest`, `preview`, `nightly`) to get the exact version strings for the + packages currently available to users. +2. **Cross-check for integrity:** For each version retrieved from NPM, the + workflow performs a critical integrity check: + - It verifies that a corresponding **git tag** exists in the repository. + - It verifies that a corresponding **GitHub release** has been created. +3. **Halt on discrepancy:** If either the git tag or the GitHub Release is + missing for a version listed on NPM, the workflow will immediately fail. + This strict check prevents promotions from a broken or incomplete previous + release and alerts the on-call engineer to a release state inconsistency + that must be manually resolved. +4. **Calculate next version:** Only after these checks pass does the workflow + proceed to calculate the next semantic version based on the trusted version + numbers retrieved from NPM. + +This NPM-first approach, backed by integrity checks, makes the release process +highly robust and prevents the kinds of versioning discrepancies that can arise +from relying solely on git history or API outputs. + +## Manual releases + +For situations requiring a release outside of the regular nightly and weekly +promotion schedule, and NOT already covered by patching process, you can use the +`Release: Manual` workflow. This workflow provides a direct way to publish a +specific version from any branch, tag, or commit SHA. + +### How to create a manual release + +1. Navigate to the **Actions** tab of the repository. +2. Select the **Release: Manual** workflow from the list. +3. Click the **Run workflow** dropdown button. +4. Fill in the required inputs: + - **Version**: The exact version to release (for example, `v0.6.1`). This + must be a valid semantic version with a `v` prefix. + - **Ref**: The branch, tag, or full commit SHA to release from. + - **NPM Channel**: The npm channel to publish to. The options are `preview`, + `nightly`, `latest` (for stable releases), and `dev`. The default is + `dev`. + - **Dry Run**: Leave as `true` to run all steps without publishing, or set + to `false` to perform a live release. + - **Force Skip Tests**: Set to `true` to skip the test suite. This is not + recommended for production releases. + - **Skip GitHub Release**: Set to `true` to skip creating a GitHub release + and create an npm release only. + - **Environment**: Select the appropriate environment. The `dev` environment + is intended for testing. The `prod` environment is intended for production + releases. `prod` is the default and will require authorization from a + release administrator. +5. Click **Run workflow**. + +The workflow will then proceed to test (if not skipped), build, and publish the +release. If the workflow fails during a non-dry run, it will automatically +create a GitHub issue with the failure details. + +## Rollback/rollforward + +In the event that a release has a critical regression, you can quickly roll back +to a previous stable version or roll forward to a new patch by changing the npm +`dist-tag`. The `Release: Change Tags` workflow provides a safe and controlled +way to do this. + +This is the preferred method for both rollbacks and rollforwards, as it does not +require a full release cycle. + +### How to change a release tag + +1. Navigate to the **Actions** tab of the repository. +2. Select the **Release: Change Tags** workflow from the list. +3. Click the **Run workflow** dropdown button. +4. Fill in the required inputs: + - **Version**: The existing package version that you want to point the tag + to (for example, `0.5.0-preview-2`). This version **must** already be + published to the npm registry. + - **Channel**: The npm `dist-tag` to apply (for example, `preview`, + `stable`). + - **Dry Run**: Leave as `true` to log the action without making changes, or + set to `false` to perform the live tag change. + - **Environment**: Select the appropriate environment. The `dev` environment + is intended for testing. The `prod` environment is intended for production + releases. `prod` is the default and will require authorization from a + release administrator. +5. Click **Run workflow**. + +The workflow will then run `npm dist-tag add` for the appropriate `gemini-cli`, +`gemini-cli-core` and `gemini-cli-a2a-server` packages, pointing the specified +channel to the specified version. + +## Patching + +If a critical bug that is already fixed on `main` needs to be patched on a +`stable` or `preview` release, the process is now highly automated. + +### How to patch + +#### 1. Create the patch pull request + +There are two ways to create a patch pull request: + +**Option A: From a GitHub comment (recommended)** + +After a pull request containing the fix has been merged, a maintainer can add a +comment on that same PR with the following format: + +`/patch [channel]` + +- **channel** (optional): + - _no channel_ - patches both stable and preview channels (default, + recommended for most fixes) + - `both` - patches both stable and preview channels (same as default) + - `stable` - patches only the stable channel + - `preview` - patches only the preview channel + +Examples: + +- `/patch` (patches both stable and preview - default) +- `/patch both` (patches both stable and preview - explicit) +- `/patch stable` (patches only stable) +- `/patch preview` (patches only preview) + +The `Release: Patch from Comment` workflow will automatically find the merge +commit SHA and trigger the `Release: Patch (1) Create PR` workflow. If the PR is +not yet merged, it will post a comment indicating the failure. + +**Option B: Manually triggering the workflow** + +Navigate to the **Actions** tab and run the **Release: Patch (1) Create PR** +workflow. + +- **Commit**: The full SHA of the commit on `main` that you want to cherry-pick. +- **Channel**: The channel you want to patch (`stable` or `preview`). + +This workflow will automatically: + +1. Find the latest release tag for the channel. +2. Create a release branch from that tag if one doesn't exist (for example, + `release/v0.5.1-pr-12345`). +3. Create a new hotfix branch from the release branch. +4. Cherry-pick your specified commit into the hotfix branch. +5. Create a pull request from the hotfix branch back to the release branch. + +#### 2. Review and merge + +Review the automatically created pull request(s) to ensure the cherry-pick was +successful and the changes are correct. Once approved, merge the pull request. + + +> [!WARNING] +> The `release/*` branches are protected by branch protection +> rules. A pull request to one of these branches requires at least one review from +> a code owner before it can be merged. This ensures that no unauthorized code is +> released. + +#### 2.5. Adding multiple commits to a hotfix (advanced) + +If you need to include multiple fixes in a single patch release, you can add +additional commits to the hotfix branch after the initial patch PR has been +created: + +1. **Start with the primary fix**: Use `/patch` (or `/patch both`) on the most + important PR to create the initial hotfix branch and PR. + +2. **Checkout the hotfix branch locally**: + + ```bash + git fetch origin + git checkout hotfix/v0.5.1/stable/cherry-pick-abc1234 # Use the actual branch name from the PR + ``` + +3. **Cherry-pick additional commits**: + + ```bash + git cherry-pick + git cherry-pick + # Add as many commits as needed + ``` + +4. **Push the updated branch**: + + ```bash + git push origin hotfix/v0.5.1/stable/cherry-pick-abc1234 + ``` + +5. **Test and review**: The existing patch PR will automatically update with + your additional commits. Test thoroughly since you're now releasing multiple + changes together. + +6. **Update the PR description**: Consider updating the PR title and description + to reflect that it includes multiple fixes. + +This approach lets you group related fixes into a single patch release while +maintaining full control over what gets included and how conflicts are resolved. + +#### 3. Automatic release + +Upon merging the pull request, the `Release: Patch (2) Trigger` workflow is +automatically triggered. It will then start the `Release: Patch (3) Release` +workflow, which will: + +1. Build and test the patched code. +2. Publish the new patch version to npm. +3. Create a new GitHub release with the patch notes. + +This fully automated process ensures that patches are created and released +consistently and reliably. + +#### Troubleshooting: Older branch workflows + +**Issue**: If the patch trigger workflow fails with errors like "Resource not +accessible by integration" or references to non-existent workflow files (for +example, `patch-release.yml`), this indicates the hotfix branch contains an +outdated version of the workflow files. + +**Root cause**: When a PR is merged, GitHub Actions runs the workflow definition +from the **source branch** (the hotfix branch), not from the target branch (the +release branch). If the hotfix branch was created from an older release branch +that predates workflow improvements, it will use the old workflow logic. + +**Solutions**: + +**Option 1: Manual trigger (quick fix)** Manually trigger the updated workflow +from the branch with the latest workflow code: + +```bash +# For a preview channel patch with tests skipped +gh workflow run release-patch-2-trigger.yml --ref \ + --field ref="hotfix/v0.6.0-preview.2/preview/cherry-pick-abc1234" \ + --field workflow_ref= \ + --field dry_run=false \ + --field force_skip_tests=true + +# For a stable channel patch +gh workflow run release-patch-2-trigger.yml --ref \ + --field ref="hotfix/v0.5.1/stable/cherry-pick-abc1234" \ + --field workflow_ref= \ + --field dry_run=false \ + --field force_skip_tests=false + +# Example using main branch (most common case) +gh workflow run release-patch-2-trigger.yml --ref main \ + --field ref="hotfix/v0.6.0-preview.2/preview/cherry-pick-abc1234" \ + --field workflow_ref=main \ + --field dry_run=false \ + --field force_skip_tests=true +``` + +**Note**: Replace `` with the branch containing +the latest workflow improvements (usually `main`, but could be a feature branch +if testing updates). + +**Option 2: Update the hotfix branch** Merge the latest main branch into your +hotfix branch to get the updated workflows: + +```bash +git checkout hotfix/v0.6.0-preview.2/preview/cherry-pick-abc1234 +git merge main +git push +``` + +Then close and reopen the PR to retrigger the workflow with the updated version. + +**Option 3: Direct release trigger** Skip the trigger workflow entirely and +directly run the release workflow: + +```bash +# Replace channel and release_ref with appropriate values +gh workflow run release-patch-3-release.yml --ref main \ + --field type="preview" \ + --field dry_run=false \ + --field force_skip_tests=true \ + --field release_ref="release/v0.6.0-preview.2" +``` + +### Docker + +We also run a Google cloud build called +[release-docker.yml](../.gcp/release-docker.yml). Which publishes the sandbox +docker to match your release. This will also be moved to GH and combined with +the main release file once service account permissions are sorted out. + +## Release validation + +After pushing a new release smoke testing should be performed to ensure that the +packages are working as expected. This can be done by installing the packages +locally and running a set of tests to ensure that they are functioning +correctly. + +- `npx -y @google/gemini-cli@latest --version` to validate the push worked as + expected if you were not doing a rc or dev tag +- `npx -y @google/gemini-cli@ --version` to validate the tag pushed + appropriately +- _This is destructive locally_ + `npm uninstall @google/gemini-cli && npm uninstall -g @google/gemini-cli && npm cache clean --force && npm install @google/gemini-cli@` +- Smoke testing a basic run through of exercising a few llm commands and tools + is recommended to ensure that the packages are working as expected. We'll + codify this more in the future. + +## Local testing and validation: Changes to the packaging and publishing process + +If you need to test the release process without actually publishing to NPM or +creating a public GitHub release, you can trigger the workflow manually from the +GitHub UI. + +1. Go to the + [Actions tab](https://github.com/google-gemini/gemini-cli/actions/workflows/release-manual.yml) + of the repository. +2. Click on the "Run workflow" dropdown. +3. Leave the `dry_run` option checked (`true`). +4. Click the "Run workflow" button. + +This will run the entire release process but will skip the `npm publish` and +`gh release create` steps. You can inspect the workflow logs to ensure +everything is working as expected. + +It is crucial to test any changes to the packaging and publishing process +locally before committing them. This ensures that the packages will be published +correctly and that they will work as expected when installed by a user. + +To validate your changes, you can perform a dry run of the publishing process. +This will simulate the publishing process without actually publishing the +packages to the npm registry. + +```bash +npm_package_version=9.9.9 SANDBOX_IMAGE_REGISTRY="registry" SANDBOX_IMAGE_NAME="thename" npm run publish:npm --dry-run +``` + +This command will do the following: + +1. Build all the packages. +2. Run all the prepublish scripts. +3. Create the package tarballs that would be published to npm. +4. Print a summary of the packages that would be published. + +You can then inspect the generated tarballs to ensure that they contain the +correct files and that the `package.json` files have been updated correctly. The +tarballs will be created in the root of each package's directory (for example, +`packages/cli/google-gemini-cli-0.1.6.tgz`). + +By performing a dry run, you can be confident that your changes to the packaging +process are correct and that the packages will be published successfully. + +## Release deep dive + +The release process creates two distinct types of artifacts for different +distribution channels: standard packages for the NPM registry and a single, +self-contained executable for GitHub Releases. + +Here are the key stages: + +**Stage 1: Pre-release sanity checks and versioning** + +- **What happens:** Before any files are moved, the process ensures the project + is in a good state. This involves running tests, linting, and type-checking + (`npm run preflight`). The version number in the root `package.json` and + `packages/cli/package.json` is updated to the new release version. + +**Stage 2: Building the source code for NPM** + +- **What happens:** The TypeScript source code in `packages/core/src` and + `packages/cli/src` is compiled into standard JavaScript. +- **File movement:** + - `packages/core/src/**/*.ts` -> compiled to -> `packages/core/dist/` + - `packages/cli/src/**/*.ts` -> compiled to -> `packages/cli/dist/` +- **Why:** The TypeScript code written during development needs to be converted + into plain JavaScript that can be run by Node.js. The `core` package is built + first as the `cli` package depends on it. + +**Stage 3: Publishing standard packages to NPM** + +- **What happens:** The `npm publish` command is run for the + `@google/gemini-cli-core` and `@google/gemini-cli` packages. +- **Why:** This publishes them as standard Node.js packages. Users installing + via `npm install -g @google/gemini-cli` will download these packages, and + `npm` will handle installing the `@google/gemini-cli-core` dependency + automatically. The code in these packages is not bundled into a single file. + +**Stage 4: Assembling and creating the GitHub release asset** + +This stage happens _after_ the NPM publish and creates the single-file +executable that enables `npx` usage directly from the GitHub repository. + +1. **The JavaScript bundle is created:** + + - **What happens:** The built JavaScript from both `packages/core/dist` and + `packages/cli/dist`, along with all third-party JavaScript dependencies, + are bundled by `esbuild` into a single, executable JavaScript file (for + example, `gemini.js`). The `node-pty` library is excluded from this bundle + as it contains native binaries. + - **Why:** This creates a single, optimized file that contains all the + necessary application code. It simplifies execution for users who want to + run the CLI without a full `npm install`, as all dependencies (including + the `core` package) are included directly. + +2. **The `bundle` directory is assembled:** + + - **What happens:** A temporary `bundle` folder is created at the project + root. The single `gemini.js` executable is placed inside it, along with + other essential files. + - **File movement:** + - `gemini.js` (from esbuild) -> `bundle/gemini.js` + - `README.md` -> `bundle/README.md` + - `LICENSE` -> `bundle/LICENSE` + - `packages/cli/src/utils/*.sb` (sandbox profiles) -> `bundle/` + - **Why:** This creates a clean, self-contained directory with everything + needed to run the CLI and understand its license and usage. + +3. **The GitHub release is created:** + - **What happens:** The contents of the `bundle` directory, including the + `gemini.js` executable, are attached as assets to a new GitHub Release. + - **Why:** This makes the single-file version of the CLI available for + direct download and enables the + `npx https://github.com/google-gemini/gemini-cli` command, which downloads + and runs this specific bundled asset. + +**Summary of artifacts** + +- **NPM:** Publishes standard, un-bundled Node.js packages. The primary artifact + is the code in `packages/cli/dist`, which depends on + `@google/gemini-cli-core`. +- **GitHub release:** Publishes a single, bundled `gemini.js` file that contains + all dependencies, for easy execution via `npx`. + +This dual-artifact process ensures that both traditional `npm` users and those +who prefer the convenience of `npx` have an optimized experience. + +## Notifications + +Failing release workflows will automatically create an issue with the label +`release-failure`. + +A notification will be posted to the maintainer's chat channel when issues with +this type are created. + +### Modifying chat notifications + +Notifications use +[GitHub for Google Chat](https://workspace.google.com/marketplace/app/github_for_google_chat/536184076190). +To modify the notifications, use `/github-settings` within the chat space. + + +> [!WARNING] +> The following instructions describe a fragile workaround that depends on the +> internal structure of the chat application's UI. It is likely to break with +> future updates. + +The list of available labels is not currently populated correctly. If you want +to add a label that does not appear alphabetically in the first 30 labels in the +repo, you must use your browser's developer tools to manually modify the UI: + +1. Open your browser's developer tools (for example, Chrome DevTools). +2. In the `/github-settings` dialog, inspect the list of labels. +3. Locate one of the `
  • ` elements representing a label. +4. In the HTML, modify the `data-option-value` attribute of that `
  • ` element + to the desired label name (for example, `release-failure`). +5. Click on your modified label in the UI to select it, then save your settings. diff --git a/docs/sidebar.json b/docs/sidebar.json new file mode 100644 index 0000000000000000000000000000000000000000..bf82bc8220bfd04fb6c9acb86aeb08f8f7c33a1d --- /dev/null +++ b/docs/sidebar.json @@ -0,0 +1,298 @@ +[ + { + "label": "docs_tab", + "items": [ + { + "label": "Get started", + "items": [ + { "label": "Overview", "slug": "docs" }, + { "label": "Quickstart", "slug": "docs/get-started" }, + { "label": "Installation", "slug": "docs/get-started/installation" }, + { + "label": "Authentication", + "slug": "docs/get-started/authentication" + }, + { "label": "CLI cheatsheet", "slug": "docs/cli/cli-reference" }, + { + "label": "Gemini 3 on Gemini CLI", + "slug": "docs/get-started/gemini-3" + } + ] + }, + { + "label": "Use Gemini CLI", + "items": [ + { + "label": "File management", + "slug": "docs/cli/tutorials/file-management" + }, + { + "label": "Get started with Agent Skills", + "slug": "docs/cli/tutorials/skills-getting-started" + }, + { + "label": "Manage context and memory", + "slug": "docs/cli/tutorials/memory-management" + }, + { + "label": "Execute shell commands", + "slug": "docs/cli/tutorials/shell-commands" + }, + { + "label": "Manage sessions and history", + "slug": "docs/cli/tutorials/session-management" + }, + { + "label": "Plan tasks with todos", + "slug": "docs/cli/tutorials/task-planning" + }, + { + "label": "Use Plan Mode with model steering", + "badge": "πŸ”¬", + "slug": "docs/cli/tutorials/plan-mode-steering" + }, + { + "label": "Web search and fetch", + "slug": "docs/cli/tutorials/web-tools" + }, + { + "label": "Set up an MCP server", + "slug": "docs/cli/tutorials/mcp-setup" + }, + { "label": "Automate tasks", "slug": "docs/cli/tutorials/automation" } + ] + }, + { + "label": "Features", + "items": [ + { + "label": "Extensions", + "collapsed": true, + "items": [ + { + "label": "Overview", + "slug": "docs/extensions" + }, + { + "label": "User guide: Install and manage", + "link": "/docs/extensions/#manage-extensions" + }, + { + "label": "Developer guide: Build extensions", + "slug": "docs/extensions/writing-extensions" + }, + { + "label": "Developer guide: Best practices", + "slug": "docs/extensions/best-practices" + }, + { + "label": "Developer guide: Releasing", + "slug": "docs/extensions/releasing" + }, + { + "label": "Developer guide: Reference", + "slug": "docs/extensions/reference" + } + ] + }, + { + "label": "Agent Skills", + "collapsed": true, + "items": [ + { "label": "Overview", "slug": "docs/cli/skills" }, + { + "label": "Get started with Agent Skills", + "slug": "docs/cli/tutorials/skills-getting-started" + }, + { + "label": "Creating Agent Skills", + "slug": "docs/cli/creating-skills" + }, + { + "label": "Using Agent Skills", + "slug": "docs/cli/using-agent-skills" + }, + { + "label": "Developer guide: Best practices", + "slug": "docs/cli/skills-best-practices" + } + ] + }, + { + "label": "Auto Memory", + "badge": "πŸ”¬", + "slug": "docs/cli/auto-memory" + }, + { "label": "Checkpointing", "slug": "docs/cli/checkpointing" }, + { "label": "Headless mode", "slug": "docs/cli/headless" }, + { + "label": "Git worktrees", + "badge": "πŸ”¬", + "slug": "docs/cli/git-worktrees" + }, + { + "label": "Hooks", + "collapsed": true, + "items": [ + { "label": "Overview", "slug": "docs/hooks" }, + { "label": "Reference", "slug": "docs/hooks/reference" } + ] + }, + { + "label": "IDE integration", + "collapsed": true, + "items": [ + { "label": "Overview", "slug": "docs/ide-integration" }, + { + "label": "Developer guide: ACP mode", + "slug": "docs/cli/acp-mode" + } + ] + }, + { + "label": "MCP servers", + "collapsed": true, + "items": [ + { "label": "Overview", "slug": "docs/tools/mcp-server" }, + { "label": "Resource tools", "slug": "docs/tools/mcp-resources" } + ] + }, + { "label": "Model routing", "slug": "docs/cli/model-routing" }, + { "label": "Model selection", "slug": "docs/cli/model" }, + { + "label": "Model steering", + "badge": "πŸ”¬", + "slug": "docs/cli/model-steering" + }, + { + "label": "Notifications", + "badge": "πŸ”¬", + "slug": "docs/cli/notifications" + }, + { "label": "Plan mode", "slug": "docs/cli/plan-mode" }, + { + "label": "Subagents", + "slug": "docs/core/subagents" + }, + { + "label": "Remote subagents", + "slug": "docs/core/remote-agents" + }, + { "label": "Rewind", "slug": "docs/cli/rewind" }, + { "label": "Sandboxing", "slug": "docs/cli/sandbox" }, + { "label": "Settings", "slug": "docs/cli/settings" }, + { "label": "Telemetry", "slug": "docs/cli/telemetry" }, + { "label": "Token caching", "slug": "docs/cli/token-caching" } + ] + }, + { + "label": "Configuration", + "items": [ + { "label": "Custom commands", "slug": "docs/cli/custom-commands" }, + { + "label": "Enterprise configuration", + "slug": "docs/cli/enterprise" + }, + { + "label": "Ignore files (.geminiignore)", + "slug": "docs/cli/gemini-ignore" + }, + { + "label": "Model configuration", + "slug": "docs/cli/generation-settings" + }, + { + "label": "Project context (GEMINI.md)", + "slug": "docs/cli/gemini-md" + }, + { "label": "Settings", "slug": "docs/cli/settings" }, + { + "label": "System prompt override", + "slug": "docs/cli/system-prompt" + }, + { "label": "Themes", "slug": "docs/cli/themes" }, + { "label": "Trusted folders", "slug": "docs/cli/trusted-folders" } + ] + }, + { + "label": "Development", + "items": [ + { + "label": "Behavioral evaluations", + "slug": "docs/behavioral-evals" + }, + { "label": "Contribution guide", "slug": "docs/contributing" }, + { "label": "Integration testing", "slug": "docs/integration-tests" }, + { + "label": "Issue and PR automation", + "slug": "docs/issue-and-pr-automation" + }, + { "label": "Local development", "slug": "docs/local-development" }, + { "label": "NPM package structure", "slug": "docs/npm" } + ] + } + ] + }, + { + "label": "reference_tab", + "items": [ + { + "label": "Reference", + "items": [ + { "label": "Command reference", "slug": "docs/reference/commands" }, + { + "label": "Configuration reference", + "slug": "docs/reference/configuration" + }, + { + "label": "Keyboard shortcuts", + "slug": "docs/reference/keyboard-shortcuts" + }, + { + "label": "Memory import processor", + "slug": "docs/reference/memport" + }, + { "label": "Policy engine", "slug": "docs/reference/policy-engine" }, + { "label": "Tools reference", "slug": "docs/reference/tools" } + ] + } + ] + }, + { + "label": "resources_tab", + "items": [ + { + "label": "Resources", + "items": [ + { "label": "FAQ", "slug": "docs/resources/faq" }, + { + "label": "Quota and pricing", + "slug": "docs/resources/quota-and-pricing" + }, + { + "label": "Terms and privacy", + "slug": "docs/resources/tos-privacy" + }, + { + "label": "Troubleshooting", + "slug": "docs/resources/troubleshooting" + }, + { "label": "Uninstall", "slug": "docs/resources/uninstall" } + ] + } + ] + }, + { + "label": "releases_tab", + "items": [ + { + "label": "Releases", + "items": [ + { "label": "Release notes", "slug": "docs/changelogs/" }, + { "label": "Stable release", "slug": "docs/changelogs/latest" }, + { "label": "Preview release", "slug": "docs/changelogs/preview" } + ] + } + ] + } +] diff --git a/integration-tests/acp-env-auth.test.ts b/integration-tests/acp-env-auth.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..65f8adbf22a4290162ba30e0864ec2bb0cc895ca --- /dev/null +++ b/integration-tests/acp-env-auth.test.ts @@ -0,0 +1,163 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { spawn, ChildProcess } from 'node:child_process'; +import { join, resolve } from 'node:path'; +import { writeFileSync, mkdirSync } from 'node:fs'; +import { Writable, Readable } from 'node:stream'; +import { env } from 'node:process'; +import * as acp from '@agentclientprotocol/sdk'; + +const sandboxEnv = env['GEMINI_SANDBOX']; +const itMaybe = sandboxEnv && sandboxEnv !== 'false' ? it.skip : it; + +class MockClient implements acp.Client { + updates: acp.SessionNotification[] = []; + sessionUpdate = async (params: acp.SessionNotification) => { + this.updates.push(params); + }; + requestPermission = async (): Promise => { + throw new Error('unexpected'); + }; +} + +describe.skip('ACP Environment and Auth', () => { + let rig: TestRig; + let child: ChildProcess | undefined; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + child?.kill(); + child = undefined; + await rig.cleanup(); + }); + + itMaybe( + 'should load .env from project directory and use the provided API key', + async () => { + rig.setup('acp-env-loading'); + + // Create a project directory with a .env file containing a recognizable invalid key + const projectDir = resolve(join(rig.testDir!, 'project')); + mkdirSync(projectDir, { recursive: true }); + writeFileSync( + join(projectDir, '.env'), + 'GEMINI_API_KEY=test-key-from-env\n', + ); + + const bundlePath = join(import.meta.dirname, '..', 'bundle/gemini.js'); + + child = spawn('node', [bundlePath, '--acp'], { + cwd: rig.homeDir!, + stdio: ['pipe', 'pipe', 'inherit'], + env: { + ...process.env, + GEMINI_CLI_HOME: rig.homeDir!, + GEMINI_API_KEY: undefined, + VERBOSE: 'true', + }, + }); + + const input = Writable.toWeb(child.stdin!); + const output = Readable.toWeb( + child.stdout!, + ) as ReadableStream; + const testClient = new MockClient(); + const stream = acp.ndJsonStream(input, output); + const connection = new acp.ClientSideConnection(() => testClient, stream); + + await connection.initialize({ + protocolVersion: acp.PROTOCOL_VERSION, + clientCapabilities: { + fs: { readTextFile: false, writeTextFile: false }, + }, + }); + + // 1. newSession should succeed because it finds the key in .env + const { sessionId } = await connection.newSession({ + cwd: projectDir, + mcpServers: [], + }); + + expect(sessionId).toBeDefined(); + + // 2. prompt should fail because the key is invalid, + // but the error should come from the API, not the internal auth check. + await expect( + connection.prompt({ + sessionId, + prompt: [{ type: 'text', text: 'hello' }], + }), + ).rejects.toSatisfy((error: unknown) => { + const acpError = error as acp.RequestError; + const errorData = acpError.data as + | { error?: { message?: string } } + | undefined; + const message = String(errorData?.error?.message || acpError.message); + // It should NOT be our internal "Authentication required" message + expect(message).not.toContain('Authentication required'); + // It SHOULD be an API error mentioning the invalid key + expect(message).toContain('API key not valid'); + return true; + }); + + child.stdin!.end(); + }, + ); + + itMaybe( + 'should fail with authRequired when no API key is found', + async () => { + rig.setup('acp-auth-failure'); + + const bundlePath = join(import.meta.dirname, '..', 'bundle/gemini.js'); + + child = spawn('node', [bundlePath, '--acp'], { + cwd: rig.homeDir!, + stdio: ['pipe', 'pipe', 'inherit'], + env: { + ...process.env, + GEMINI_CLI_HOME: rig.homeDir!, + GEMINI_API_KEY: undefined, + VERBOSE: 'true', + }, + }); + + const input = Writable.toWeb(child.stdin!); + const output = Readable.toWeb( + child.stdout!, + ) as ReadableStream; + const testClient = new MockClient(); + const stream = acp.ndJsonStream(input, output); + const connection = new acp.ClientSideConnection(() => testClient, stream); + + await connection.initialize({ + protocolVersion: acp.PROTOCOL_VERSION, + clientCapabilities: { + fs: { readTextFile: false, writeTextFile: false }, + }, + }); + + await expect( + connection.newSession({ + cwd: resolve(rig.testDir!), + mcpServers: [], + }), + ).rejects.toMatchObject({ + message: expect.stringContaining( + 'Gemini API key is missing or not configured.', + ), + }); + + child.stdin!.end(); + }, + ); +}); diff --git a/integration-tests/acp-telemetry.test.ts b/integration-tests/acp-telemetry.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..487dac474db6dd7584c3cdf0ecd0b36eabf90cb3 --- /dev/null +++ b/integration-tests/acp-telemetry.test.ts @@ -0,0 +1,116 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { spawn, ChildProcess } from 'node:child_process'; +import { join } from 'node:path'; +import { readFileSync, existsSync } from 'node:fs'; +import { Writable, Readable } from 'node:stream'; +import { env } from 'node:process'; +import * as acp from '@agentclientprotocol/sdk'; + +// Skip in sandbox mode - test spawns CLI directly which behaves differently in containers +const sandboxEnv = env['GEMINI_SANDBOX']; +const itMaybe = sandboxEnv && sandboxEnv !== 'false' ? it.skip : it; + +// Reuse existing fake responses that return a simple "Hello" response +const SIMPLE_RESPONSE_PATH = 'hooks-system.session-startup.responses'; + +class SessionUpdateCollector implements acp.Client { + updates: acp.SessionNotification[] = []; + + sessionUpdate = async (params: acp.SessionNotification) => { + this.updates.push(params); + }; + + requestPermission = async (): Promise => { + throw new Error('unexpected'); + }; +} + +describe('ACP telemetry', () => { + let rig: TestRig; + let child: ChildProcess | undefined; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + child?.kill(); + child = undefined; + await rig.cleanup(); + }); + + itMaybe('should flush telemetry when connection closes', async () => { + rig.setup('acp-telemetry-flush', { + fakeResponsesPath: join(import.meta.dirname, SIMPLE_RESPONSE_PATH), + }); + + const telemetryPath = join(rig.homeDir!, 'telemetry.log'); + const bundlePath = join(import.meta.dirname, '..', 'bundle/gemini.js'); + + child = spawn( + 'node', + [ + bundlePath, + '--acp', + '--fake-responses', + join(rig.testDir!, 'fake-responses.json'), + ], + { + cwd: rig.testDir!, + stdio: ['pipe', 'pipe', 'inherit'], + env: { + ...process.env, + GEMINI_API_KEY: 'fake-key', + GEMINI_CLI_HOME: rig.homeDir!, + GEMINI_TELEMETRY_ENABLED: 'true', + GEMINI_TELEMETRY_TRACES_ENABLED: 'true', + GEMINI_TELEMETRY_TARGET: 'local', + GEMINI_TELEMETRY_OUTFILE: telemetryPath, + }, + }, + ); + + const input = Writable.toWeb(child.stdin!); + const output = Readable.toWeb(child.stdout!) as ReadableStream; + const testClient = new SessionUpdateCollector(); + const stream = acp.ndJsonStream(input, output); + const connection = new acp.ClientSideConnection(() => testClient, stream); + + await connection.initialize({ + protocolVersion: acp.PROTOCOL_VERSION, + clientCapabilities: { fs: { readTextFile: false, writeTextFile: false } }, + }); + + const { sessionId } = await connection.newSession({ + cwd: rig.testDir!, + mcpServers: [], + }); + + await connection.prompt({ + sessionId, + prompt: [{ type: 'text', text: 'Say hello' }], + }); + + expect(JSON.stringify(testClient.updates)).toContain('Hello'); + + // Close stdin to trigger telemetry flush via runExitCleanup() + child.stdin!.end(); + await new Promise((resolve) => { + child!.on('close', () => resolve()); + }); + child = undefined; + + // gen_ai.output.messages is the last OTEL log emitted (after prompt response) + expect(existsSync(telemetryPath)).toBe(true); + expect(readFileSync(telemetryPath, 'utf-8')).toContain( + 'gen_ai.output.messages', + ); + }); +}); diff --git a/integration-tests/api-resilience.responses b/integration-tests/api-resilience.responses new file mode 100644 index 0000000000000000000000000000000000000000..d0520047f7e648e7216e14a9948e7cf268e53c79 --- /dev/null +++ b/integration-tests/api-resilience.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Part 1. "}],"role":"model"},"index":0}]},{"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":10,"totalTokenCount":110}},{"candidates":[{"content":{"parts":[{"text":"Part 2."}],"role":"model"},"index":0,"finishReason":"STOP"}]}]} diff --git a/integration-tests/api-resilience.test.ts b/integration-tests/api-resilience.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..870adf701a3063a371770e4275857a43193e0044 --- /dev/null +++ b/integration-tests/api-resilience.test.ts @@ -0,0 +1,50 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +describe('API Resilience E2E', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it('should not crash when receiving metadata-only chunks in a stream', async () => { + await rig.setup('api-resilience-metadata-only', { + fakeResponsesPath: join( + dirname(fileURLToPath(import.meta.url)), + 'api-resilience.responses', + ), + settings: { + planSettings: { modelRouting: false }, + }, + }); + + // Run the CLI with a simple prompt. + // The fake responses will provide a stream with a metadata-only chunk in the middle. + // We use gemini-3-pro-preview to minimize internal service calls. + const result = await rig.run({ + args: ['hi', '--model', 'gemini-3-pro-preview'], + }); + + // Verify the output contains text from the normal chunks. + // If the CLI crashed on the metadata chunk, rig.run would throw. + expect(result).toContain('Part 1.'); + expect(result).toContain('Part 2.'); + + // Verify telemetry event for the prompt was still generated + const hasUserPromptEvent = await rig.waitForTelemetryEvent('user_prompt'); + expect(hasUserPromptEvent).toBe(true); + }); +}); diff --git a/integration-tests/browser-agent-localhost.multistep.responses b/integration-tests/browser-agent-localhost.multistep.responses new file mode 100644 index 0000000000000000000000000000000000000000..3ed786578f3c54547cc712885f5a009cf1e26c1c --- /dev/null +++ b/integration-tests/browser-agent-localhost.multistep.responses @@ -0,0 +1,9 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll go through the multi-step flow on the localhost server."},{"functionCall":{"name":"invoke_agent","args":{"agent_name":"browser_agent","prompt":"Navigate to http://127.0.0.1:18923/multi-step/step1.html, fill in 'testuser' as the username, click Next, then on step 2 select 'Option B' and click Finish. Report the final result page content."}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"navigate_page","args":{"url":"http://127.0.0.1:18923/multi-step/step1.html"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":20,"totalTokenCount":120}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"fill","args":{"selector":"#username","value":"testuser"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":150,"candidatesTokenCount":25,"totalTokenCount":175}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"click","args":{"selector":"#next-btn"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":180,"candidatesTokenCount":20,"totalTokenCount":200}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"take_snapshot","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":210,"candidatesTokenCount":15,"totalTokenCount":225}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"click","args":{"selector":"#finish-btn"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":240,"candidatesTokenCount":20,"totalTokenCount":260}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"take_snapshot","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":270,"candidatesTokenCount":15,"totalTokenCount":285}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"complete_task","args":{"result":{"success":true,"summary":"Completed all steps. Step 1: entered username 'testuser'. Step 2: selected default option. Final result page shows 'Multi-Step Complete' with 'βœ“ Complete' status badge."}}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":300,"candidatesTokenCount":40,"totalTokenCount":340}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I've completed the multi-step flow:\n\n1. **Step 1**: Entered 'testuser' as username and clicked Next\n2. **Step 2**: Confirmed selection and clicked Finish\n3. **Result**: Final page shows 'Multi-Step Complete' with a 'βœ“ Complete' status badge\n\nAll steps were successfully navigated."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":300,"candidatesTokenCount":60,"totalTokenCount":360}}]} diff --git a/integration-tests/browser-agent-localhost.navigate.responses b/integration-tests/browser-agent-localhost.navigate.responses new file mode 100644 index 0000000000000000000000000000000000000000..7c25e8294570fbeaf7d0a83685c09b433ab06f27 --- /dev/null +++ b/integration-tests/browser-agent-localhost.navigate.responses @@ -0,0 +1,5 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll navigate to the localhost page and read its content using the browser agent."},{"functionCall":{"name":"invoke_agent","args":{"agent_name":"browser_agent","prompt":"Navigate to http://127.0.0.1:18923/index.html and tell me the page title and list all links on the page"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":40,"totalTokenCount":140}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"navigate_page","args":{"url":"http://127.0.0.1:18923/index.html"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":20,"totalTokenCount":120}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"take_snapshot","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":150,"candidatesTokenCount":20,"totalTokenCount":170}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"complete_task","args":{"result":{"success":true,"summary":"Page title is 'Test Fixture - Home'. Found 3 links: Contact Form (/form.html), Multi-Step Flow (/multi-step/step1.html), Dynamic Content (/dynamic.html)."}}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":200,"candidatesTokenCount":40,"totalTokenCount":240}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The localhost test fixture page has:\n\n**Title**: Test Fixture - Home\n\n**Links**:\n1. Contact Form (form.html)\n2. Multi-Step Flow (multi-step/step1.html)\n3. Dynamic Content (dynamic.html)\n\nThe page also has a heading 'Test Fixture Home Page' and footer content."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":200,"candidatesTokenCount":60,"totalTokenCount":260}}]} diff --git a/integration-tests/browser-agent.cleanup.responses b/integration-tests/browser-agent.cleanup.responses new file mode 100644 index 0000000000000000000000000000000000000000..755341ef0f9e842dba2293580116c1de8fda92b4 --- /dev/null +++ b/integration-tests/browser-agent.cleanup.responses @@ -0,0 +1,5 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll open https://example.com and check the page title for you."},{"functionCall":{"name":"invoke_agent","args":{"agent_name":"browser_agent","prompt":"Open https://example.com and get the page title"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":35,"totalTokenCount":135}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"navigate_page","args":{"url":"https://example.com"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":20,"totalTokenCount":120}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"take_snapshot","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":150,"candidatesTokenCount":20,"totalTokenCount":170}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"complete_task","args":{"result":{"success":true,"summary":"The page title is 'Example Domain'."}}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":200,"candidatesTokenCount":30,"totalTokenCount":230}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I have opened the page and the title is 'Example Domain'. The browser session has been cleaned up successfully."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":200,"candidatesTokenCount":30,"totalTokenCount":230}}]} diff --git a/integration-tests/browser-agent.confirmation.responses b/integration-tests/browser-agent.confirmation.responses new file mode 100644 index 0000000000000000000000000000000000000000..4f645c6531ff27e46d53d43619d89ae458a69a27 --- /dev/null +++ b/integration-tests/browser-agent.confirmation.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"file_path":"test.txt","content":"hello"}}},{"text":"I've successfully written \"hello\" to test.txt. The file has been created with the specified content."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} diff --git a/integration-tests/browser-policy.responses b/integration-tests/browser-policy.responses new file mode 100644 index 0000000000000000000000000000000000000000..95b055d5c716ed2fb91697f1b3c75d01d0133e7b --- /dev/null +++ b/integration-tests/browser-policy.responses @@ -0,0 +1,5 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll help you with that."},{"functionCall":{"name":"invoke_agent","args":{"agent_name":"browser_agent","prompt":"Open https://example.com and check if there is a heading"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"new_page","args":{"url":"https://example.com"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"take_snapshot","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"complete_task","args":{"success":true,"summary":"SUCCESS_POLICY_TEST_COMPLETED"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":50,"totalTokenCount":150}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Task completed successfully. The page has the heading \"Example Domain\"."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":200,"candidatesTokenCount":50,"totalTokenCount":250}}]} diff --git a/integration-tests/browser-policy.test.ts b/integration-tests/browser-policy.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..d727ca2fc126f2381396e14116e8d3374b1f42c2 --- /dev/null +++ b/integration-tests/browser-policy.test.ts @@ -0,0 +1,240 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig, poll } from './test-helper.js'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { execSync } from 'node:child_process'; +import { existsSync, writeFileSync, readFileSync, mkdirSync } from 'node:fs'; +import { env } from 'node:process'; +import stripAnsi from 'strip-ansi'; + +// Browser agent Chrome DevTools MCP connection is flaky in Docker sandbox. +// See: https://github.com/google-gemini/gemini-cli/issues/24382 +const isDockerSandbox = env['GEMINI_SANDBOX'] === 'docker'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +const chromeAvailable = (() => { + try { + if (process.platform === 'darwin') { + execSync( + 'test -d "/Applications/Google Chrome.app" || test -d "/Applications/Chromium.app"', + { + stdio: 'ignore', + }, + ); + } else if (process.platform === 'linux') { + execSync( + 'which google-chrome || which chromium-browser || which chromium', + { stdio: 'ignore' }, + ); + } else if (process.platform === 'win32') { + const chromePaths = [ + 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', + 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe', + `${process.env['LOCALAPPDATA'] ?? ''}\\Google\\Chrome\\Application\\chrome.exe`, + ]; + const found = chromePaths.some((p) => existsSync(p)); + if (!found) { + execSync('where chrome || where chromium', { stdio: 'ignore' }); + } + } else { + return false; + } + return true; + } catch { + return false; + } +})(); + +describe.skipIf(!chromeAvailable)('browser-policy', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it.skipIf(isDockerSandbox)( + 'should skip confirmation when "Allow all server tools for this session" is chosen', + async () => { + rig.setup('browser-policy-skip-confirmation', { + fakeResponsesPath: join(__dirname, 'browser-policy.responses'), + settings: { + agents: { + overrides: { + browser_agent: { + enabled: true, + }, + }, + browser: { + headless: true, + sessionMode: 'isolated', + allowedDomains: ['example.com'], + }, + }, + }, + }); + + // Manually trust the folder to avoid the dialog and enable option 3 + const geminiDir = join(rig.homeDir!, '.gemini'); + mkdirSync(geminiDir, { recursive: true }); + + // Write to trustedFolders.json + const trustedFoldersPath = join(geminiDir, 'trustedFolders.json'); + const trustedFolders = { + [rig.testDir!]: 'TRUST_FOLDER', + }; + writeFileSync( + trustedFoldersPath, + JSON.stringify(trustedFolders, null, 2), + ); + + // Force confirmation for browser agent. + // NOTE: We don't force confirm browser tools here because "Allow all server tools" + // adds a rule with ALWAYS_ALLOW_PRIORITY (3.9x) which would be overshadowed by + // a rule in the user tier (4.x) like the one from this TOML. + // By removing the explicit mcp rule, the first MCP tool will still prompt + // due to default approvalMode = 'default', and then "Allow all" will correctly + // bypass subsequent tools. + const policyFile = join(rig.testDir!, 'force-confirm.toml'); + writeFileSync( + policyFile, + ` +[[rule]] +name = "Force confirm browser_agent" +toolName = "invoke_agent" +argsPattern = "\\"agent_name\\":\\\\s*\\"browser_agent\\"" +decision = "ask_user" +priority = 200 +`, + ); + + // Update settings.json in both project and home directories to point to the policy file + for (const baseDir of [rig.testDir!, rig.homeDir!]) { + const settingsPath = join(baseDir, '.gemini', 'settings.json'); + if (existsSync(settingsPath)) { + const settings = JSON.parse(readFileSync(settingsPath, 'utf-8')); + settings.policyPaths = [policyFile]; + // Ensure folder trust is enabled + settings.security = settings.security || {}; + settings.security.folderTrust = settings.security.folderTrust || {}; + settings.security.folderTrust.enabled = true; + writeFileSync(settingsPath, JSON.stringify(settings, null, 2)); + } + } + + const run = await rig.runInteractive({ + approvalMode: 'default', + env: { + GEMINI_CLI_INTEGRATION_TEST: 'true', + }, + }); + + await run.sendKeys( + 'Open https://example.com and check if there is a heading\r', + ); + await run.sendKeys('\r'); + + // Handle confirmations. + // 1. Initial browser_agent delegation (likely only 3 options, so use option 1: Allow once) + await poll( + () => stripAnsi(run.output).toLowerCase().includes('action required'), + 60000, + 1000, + ); + await run.sendKeys('1\r'); + await new Promise((r) => setTimeout(r, 2000)); + + // Handle privacy notice + await poll( + () => stripAnsi(run.output).toLowerCase().includes('privacy notice'), + 5000, + 100, + ); + await run.sendKeys('1\r'); + await new Promise((r) => setTimeout(r, 5000)); + + // new_page (MCP tool, should have 4 options, use option 3: Allow all server tools) + await poll( + () => { + const stripped = stripAnsi(run.output).toLowerCase(); + return ( + stripped.includes('new_page') && + stripped.includes('allow all server tools for this session') + ); + }, + 60000, + 1000, + ); + + // Select "Allow all server tools for this session" (option 3) + await run.sendKeys('3\r'); + + // Wait for the browser agent to finish (success or failure) + await poll( + () => { + const stripped = stripAnsi(run.output).toLowerCase(); + return ( + stripped.includes('completed successfully') || + stripped.includes('agent error') + ); + }, + 120000, + 1000, + ); + + const output = stripAnsi(run.output).toLowerCase(); + + expect(output).toContain('browser_agent'); + // The test validates that "Allow all server tools" skips subsequent + // tool confirmations β€” the browser agent may still fail due to + // Chrome/MCP issues in CI, which is acceptable for this policy test. + expect( + output.includes('completed successfully') || + output.includes('agent error'), + ).toBe(true); + }, + ); + + it('should show the visible warning when browser agent starts in existing session mode', async () => { + rig.setup('browser-session-warning', { + fakeResponsesPath: join(__dirname, 'browser-agent.cleanup.responses'), + settings: { + general: { + enableAutoUpdateNotification: false, + }, + agents: { + overrides: { + browser_agent: { + enabled: true, + }, + }, + browser: { + sessionMode: 'existing', + headless: true, + }, + }, + }, + }); + + const stdout = await rig.runCommand(['Open https://example.com'], { + env: { + GEMINI_API_KEY: 'fake-key', + GEMINI_TELEMETRY_DISABLED: 'true', + DEV: 'true', + }, + }); + + expect(stdout).toContain('saved logins will be visible'); + }); +}); diff --git a/integration-tests/checkpointing.test.ts b/integration-tests/checkpointing.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..72277f25dafc8415d31afae03932b419528f23d7 --- /dev/null +++ b/integration-tests/checkpointing.test.ts @@ -0,0 +1,155 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'node:fs/promises'; +import * as path from 'node:path'; +import * as os from 'node:os'; +import { GitService, Storage } from '@google/gemini-cli-core'; + +describe('Checkpointing Integration', () => { + let tmpDir: string; + let projectRoot: string; + let fakeHome: string; + let originalEnv: NodeJS.ProcessEnv; + + beforeEach(async () => { + tmpDir = await fs.mkdtemp( + path.join(os.tmpdir(), 'gemini-checkpoint-test-'), + ); + projectRoot = path.join(tmpDir, 'project'); + fakeHome = path.join(tmpDir, 'home'); + + await fs.mkdir(projectRoot, { recursive: true }); + await fs.mkdir(fakeHome, { recursive: true }); + + // Save original env + originalEnv = { ...process.env }; + + // Simulate environment with NO global gitconfig + process.env['HOME'] = fakeHome; + delete process.env['GIT_CONFIG_GLOBAL']; + delete process.env['GIT_CONFIG_SYSTEM']; + }); + + afterEach(async () => { + // Restore env + process.env = originalEnv; + + // Cleanup + try { + await fs.rm(tmpDir, { recursive: true, force: true }); + } catch (e) { + console.error('Failed to cleanup temp dir', e); + } + }); + + it('should successfully create and restore snapshots without global git config', async () => { + const storage = new Storage(projectRoot); + const gitService = new GitService(projectRoot, storage); + + // 1. Initialize + await gitService.initialize(); + + // Verify system config empty file creation + // We need to access getHistoryDir logic or replicate it. + // Since we don't have access to private getHistoryDir, we can infer it or just trust the functional test. + + // 2. Create initial state + await fs.writeFile(path.join(projectRoot, 'file1.txt'), 'version 1'); + await fs.writeFile(path.join(projectRoot, 'file2.txt'), 'permanent file'); + + // 3. Create Snapshot + const snapshotHash = await gitService.createFileSnapshot('Checkpoint 1'); + expect(snapshotHash).toBeDefined(); + + // 4. Modify files + await fs.writeFile( + path.join(projectRoot, 'file1.txt'), + 'version 2 (BAD CHANGE)', + ); + await fs.writeFile( + path.join(projectRoot, 'file3.txt'), + 'new file (SHOULD BE GONE)', + ); + await fs.rm(path.join(projectRoot, 'file2.txt')); + + // 5. Restore + await gitService.restoreProjectFromSnapshot(snapshotHash); + + // 6. Verify state + const file1Content = await fs.readFile( + path.join(projectRoot, 'file1.txt'), + 'utf-8', + ); + expect(file1Content).toBe('version 1'); + + const file2Exists = await fs + .stat(path.join(projectRoot, 'file2.txt')) + .then(() => true) + .catch(() => false); + expect(file2Exists).toBe(true); + const file2Content = await fs.readFile( + path.join(projectRoot, 'file2.txt'), + 'utf-8', + ); + expect(file2Content).toBe('permanent file'); + + const file3Exists = await fs + .stat(path.join(projectRoot, 'file3.txt')) + .then(() => true) + .catch(() => false); + expect(file3Exists).toBe(false); + }); + + it('should ignore user global git config and use isolated identity', async () => { + // 1. Create a fake global gitconfig with a specific user + const globalConfigPath = path.join(fakeHome, '.gitconfig'); + const globalConfigContent = `[user] + name = Global User + email = global@example.com +`; + await fs.writeFile(globalConfigPath, globalConfigContent); + + // Point HOME to fakeHome so git picks up this global config (if we didn't isolate it) + process.env['HOME'] = fakeHome; + // Ensure GIT_CONFIG_GLOBAL is NOT set for the process initially, + // so it would default to HOME/.gitconfig if GitService didn't override it. + delete process.env['GIT_CONFIG_GLOBAL']; + + const storage = new Storage(projectRoot); + const gitService = new GitService(projectRoot, storage); + + await gitService.initialize(); + + // 2. Create a file and snapshot + await fs.writeFile(path.join(projectRoot, 'test.txt'), 'content'); + await gitService.createFileSnapshot('Snapshot with global config present'); + + // 3. Verify the commit author in the shadow repo + const historyDir = storage.getHistoryDir(); + + const { execFileSync } = await import('node:child_process'); + + const logOutput = execFileSync( + 'git', + ['log', '-1', '--pretty=format:%an <%ae>'], + { + cwd: historyDir, + env: { + ...process.env, + GIT_DIR: path.join(historyDir, '.git'), + GIT_CONFIG_GLOBAL: path.join(historyDir, '.gitconfig'), + GIT_CONFIG_SYSTEM: path.join(historyDir, '.gitconfig_system_empty'), + }, + encoding: 'utf-8', + }, + ); + + expect(logOutput).toBe('Gemini CLI '); + expect(logOutput).not.toContain('Global User'); + }); +}); diff --git a/integration-tests/concurrency-limit.responses b/integration-tests/concurrency-limit.responses new file mode 100644 index 0000000000000000000000000000000000000000..e2bd5efe2aefb3735df57b65108f9e5cc1796416 --- /dev/null +++ b/integration-tests/concurrency-limit.responses @@ -0,0 +1,12 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/1"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/2"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/3"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/4"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/5"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/6"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/7"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/8"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/9"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/10"}}},{"functionCall":{"name":"web_fetch","args":{"prompt":"fetch https://example.com/11"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":100,"candidatesTokenCount":500,"totalTokenCount":600}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 1 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 2 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 3 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 4 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 5 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 6 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 7 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 8 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 9 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Page 10 content"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Some requests were rate limited: Rate limit exceeded for host. Please wait 60 seconds before trying again."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":1000,"candidatesTokenCount":50,"totalTokenCount":1050}}]} diff --git a/integration-tests/context-compress-interactive.compress-empty.responses b/integration-tests/context-compress-interactive.compress-empty.responses new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/integration-tests/context-compress-interactive.compress-failure.responses b/integration-tests/context-compress-interactive.compress-failure.responses new file mode 100644 index 0000000000000000000000000000000000000000..7ba10591a6251a62385433846efd1f6d25624741 --- /dev/null +++ b/integration-tests/context-compress-interactive.compress-failure.responses @@ -0,0 +1,3 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"thought":true,"text":"**Observing Initial Conditions**\n\nI'm currently focused on the initial context. I've taken note of the provided date, OS, and working directory. I'm also carefully examining the file structure presented within the current working directory. It's helping me understand the starting point for further analysis.\n\n\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12270,"totalTokenCount":12316,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12270}],"thoughtsTokenCount":46}},{"candidates":[{"content":{"parts":[{"thought":true,"text":"**Assessing User Intent**\n\nI'm now shifting my focus. I've successfully registered the provided data and file structure. My current task is to understand the user's ultimate goal, given the information provided. The \"Hello.\" command is straightforward, but I'm checking if there's an underlying objective.\n\n\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12270,"totalTokenCount":12341,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12270}],"thoughtsTokenCount":71}},{"candidates":[{"content":{"parts":[{"thoughtSignature":"CiQB0e2Kb3dRh+BYdbZvmulSN2Pwbc75DfQOT3H4EN0rn039hoMKfwHR7YpvvyqNKoxXAiCbYw3gbcTr/+pegUpgnsIrt8oQPMytFMjKSsMyshfygc21T2MkyuI6Q5I/fNCcHROWexdZnIeppVCDB2TarN4LGW4T9Yci6n/ynMMFT2xc2/vyHpkDgRM7avhMElnBhuxAY+e4TpxkZIncGWCEHP1TouoKpgEB0e2Kb8Xpwm0hiKhPt2ZLizpxjk+CVtcbnlgv69xo5VsuQ+iNyrVGBGRwNx+eTeNGdGpn6e73WOCZeP91FwOZe7URyL12IA6E6gYWqw0kXJR4hO4p6Lwv49E3+FRiG2C4OKDF8LF5XorYyCHSgBFT1/RUAVj81GDTx1xxtmYKN3xq8Ri+HsPbqU/FM/jtNZKkXXAtufw2Bmw8lJfmugENIv/TQI7xCo8BAdHtim8KgAXJfZ7ASfutVLKTylQeaslyB/SmcHJ0ZiNr5j8WP1prZdb6XnZZ1ZNbhjxUf/ymoxHKGvtTPBgLE9azMj8Lx/k0clhd2a+wNsiIqW9qCzlVah0tBMytpQUjIDtQe9Hj4LLUprF9PUe/xJkj000Z0ZzsgFm2ncdTWZTdkhCQDpyETVAxdE+oklwKJAHR7YpvUjSkD6KwY1gLrOsHKy0UNfn2lMbxjVetKNMVBRqsTg==","text":"Hello."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12270,"totalTokenCount":12341,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12270}],"thoughtsTokenCount":71}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"\n \n \n \n\n \n - OS: linux\n - Date: Friday, October 24, 2025\n \n\n \n - OBSERVED: The directory contains `telemetry.log` and a `.gemini/` directory.\n - OBSERVED: The `.gemini/` directory contains `settings.json` and `settings.json.orig`.\n \n\n \n - The user initiated the chat.\n \n\n \n 1. [TODO] Await the user's first instruction to formulate a plan.\n \n"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":983,"candidatesTokenCount":299,"totalTokenCount":1637,"promptTokensDetails":[{"modality":"TEXT","tokenCount":983}],"thoughtsTokenCount":355}}} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"\n \n \n \n\n \n - OS: linux\n - Date: Friday, October 24, 2025\n \n\n \n - OBSERVED: The directory contains `telemetry.log` and a `.gemini/` directory.\n - OBSERVED: The `.gemini/` directory contains `settings.json` and `settings.json.orig`.\n \n\n \n - The user initiated the chat.\n \n\n \n 1. [TODO] Await the user's first instruction to formulate a plan.\n \n"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":983,"candidatesTokenCount":299,"totalTokenCount":1637,"promptTokensDetails":[{"modality":"TEXT","tokenCount":983}],"thoughtsTokenCount":355}}} diff --git a/integration-tests/context-fidelity.test.ts b/integration-tests/context-fidelity.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..845b25b22f42c16caade16af9031a073f0e9de0e --- /dev/null +++ b/integration-tests/context-fidelity.test.ts @@ -0,0 +1,287 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import * as path from 'node:path'; +import * as fs from 'node:fs'; +import { FinishReason, GenerateContentResponse } from '@google/genai'; +import type { FakeResponse, HistoryTurn } from '@google/gemini-cli-core'; + +describe('Context Management Fidelity E2E', () => { + let rig: TestRig; + + function generateRandomString(length: number): string { + const characters = + 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'; + let result = ''; + for (let i = 0; i < length; i++) { + result += characters.charAt( + Math.floor(Math.random() * characters.length), + ); + } + return result; + } + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it( + 'should reproduce the exact context working buffer on resume', + { timeout: 300000 }, + async () => { + // Mock responses to trigger GC (summarization) + const snapshotResponse: FakeResponse = { + method: 'generateContent', + response: { + candidates: [ + { + content: { + parts: [ + { + text: JSON.stringify({ + new_facts: ['GC Triggered.'], + new_constraints: [], + new_tasks: [], + resolved_task_ids: [], + obsolete_fact_indices: [], + obsolete_constraint_indices: [], + chronological_summary: 'Snapshot created.', + }), + }, + ], + role: 'model', + }, + finishReason: FinishReason.STOP, + index: 0, + }, + ], + } as unknown as GenerateContentResponse, + }; + + const countTokensResponse: FakeResponse = { + method: 'countTokens', + response: { totalTokens: 1000 }, + }; + + const streamResponse = (text: string): FakeResponse => ({ + method: 'generateContentStream', + response: [ + { + candidates: [ + { + content: { parts: [{ text }], role: 'model' }, + finishReason: FinishReason.STOP, + index: 0, + }, + ], + }, + ] as unknown as GenerateContentResponse[], + }); + + const setupResponses = (fileName: string, mocks: FakeResponse[]) => { + const filePath = path.join(rig.testDir!, fileName); + fs.writeFileSync( + filePath, + mocks.map((m) => JSON.stringify(m)).join('\n'), + ); + return filePath; + }; + + await rig.setup('context-fidelity', { + settings: { + experimental: { + stressTestProfile: true, // Lowers thresholds to trigger GC easily + }, + }, + }); + + const traceDir = path.join(rig.testDir!, 'traces'); + fs.mkdirSync(traceDir, { recursive: true }); + const traceLog = path.join(traceDir, 'trace.log'); + + // Ignore trace and response files to keep environment context clean and stable + fs.writeFileSync( + path.join(rig.testDir!, '.geminiignore'), + 'traces/\nresp*.json\ndebug.log\n', + ); + + const commonEnv = { + GEMINI_API_KEY: 'mock-key', + GEMINI_CONTEXT_TRACE_DIR: traceDir, + GEMINI_CONTEXT_TRACE_ENABLED: 'true', + GEMINI_DEBUG_LOG_FILE: path.join(rig.testDir!, 'debug.log'), + }; + + const runMocks: FakeResponse[] = [ + streamResponse('Ack 1'), + streamResponse('Ack 2'), + streamResponse('Ack 3'), + streamResponse('Ack 4'), + streamResponse('Ack 5'), + streamResponse('Ack 6'), + streamResponse('Ack 7'), + streamResponse('Ack 8'), + streamResponse('Ack 9'), + streamResponse('Ack 10'), + streamResponse('Ack 11'), + streamResponse('Ack 12'), + ]; + for (let i = 0; i < 50; i++) { + runMocks.push(snapshotResponse); + runMocks.push(countTokensResponse); + } + + // Turns 1-10: Build up history + for (let i = 1; i <= 10; i++) { + await rig.run({ + args: [ + '--debug', + i === 1 ? '' : '--resume', + i === 1 ? '' : 'latest', + '--fake-responses-non-strict', + setupResponses(`resp_init_${i}.json`, runMocks), + ].filter(Boolean), + stdin: `Turn ${i}: ` + generateRandomString(900), + env: commonEnv, + }); + } + + // Turn 11: Penultimate turn + await rig.run({ + args: [ + '--debug', + '--resume', + 'latest', + '--fake-responses-non-strict', + setupResponses('resp2.json', runMocks), + ], + stdin: 'Turn 11: ' + generateRandomString(900), + env: commonEnv, + }); + + // Turn 12: Breach threshold and force GC + await rig.run({ + args: [ + '--debug', + '--resume', + 'latest', + '--fake-responses-non-strict', + setupResponses('resp3.json', runMocks), + ], + stdin: 'Turn 12: ' + generateRandomString(900), + env: commonEnv, + }); + + // Extract the rendered context asset from the log + const getRenderedContext = (logContent: string): HistoryTurn[] | null => { + const lines = logContent.split('\n'); + const renderLines = lines.filter( + (l) => + l.includes('[Render] Render Sanitized Context for LLM') || + l.includes('[Render] Render Context for LLM'), + ); + if (renderLines.length === 0) return null; + + const lastRender = renderLines[renderLines.length - 1]; + const detailsMatch = lastRender.match(/\| Details: (.*)$/); + if (!detailsMatch) return null; + + const details = JSON.parse(detailsMatch[1]); + const assetInfo = + details.renderedContextSanitized || details.renderedContext; + if (assetInfo && assetInfo.$asset) { + const assetPath = path.join(traceDir, 'assets', assetInfo.$asset); + return JSON.parse(fs.readFileSync(assetPath, 'utf-8')); + } + return assetInfo; + }; + + const log1 = fs.readFileSync(traceLog, 'utf-8'); + const contextBeforeExit = getRenderedContext(log1); + expect(contextBeforeExit).toBeDefined(); + console.log( + 'Context Before Exit (First 2 turns):', + JSON.stringify(contextBeforeExit!.slice(0, 2), null, 2), + ); + + // Turn 4: Resume and run a small command + await rig.run({ + args: [ + '--debug', + '--resume', + 'latest', + '--fake-responses-non-strict', + setupResponses('resp4.json', runMocks), + 'continue', + ], + env: commonEnv, + }); + + const log2 = fs.readFileSync(traceLog, 'utf-8'); + const contextAfterResume = getRenderedContext(log2); + expect(contextAfterResume).toBeDefined(); + console.log( + 'Context After Resume (First 2 turns):', + JSON.stringify(contextAfterResume!.slice(0, 2), null, 2), + ); + + expect(contextAfterResume!.length).toBeGreaterThanOrEqual( + contextBeforeExit!.length, + ); + + // The environment context is intentionally refreshed on resume to reflect + // the current state of the workspace (e.g. new files, current date). + // We allow its content to differ but ensure it's still an environment context. + const isEnvContext = (turn: HistoryTurn) => + turn.content.parts?.some((p) => p.text?.includes('')); + + for (let i = 0; i < contextBeforeExit!.length; i++) { + expect(contextAfterResume![i].id).toBe(contextBeforeExit![i].id); + + const turnBefore = contextBeforeExit![i]; + const turnAfter = contextAfterResume![i]; + + if (isEnvContext(turnBefore)) { + expect(isEnvContext(turnAfter)).toBe(true); + continue; + } + + expect(turnAfter.content).toEqual(turnBefore.content); + } + + // Most importantly, synthetic IDs (like summaries) must be stable. + const syntheticTurns = contextBeforeExit!.filter( + (t: HistoryTurn) => + t.content.parts?.some((p) => p.text?.includes('active_tasks')) || + (t.id && t.id.length === 32), + ); + expect(syntheticTurns.length).toBeGreaterThan(0); + + const syntheticTurnsAfter = contextAfterResume!.filter( + (t: HistoryTurn) => + t.content.parts?.some((p) => p.text?.includes('active_tasks')) || + (t.id && t.id.length === 32), + ); + expect(syntheticTurnsAfter.length).toBeGreaterThanOrEqual( + syntheticTurns.length, + ); + + // Check if the first synthetic turn is identical (with relaxation for environment context) + expect(syntheticTurnsAfter[0].id).toBe(syntheticTurns[0].id); + if (isEnvContext(syntheticTurns[0])) { + expect(isEnvContext(syntheticTurnsAfter[0])).toBe(true); + } else { + expect(syntheticTurnsAfter[0].content).toEqual( + syntheticTurns[0].content, + ); + } + }, + ); +}); diff --git a/integration-tests/extensions-install.test.ts b/integration-tests/extensions-install.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..e9f1cdbf49ea1ccb9df6be5584c0af07477d7d5d --- /dev/null +++ b/integration-tests/extensions-install.test.ts @@ -0,0 +1,62 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, expect, it, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; + +const extension = `{ + "name": "test-extension-install", + "version": "0.0.1" +}`; + +const extensionUpdate = `{ + "name": "test-extension-install", + "version": "0.0.2" +}`; + +describe('extension install', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('installs a local extension, verifies a command, and updates it', async () => { + rig.setup('extension install test'); + const testServerPath = join(rig.testDir!, 'gemini-extension.json'); + writeFileSync(testServerPath, extension); + try { + const result = await rig.runCommand( + ['--debug', 'extensions', 'install', `${rig.testDir!}`], + { stdin: 'y\n' }, + ); + expect(result).toContain('test-extension-install'); + + const listResult = await rig.runCommand([ + '--debug', + 'extensions', + 'list', + ]); + expect(listResult).toContain('test-extension-install'); + writeFileSync(testServerPath, extensionUpdate); + const updateResult = await rig.runCommand( + ['--debug', 'extensions', 'update', `test-extension-install`], + { stdin: 'y\n' }, + ); + expect(updateResult).toContain('0.0.2'); + } finally { + await rig.runCommand([ + 'extensions', + 'uninstall', + 'test-extension-install', + ]); + } + }); +}); diff --git a/integration-tests/extensions-reload.test.ts b/integration-tests/extensions-reload.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..4a1250fd00fe9180c37b32d4ca68aeeda845d883 --- /dev/null +++ b/integration-tests/extensions-reload.test.ts @@ -0,0 +1,151 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { expect, it, describe, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { TestMcpServer } from './test-mcp-server.js'; +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { safeJsonStringify } from '@google/gemini-cli-core/src/utils/safeJsonStringify.js'; + +import stripAnsi from 'strip-ansi'; + +describe('extension reloading', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + // always fails + // TODO(#14527): Re-enable this once fixed + it.skip('installs a local extension, updates it, checks it was reloaded properly', async () => { + const serverA = new TestMcpServer(); + const portA = await serverA.start({ + hello: () => ({ content: [{ type: 'text', text: 'world' }] }), + }); + const extension = { + name: 'test-extension', + version: '0.0.1', + mcpServers: { + 'test-server': { + httpUrl: `http://localhost:${portA}/mcp`, + }, + }, + }; + + rig.setup('extension reload test', { + settings: { + experimental: { extensionReloading: true }, + }, + }); + const testServerPath = join(rig.testDir!, 'gemini-extension.json'); + writeFileSync(testServerPath, safeJsonStringify(extension, 2)); + // defensive cleanup from previous tests. + try { + await rig.runCommand(['extensions', 'uninstall', 'test-extension']); + } catch { + /* empty */ + } + + const result = await rig.runCommand( + ['--debug', 'extensions', 'install', `${rig.testDir!}`], + { stdin: 'y\n' }, + ); + expect(result).toContain('test-extension'); + + // Now create the update, but its not installed yet + const serverB = new TestMcpServer(); + const portB = await serverB.start({ + goodbye: () => ({ content: [{ type: 'text', text: 'world' }] }), + }); + extension.version = '0.0.2'; + extension.mcpServers['test-server'].httpUrl = + `http://localhost:${portB}/mcp`; + writeFileSync(testServerPath, safeJsonStringify(extension, 2)); + + // Start the CLI. + const run = await rig.runInteractive({ args: '--debug' }); + await run.expectText('You have 1 extension with an update available'); + // See the outdated extension + await run.sendText('/extensions list'); + await run.type('\r'); + await run.expectText('test-extension (v0.0.1) - active (update available)'); + // Wait for the UI to settle and retry the command until we see the update + await new Promise((resolve) => setTimeout(resolve, 1000)); + + // Poll for the updated list + await rig.pollCommand( + async () => { + await run.sendText('/mcp list'); + await run.type('\r'); + }, + () => { + const output = stripAnsi(run.output); + return ( + output.includes( + 'test-server (from test-extension) - Ready (1 tool)', + ) && output.includes('- mcp_test-server_hello') + ); + }, + 30000, // 30s timeout + ); + + // Update the extension, expect the list to update, and mcp servers as well. + await run.sendKeys('\u0015/extensions update test-extension'); + await run.expectText('/extensions update test-extension'); + await run.type('\r'); + await new Promise((resolve) => setTimeout(resolve, 500)); + await run.type('\r'); + await run.expectText( + ` * test-server (remote): http://localhost:${portB}/mcp`, + ); + await run.type('\r'); // consent + await run.expectText( + 'Extension "test-extension" successfully updated: 0.0.1 β†’ 0.0.2', + ); + + // Poll for the updated extension version + await rig.pollCommand( + async () => { + await run.sendText('/extensions list'); + await run.type('\r'); + }, + () => + stripAnsi(run.output).includes( + 'test-extension (v0.0.2) - active (updated)', + ), + 30000, + ); + + // Poll for the updated mcp tool + await rig.pollCommand( + async () => { + await run.sendText('/mcp list'); + await run.type('\r'); + }, + () => { + const output = stripAnsi(run.output); + return ( + output.includes( + 'test-server (from test-extension) - Ready (1 tool)', + ) && output.includes('- mcp_test-server_goodbye') + ); + }, + 30000, + ); + + await run.sendText('/quit'); + await run.type('\r'); + + // Clean things up. + await serverA.stop(); + await serverB.stop(); + await rig.runCommand(['extensions', 'uninstall', 'test-extension']); + }); +}); diff --git a/integration-tests/file-system-interactive.test.ts b/integration-tests/file-system-interactive.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..8d90b8a6779cc33d65eaf733425409c357f0b127 --- /dev/null +++ b/integration-tests/file-system-interactive.test.ts @@ -0,0 +1,67 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { expect, describe, it, beforeEach, afterEach } from 'vitest'; +import { TestRig, skipFlaky } from './test-helper.js'; + +describe.skipIf(skipFlaky)('Interactive file system', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it('should perform a read-then-write sequence', async () => { + const fileName = 'version.txt'; + await rig.setup('interactive-read-then-write', { + settings: { + security: { + auth: { + selectedType: 'gemini-api-key', + }, + disableYoloMode: false, + }, + }, + }); + rig.createFile(fileName, '1.0.0'); + + const run = await rig.runInteractive({ + env: { + GEMINI_CLI_TRUST_WORKSPACE: 'true', + }, + }); + + // Step 1: Read the file + const readPrompt = `Read the version from ${fileName} using the read_file tool`; + await run.type(readPrompt); + await run.type('\r'); + + const readCall = await rig.waitForToolCall('read_file', 30000); + expect(readCall, 'Expected to find a read_file tool call').toBe(true); + + // Wait for the CLI to finish outputting the response and show the prompt again + await run.expectText('Type your message', 30000); + + // Step 2: Write the file + const writePrompt = `now change the version to 1.0.1 in ${fileName} using the write_file tool`; + await run.type(writePrompt); + await run.type('\r'); + + // Check tool calls made with right args + await rig.expectToolCallSuccess( + ['write_file', 'replace'], + 30000, + (args) => args.includes('1.0.1') && args.includes(fileName), + ); + + // Wait for telemetry to flush and file system to sync, especially in sandboxed environments + await rig.waitForTelemetryReady(); + }, 120000); +}); diff --git a/integration-tests/flicker-detector.max-height.responses b/integration-tests/flicker-detector.max-height.responses new file mode 100644 index 0000000000000000000000000000000000000000..b905b3d866c4cf13bd3fac7a54187b9ef81f2186 --- /dev/null +++ b/integration-tests/flicker-detector.max-height.responses @@ -0,0 +1,3 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"{\n \"reasoning\": \"The user is asking for a simple piece of information ('a fun fact'). This is a direct, bounded request with low operational complexity and does not require strategic planning, extensive investigation, or debugging.\",\n \"model_choice\": \"flash\"\n}"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":1173,"candidatesTokenCount":59,"totalTokenCount":1344,"promptTokensDetails":[{"modality":"TEXT","tokenCount":1173}],"thoughtsTokenCount":112}}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"thought":true,"text":"**Locating a fun fact**\n\nI'm now searching for a fun fact using the web search tool, focusing on finding something engaging and potentially surprising. The goal is to provide a brief, interesting piece of information.\n\n\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12226,"totalTokenCount":12255,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12226}],"thoughtsTokenCount":29}},{"candidates":[{"content":{"parts":[{"thoughtSignature":"CikB0e2Kb1vYSbIdmBfclWY7z4mOZgPxUGi3CtNXYYV9CSmG+SpVXZZkmQpZAdHtim9HVruyrUZZcHKDvIfn3j6/zLMgepC4Pqd79pG641PkPJnnCqEfVFRxmE2NX3Tj2lwRhtuIYT9Cc3CfvWGjbuuvwzynMCApxpIvxdXac/fXJYeRHTsKQQHR7Ypv6eOvWUFUTRGm1x29v8ZnGjtudG31H/Dgc65Y47c594ZJfX9RqJJil0I52Bxsm8UQ74rbARqwT7zYEbNO","functionCall":{"name":"google_web_search","args":{"query":"fun fact"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12226,"candidatesTokenCount":17,"totalTokenCount":12272,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12226}],"thoughtsTokenCount":29}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"Here's a fun fact: A day on Venus is longer than a year on Venus. It takes approximately 243 Earth days for Venus to rotate once on its axis, while its orbit around the Sun is about 225 Earth days."}],"role":"model"},"finishReason":"STOP","groundingMetadata":{"searchEntryPoint":{"renderedContent":"\n
    \n
    \n \n \n \n \n \n \n \n \n \n \n \n \n \n
    \n
    \n
    \n fun fact\n
    \n
    \n"},"groundingChunks":[{"web":{"uri":"https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQF-NBVWZeEqhT2BixBuiSaCHeF50iewha2f3M2FfpiNsStuPxhc3sLEzXLR7IFsBbzUBO2kbUmm-usnToWabMSvOIT4ZnTXedj5ZkpwFlYyuadyuBhLNKKJQtOGgg9JTNiwvKxBWt2beHYUjelTJXfVPb0Iy8SVJTahtA3GDA==","title":"hellosubs.co"}}],"groundingSupports":[{"segment":{"startIndex":66,"endIndex":197,"text":"It takes approximately 243 Earth days for Venus to rotate once on its axis, while its orbit around the Sun is about 225 Earth days."},"groundingChunkIndices":[0]}],"webSearchQueries":["fun fact"]},"index":0}],"usageMetadata":{"promptTokenCount":8186,"candidatesTokenCount":65,"totalTokenCount":16468,"cachedContentTokenCount":5360,"promptTokensDetails":[{"modality":"TEXT","tokenCount":8186}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":5360}],"toolUsePromptTokenCount":8207,"toolUsePromptTokensDetails":[{"modality":"TEXT","tokenCount":8207}],"thoughtsTokenCount":10}}} diff --git a/integration-tests/globalSetup.ts b/integration-tests/globalSetup.ts new file mode 100644 index 0000000000000000000000000000000000000000..b05d0dd8d1fcbb56abdd3c11a76e14c4b32be94c --- /dev/null +++ b/integration-tests/globalSetup.ts @@ -0,0 +1,155 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +// Unset NO_COLOR environment variable to ensure consistent theme behavior between local and CI test runs +if (process.env['NO_COLOR'] !== undefined) { + delete process.env['NO_COLOR']; +} + +import { mkdir, readdir, rm, readFile } from 'node:fs/promises'; +import { join, dirname, extname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { resolveRipgrepPath } from '../packages/core/src/tools/ripGrep.js'; +import { disableMouseTracking } from '@google/gemini-cli-core'; +import { isolateTestEnv } from '../packages/test-utils/src/env-setup.js'; +import { createServer, type Server } from 'node:http'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const rootDir = join(__dirname, '..'); +const integrationTestsDir = join(rootDir, '.integration-tests'); +let runDir = ''; // Make runDir accessible in teardown +let fixtureServer: Server | undefined; + +const FIXTURE_PORT = 18923; +const FIXTURE_DIR = join(__dirname, 'test-fixtures'); + +const MIME_TYPES: Record = { + '.html': 'text/html', + '.css': 'text/css', + '.js': 'application/javascript', + '.json': 'application/json', + '.png': 'image/png', + '.jpg': 'image/jpeg', + '.svg': 'image/svg+xml', +}; + +async function startFixtureServer(): Promise { + return new Promise((resolve, reject) => { + const server = createServer(async (req, res) => { + const urlPath = req.url?.split('?')[0] || '/'; + const relativePath = urlPath === '/' ? 'index.html' : urlPath; + const filePath = join(FIXTURE_DIR, relativePath); + + if (!filePath.startsWith(FIXTURE_DIR)) { + res.writeHead(403, { 'Content-Type': 'text/html' }); + res.end('

    403 Forbidden

    '); + return; + } + + try { + const content = await readFile(filePath); + const ext = extname(filePath); + res.writeHead(200, { + 'Content-Type': MIME_TYPES[ext] || 'application/octet-stream', + }); + res.end(content); + } catch { + res.writeHead(404, { 'Content-Type': 'text/html' }); + res.end('

    404 Not Found

    '); + } + }); + + server.on('error', (err: NodeJS.ErrnoException) => { + if (err.code === 'EADDRINUSE') { + console.warn( + `Port ${FIXTURE_PORT} in use, trying ${FIXTURE_PORT + 1}...`, + ); + server.listen(FIXTURE_PORT + 1, '127.0.0.1'); + } else { + reject(err); + } + }); + + server.on('listening', () => { + const addr = server.address(); + const port = typeof addr === 'object' && addr ? addr.port : FIXTURE_PORT; + fixtureServer = server; + console.log(`Test fixture server listening on http://127.0.0.1:${port}`); + resolve(port); + }); + + server.listen(FIXTURE_PORT, '127.0.0.1'); + }); +} + +export async function setup() { + runDir = join(integrationTestsDir, `${Date.now()}`); + await mkdir(runDir, { recursive: true }); + + // Isolate environment variables + isolateTestEnv(runDir); + + // Download ripgrep to avoid race conditions in parallel tests + const available = await resolveRipgrepPath(); + if (!available) { + throw new Error('Failed to download ripgrep binary'); + } + + // Start the test fixture server + const port = await startFixtureServer(); + process.env['TEST_FIXTURE_PORT'] = String(port); + + // Clean up old test runs, but keep the latest few for debugging + try { + const testRuns = await readdir(integrationTestsDir); + if (testRuns.length > 5) { + const oldRuns = testRuns.sort().slice(0, testRuns.length - 5); + await Promise.all( + oldRuns.map((oldRun) => + rm(join(integrationTestsDir, oldRun), { + recursive: true, + force: true, + }), + ), + ); + } + } catch (e) { + console.error('Error cleaning up old test runs:', e); + } + + process.env['INTEGRATION_TEST_FILE_DIR'] = runDir; + + if (process.env['KEEP_OUTPUT']) { + console.log(`Keeping output for test run in: ${runDir}`); + } + process.env['VERBOSE'] = process.env['VERBOSE'] ?? 'false'; + + console.log(`\nIntegration test output directory: ${runDir}`); +} + +export async function teardown() { + // Stop the fixture server + if (fixtureServer) { + await new Promise((resolve) => { + fixtureServer!.close(() => resolve()); + }); + fixtureServer = undefined; + } + + // Disable mouse tracking + if (process.stdout.isTTY) { + disableMouseTracking(); + } + + // Cleanup the test run directory unless KEEP_OUTPUT is set + if (process.env['KEEP_OUTPUT'] !== 'true' && runDir) { + try { + await rm(runDir, { recursive: true, force: true }); + } catch (e) { + console.warn('Failed to clean up test run directory:', e); + } + } +} diff --git a/integration-tests/google_web_search.test.ts b/integration-tests/google_web_search.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..dc19d2df90810a6a8d14ce2c76a9d35fcf433c22 --- /dev/null +++ b/integration-tests/google_web_search.test.ts @@ -0,0 +1,95 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { WEB_SEARCH_TOOL_NAME } from '../packages/core/src/tools/tool-names.js'; +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + TestRig, + printDebugInfo, + assertModelHasOutput, + checkModelOutputContent, +} from './test-helper.js'; + +describe('web search tool', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should be able to search the web', async () => { + await rig.setup('should be able to search the web', { + settings: { tools: { core: [WEB_SEARCH_TOOL_NAME] } }, + }); + + let result; + try { + result = await rig.run({ args: `what is the weather in London` }); + } catch (error) { + // Network errors can occur in CI environments + if ( + error instanceof Error && + (error.message.includes('network') || error.message.includes('timeout')) + ) { + console.warn( + 'Skipping test due to network error:', + (error as Error).message, + ); + return; // Skip the test + } + throw error; // Re-throw if not a network error + } + + const foundToolCall = await rig.waitForToolCall(WEB_SEARCH_TOOL_NAME); + + // Add debugging information + if (!foundToolCall) { + const allTools = printDebugInfo(rig, result); + + // Check if the tool call failed due to network issues + const failedSearchCalls = allTools.filter( + (t) => + t.toolRequest.name === WEB_SEARCH_TOOL_NAME && !t.toolRequest.success, + ); + if (failedSearchCalls.length > 0) { + console.warn( + `${WEB_SEARCH_TOOL_NAME} tool was called but failed, possibly due to network issues`, + ); + console.warn( + 'Failed calls:', + failedSearchCalls.map((t) => t.toolRequest.args), + ); + return; // Skip the test if network issues + } + } + + expect( + foundToolCall, + `Expected to find a call to ${WEB_SEARCH_TOOL_NAME}`, + ).toBeTruthy(); + + assertModelHasOutput(result); + const hasExpectedContent = checkModelOutputContent(result, { + expectedContent: ['weather', 'london'], + testName: 'Google web search test', + }); + + // If content was missing, log the search queries used + if (!hasExpectedContent) { + const searchCalls = rig + .readToolLogs() + .filter((t) => t.toolRequest.name === WEB_SEARCH_TOOL_NAME); + if (searchCalls.length > 0) { + console.warn( + 'Search queries used:', + searchCalls.map((t) => t.toolRequest.args), + ); + } + } + }); +}); diff --git a/integration-tests/hooks-agent-flow-multistep.responses b/integration-tests/hooks-agent-flow-multistep.responses new file mode 100644 index 0000000000000000000000000000000000000000..89718faa62bbc74e60835f2b669540b32d79a9ba --- /dev/null +++ b/integration-tests/hooks-agent-flow-multistep.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"list_dir","args":{"path":"."}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":10,"candidatesTokenCount":10,"totalTokenCount":20}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Final Answer"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":10,"candidatesTokenCount":10,"totalTokenCount":20}}]} diff --git a/integration-tests/hooks-agent-flow.test.ts b/integration-tests/hooks-agent-flow.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..b602737a3919400d149152bcb17e6063291b923e --- /dev/null +++ b/integration-tests/hooks-agent-flow.test.ts @@ -0,0 +1,338 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig, normalizePath } from './test-helper.js'; +import { join } from 'node:path'; +import { writeFileSync } from 'node:fs'; + +describe('Hooks Agent Flow', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + if (rig) { + await rig.cleanup(); + } + }); + + describe('BeforeAgent Hooks', () => { + it('should inject additional context via BeforeAgent hook', async () => { + await rig.setup('should inject additional context via BeforeAgent hook', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-agent-flow.responses', + ), + }); + + const hookScript = ` + try { + const output = { + decision: "allow", + hookSpecificOutput: { + hookEventName: "BeforeAgent", + additionalContext: "SYSTEM INSTRUCTION: This is injected context." + } + }; + process.stdout.write(JSON.stringify(output)); + } catch (e) { + console.error('Failed to write stdout:', e); + process.exit(1); + } + console.error('DEBUG: BeforeAgent hook executed'); + `; + + const scriptPath = join(rig.testDir!, 'before_agent_context.cjs'); + writeFileSync(scriptPath, hookScript); + + await rig.setup('should inject additional context via BeforeAgent hook', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeAgent: [ + { + hooks: [ + { + type: 'command', + command: `node "${scriptPath}"`, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ args: 'Hello test' }); + + // Verify hook execution and telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + const hookLogs = rig.readHookLogs(); + const beforeAgentLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'BeforeAgent', + ); + + expect(beforeAgentLog).toBeDefined(); + expect(beforeAgentLog?.hookCall.stdout).toContain('injected context'); + expect(beforeAgentLog?.hookCall.stdout).toContain('"decision":"allow"'); + expect(beforeAgentLog?.hookCall.stdout).toContain( + 'SYSTEM INSTRUCTION: This is injected context.', + ); + }); + }); + + describe('AfterAgent Hooks', () => { + it('should receive prompt and response in AfterAgent hook', async () => { + await rig.setup('should receive prompt and response in AfterAgent hook', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-agent-flow.responses', + ), + }); + + const hookScript = ` + const fs = require('fs'); + try { + const input = fs.readFileSync(0, 'utf-8'); + console.error('DEBUG: AfterAgent hook input received'); + process.stdout.write("Received Input: " + input); + } catch (err) { + console.error('Hook Failed:', err); + process.exit(1); + } + `; + + const scriptPath = rig.createScript('after_agent_verify.cjs', hookScript); + + rig.setup('should receive prompt and response in AfterAgent hook', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + AfterAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ args: 'Hello validation' }); + + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + const hookLogs = rig.readHookLogs(); + const afterAgentLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'AfterAgent', + ); + + expect(afterAgentLog).toBeDefined(); + // Verify the hook stdout contains the input we echoed which proves the + // hook received the prompt and response + expect(afterAgentLog?.hookCall.stdout).toContain('Received Input'); + expect(afterAgentLog?.hookCall.stdout).toContain('Hello validation'); + // The fake response contains "Hello World" + expect(afterAgentLog?.hookCall.stdout).toContain('Hello World'); + }); + + it('should process clearContext in AfterAgent hook output', async () => { + rig.setup('should process clearContext in AfterAgent hook output', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.after-agent.responses', + ), + }); + + // BeforeModel hook to track message counts across LLM calls + const messageCountFile = join(rig.testDir!, 'message-counts.json'); + const escapedPath = JSON.stringify(messageCountFile); + const beforeModelScript = ` + const fs = require('fs'); + const input = JSON.parse(fs.readFileSync(0, 'utf-8')); + const messageCount = input.llm_request?.contents?.length || 0; + let counts = []; + try { counts = JSON.parse(fs.readFileSync(${escapedPath}, 'utf-8')); } catch (e) {} + counts.push(messageCount); + fs.writeFileSync(${escapedPath}, JSON.stringify(counts)); + console.log(JSON.stringify({ decision: 'allow' })); + `; + const beforeModelScriptPath = rig.createScript( + 'before_model_counter.cjs', + beforeModelScript, + ); + + const afterAgentScript = ` + const fs = require('fs'); + const input = JSON.parse(fs.readFileSync(0, 'utf-8')); + if (input.stop_hook_active) { + // Retry turn: allow execution to proceed (breaks the loop) + console.log(JSON.stringify({ decision: 'allow' })); + } else { + // First call: block and clear context to trigger the retry + console.log(JSON.stringify({ + decision: 'block', + reason: 'Security policy triggered', + hookSpecificOutput: { + hookEventName: 'AfterAgent', + clearContext: true + } + })); + } + `; + const afterAgentScriptPath = rig.createScript( + 'after_agent_clear.cjs', + afterAgentScript, + ); + + rig.setup('should process clearContext in AfterAgent hook output', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeModel: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${beforeModelScriptPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + AfterAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${afterAgentScriptPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ args: 'Hello test' }); + + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + const hookLogs = rig.readHookLogs(); + const afterAgentLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'AfterAgent', + ); + + expect(afterAgentLog).toBeDefined(); + expect(afterAgentLog?.hookCall.stdout).toContain('clearContext'); + expect(afterAgentLog?.hookCall.stdout).toContain('true'); + expect(result).toContain('Security policy triggered'); + + // Verify context was cleared: second call should not have more messages than first + const countsRaw = rig.readFile('message-counts.json'); + const counts = JSON.parse(countsRaw) as number[]; + expect(counts.length).toBeGreaterThanOrEqual(2); + expect(counts[1]).toBeLessThanOrEqual(counts[0]); + }); + }); + + describe('Multi-step Loops', () => { + it('should fire BeforeAgent and AfterAgent exactly once per turn despite tool calls', async () => { + await rig.setup( + 'should fire BeforeAgent and AfterAgent exactly once per turn despite tool calls', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-agent-flow-multistep.responses', + ), + }, + ); + + // Create script files for hooks + const baPath = rig.createScript( + 'ba_fired.cjs', + "console.log('BeforeAgent Fired');", + ); + const aaPath = rig.createScript( + 'aa_fired.cjs', + "console.log('AfterAgent Fired');", + ); + + await rig.setup( + 'should fire BeforeAgent and AfterAgent exactly once per turn despite tool calls', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${baPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + AfterAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${aaPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + await rig.run({ args: 'Do a multi-step task' }); + + const hookLogs = rig.readHookLogs(); + const beforeAgentLogs = hookLogs.filter( + (log) => log.hookCall.hook_event_name === 'BeforeAgent', + ); + const afterAgentLogs = hookLogs.filter( + (log) => log.hookCall.hook_event_name === 'AfterAgent', + ); + + expect(beforeAgentLogs).toHaveLength(1); + + expect(afterAgentLogs).toHaveLength(1); + + const afterAgentLog = afterAgentLogs[0]; + expect(afterAgentLog).toBeDefined(); + expect(afterAgentLog?.hookCall.stdout).toContain('AfterAgent Fired'); + }); + }); +}); diff --git a/integration-tests/hooks-system.after-agent.responses b/integration-tests/hooks-system.after-agent.responses new file mode 100644 index 0000000000000000000000000000000000000000..526c59362dc77202ba4ca341c4f328f0295a9390 --- /dev/null +++ b/integration-tests/hooks-system.after-agent.responses @@ -0,0 +1,3 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hi there!"}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Clarification: I am a bot."}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Security policy triggered"}],"role":"model"},"finishReason":"STOP","index":0}]}]} diff --git a/integration-tests/hooks-system.after-model.responses b/integration-tests/hooks-system.after-model.responses new file mode 100644 index 0000000000000000000000000000000000000000..b60eb23cda99f9417cc522d04b0a6a38862fd399 --- /dev/null +++ b/integration-tests/hooks-system.after-model.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Addressing the Inquiry**\n\nI've grasped the core of the user's question and identified that no tools are needed. My focus is now on crafting a straightforward, direct response that fully addresses their query without any unnecessary complexity. The goal is to provide a clear and concise answer.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12777,"totalTokenCount":12802,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12777}],"thoughtsTokenCount":25}},{"candidates":[{"content":{"parts":[{"text":"4","thoughtSignature":"CiQBcsjafBFqw6veocEvtOGGuQcsyHdcNrXDIn19n9ImwBBwcYQKdgFyyNp8g7o8Ji++OXoqml4gbLPIB2DQbXcaRQfRuYefF8RxMEpzJSITZBlT1VpJQoeYmQcb9c8dg/POmo5d3ZcuLbpVJpbjMIV1SoUI4KEn3zqz7a8BFuyq3zY4VEliRWMZO21JMd8qp59M9m64hX7W1YPyzu8KPwFyyNp8aNCD7P1NJDG3csQkiMW/0jWdPkh+7+XxT7i3ku/lYH4yTEShdicPcmnzoPGhEWTUDr/4Lx+A0DnVGQ=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12777,"totalTokenCount":12802,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12777}],"thoughtsTokenCount":25}}]} diff --git a/integration-tests/hooks-system.after-tool-context.responses b/integration-tests/hooks-system.after-tool-context.responses new file mode 100644 index 0000000000000000000000000000000000000000..ef741e7013f011d1c5afc5b7bb9e34031226dcc5 --- /dev/null +++ b/integration-tests/hooks-system.after-tool-context.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Analyzing File Access**\n\nI've realized the `read_file` tool is perfect for accessing the contents of `test-file.txt`. My next step is to call this tool and set the `file_path` parameter to `test-file.txt`.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12785,"totalTokenCount":12841,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12785}],"thoughtsTokenCount":56}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"read_file","args":{"file_path":"test-file.txt"}},"thoughtSignature":"CiQBcsjafE9D7iAF+V3wpXP81/VmxiMeSFA6afML/lAB76U6QFQKXgFyyNp8i/vhxpkTQ5Cq81QTeEJDDMaYihzSTFMqO4Vj0+CLNtoy+SC/LmqA+WaXh4tm6UCNFTzB2fpVW13YOU1oVYhLpVpeck746YExu1MOSTAq7AC9Yz8ZoelXdecKdwFyyNp8q0PejiY9K1osdOJ02tOHAzAb8ZCSFHtHamEPxRB93krGMNvuIYC1jM1JnC/fzpH8gYV+0/xkoPJMHpF/aSzWq4kZ/j5cUhMYaqKJTulY8ZZGfawnXG7z0spmmr06gwfgILa+HK++xQhhTphMQCobX5hyCjUBcsjafHY6eJfVNitYmfruLV1mnoYnNViHuAOOOni9jIz4VMIjLbClKkb2rpVfHIjx+vZSHA=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12785,"candidatesTokenCount":20,"totalTokenCount":12861,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12785}],"thoughtsTokenCount":56}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"This"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12889,"candidatesTokenCount":1,"totalTokenCount":12890,"cachedContentTokenCount":12206,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12889}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12206}]}},{"candidates":[{"content":{"parts":[{"text":" is test content"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12889,"candidatesTokenCount":4,"totalTokenCount":12893,"cachedContentTokenCount":12206,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12889}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12206}]}}]} diff --git a/integration-tests/hooks-system.allow-tool.responses b/integration-tests/hooks-system.allow-tool.responses new file mode 100644 index 0000000000000000000000000000000000000000..9241bc0e786e6ed2c282492a11f1c874a39042a2 --- /dev/null +++ b/integration-tests/hooks-system.allow-tool.responses @@ -0,0 +1,4 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Formulating the Write Operation**\n\nOkay, I'm now clear on the user's intent: they want a file called `approved.txt`, containing the words \"Approved content.\" I've decided to leverage the `write_file` tool. The specific parameter assignments seem straightforward; `file_path` will be \"approved.txt\", and the file's `content` will precisely mirror the desired output string.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12778,"totalTokenCount":12838,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12778}],"thoughtsTokenCount":60}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"content":"Approved content","file_path":"approved.txt"}},"thoughtSignature":"CiQBcsjafF4NswdygCBTU7cA/yXVRcUI3XHwV+E8BDg/hRr1MaoKZQFyyNp8HRY1qEvivtg0LpYPo1022IfTY3QIeigqGvSoRVospxT5MBggc9nRbwH2vrdhZ772IdqOCrpjNHs3wc+h0AF4JzjlBet6+yC2m7TdenVOkzVAtqnNDMQAIS1gDZyKs8w/CngBcsjafOeuyDQtxuK7JCafKjtfvPvoKOkVxzDetQtHesBkPtv1Xng9dkP77jLH44hn9rrg7yA+za6vssiFZUjC/FU25pCWQgIhM+K7nt3wbAgoOZRqra2gRr3od2D3osV/UpYhy8MoloykqrWvHDOzT/0KScpHarwKXQFyyNp8qabyDYlfElywQBjqQT4f6My7+Ln9AbKZQz4NaEe90ESg4jr4jjANxyd/WKzRheaBq7BYxTHQSeShgQbVjk2D0tZO4hAN+CToMtQwJl95Ss4ZEov6gAwMNA=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12778,"candidatesTokenCount":24,"totalTokenCount":12862,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12778}],"thoughtsTokenCount":60}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12939,"totalTokenCount":12939,"cachedContentTokenCount":12203,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12939}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12203}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12939,"totalTokenCount":12939,"cachedContentTokenCount":12203,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12939}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12203}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Confirming File Creation**\n\nI've successfully created the file `approved.txt`, and I've verified that it contains the intended content, \"Approved content\". Moving on.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12887,"totalTokenCount":12932,"cachedContentTokenCount":12198,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12887}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12198}],"thoughtsTokenCount":45}},{"candidates":[{"content":{"parts":[{"text":"**Assessing File Contents**\n\nI'm now checking the content of `approved.txt`. I used `cat` to display its contents, and it confirms the initial content of \"Approved content\" is present. My next step will be based on this verification.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12887,"totalTokenCount":12946,"cachedContentTokenCount":12198,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12887}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12198}],"thoughtsTokenCount":59}},{"candidates":[{"content":{"parts":[{"text":"I have created the file. What would you like me to do next?","thoughtSignature":"CiQBcsjafEAq9BWRwBqUousKwXME0A2Wh1tJI5cJC9ROpr9Cix8KagFyyNp81VagWC/YxtY8zCAiThU3BHMVh5wZIsGIWv1NNIXqACLQLoSeLhWEneb6CBkKdbKBugy6g9+jP5phYt+Vz5oYuO1Op2kM1qWjFmEQyr71TUISNtZ9zrOHNQKKW7K9ukUi0paw85YKoAEBcsjafF6QLINjBWwQPZh6EPVNGk4wojTKglNp7xy5vclYBbq58A6A8AtZUHKYA2cV32SLb2TGcPnkE4iKunvPf6sZy9Uc7gKA+x/OgSl7i5m0wSpMOh9fLpGt4CNtieigpxHkNAdxdZ5qzGvCkBFWYhaZAWGbj7+1YibIKJFNjX9yEz1T5dOQmVmceu80dFyz+fwl7RiOXSGR5xK4J7DeClYBcsjafPUccUubdSVLFmRohU4bBtQzLvXxw25mqm5TKANLKINQoloZ+xfXzfe8xw/WZL/mg30AqQErBXPNnLk5vIWLK7suuFAZ7oXdisTCj3MRa1HQmQ=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12887,"candidatesTokenCount":13,"totalTokenCount":12959,"cachedContentTokenCount":12198,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12887}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12198}],"thoughtsTokenCount":59}}]} diff --git a/integration-tests/hooks-system.before-model.responses b/integration-tests/hooks-system.before-model.responses new file mode 100644 index 0000000000000000000000000000000000000000..3936e7d37a2f3cb666dd0611576d231341142a6a --- /dev/null +++ b/integration-tests/hooks-system.before-model.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Initiating String Output**\n\nI'm now fully focused on directly outputting the specified string. The process has been simplified to its core objective, eliminating extraneous steps. All systems are go for immediate execution of the requested string output.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12419,"totalTokenCount":12439,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12419}],"thoughtsTokenCount":20}},{"candidates":[{"content":{"parts":[{"text":"The security hook modified this request successfully.","thoughtSignature":"CiQBcsjafAsmW87n4ndCW3YiNIqK6jp0zaTwTjz12vWiwbCFNAUKdQFyyNp808SX5BqCBNZt+dlgsPf74u9W6ofevKGwkTTHQZWJEQiJR2j4uRfESTazuawuWfzKfNJq5Zml6fokNR9jzmQM+Jf4FHw95Jd4lneap+YGO9x5nZMNDI1cHRx0vs4BYW9GWY7lBIM8xKtaEkPrwqc88goiAXLI2nx5o6VrBpXs6jzf5maZIauSYw42zlnkqdDEMI20rg=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12419,"candidatesTokenCount":7,"totalTokenCount":12446,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12419}],"thoughtsTokenCount":20}}]} diff --git a/integration-tests/hooks-system.before-tool-stop.responses b/integration-tests/hooks-system.before-tool-stop.responses new file mode 100644 index 0000000000000000000000000000000000000000..e27a39301374e51b0ccae89646e28d396f681b22 --- /dev/null +++ b/integration-tests/hooks-system.before-tool-stop.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Initializing File Creation**\n\nI'm starting to think about how to make a new file called `test.txt`. My plan is to use a `write_file` tool. I'll need to specify the location and what the file should contain. For now, it will be empty.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":13216,"totalTokenCount":13269,"promptTokensDetails":[{"modality":"TEXT","tokenCount":13216}],"thoughtsTokenCount":53}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"file_path":"test.txt","content":""}},"thoughtSignature":"CiQBcsjafJ20Qbx0YvING6aZ0wYoGWJh3eqornOG4E4AfBLiVsQKXwFyyNp8UlwYs/pv9IRQQGhDlrmlOJF2hfQijryyUYLI+qjDYTpZ6KKIfZF4+vS0soL2BJ3eTXA6gaadFEfNQem3WQVeQoKLFoW4Hv4mbasXqQc0K3p15DuSAtZZENTbCnsBcsjafGK+BJyF/Npnd7gyU0TL5PXePT0nuDFjhJDxlSRUJHDP315TewD3PUYsXd10oWsfhy4B5AngyUiBPUoajdsxg8WxaxnOZYqcp8EIuwtGZrCTev6IihT5nE5jj7u0P9vtnCmkAc6p+4O7Q7Jku1uVGqeJChgzI4YKSAFyyNp8EXSdbttV4xzX+NLKkc276L8Y63tnKU6/Y7fc9/58tU29DSdrgwfe9qmvwtTsO0piFXSLazqHJt8h2bgR7A7GnKDiIA=="}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":13216,"candidatesTokenCount":21,"totalTokenCount":13290,"promptTokensDetails":[{"modality":"TEXT","tokenCount":13216}],"thoughtsTokenCount":53}},{"candidates":[{"content":{"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":13216,"candidatesTokenCount":21,"totalTokenCount":13290,"promptTokensDetails":[{"modality":"TEXT","tokenCount":13216}],"thoughtsTokenCount":53}}]} diff --git a/integration-tests/hooks-system.compress-auto.responses b/integration-tests/hooks-system.compress-auto.responses new file mode 100644 index 0000000000000000000000000000000000000000..125928ae5e943b8ff4241cf77f03f92ecb5bf1ba --- /dev/null +++ b/integration-tests/hooks-system.compress-auto.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Devising a Greeting Phrase**\n\nI've been occupied by the constraint of constructing a five-word salutation. My goal is to make it natural and concise. I'm exploring various combinations to meet the specified word count precisely.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12587,"totalTokenCount":12612,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12587}],"thoughtsTokenCount":25}},{"candidates":[{"content":{"parts":[{"text":"Hello! How can I help you?","thoughtSignature":"CiQBcsjafHso9FUsdYOCTv1xOLlW4MnjbeYnUUBocz0KNgHSzOcKZAFyyNp8XuI6j2afRczgPL8v1dxfVwAJ+5XDKhWKIYf1/8TKGVHh7xXnPfdYBdQ07Ohe7OZXr92xL/IC7B1U2SHDuAOozC0CCW7aiDysu6Hbo6jzYfW5epKht4QjdxYgcKHySrkKMQFyyNp8jXWlHmox53O/CJPXXz2FAmw+ubHKBpYgRezBpA+byyEY2RbVYlZlEMSNkhs="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12587,"candidatesTokenCount":7,"totalTokenCount":12619,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12587}],"thoughtsTokenCount":25}}]} diff --git a/integration-tests/hooks-system.input-modification.responses b/integration-tests/hooks-system.input-modification.responses new file mode 100644 index 0000000000000000000000000000000000000000..aa9744466375f05f1a2e0ef1715463c4e55432c6 --- /dev/null +++ b/integration-tests/hooks-system.input-modification.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"content":"original content","file_path":"original.txt"}}}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I have created the file."}],"role":"model"},"finishReason":"STOP","index":0}]}]} diff --git a/integration-tests/hooks-system.input-validation.responses b/integration-tests/hooks-system.input-validation.responses new file mode 100644 index 0000000000000000000000000000000000000000..d7f6c34d89879c7714ca65760942bf61660a2404 --- /dev/null +++ b/integration-tests/hooks-system.input-validation.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Defining File Creation**\n\nI'm thinking about the user's intent to generate a file named \"input-test.txt\" with the content \" test\". I've determined that the `write_file` tool is suitable. I've parsed `file_path` as \"input-test.txt\" and `content` as \" test\". This should accomplish the user's need.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12778,"totalTokenCount":12840,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12778}],"thoughtsTokenCount":62}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"file_path":"input-test.txt","content":"test"}},"thoughtSignature":"CiQBcsjafCO/Ifs3Lj/Gtzy2ylSYoGB3GXjJby4F3R8FxWp+hP0KZwFyyNp8oD7KvcSYXDimGOiqAdxtdOJpc2tFJbHm2Jw7ahiuKLtoKZWE+1bBZEWVKxC0dCQIeIcxZ0SaLn7tDbfc2qPzhyUA46d/T1+e314SFLWW1asIOBkQ4T0sFDAFPZ4m9bFm3UkKbAFyyNp8EAnclI0wYCGwpg0AOOV52F5J9Hc2EeaXkGsc6hCnba7aNhPucWYIn2Da8FK2IJAWUWaNvGNGoNUZETaG+iL9+6KRJgN3Ql/wQzQ2pHUvTGHC3RkfMGTQ+YCQKvlOReilps5lDmMnhQpTAXLI2nzcl9Aqd0Nb/w934w+tqz1Jth7GlQVMYktHOl7Hgkoykfh3NzM67SEAilxjowfBL6MY7UBUP3YGwi1CXVVa4d0wHnMD9BJYp2w8ztZch8I="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12778,"candidatesTokenCount":25,"totalTokenCount":12865,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12778}],"thoughtsTokenCount":62}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**File Creation Achieved**\n\nI've successfully created the file as requested. Now, I'm ready to move on to the next instruction whenever it arrives. I am now awaiting the next task.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12940,"totalTokenCount":12965,"cachedContentTokenCount":12203,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12940}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12203}],"thoughtsTokenCount":25}},{"candidates":[{"content":{"parts":[{"text":"Done.","thoughtSignature":"CiQBcsjafEwfH5zTnAjEjloMcDDflS/MmoH03HXVl8HoQ04vmVIKcQFyyNp8/6HrBz8vokXB1Ms1zW51p32T3Ni3HEbgSFPHMGZt9LHFtLkLzuFrxym66z1Tcb5tqj+7jAdpM/dIUb6ecrKj9FWqMB+QR4BSxdAiJSiL8Rp+Pc5ckCtT1nrv4C5w3/fhCNE4WvZzeyGPt+PACjsBcsjafNWzUJcHxgKp6MYWQ8RW0QrGerM51nkgXHBafxY5KwTznX4B/ETccGnXX3zSciaJiZR1FfudVw=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12940,"candidatesTokenCount":1,"totalTokenCount":12966,"cachedContentTokenCount":12203,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12940}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12203}],"thoughtsTokenCount":25}}]} diff --git a/integration-tests/hooks-system.multiple-events.responses b/integration-tests/hooks-system.multiple-events.responses new file mode 100644 index 0000000000000000000000000000000000000000..0818d13de9b2ec7d9c2ad694013fb7a31bfa1724 --- /dev/null +++ b/integration-tests/hooks-system.multiple-events.responses @@ -0,0 +1,4 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Formulating a Plan**\n\nOkay, I've outlined the initial steps: I'll use the `write_file` tool to make a file named `multi-event-test.txt` containing the text \"testing multiple events\". After that, I'll need to remember to reply with the phrase as requested. It seems straightforward so far.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12622,"totalTokenCount":12692,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12622}],"thoughtsTokenCount":70}},{"candidates":[{"content":{"parts":[{"text":"**Confirming the Procedure**\n\nI've solidified the steps. First, I'll create `multi-event-test.txt` using the `write_file` tool with the required content. Following that, my response will be \"BeforeAgent: User request processed.\" This ensures I fulfill both parts of the request.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12622,"totalTokenCount":12713,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12622}],"thoughtsTokenCount":91}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"file_path":"multi-event-test.txt","content":"testing multiple events"}},"thoughtSignature":"CiQBcsjafIqcYtNLIeBwJi3k5k8jho3QiWM+51Kw5vTQ7/V4qVQKZgFyyNp8mIIB0+Mvwhvo2fACDpTWpRYeOFPGrjZrc+N05S0WGEHzE4Dv9peHKdvZkjGNW+HyYHXoRpd5c/ScdhPxQoVZmZ9K7sRjVxv/nWVDoKnHlSsn94nJ8acjLnj1oqt9cHni0ApyAXLI2nwj5WuLHr+UFIxnqRKCUJboLo6bQMkqR1TsqXbjsgHp3zNQYT+xzbse4PKPLJV48FN6cL9MrrZ81E7k7AVo1cKyrC7ky7tdRH6gYHewIqgQWBIUgMKhLkePH/fYZ6fS7SMrf4Q6DFGHh6pIAAdRCooBAXLI2nxpudEZr+5jZAaAcCMIdij5oZq3s0xsQv/7iWVh8IossRuR0J4eMMSN8fV6+fjbSQ6YtJQfrxsm3a6gVIkJNno2b2PRZestS/0Z7DvPDGE6r1sGchvbcz8EW7Z/pvJvPBRFWlMTJ1eqY9vuyuNYMKeWlyt+5V9y2GUbcLWvcNDZSC43vQEKCo0BAXLI2nxP4INgBaSHInyFrG1/SEP0SUimKvP69FkcIBxx60x3iKqdtb2flLIhoOr/QuesASlflRfzNo3J5LOudrjZzNlRfVRqOZIyOVxZlviXtO7+w/oPCV61Sby6xPTGtFsWlt6GxEGF7iYLfvi4KWN9q/W9tlqEqUrpl/WMwS/4pYBi1xPcvXZNlJ6g"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12622,"candidatesTokenCount":28,"totalTokenCount":12741,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12622}],"thoughtsTokenCount":91}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12836,"totalTokenCount":12836,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12836}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}]}},{"candidates":[{"content":{"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12836,"totalTokenCount":12836,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12836}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12836,"totalTokenCount":12836,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12836}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}]}},{"candidates":[{"content":{"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12836,"totalTokenCount":12836,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12836}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Echoing User Commands**\n\nI'm now tasked with echoing a specific phrase following a particular signal, but it's becoming complex. The user wants me to repeat \"BeforeAgent: User request processed\" when prompted. It appears I need to retain context from the previous turn, the user's initial request to create a file, to correctly respond now.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12759,"totalTokenCount":12827,"cachedContentTokenCount":12199,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12759}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12199}],"thoughtsTokenCount":68}},{"candidates":[{"content":{"parts":[{"text":"**Responding Precisely to Prompt**\n\nI've determined I need to repeat the phrase \"BeforeAgent: User request processed,\" even though the overall context and turn history are complex. The user has given several prompts, but has now provided a more direct command, which I believe is to follow up on the previous request. I am taking care to match the specific instructions the user provided.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12759,"totalTokenCount":12982,"cachedContentTokenCount":12199,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12759}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12199}],"thoughtsTokenCount":223}},{"candidates":[{"content":{"parts":[{"text":"BeforeAgent: User request processed","thoughtSignature":"CiQBcsjafAntJrb1JBgpnZaCNeYhOJXtbH6dKTeM1llglCdoOvUKYwFyyNp8PUj5sihYyITQJhdz4MqEeftyuUc4G+iTprve11gPN04eK9Y1Wi/wyln4RjRgroIrV5kByKzdGhECoyCeInpiILGhY0peIM7dZOKFdIOL7xAR9pmn4wMreqyH7l5WSAqJAQFyyNp8Cugemkt4YZWkIwEJYmUukLFx4d5EwP/9k/e4OH/svpM+uyuN3n1KVN3bFgRV5yuF0HnDLl+P7WVSSxMmWvXO2f7A1HALg+gCvZw9IV7Btgg1qp81dDoNcVkzSbTBtT4UrlJ5R6sclvHZOLUtKGwBEQ6zRonBugAgj9RV4BT1AJNOgdSsCokBAXLI2nyDGU1Iq30QVbqhgEwFa5sB6uPC+35BV8ZKGwK+YglO9rqXMrkXM+GcQi2hVIsOFXBYGTS6E2/mQfFbIKDytrb1JgP3q5xVd/bE23M2Nnf+q5TLbRpLAPmyfg0AGwhN0L7d5W6b/3ydqEPeA1/Vw/cnBzz5ND1LOTOX6BFqEs33/WHj7HIKpAEBcsjafEsn8//cZMWUQcSAucBQauojv/f7h11nbeMrZK84nEotR30BgMIWYiiWM6sGDy/4MzHwr+z2YdAz4PSgRvEf7DPxHps2nvZfAdtskgtdPl2JD81WpokSnJvCqU+cOuz+Nh3+fIiZ6vEsVpi/5cwEiGT0g3Z3I2ubyzv58oH8YnVQlKT3MsKRGb5//aXZJY57jNrexgDPzYAQsBgSuGBmqwqaAQFyyNp8sSIYw3It6GpZqC+oxJCC26pt4RxhG8rDZ3zuoADYlOpoUdSzbNuDB+iVHeen5OoCEAaH0GrFV4iZxgu40wu4ZD/VMfHi/Vm7vku23EUV/94U8mT+VEwPfd2gqv+3xPZ9MEHjOOox1Xq1984w2cA6u0Qn7wWHXeOGFVGSOHtdJtQ7ToNT8VEecblAVq8lm42sSccXQEEKmAEBcsjafONCvBhW2s8Bset20YFdbeSHelnILFDxXlCoYla5nP5UjGk4vpXu2+7RCFtKXfoyYEVEkmiGBRsmwJ82Q1nMkGkXMhuTdNhu4aCwI5m+STGxx26vkp9bcqGwMDHBotZL63PSrJacRoW8zfpDXD1PABLeTIfh5jgipQdgltyjlbc+3qfIfjBYNRSkE8ByErSz5rT7SwqSAQFyyNp8W2kut1PSJISxM7YJtbRdFqPBTikGDM6F/3l6ba6LpeRBfHdtueLChqFpwLH41VdIPQ7lRZflOq3KaZz+TQ11eDnYQbiaIdGOPgHJ/HH/0iQv2hnoOY5vg3gubFWFuZh9Bfun2VCYUI39tIxGC46TZWfgCdiP/O9CFOlpDfidPiz5ZS/4LhG9FA4Q85OuCpEBAXLI2nzpoEUA6jCZopeNTRA2uZ1r0DMm5cWVVXtFO4CoRS+19BbADNBRyNrR5qcf7bUflJBvMRVxx3mtmgK9aE5VmKYxK2Dqg15l9RUxjtqspC3VVmszVd6lOkf1BBQ/VtWDulqRetKE2u62Is9NNGuK9HsLzIBLRRc8QoML41WffuXQ+uxwyXpjx2USC44MGAqIAQFyyNp8gN3lOyHyk674W3Pyv+Egw1ZDUQK4xpvAfgnK+y53gclMGJ2IjOSvg4j0f1WO1OGqY2TBUFS7w21PXasvCkfxpqeStEb+U7Vm0r63LzXdGdug5/b1Ap6Phn4/vAYmfaKISKG4+QpjI+ehgEJzsIee2rgqOaePTP18fq8T7EDbF/B/iscKNQFyyNp8DWt2a8OetaCc5E/KsntbbOcNc7yikPZBdUezphrqIH4ztpicsHvEicYF002qWHoY"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12759,"candidatesTokenCount":4,"totalTokenCount":12986,"cachedContentTokenCount":12199,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12759}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12199}],"thoughtsTokenCount":223}}]} diff --git a/integration-tests/hooks-system.notification.responses b/integration-tests/hooks-system.notification.responses new file mode 100644 index 0000000000000000000000000000000000000000..5bee660852036188dbc88968ecdeee424f448a5c --- /dev/null +++ b/integration-tests/hooks-system.notification.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Executing the Command**\n\nI've got the command, \"echo test,\" ready to go. My focus is entirely on calling the `run_shell_command` tool now. The user's input is processed, and the next step is straightforward: using the tool to execute the supplied command.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12751,"totalTokenCount":12801,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12751}],"thoughtsTokenCount":50}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"run_shell_command","args":{"command":"echo test","description":"Running the command 'echo test'"}},"thoughtSignature":"CiQBcsjafL1lDlnUGmt38n1/gjwecXzy9S3qEW5sYMEno5Mr7LEKZgFyyNp8jMABmMAatt49FTdh7UiM62SI1GnjcyG+kV7xzcD73uMKHST/0D0vKP7x1equv5d6YiXnOslhVnnHotYPtVl0/kI/0unBZRdMzkBNrJXKUoSWXJXxNpV6JhJav3Uh9h1sPQqOAQFyyNp8PFeESLk0J5cPFP0EA7a13iA/rXTiKoHnjSCzDV9ALcXM78xv10/V028ZtDeQslYfT82q4++W8AlJwTQRTIrdscu2y+nCS8jnQizYN1V1yR42eMzuBU3txXcqEV8bmP6GGOe58vrqyS2zdnJKCgMntMB/niwlJlr5frhDestSOJk62tVDWKFzOiAKOAFyyNp81FtGXQTX+OSio/2PbzpCCuaQFqpEgCZpkaXXyvmXYDAI1qCq1tA+m/e5ozWdm8zTGuyb"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12751,"candidatesTokenCount":28,"totalTokenCount":12829,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12751}],"thoughtsTokenCount":50}}]} diff --git a/integration-tests/hooks-system.sequential-execution.responses b/integration-tests/hooks-system.sequential-execution.responses new file mode 100644 index 0000000000000000000000000000000000000000..66a68e9fbd4e7ffeb87bd72ea05e83c0e6491aa0 --- /dev/null +++ b/integration-tests/hooks-system.sequential-execution.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Seeking Task Clarity**\n\nI'm currently focused on identifying the precise task. My initial assessment indicates the user is seeking assistance, but the specific requirements remain undefined. I will directly solicit a detailed task description from the user to clarify this.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12604,"totalTokenCount":12633,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12604}],"thoughtsTokenCount":29}},{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help. Please describe the task you'd like me to assist you with.","thoughtSignature":"CiQBcsjafM2CL00L595T19DK8M8zP5p9/tbFPPwdM2S6669z2FgKYQFyyNp8Ya0YVCtft9Asr/45XOCfNdPWbwZt8SvIeX3IxYzOFcOK14+DnoDIuTIrmRQBeUvdxD59QmEWx+/OaSxj9564L0IU703C1JX20buEtYhkRM4LhK0G4LG/z6IJauEKSQFyyNp8n784BnEcDTQGfZ8/s3pl/TNaNzjQx0o8wYCYZH1qsRbVa3YJAvRGrVXL6y9ka10w0lhEsrQ8vOiw6ilZKirA5DjLz4U="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12604,"candidatesTokenCount":22,"totalTokenCount":12655,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12604}],"thoughtsTokenCount":29}}]} diff --git a/integration-tests/hooks-system.session-clear.responses b/integration-tests/hooks-system.session-clear.responses new file mode 100644 index 0000000000000000000000000000000000000000..14906f4eb15650e0f04bcddda94523e796977e41 --- /dev/null +++ b/integration-tests/hooks-system.session-clear.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Greeting the User**\n\nI've registered the user's greeting. I'm primed to respond with a friendly welcome and signal my availability to assist. My focus now is drafting a suitable response.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12761,"totalTokenCount":12787,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12761}],"thoughtsTokenCount":26}},{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help. What can I do for you?","thoughtSignature":"CikBcsjafBz/0rqJuIv9woxRvivjZyAqBjpoJhOTSPfcbMWCawTfcyKImQpxAXLI2nxyuBo6dqZmTxkH7XxPxjq7mNoacRa48wc/eT5caK/4tu0Y9fJ1ScpJZb+tCNzrqTNwVXa98ppjB2O/X4eejJN+hUr3LCalDFRdRLO17PFUI5qgYSbSgIGzhbnQASgzOArvvqzDPPgqXWVIDj8KMQFyyNp8ayfqBNRkBykRSTDtzOKVGkjLW1dXWamLB4ojeEVHSOgne4vlYaKs44pitsg="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12761,"candidatesTokenCount":15,"totalTokenCount":12802,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12761}],"thoughtsTokenCount":26}}]} diff --git a/integration-tests/hooks-system.session-startup.responses b/integration-tests/hooks-system.session-startup.responses new file mode 100644 index 0000000000000000000000000000000000000000..83770d6762393129d54da3f19bd993766b9a7c6a --- /dev/null +++ b/integration-tests/hooks-system.session-startup.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Initiating a Dialogue**\n\nI've successfully received and understood the user's initial request. My next move will be to output a simple \"Hello\" as a greeting, fulfilling the basic instruction I was given. This constitutes the first step in the interaction, and I'm ready to move forward based on the user's subsequent input.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12588,"totalTokenCount":12607,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12588}],"thoughtsTokenCount":19}},{"candidates":[{"content":{"parts":[{"text":"Hello","thoughtSignature":"CikBcsjafB9jXawgyqQ5mpEJ4ihpLD/B2i8GR75sod00ZF3TCbrLHS9YjgpeAXLI2nx1fmJO2VIiwBpF+vLBPhYE/B2992PVW6XM20cEYx4g0leDNs6BIhzEipm6RYOxzgz8KxH9+ZkCnd8bVZr59lbDCgqSCSB6IKA+csXHKsF9g3UMRAtoSBwiBw=="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12588,"totalTokenCount":12607,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12588}],"thoughtsTokenCount":19}}]} diff --git a/integration-tests/hooks-system.tail-tool-call.responses b/integration-tests/hooks-system.tail-tool-call.responses new file mode 100644 index 0000000000000000000000000000000000000000..13dc3fde4db082af9d1e8ff8f68f6744ca3e4ef5 --- /dev/null +++ b/integration-tests/hooks-system.tail-tool-call.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"read_file","args":{"file_path":"original.txt"}}}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Tail call completed successfully."}],"role":"model"},"finishReason":"STOP","index":0}]}]} \ No newline at end of file diff --git a/integration-tests/hooks-system.telemetry.responses b/integration-tests/hooks-system.telemetry.responses new file mode 100644 index 0000000000000000000000000000000000000000..efe1cc3fd3bdbeab054eeaa184524a6c2554dc89 --- /dev/null +++ b/integration-tests/hooks-system.telemetry.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"**Initializing File Creation**\n\nI've decided on the `write_file` tool to create the telemetry file. I'll pass \"telemetry-test.txt\" as the file path, and an empty string for the content, as the user didn't specify anything to include. This is the initial setup; the file should now exist.\n\n\n","thought":true}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":12779,"totalTokenCount":12850,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12779}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}],"thoughtsTokenCount":71}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"write_file","args":{"content":"","file_path":"telemetry-test.txt"}},"thoughtSignature":"CiQBcsjafG+JDSqtKOK+ZvSjZQZmS91c1Gz0YyTiirI2u5+rhEIKZAFyyNp8DNXb+xHILTC+FVlEifqEHdrmfFNBLKojci1UIBhcZpQ4UXCMkxUXYKO34IjTlyLgSsjVbbXWEFXatb/z/RtTDcf51uc3YOEwlDScGempkJxfFgcPfIiD7bhuHBqdQfUKfAFyyNp8wZ71h+QjdfVw12PwDXWgGZ0Xed1GuyJXuqAwpWnwxDIvsDaPwDFYyLR1XDiIZZk4AvFCGt6HGMSLRuPh4K3i9CVnDc5hcjyvMIde0idAFMrgs2Mq5SARfCPrWkqyq2f0Q0WonUl2n7yr/sDQ78rx2E6qXyUJ8XMKfAFyyNp8DdTYLttyI0jknqAeZDxdFmHtpJUI8UKP5YHzpQc8Qn80OJcwhZSRH4HRKCqoC7Sukq/A5vJ5T468WqgjOoLlPLq02bYRTf/q6LC1ogEhdLHrcFv2jDeCdXJJ8NHv3O4DZAUAk1W5Gd0428zMFOxH3AgkWwEGuow="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12779,"candidatesTokenCount":24,"totalTokenCount":12874,"cachedContentTokenCount":12204,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12779}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12204}],"thoughtsTokenCount":71}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"OK."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":12951,"candidatesTokenCount":2,"totalTokenCount":12953,"cachedContentTokenCount":12202,"promptTokensDetails":[{"modality":"TEXT","tokenCount":12951}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":12202}]}}]} diff --git a/integration-tests/hooks-system.test.ts b/integration-tests/hooks-system.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..cdead08b17f525d3e1d120a7e5385826eb08447a --- /dev/null +++ b/integration-tests/hooks-system.test.ts @@ -0,0 +1,2484 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig, poll, normalizePath, skipFlaky } from './test-helper.js'; +import { join } from 'node:path'; +import { writeFileSync, existsSync, mkdirSync } from 'node:fs'; +import os from 'node:os'; + +describe.skipIf(skipFlaky)( + 'Hooks System Integration', + { timeout: 120000 }, + () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + if (rig) { + await rig.cleanup(); + } + }); + + describe('Command Hooks - Blocking Behavior', () => { + it('should block tool execution when hook returns block decision', async () => { + rig.setup( + 'should block tool execution when hook returns block decision', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.block-tool.responses', + ), + }, + ); + + const scriptPath = rig.createScript( + 'block_hook.cjs', + "console.log(JSON.stringify({decision: 'block', reason: 'File writing blocked by security policy'}));", + ); + + rig.setup( + 'should block tool execution when hook returns block decision', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const result = await rig.run({ + args: 'Create a file called test.txt with content "Hello World"', + }); + + // The hook should block the write_file tool + const toolLogs = rig.readToolLogs(); + const writeFileCalls = toolLogs.filter( + (t) => + t.toolRequest.name === 'write_file' && + t.toolRequest.success === true, + ); + + // Tool should not be called due to blocking hook + expect(writeFileCalls).toHaveLength(0); + + // Result should mention the blocking reason + expect(result).toContain('File writing blocked by security policy'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + + it('should block tool execution and use stderr as reason when hook exits with code 2', async () => { + rig.setup( + 'should block tool execution and use stderr as reason when hook exits with code 2', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.block-tool.responses', + ), + }, + ); + + const blockMsg = 'File writing blocked by security policy'; + + const scriptPath = rig.createScript( + 'stderr_block_hook.cjs', + `process.stderr.write(JSON.stringify({ decision: 'deny', reason: '${blockMsg}' })); process.exit(2);`, + ); + + rig.setup( + 'should block tool execution and use stderr as reason when hook exits with code 2', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`)!, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const result = await rig.run({ + args: 'Create a file called test.txt with content "Hello World"', + }); + + // The hook should block the write_file tool + const toolLogs = rig.readToolLogs(); + const writeFileCalls = toolLogs.filter( + (t) => + t.toolRequest.name === 'write_file' && + t.toolRequest.success === true, + ); + + // Tool should not be called due to blocking hook + expect(writeFileCalls).toHaveLength(0); + + // Result should mention the blocking reason + expect(result).toContain(blockMsg); + + // Verify hook telemetry shows the deny decision + const hookLogs = rig.readHookLogs(); + const blockHook = hookLogs.find( + (log) => + log.hookCall.hook_event_name === 'BeforeTool' && + ((log.hookCall.stdout?.includes('"decision":"deny"') ?? false) || + (log.hookCall.stderr?.includes('"decision":"deny"') ?? false)), + ); + expect(blockHook).toBeDefined(); + expect( + (blockHook?.hookCall.stdout || '') + + (blockHook?.hookCall.stderr || ''), + ).toContain(blockMsg); + }); + + it('should allow tool execution when hook returns allow decision', async () => { + rig.setup( + 'should allow tool execution when hook returns allow decision', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.allow-tool.responses', + ), + }, + ); + + const scriptPath = rig.createScript( + 'allow_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', reason: 'File writing approved'}));", + ); + + rig.setup( + 'should allow tool execution when hook returns allow decision', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + await rig.run({ + args: 'Create a file called approved.txt with content "Approved content"', + }); + + // The hook should allow the write_file tool + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // File should be created + const fileContent = rig.readFile('approved.txt'); + expect(fileContent).toContain('Approved content'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + }); + + describe('Command Hooks - Additional Context', () => { + it('should add additional context from AfterTool hooks', async () => { + rig.setup('should add additional context from AfterTool hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.after-tool-context.responses', + ), + }); + + const scriptPath = rig.createScript( + 'after_tool_context.cjs', + "console.log(JSON.stringify({hookSpecificOutput: {hookEventName: 'AfterTool', additionalContext: 'Security scan: File content appears safe'}}));", + ); + + const command = `node "${scriptPath}"`; + rig.setup('should add additional context from AfterTool hooks', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + AfterTool: [ + { + matcher: 'read_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(command), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Create a test file to read + rig.createFile('test-file.txt', 'This is test content'); + + await rig.run({ + args: 'Read the contents of test-file.txt and tell me what it contains', + }); + + // Should find read_file tool call + const foundReadFile = await rig.waitForToolCall('read_file'); + expect(foundReadFile).toBeTruthy(); + + // Should generate hook telemetry + const hookTelemetryFound = rig.readHookLogs(); + expect(hookTelemetryFound.length).toBeGreaterThan(0); + expect(hookTelemetryFound[0].hookCall.hook_event_name).toBe( + 'AfterTool', + ); + expect(hookTelemetryFound[0].hookCall.hook_name).toBe( + normalizePath(command), + ); + expect(hookTelemetryFound[0].hookCall.hook_input).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.hook_output).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.exit_code).toBe(0); + expect(hookTelemetryFound[0].hookCall.stdout).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.stderr).toBeDefined(); + }); + }); + + describe('Command Hooks - Tail Tool Calls', () => { + it('should execute a tail tool call from AfterTool hooks and replace original response', async () => { + // Create a script that acts as the hook. + // It will trigger on "read_file" and issue a tail call to "write_file". + rig.setup('should execute a tail tool call from AfterTool hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.tail-tool-call.responses', + ), + }); + + const hookOutput = { + decision: 'allow', + hookSpecificOutput: { + hookEventName: 'AfterTool', + tailToolCallRequest: { + name: 'write_file', + args: { + file_path: 'tail-called-file.txt', + content: 'Content from tail call', + }, + }, + }, + }; + + const hookScript = `console.log(JSON.stringify(${JSON.stringify( + hookOutput, + )})); process.exit(0);`; + + const scriptPath = join(rig.testDir!, 'tail_call_hook.js'); + writeFileSync(scriptPath, hookScript); + const commandPath = scriptPath.replace(/\\/g, '/'); + + rig.setup('should execute a tail tool call from AfterTool hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.tail-tool-call.responses', + ), + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + AfterTool: [ + { + matcher: 'read_file', + hooks: [ + { + type: 'command', + command: `node "${commandPath}"`, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Create a test file to trigger the read_file tool + rig.createFile('original.txt', 'Original content'); + + const cliOutput = await rig.run({ + args: 'Read original.txt', // Fake responses should trigger read_file on this + }); + + // 1. Verify that write_file was called (as a tail call replacing read_file) + // Since read_file was replaced before finalizing, it will not appear in the tool logs. + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // Ensure hook logs are flushed and the final LLM response is received. + // The mock LLM is configured to respond with "Tail call completed successfully." + expect(cliOutput).toContain('Tail call completed successfully.'); + + // Ensure telemetry is written to disk + await rig.waitForTelemetryReady(); + + // Read hook logs to debug + const hookLogs = rig.readHookLogs(); + const relevantHookLog = hookLogs.find( + (l) => l.hookCall.hook_event_name === 'AfterTool', + ); + + expect(relevantHookLog).toBeDefined(); + + // 2. Verify write_file was executed. + // In non-interactive mode, the CLI deduplicates tool execution logs by callId. + // Since a tail call reuses the original callId, "Tool: write_file" is not printed. + // Instead, we verify the side-effect (file creation) and the telemetry log. + + // 3. Verify the tail-called tool actually wrote the file + const modifiedContent = rig.readFile('tail-called-file.txt'); + expect(modifiedContent).toBe('Content from tail call'); + + // 4. Verify telemetry for the final tool call. + // The original 'read_file' call is replaced, so only 'write_file' is finalized and logged. + const toolLogs = rig.readToolLogs(); + const successfulTools = toolLogs.filter((t) => t.toolRequest.success); + expect( + successfulTools.some((t) => t.toolRequest.name === 'write_file'), + ).toBeTruthy(); + // The original request name should be preserved in the log payload if possible, + // but the executed tool name is 'write_file'. + }); + }); + + describe('BeforeModel Hooks - LLM Request Modification', () => { + it('should modify LLM requests with BeforeModel hooks', async () => { + // Create a hook script that replaces the LLM request with a modified version + // Note: Providing messages in the hook output REPLACES the entire conversation + rig.setup('should modify LLM requests with BeforeModel hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.before-model.responses', + ), + }); + const hookScript = `const fs = require('fs'); +console.log(JSON.stringify({ + decision: "allow", + hookSpecificOutput: { + hookEventName: "BeforeModel", + llm_request: { + messages: [ + { + role: "user", + content: "Please respond with exactly: The security hook modified this request successfully." + } + ] + } + } +}));`; + + const scriptPath = rig.createScript( + 'before_model_hook.cjs', + hookScript, + ); + + rig.setup('should modify LLM requests with BeforeModel hooks', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeModel: [ + { + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ args: 'Tell me a story' }); + + // The hook should have replaced the request entirely + // Verify that the model responded to the modified request, not the original + expect(result).toBeDefined(); + expect(result.length).toBeGreaterThan(0); + // The response should contain the expected text from the modified request + expect(result.toLowerCase()).toContain('security hook modified'); + + // Should generate hook telemetry + + // Should generate hook telemetry + const hookTelemetryFound = rig.readHookLogs(); + expect(hookTelemetryFound.length).toBeGreaterThan(0); + expect(hookTelemetryFound[0].hookCall.hook_event_name).toBe( + 'BeforeModel', + ); + expect(hookTelemetryFound[0].hookCall.hook_name).toBe( + `node "${scriptPath}"`, + ); + expect(hookTelemetryFound[0].hookCall.hook_input).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.hook_output).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.exit_code).toBe(0); + expect(hookTelemetryFound[0].hookCall.stdout).toBeDefined(); + expect(hookTelemetryFound[0].hookCall.stderr).toBeDefined(); + }); + + it('should block model execution when BeforeModel hook returns deny decision', async () => { + rig.setup( + 'should block model execution when BeforeModel hook returns deny decision', + ); + const hookScript = `console.log(JSON.stringify({ + decision: "deny", + reason: "Model execution blocked by security policy" +}));`; + const scriptPath = rig.createScript( + 'before_model_deny_hook.cjs', + hookScript, + ); + + rig.setup( + 'should block model execution when BeforeModel hook returns deny decision', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeModel: [ + { + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const result = await rig.run({ args: 'Hello' }); + + // The hook should have blocked the request + expect(result).toContain('Model execution blocked by security policy'); + + // Verify no API requests were made to the LLM + const apiRequests = rig.readAllApiRequest(); + expect(apiRequests).toHaveLength(0); + }); + + it('should block model execution when BeforeModel hook returns block decision', async () => { + rig.setup( + 'should block model execution when BeforeModel hook returns block decision', + ); + const hookScript = `console.log(JSON.stringify({ + decision: "block", + reason: "Model execution blocked by security policy" +}));`; + const scriptPath = rig.createScript( + 'before_model_block_hook.cjs', + hookScript, + ); + + rig.setup( + 'should block model execution when BeforeModel hook returns block decision', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeModel: [ + { + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const result = await rig.run({ args: 'Hello' }); + + // The hook should have blocked the request + expect(result).toContain('Model execution blocked by security policy'); + + // Verify no API requests were made to the LLM + const apiRequests = rig.readAllApiRequest(); + expect(apiRequests).toHaveLength(0); + }); + }); + + describe('AfterModel Hooks - LLM Response Modification', () => { + it.skipIf(process.platform === 'win32')( + 'should modify LLM responses with AfterModel hooks', + async () => { + rig.setup('should modify LLM responses with AfterModel hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.after-model.responses', + ), + }); + // Create a hook script that modifies the LLM response + const hookScript = `const fs = require('fs'); +console.log(JSON.stringify({ + hookSpecificOutput: { + hookEventName: "AfterModel", + llm_response: { + candidates: [ + { + content: { + role: "model", + parts: [ + "[FILTERED] Response has been filtered for security compliance." + ] + }, + finishReason: "STOP" + } + ] + } + } +}));`; + + const scriptPath = rig.createScript( + 'after_model_hook.cjs', + hookScript, + ); + + rig.setup('should modify LLM responses with AfterModel hooks', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + AfterModel: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ args: 'What is 2 + 2?' }); + + // The hook should have replaced the model response + expect(result).toContain( + '[FILTERED] Response has been filtered for security compliance', + ); + + // Should generate hook telemetry + const hookTelemetryFound = + await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }, + ); + }); + + describe('BeforeToolSelection Hooks - Tool Configuration', () => { + it('should modify tool selection with BeforeToolSelection hooks', async () => { + // 1. Initial setup to establish test directory + rig.setup('BeforeToolSelection Hooks'); + + const toolConfigJson = JSON.stringify({ + decision: 'allow', + hookSpecificOutput: { + hookEventName: 'BeforeToolSelection', + toolConfig: { + mode: 'ANY', + allowedFunctionNames: ['read_file'], + }, + }, + }); + + // Use file-based hook to avoid quoting issues + const hookScript = `console.log(JSON.stringify(${toolConfigJson}));`; + const hookFilename = 'before_tool_selection_hook.js'; + const scriptPath = rig.createScript(hookFilename, hookScript); + + // 2. Final setup with script path + rig.setup('BeforeToolSelection Hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.before-tool-selection.responses', + ), + settings: { + debugMode: true, + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeToolSelection: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 60000, + }, + ], + }, + ], + }, + }, + }); + + // Create a test file + rig.createFile('new_file_data.txt', 'test data'); + + await rig.run({ + args: 'Check the content of new_file_data.txt', + }); + + // Verify the hook was called for BeforeToolSelection event + const hookLogs = rig.readHookLogs(); + const beforeToolSelectionHook = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'BeforeToolSelection', + ); + expect(beforeToolSelectionHook).toBeDefined(); + expect(beforeToolSelectionHook?.hookCall.success).toBe(true); + + // Verify hook telemetry shows it modified the config + expect( + JSON.stringify(beforeToolSelectionHook?.hookCall.hook_output), + ).toContain('read_file'); + }); + }); + + describe('BeforeAgent Hooks - Prompt Augmentation', () => { + it('should augment prompts with BeforeAgent hooks', async () => { + // Create a hook script that adds context to the prompt + const hookScript = `const fs = require('fs'); +console.log(JSON.stringify({ + decision: "allow", + hookSpecificOutput: { + hookEventName: "BeforeAgent", + additionalContext: "SYSTEM INSTRUCTION: You are in a secure environment. Always mention security compliance in your responses." + } +}));`; + + rig.setup('should augment prompts with BeforeAgent hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.before-agent.responses', + ), + }); + + const scriptPath = rig.createScript( + 'before_agent_hook.cjs', + hookScript, + ); + + rig.setup('should augment prompts with BeforeAgent hooks', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ args: 'Hello, how are you?' }); + + // The hook should have added security context, which should influence the response + expect(result).toContain('security'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + }); + + describe('Notification Hooks - Permission Handling', () => { + it('should handle notification hooks for tool permissions', async () => { + rig.setup('should handle notification hooks for tool permissions', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.notification.responses', + ), + }); + + // Create script file for hook + const scriptPath = rig.createScript( + 'notification_hook.cjs', + "console.log(JSON.stringify({suppressOutput: false, systemMessage: 'Permission request logged by security hook'}));", + ); + + const hookCommand = `node "${scriptPath}"`; + + rig.setup('should handle notification hooks for tool permissions', { + settings: { + // Configure tools to enable hooks and require confirmation to trigger notifications + tools: { + approval: 'ASK', // Disable YOLO mode to show permission prompts + confirmationRequired: ['run_shell_command'], + }, + hooksConfig: { + enabled: true, + }, + hooks: { + Notification: [ + { + matcher: 'ToolPermission', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(hookCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const run = await rig.runInteractive({ approvalMode: 'default' }); + + // Send prompt that will trigger a permission request + await run.type('Run the command "echo test"'); + await run.type('\r'); + + // Wait for permission prompt to appear + await run.expectText('Allow', 10000); + + // Approve the permission + await run.type('y'); + await run.type('\r'); + + // Wait for command to execute + await run.expectText('test', 10000); + + // Should find the shell command execution + const foundShellCommand = + await rig.waitForToolCall('run_shell_command'); + expect(foundShellCommand).toBeTruthy(); + + // Verify Notification hook executed + const hookLogs = rig.readHookLogs(); + const notificationLog = hookLogs.find( + (log) => + log.hookCall.hook_event_name === 'Notification' && + log.hookCall.hook_name === normalizePath(hookCommand), + ); + + expect(notificationLog).toBeDefined(); + if (notificationLog) { + expect(notificationLog.hookCall.exit_code).toBe(0); + expect(notificationLog.hookCall.stdout).toContain( + 'Permission request logged by security hook', + ); + + // Verify hook input contains notification details + const hookInputStr = + typeof notificationLog.hookCall.hook_input === 'string' + ? notificationLog.hookCall.hook_input + : JSON.stringify(notificationLog.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + + // Should have notification type (uses snake_case) + expect(hookInput['notification_type']).toBe('ToolPermission'); + + // Should have message + expect(hookInput['message']).toBeDefined(); + + // Should have details with tool info + expect(hookInput['details']).toBeDefined(); + const details = hookInput['details'] as Record; + // For 'exec' type confirmations, details contains: type, title, command, rootCommand + expect(details['type']).toBe('exec'); + expect(details['command']).toBeDefined(); + expect(details['title']).toBeDefined(); + } + }); + }); + + describe('Sequential Hook Execution', () => { + it('should execute hooks sequentially when configured', async () => { + rig.setup('should execute hooks sequentially when configured', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.sequential-execution.responses', + ), + }); + + // Create script files for hooks + const hook1Path = rig.createScript( + 'seq_hook1.cjs', + "console.log(JSON.stringify({decision: 'allow', hookSpecificOutput: {hookEventName: 'BeforeAgent', additionalContext: 'Step 1: Initial validation passed.'}}));", + ); + const hook2Path = rig.createScript( + 'seq_hook2.cjs', + "console.log(JSON.stringify({decision: 'allow', hookSpecificOutput: {hookEventName: 'BeforeAgent', additionalContext: 'Step 2: Security check completed.'}}));", + ); + + const hook1Command = `node "${hook1Path}"`; + const hook2Command = `node "${hook2Path}"`; + + rig.setup('should execute hooks sequentially when configured', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeAgent: [ + { + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(hook1Command), + timeout: 5000, + }, + { + type: 'command', + command: normalizePath(hook2Command), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ args: 'Hello, please help me with a task' }); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + // Verify both hooks executed + const hookLogs = rig.readHookLogs(); + const hook1Log = hookLogs.find( + (log) => log.hookCall.hook_name === normalizePath(hook1Command), + ); + const hook2Log = hookLogs.find( + (log) => log.hookCall.hook_name === normalizePath(hook2Command), + ); + + expect(hook1Log).toBeDefined(); + expect(hook1Log?.hookCall.exit_code).toBe(0); + expect(hook1Log?.hookCall.stdout).toContain( + 'Step 1: Initial validation passed', + ); + + expect(hook2Log).toBeDefined(); + expect(hook2Log?.hookCall.exit_code).toBe(0); + expect(hook2Log?.hookCall.stdout).toContain( + 'Step 2: Security check completed', + ); + }); + }); + + describe('Hook Input/Output Validation', () => { + it('should provide correct input format to hooks', async () => { + rig.setup('should provide correct input format to hooks', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.input-validation.responses', + ), + }); + // Create a hook script that validates the input format + const hookScript = `const fs = require('fs'); +const input = fs.readFileSync(0, 'utf-8'); +try { + const json = JSON.parse(input); + // Check fields + if (json.session_id && json.cwd && json.hook_event_name && json.timestamp && json.tool_name && json.tool_input) { + console.log(JSON.stringify({decision: "allow", reason: "Input format is correct"})); + } else { + console.log(JSON.stringify({decision: "block", reason: "Input format is invalid"})); + } +} catch (e) { + console.log(JSON.stringify({decision: "block", reason: "Invalid JSON"})); +}`; + + const scriptPath = rig.createScript( + 'input_validation_hook.cjs', + hookScript, + ); + + rig.setup('should provide correct input format to hooks', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ + args: 'Create a file called input-test.txt with content "test"', + }); + + // Hook should validate input format successfully + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // Check that the file was created (hook allowed it) + const fileContent = rig.readFile('input-test.txt'); + expect(fileContent).toContain('test'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + + it('should treat mixed stdout (text + JSON) as system message and allow execution when exit code is 0', async () => { + rig.setup( + 'should treat mixed stdout (text + JSON) as system message and allow execution when exit code is 0', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.allow-tool.responses', + ), + }, + ); + + // Create script file for hook + const scriptPath = rig.createScript( + 'pollution_hook.cjs', + "console.log('Pollution'); console.log(JSON.stringify({decision: 'deny', reason: 'Should be ignored'}));", + ); + + rig.setup( + 'should treat mixed stdout (text + JSON) as system message and allow execution when exit code is 0', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + // Output plain text then JSON. + // This breaks JSON parsing, so it falls back to 'allow' with the whole stdout as systemMessage. + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const result = await rig.run({ + args: 'Create a file called approved.txt with content "Approved content"', + }); + + // The hook logic fails to parse JSON, so it allows the tool. + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // The entire stdout (including the JSON part) becomes the systemMessage + expect(result).toContain('Pollution'); + expect(result).toContain('Should be ignored'); + }); + }); + + describe('Multiple Event Types', () => { + it('should handle hooks for all major event types', async () => { + rig.setup('should handle hooks for all major event types', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.multiple-events.responses', + ), + }); + + // Create script files for hooks + const btPath = rig.createScript( + 'bt_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'BeforeTool: File operation logged'}));", + ); + const atPath = rig.createScript( + 'at_hook.cjs', + "console.log(JSON.stringify({hookSpecificOutput: {hookEventName: 'AfterTool', additionalContext: 'AfterTool: Operation completed successfully'}}));", + ); + const baPath = rig.createScript( + 'ba_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', hookSpecificOutput: {hookEventName: 'BeforeAgent', additionalContext: 'BeforeAgent: User request processed'}}));", + ); + + const beforeToolCommand = `node "${btPath}"`; + const afterToolCommand = `node "${atPath}"`; + const beforeAgentCommand = `node "${baPath}"`; + + rig.setup('should handle hooks for all major event types', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeAgent: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(beforeAgentCommand), + timeout: 5000, + }, + ], + }, + ], + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(beforeToolCommand), + timeout: 5000, + }, + ], + }, + ], + AfterTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(afterToolCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ + args: + 'Create a file called multi-event-test.txt with content ' + + '"testing multiple events", and then please reply with ' + + 'everything I say just after this:"', + }); + + // Should execute write_file tool + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // File should be created + const fileContent = rig.readFile('multi-event-test.txt'); + expect(fileContent).toContain('testing multiple events'); + + // Result should contain context from all hooks + expect(result).toContain('BeforeTool: File operation logged'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + // Verify all three hooks executed + const hookLogs = rig.readHookLogs(); + const beforeAgentLog = hookLogs.find( + (log) => log.hookCall.hook_name === normalizePath(beforeAgentCommand), + ); + const beforeToolLog = hookLogs.find( + (log) => log.hookCall.hook_name === normalizePath(beforeToolCommand), + ); + const afterToolLog = hookLogs.find( + (log) => log.hookCall.hook_name === normalizePath(afterToolCommand), + ); + + expect(beforeAgentLog).toBeDefined(); + expect(beforeAgentLog?.hookCall.exit_code).toBe(0); + expect(beforeAgentLog?.hookCall.stdout).toContain( + 'BeforeAgent: User request processed', + ); + + expect(beforeToolLog).toBeDefined(); + expect(beforeToolLog?.hookCall.exit_code).toBe(0); + expect(beforeToolLog?.hookCall.stdout).toContain( + 'BeforeTool: File operation logged', + ); + + expect(afterToolLog).toBeDefined(); + expect(afterToolLog?.hookCall.exit_code).toBe(0); + expect(afterToolLog?.hookCall.stdout).toContain( + 'AfterTool: Operation completed successfully', + ); + }); + }); + + describe('Hook Error Handling', () => { + it('should handle hook failures gracefully', async () => { + rig.setup('should handle hook failures gracefully', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.error-handling.responses', + ), + }); + // Create script files for hooks + const failingPath = join(rig.testDir!, 'fail_hook.cjs'); + writeFileSync(failingPath, 'process.exit(1);'); + const workingPath = join(rig.testDir!, 'work_hook.cjs'); + writeFileSync( + workingPath, + "console.log(JSON.stringify({decision: 'allow', reason: 'Working hook succeeded'}));", + ); + + // Failing hook: exits with non-zero code + const failingCommand = `node "${failingPath}"`; + // Working hook: returns success with JSON + const workingCommand = `node "${workingPath}"`; + + rig.setup('should handle hook failures gracefully', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(failingCommand), + timeout: 5000, + }, + { + type: 'command', + command: normalizePath(workingCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ + args: 'Create a file called error-test.txt with content "testing error handling"', + }); + + // Despite one hook failing, the working hook should still allow the operation + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // File should be created + const fileContent = rig.readFile('error-test.txt'); + expect(fileContent).toContain('testing error handling'); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + }); + + describe('Hook Telemetry and Observability', () => { + it('should generate telemetry events for hook executions', async () => { + rig.setup('should generate telemetry events for hook executions', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.telemetry.responses', + ), + }); + + // Create script file for hook + const scriptPath = rig.createScript( + 'telemetry_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', reason: 'Telemetry test hook'}));", + ); + + const hookCommand = `node "${scriptPath}"`; + + rig.setup('should generate telemetry events for hook executions', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + hooks: [ + { + type: 'command', + command: normalizePath(hookCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + await rig.run({ args: 'Create a file called telemetry-test.txt' }); + + // Should execute the tool + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // Should generate hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + }); + }); + + describe('Session Lifecycle Hooks', () => { + it('should fire SessionStart hook on app startup', async () => { + rig.setup('should fire SessionStart hook on app startup', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.session-startup.responses', + ), + }); + + // Create script file for hook + const scriptPath = rig.createScript( + 'session_start_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'Session starting on startup'}));", + ); + + const sessionStartCommand = `node "${scriptPath}"`; + + rig.setup('should fire SessionStart hook on app startup', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + SessionStart: [ + { + matcher: 'startup', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(sessionStartCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Run a simple query - the SessionStart hook will fire during app initialization + await rig.run({ args: 'Say hello' }); + + // Verify hook executed with correct parameters + const hookLogs = rig.readHookLogs(); + const sessionStartLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'SessionStart', + ); + + expect(sessionStartLog).toBeDefined(); + if (sessionStartLog) { + expect(sessionStartLog.hookCall.hook_name).toBe( + normalizePath(sessionStartCommand), + ); + expect(sessionStartLog.hookCall.exit_code).toBe(0); + expect(sessionStartLog.hookCall.hook_input).toBeDefined(); + + // hook_input is a string that needs to be parsed + const hookInputStr = + typeof sessionStartLog.hookCall.hook_input === 'string' + ? sessionStartLog.hookCall.hook_input + : JSON.stringify(sessionStartLog.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + + expect(hookInput['source']).toBe('startup'); + expect(sessionStartLog.hookCall.stdout).toContain( + 'Session starting on startup', + ); + } + }); + + it('should fire SessionStart hook and inject context', async () => { + // Create hook script that outputs JSON with additionalContext + const hookScript = `const fs = require('fs'); +console.log(JSON.stringify({ + decision: 'allow', + systemMessage: 'Context injected via SessionStart hook', + hookSpecificOutput: { + hookEventName: 'SessionStart', + additionalContext: 'If asked about your identity, you must respond with the following: I am a protocol droid that speaks the bacchi.' + } +}));`; + + rig.setup('should fire SessionStart hook and inject context', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.session-startup.responses', + ), + }); + + const scriptPath = rig.createScript( + 'session_start_context_hook.cjs', + hookScript, + ); + + rig.setup('should fire SessionStart hook and inject context', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + SessionStart: [ + { + matcher: 'startup', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Run a query - the SessionStart hook will fire during app initialization + const result = await rig.run({ args: 'Who are you?' }); + + // Check if systemMessage was displayed (in stderr, which rig.run captures) + expect(result).toContain('Context injected via SessionStart hook'); + + // Check if additionalContext influenced the model response + // Note: We use fake responses, but the rig records interactions. + // If we are using fake responses, the model won't actually respond unless we provide a fake response for the injected context. + // But the test rig setup uses 'hooks-system.session-startup.responses'. + // If I'm adding a new test, I might need to generate new fake responses or expect the context to be sent to the model (verify API logs). + + // Verify hook executed + const hookLogs = rig.readHookLogs(); + const sessionStartLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'SessionStart', + ); + + expect(sessionStartLog).toBeDefined(); + + // Verify the API request contained the injected context + // rig.readAllApiRequest() gives us telemetry on API requests. + const apiRequests = rig.readAllApiRequest(); + // We expect at least one API request + expect(apiRequests.length).toBeGreaterThan(0); + + // The injected context should be in the request text + // For non-interactive mode, I prepended it to input: "context\n\ninput" + // The telemetry `request_text` should contain it. + const requestText = apiRequests[0].attributes?.request_text || ''; + expect(requestText).toContain('protocol droid'); + }); + + it('should fire SessionStart hook and display systemMessage in interactive mode', async () => { + // Create hook script that outputs JSON with systemMessage and additionalContext + const hookScript = `const fs = require('fs'); +console.log(JSON.stringify({ + decision: 'allow', + systemMessage: 'Interactive Session Start Message', + hookSpecificOutput: { + hookEventName: 'SessionStart', + additionalContext: 'The user is a Jedi Master.' + } +}));`; + + rig.setup( + 'should fire SessionStart hook and display systemMessage in interactive mode', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.session-startup.responses', + ), + }, + ); + + const scriptPath = rig.createScript( + 'session_start_interactive_hook.cjs', + hookScript, + ); + + rig.setup( + 'should fire SessionStart hook and display systemMessage in interactive mode', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + SessionStart: [ + { + matcher: 'startup', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const run = await rig.runInteractive(); + + // Verify systemMessage is displayed + await run.expectText('Interactive Session Start Message', 10000); + + // Send a prompt to establish a session and trigger an API call + await run.sendKeys('Hello'); + await run.type('\r'); + + // Wait for response to ensure API call happened + await run.expectText('Hello', 15000); + + // Wait for telemetry to be written to disk + await rig.waitForTelemetryReady(); + + // Verify the API request contained the injected context + // We may need to poll for API requests as they are written asynchronously + const pollResult = await poll( + () => { + const apiRequests = rig.readAllApiRequest(); + return apiRequests.length > 0; + }, + 15000, + 500, + ); + + expect(pollResult).toBe(true); + + const apiRequests = rig.readAllApiRequest(); + // The injected context should be in the request_text of the API request + const requestText = apiRequests[0].attributes?.request_text || ''; + expect(requestText).toContain('Jedi Master'); + }); + + it('should fire SessionEnd and SessionStart hooks on /clear command', async () => { + rig.setup( + 'should fire SessionEnd and SessionStart hooks on /clear command', + { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.session-clear.responses', + ), + }, + ); + + // Create script files for hooks + const endScriptPath = rig.createScript( + 'session_end_clear.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'Session ending due to clear'}));", + ); + const startScriptPath = rig.createScript( + 'session_start_clear.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'Session starting after clear'}));", + ); + + const sessionEndCommand = `node "${endScriptPath}"`; + const sessionStartCommand = `node "${startScriptPath}"`; + + rig.setup( + 'should fire SessionEnd and SessionStart hooks on /clear command', + { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + SessionEnd: [ + { + matcher: '*', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(sessionEndCommand), + timeout: 5000, + }, + ], + }, + ], + SessionStart: [ + { + matcher: '*', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(sessionStartCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }, + ); + + const run = await rig.runInteractive(); + + // Send an initial prompt to establish a session + await run.sendKeys('Say hello'); + await run.type('\r'); + + // Wait for the response + await run.expectText('Hello', 10000); + + // Execute /clear command multiple times to generate more hook events + // This makes the test more robust by creating multiple start/stop cycles + const numClears = 3; + for (let i = 0; i < numClears; i++) { + await run.sendKeys('/clear'); + await run.type('\r'); + + // Wait a bit for clear to complete + await new Promise((resolve) => setTimeout(resolve, 2000)); + + // Send a prompt to establish an active session before next clear + await run.sendKeys('Say hello'); + await run.type('\r'); + + // Wait for response + await run.expectText('Hello', 10000); + } + + // Wait for all clears to complete + // BatchLogRecordProcessor exports telemetry every 10 seconds by default + // Use generous wait time across all platforms (CI, Docker, Mac, Linux) + await new Promise((resolve) => setTimeout(resolve, 15000)); + + // Wait for telemetry to be written to disk + await rig.waitForTelemetryReady(); + + // Wait for hook telemetry events to be flushed to disk + // In interactive mode, telemetry may be buffered, so we need to poll for the events + // We execute multiple clears to generate more hook events (total: 1 + numClears * 2) + // But we only require >= 1 hooks to pass, making the test more permissive + const expectedMinHooks = 1; // SessionStart (startup), SessionEnd (clear), SessionStart (clear) + const pollResult = await poll( + () => { + const hookLogs = rig.readHookLogs(); + return hookLogs.length >= expectedMinHooks; + }, + 90000, // 90 second timeout for all platforms + 1000, // check every 1s to reduce I/O overhead + ); + + // If polling failed, log diagnostic info + if (!pollResult) { + const hookLogs = rig.readHookLogs(); + const hookEvents = hookLogs.map( + (log) => log.hookCall.hook_event_name, + ); + console.error( + `Polling timeout after 90000ms: Expected >= ${expectedMinHooks} hooks, got ${hookLogs.length}`, + ); + console.error( + 'Hooks found:', + hookEvents.length > 0 ? hookEvents.join(', ') : 'NONE', + ); + console.error('Full hook logs:', JSON.stringify(hookLogs, null, 2)); + } + + // Verify hooks executed + const hookLogs = rig.readHookLogs(); + + // Diagnostic: Log which hooks we actually got + const hookEvents = hookLogs.map((log) => log.hookCall.hook_event_name); + if (hookLogs.length < expectedMinHooks) { + console.error( + `TEST FAILURE: Expected >= ${expectedMinHooks} hooks, got ${hookLogs.length}: [${hookEvents.length > 0 ? hookEvents.join(', ') : 'NONE'}]`, + ); + } + + expect(hookLogs.length).toBeGreaterThanOrEqual(expectedMinHooks); + + // Find SessionEnd hook log + const sessionEndLog = hookLogs.find( + (log) => + log.hookCall.hook_event_name === 'SessionEnd' && + log.hookCall.hook_name === normalizePath(sessionEndCommand), + ); + // Because the flakiness of the test, we relax this check + // expect(sessionEndLog).toBeDefined(); + if (sessionEndLog) { + expect(sessionEndLog.hookCall.exit_code).toBe(0); + expect(sessionEndLog.hookCall.stdout).toContain( + 'Session ending due to clear', + ); + + // Verify hook input contains reason + const hookInputStr = + typeof sessionEndLog.hookCall.hook_input === 'string' + ? sessionEndLog.hookCall.hook_input + : JSON.stringify(sessionEndLog.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + expect(hookInput['reason']).toBe('clear'); + } + + // Find SessionStart hook log after clear + const sessionStartAfterClearLogs = hookLogs.filter( + (log) => + log.hookCall.hook_event_name === 'SessionStart' && + log.hookCall.hook_name === normalizePath(sessionStartCommand), + ); + // Should have at least one SessionStart from after clear + // Because the flakiness of the test, we relax this check + // expect(sessionStartAfterClearLogs.length).toBeGreaterThanOrEqual(1); + + const sessionStartLog = sessionStartAfterClearLogs.find((log) => { + const hookInputStr = + typeof log.hookCall.hook_input === 'string' + ? log.hookCall.hook_input + : JSON.stringify(log.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + return hookInput['source'] === 'clear'; + }); + + // Because the flakiness of the test, we relax this check + // expect(sessionStartLog).toBeDefined(); + if (sessionStartLog) { + expect(sessionStartLog.hookCall.exit_code).toBe(0); + expect(sessionStartLog.hookCall.stdout).toContain( + 'Session starting after clear', + ); + } + }); + }); + + describe('Compression Hooks', () => { + it('should fire PreCompress hook on automatic compression', async () => { + rig.setup('should fire PreCompress hook on automatic compression', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.compress-auto.responses', + ), + }); + + // Create script file for hook + const scriptPath = rig.createScript( + 'pre_compress_hook.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'PreCompress hook executed for automatic compression'}));", + ); + + const preCompressCommand = `node "${scriptPath}"`; + + rig.setup('should fire PreCompress hook on automatic compression', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + PreCompress: [ + { + matcher: 'auto', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(preCompressCommand), + timeout: 5000, + }, + ], + }, + ], + }, + // Configure automatic compression with a very low threshold + // This will trigger auto-compression after the first response + contextCompression: { + // enabled: true, + targetTokenCount: 10, // Very low threshold to trigger compression + }, + }, + }); + + // Run a simple query that will trigger automatic compression + await rig.run({ args: 'Say hello in exactly 5 words' }); + + // Verify hook executed with correct parameters + const hookLogs = rig.readHookLogs(); + const preCompressLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'PreCompress', + ); + + expect(preCompressLog).toBeDefined(); + if (preCompressLog) { + expect(preCompressLog.hookCall.hook_name).toBe( + normalizePath(preCompressCommand), + ); + expect(preCompressLog.hookCall.exit_code).toBe(0); + expect(preCompressLog.hookCall.hook_input).toBeDefined(); + + // hook_input is a string that needs to be parsed + const hookInputStr = + typeof preCompressLog.hookCall.hook_input === 'string' + ? preCompressLog.hookCall.hook_input + : JSON.stringify(preCompressLog.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + + expect(hookInput['trigger']).toBe('auto'); + expect(preCompressLog.hookCall.stdout).toContain( + 'PreCompress hook executed for automatic compression', + ); + } + }); + }); + + describe('SessionEnd on Exit', () => { + it('should fire SessionEnd hook on graceful exit in non-interactive mode', async () => { + rig.setup('should fire SessionEnd hook on graceful exit', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.session-startup.responses', + ), + }); + + // Create script file for hook + const scriptPath = rig.createScript( + 'session_end_exit.cjs', + "console.log(JSON.stringify({decision: 'allow', systemMessage: 'SessionEnd hook executed on exit'}));", + ); + + const sessionEndCommand = `node "${scriptPath}"`; + + rig.setup('should fire SessionEnd hook on graceful exit', { + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + SessionEnd: [ + { + matcher: 'exit', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(sessionEndCommand), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Run in non-interactive mode with a simple prompt + await rig.run({ args: 'Hello' }); + + // The process should exit gracefully, firing the SessionEnd hook + // Wait for telemetry to be written to disk + await rig.waitForTelemetryReady(); + + // Poll for the hook log to appear + const isCI = process.env['CI'] === 'true'; + const pollTimeout = isCI ? 30000 : 10000; + const pollResult = await poll( + () => { + const hookLogs = rig.readHookLogs(); + return hookLogs.some( + (log) => log.hookCall.hook_event_name === 'SessionEnd', + ); + }, + pollTimeout, + 200, + ); + + if (!pollResult) { + const hookLogs = rig.readHookLogs(); + console.error( + 'Polling timeout: Expected SessionEnd hook, got:', + JSON.stringify(hookLogs, null, 2), + ); + } + + expect(pollResult).toBe(true); + + const hookLogs = rig.readHookLogs(); + const sessionEndLog = hookLogs.find( + (log) => log.hookCall.hook_event_name === 'SessionEnd', + ); + + expect(sessionEndLog).toBeDefined(); + if (sessionEndLog) { + expect(sessionEndLog.hookCall.hook_name).toBe( + normalizePath(sessionEndCommand), + ); + expect(sessionEndLog.hookCall.exit_code).toBe(0); + expect(sessionEndLog.hookCall.hook_input).toBeDefined(); + + const hookInputStr = + typeof sessionEndLog.hookCall.hook_input === 'string' + ? sessionEndLog.hookCall.hook_input + : JSON.stringify(sessionEndLog.hookCall.hook_input); + const hookInput = JSON.parse(hookInputStr) as Record; + + expect(hookInput['reason']).toBe('exit'); + expect(sessionEndLog.hookCall.stdout).toContain( + 'SessionEnd hook executed', + ); + } + }); + }); + + describe('Hook Disabling', () => { + it('should not execute hooks disabled in settings file', async () => { + const enabledMsg = 'EXECUTION_ALLOWED_BY_HOOK_A'; + const disabledMsg = 'EXECUTION_BLOCKED_BY_HOOK_B'; + + const enabledJson = JSON.stringify({ + decision: 'allow', + systemMessage: enabledMsg, + }); + const disabledJson = JSON.stringify({ + decision: 'block', + reason: disabledMsg, + }); + + const enabledScript = `console.log(JSON.stringify(${enabledJson}));`; + const disabledScript = `console.log(JSON.stringify(${disabledJson}));`; + const enabledFilename = 'enabled_hook.js'; + const disabledFilename = 'disabled_hook.js'; + const enabledCmd = `node ${enabledFilename}`; + const disabledCmd = `node ${disabledFilename}`; + + // 3. Final setup with full settings + rig.setup('Hook Disabling Settings', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.disabled-via-settings.responses', + ), + settings: { + hooksConfig: { + enabled: true, + disabled: ['hook-b'], + }, + hooks: { + BeforeTool: [ + { + hooks: [ + { + type: 'command', + name: 'hook-a', + command: enabledCmd, + timeout: 60000, + }, + { + type: 'command', + name: 'hook-b', + command: disabledCmd, + timeout: 60000, + }, + ], + }, + ], + }, + }, + }); + + rig.createScript(enabledFilename, enabledScript); + rig.createScript(disabledFilename, disabledScript); + + await rig.run({ + args: 'Create a file called disabled-test.txt with content "test"', + }); + + // Tool should execute (enabled hook allows it) + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // Check hook telemetry - only enabled hook should have executed + const hookLogs = rig.readHookLogs(); + const enabledHookLog = hookLogs.find((log) => + JSON.stringify(log.hookCall.hook_output).includes(enabledMsg), + ); + const disabledHookLog = hookLogs.find((log) => + JSON.stringify(log.hookCall.hook_output).includes(disabledMsg), + ); + + expect(enabledHookLog).toBeDefined(); + expect(disabledHookLog).toBeUndefined(); + }); + + it('should respect disabled hooks across multiple operations', async () => { + const activeMsg = 'MULTIPLE_OPS_ENABLED_HOOK'; + const disabledMsg = 'MULTIPLE_OPS_DISABLED_HOOK'; + + const activeJson = JSON.stringify({ + decision: 'allow', + systemMessage: activeMsg, + }); + const disabledJson = JSON.stringify({ + decision: 'block', + reason: disabledMsg, + }); + + const activeScript = `console.log(JSON.stringify(${activeJson}));`; + const disabledScript = `console.log(JSON.stringify(${disabledJson}));`; + const activeFilename = 'active_hook.js'; + const disabledFilename = 'disabled_hook.js'; + const activeCmd = `node ${activeFilename}`; + const disabledCmd = `node ${disabledFilename}`; + + // 3. Final setup with full settings + rig.setup('Hook Disabling Multiple Ops', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.disabled-via-command.responses', + ), + settings: { + hooksConfig: { + enabled: true, + disabled: ['multi-hook-disabled'], + }, + hooks: { + BeforeTool: [ + { + hooks: [ + { + type: 'command', + name: 'multi-hook-active', + command: activeCmd, + timeout: 60000, + }, + { + type: 'command', + name: 'multi-hook-disabled', + command: disabledCmd, + timeout: 60000, + }, + ], + }, + ], + }, + }, + }); + + rig.createScript(activeFilename, activeScript); + rig.createScript(disabledFilename, disabledScript); + + // First run - only active hook should execute + await rig.run({ + args: 'Create a file called first-run.txt with "test1"', + }); + + // Tool should execute (active hook allows it) + const foundWriteFile1 = await rig.waitForToolCall('write_file'); + expect(foundWriteFile1).toBeTruthy(); + + // Check hook telemetry - only active hook should have executed + const hookLogs1 = rig.readHookLogs(); + const activeHookLog1 = hookLogs1.find((log) => + JSON.stringify(log.hookCall.hook_output).includes(activeMsg), + ); + const disabledHookLog1 = hookLogs1.find((log) => + JSON.stringify(log.hookCall.hook_output).includes(disabledMsg), + ); + + expect(activeHookLog1).toBeDefined(); + expect(disabledHookLog1).toBeUndefined(); + + // Second run - verify disabled hook stays disabled + await rig.run({ + args: 'Create a file called second-run.txt with "test2"', + }); + + const foundWriteFile2 = await rig.waitForToolCall('write_file'); + expect(foundWriteFile2).toBeTruthy(); + + // Verify disabled hook still hasn't executed + const hookLogs2 = rig.readHookLogs(); + const disabledHookLog2 = hookLogs2.find((log) => + JSON.stringify(log.hookCall.hook_output).includes(disabledMsg), + ); + expect(disabledHookLog2).toBeUndefined(); + }); + }); + + describe('BeforeTool Hooks - Input Override', () => { + it('should override tool input parameters via BeforeTool hook', async () => { + // 1. First setup to get the test directory and prepare the hook script + rig.setup('should override tool input parameters via BeforeTool hook'); + + // Create a hook script that overrides the tool input + const hookOutput = { + decision: 'allow', + hookSpecificOutput: { + hookEventName: 'BeforeTool', + tool_input: { + file_path: 'modified.txt', + content: 'modified content', + }, + }, + }; + + const hookScript = `process.stdout.write(JSON.stringify(${JSON.stringify( + hookOutput, + )}));`; + + const scriptPath = rig.createScript( + 'input_override_hook.js', + hookScript, + ); + + // 2. Full setup with settings and fake responses + rig.setup('should override tool input parameters via BeforeTool hook', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.input-modification.responses', + ), + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Run the agent. The fake response will attempt to call write_file with + // file_path="original.txt" and content="original content" + await rig.run({ + args: 'Create a file called original.txt with content "original content"', + }); + + // 1. Verify that 'modified.txt' was created with 'modified content' (Override successful) + const modifiedContent = rig.readFile('modified.txt'); + expect(modifiedContent).toBe('modified content'); + + // 2. Verify that 'original.txt' was NOT created (Override replaced original) + let originalExists = false; + try { + rig.readFile('original.txt'); + originalExists = true; + } catch { + originalExists = false; + } + expect(originalExists).toBe(false); + + // 3. Verify hook telemetry + const hookTelemetryFound = await rig.waitForTelemetryEvent('hook_call'); + expect(hookTelemetryFound).toBeTruthy(); + + const hookLogs = rig.readHookLogs(); + expect(hookLogs.length).toBe(1); + expect(hookLogs[0].hookCall.hook_name).toContain( + 'input_override_hook.js', + ); + + // 4. Verify that the agent didn't try to work-around the hook input change + const toolLogs = rig.readToolLogs(); + expect(toolLogs.length).toBe(1); + expect(toolLogs[0].toolRequest.name).toBe('write_file'); + expect(JSON.parse(toolLogs[0].toolRequest.args).file_path).toBe( + 'modified.txt', + ); + }); + }); + + describe('BeforeTool Hooks - Stop Execution', () => { + it('should stop agent execution via BeforeTool hook', async () => { + // Create a hook script that stops execution + const hookOutput = { + continue: false, + reason: 'Emergency Stop triggered by hook', + hookSpecificOutput: { + hookEventName: 'BeforeTool', + }, + }; + + const hookScript = `console.log(JSON.stringify(${JSON.stringify( + hookOutput, + )}));`; + + rig.setup('should stop agent execution via BeforeTool hook'); + const scriptPath = rig.createScript( + 'before_tool_stop_hook.js', + hookScript, + ); + + rig.setup('should stop agent execution via BeforeTool hook', { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.before-tool-stop.responses', + ), + settings: { + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + sequential: true, + hooks: [ + { + type: 'command', + command: normalizePath(`node "${scriptPath}"`), + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + const result = await rig.run({ + args: 'Use write_file to create test.txt', + }); + + // The hook should have stopped execution message (returned from tool) + expect(result).toContain( + 'Agent execution stopped by hook: Emergency Stop triggered by hook', + ); + + // Tool should NOT be called successfully (it was blocked/stopped) + const toolLogs = rig.readToolLogs(); + const writeFileCalls = toolLogs.filter( + (t) => + t.toolRequest.name === 'write_file' && + t.toolRequest.success === true, + ); + expect(writeFileCalls).toHaveLength(0); + }); + }); + + describe('Hooks "ask" Decision Integration', () => { + it( + 'should force confirmation prompt when hook returns "ask" decision even in YOLO mode', + { timeout: 60000 }, + async () => { + const testName = + 'should force confirmation prompt when hook returns "ask" decision even in YOLO mode'; + + // 1. Setup hook script that returns 'ask' decision + const hookOutput = { + decision: 'ask', + systemMessage: 'Confirmation forced by security hook', + hookSpecificOutput: { + hookEventName: 'BeforeTool', + }, + }; + + const hookScript = `console.log(JSON.stringify(${JSON.stringify( + hookOutput, + )}));`; + + // Create script path predictably + const scriptPath = join(os.tmpdir(), 'gemini-cli-tests-ask-hook.js'); + writeFileSync(scriptPath, hookScript); + + // 2. Setup rig with YOLO mode enabled but with the 'ask' hook + rig.setup(testName, { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.allow-tool.responses', + ), + settings: { + debugMode: true, + tools: { + approval: 'yolo', + }, + general: { + enableAutoUpdateNotification: false, + }, + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + hooks: [ + { + type: 'command', + command: `node "${scriptPath}"`, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Bypass terminal setup prompt and other startup banners + const stateDir = join(rig.homeDir!, '.gemini'); + if (!existsSync(stateDir)) mkdirSync(stateDir, { recursive: true }); + writeFileSync( + join(stateDir, 'state.json'), + JSON.stringify({ + terminalSetupPromptShown: true, + hasSeenScreenReaderNudge: true, + tipsShown: 100, + }), + ); + + // 3. Run interactive and verify prompt appears despite YOLO mode + const run = await rig.runInteractive(); + + // Wait for prompt to appear + await run.expectText('Type your message', 30000); + + // Send prompt that will trigger write_file + await run.type( + 'Create a file called ask-test.txt with content "test"', + ); + await run.type('\r'); + + // Wait for the FORCED confirmation prompt to appear + // It should contain the system message from the hook + await run.expectText('Confirmation forced by security hook', 30000); + await run.expectText('Allow', 5000); + + // 4. Approve the permission + await run.type('y'); + await run.type('\r'); + + // Wait for command to execute + await run.expectText('approved.txt', 30000); + + // Should find the tool call + const foundWriteFile = await rig.waitForToolCall('write_file'); + expect(foundWriteFile).toBeTruthy(); + + // File should be created + const fileContent = rig.readFile('approved.txt'); + expect(fileContent).toBe('Approved content'); + }, + ); + + it( + 'should allow cancelling when hook forces "ask" decision', + { timeout: 60000 }, + async () => { + const testName = + 'should allow cancelling when hook forces "ask" decision'; + const hookOutput = { + decision: 'ask', + systemMessage: 'Confirmation forced for cancellation test', + hookSpecificOutput: { + hookEventName: 'BeforeTool', + }, + }; + + const hookScript = `console.log(JSON.stringify(${JSON.stringify( + hookOutput, + )}));`; + + const scriptPath = join( + os.tmpdir(), + 'gemini-cli-tests-ask-cancel-hook.js', + ); + writeFileSync(scriptPath, hookScript); + + rig.setup(testName, { + fakeResponsesPath: join( + import.meta.dirname, + 'hooks-system.allow-tool.responses', + ), + settings: { + debugMode: true, + tools: { + approval: 'yolo', + }, + general: { + enableAutoUpdateNotification: false, + }, + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'write_file', + hooks: [ + { + type: 'command', + command: `node "${scriptPath}"`, + timeout: 5000, + }, + ], + }, + ], + }, + }, + }); + + // Bypass terminal setup prompt and other startup banners + const stateDir = join(rig.homeDir!, '.gemini'); + if (!existsSync(stateDir)) mkdirSync(stateDir, { recursive: true }); + writeFileSync( + join(stateDir, 'state.json'), + JSON.stringify({ + terminalSetupPromptShown: true, + hasSeenScreenReaderNudge: true, + tipsShown: 100, + }), + ); + + const run = await rig.runInteractive(); + + // Wait for prompt to appear + await run.expectText('Type your message', 30000); + + await run.type( + 'Create a file called cancel-test.txt with content "test"', + ); + await run.type('\r'); + + await run.expectText( + 'Confirmation forced for cancellation test', + 30000, + ); + + // 4. Deny the permission using option 4 + await run.type('4'); + await run.type('\r'); + + // Wait for cancellation message + await run.expectText('Cancelled', 15000); + + // Tool should NOT be called successfully + const toolLogs = rig.readToolLogs(); + const writeFileCalls = toolLogs.filter( + (t) => + t.toolRequest.name === 'write_file' && + t.toolRequest.success === true, + ); + expect(writeFileCalls).toHaveLength(0); + }, + ); + }); + }, +); diff --git a/integration-tests/json-output.france.responses b/integration-tests/json-output.france.responses new file mode 100644 index 0000000000000000000000000000000000000000..5c9edce8884957bcfdd1f915383826f152563abf --- /dev/null +++ b/integration-tests/json-output.france.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The capital of France is Paris."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":7,"candidatesTokenCount":7,"totalTokenCount":14,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7}]}}]} diff --git a/integration-tests/json-output.session-id.responses b/integration-tests/json-output.session-id.responses new file mode 100644 index 0000000000000000000000000000000000000000..c96cbccea4fca5d7712096b59f82cadaa520eb6a --- /dev/null +++ b/integration-tests/json-output.session-id.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! How can I help you today?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":9,"totalTokenCount":14,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]} \ No newline at end of file diff --git a/integration-tests/mcp-resources.test.ts b/integration-tests/mcp-resources.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..ac04e36e38142b3df6bc6d961b26b38092dfb47d --- /dev/null +++ b/integration-tests/mcp-resources.test.ts @@ -0,0 +1,178 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import fs from 'node:fs'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); + +describe('mcp-resources-integration', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should list mcp resources', async () => { + await rig.setup('mcp-list-resources-test', { + settings: { + model: { + name: 'gemini-3-flash-preview', + }, + }, + fakeResponsesPath: join(__dirname, 'mcp-list-resources.responses'), + }); + + // Workaround for ProjectRegistry save issue + const userGeminiDir = join(rig.homeDir!, '.gemini'); + fs.writeFileSync(join(userGeminiDir, 'projects.json'), '{"projects":{}}'); + + // Add a dummy server to get setup done + rig.addTestMcpServer('resource-server', { + name: 'resource-server', + tools: [], + }); + + // Overwrite the script with resource support + const scriptPath = join(rig.testDir!, 'test-mcp-resource-server.mjs'); + const scriptContent = ` +import { Server } from '@modelcontextprotocol/sdk/server/index.js'; +import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; +import { + ListResourcesRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +const server = new Server( + { + name: 'resource-server', + version: '1.0.0', + }, + { + capabilities: { + resources: {}, + }, + }, +); + +server.setRequestHandler(ListResourcesRequestSchema, async () => { + return { + resources: [ + { + uri: 'test://resource1', + name: 'Resource 1', + mimeType: 'text/plain', + description: 'A test resource', + } + ], + }; +}); + +const transport = new StdioServerTransport(); +await server.connect(transport); +`; + fs.writeFileSync(scriptPath, scriptContent); + + const output = await rig.run({ + args: 'List all available MCP resources.', + env: { GEMINI_API_KEY: 'dummy' }, + }); + + const foundCall = await rig.waitForToolCall('list_mcp_resources'); + expect(foundCall).toBeTruthy(); + expect(output).toContain('test://resource1'); + }, 60000); + + it('should read mcp resource', async () => { + await rig.setup('mcp-read-resource-test', { + settings: { + model: { + name: 'gemini-3-flash-preview', + }, + }, + fakeResponsesPath: join(__dirname, 'mcp-read-resource.responses'), + }); + + // Workaround for ProjectRegistry save issue + const userGeminiDir = join(rig.homeDir!, '.gemini'); + fs.writeFileSync(join(userGeminiDir, 'projects.json'), '{"projects":{}}'); + + // Add a dummy server to get setup done + rig.addTestMcpServer('resource-server', { + name: 'resource-server', + tools: [], + }); + + // Overwrite the script with resource support + const scriptPath = join(rig.testDir!, 'test-mcp-resource-server.mjs'); + const scriptContent = ` +import { Server } from '@modelcontextprotocol/sdk/server/index.js'; +import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; +import { + ListResourcesRequestSchema, + ReadResourceRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +const server = new Server( + { + name: 'resource-server', + version: '1.0.0', + }, + { + capabilities: { + resources: {}, + }, + }, +); + +// Need to provide list resources so the tool is active! +server.setRequestHandler(ListResourcesRequestSchema, async () => { + return { + resources: [ + { + uri: 'test://resource1', + name: 'Resource 1', + mimeType: 'text/plain', + description: 'A test resource', + } + ], + }; +}); + +server.setRequestHandler(ReadResourceRequestSchema, async (request) => { + if (request.params.uri === 'test://resource1') { + return { + contents: [ + { + uri: 'test://resource1', + mimeType: 'text/plain', + text: 'This is the content of resource 1', + } + ], + }; + } + throw new Error('Resource not found'); +}); + +const transport = new StdioServerTransport(); +await server.connect(transport); +`; + fs.writeFileSync(scriptPath, scriptContent); + + const output = await rig.run({ + args: 'Read the MCP resource test://resource1.', + env: { GEMINI_API_KEY: 'dummy' }, + }); + + const foundCall = await rig.waitForToolCall('read_mcp_resource'); + expect(foundCall).toBeTruthy(); + expect(output).toContain('content of resource 1'); + }, 60000); +}); diff --git a/integration-tests/mcp_server_cyclic_schema.test.ts b/integration-tests/mcp_server_cyclic_schema.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..29373dbac44f5bfbb1d88e2d2a64a4aa0c08da41 --- /dev/null +++ b/integration-tests/mcp_server_cyclic_schema.test.ts @@ -0,0 +1,207 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * This test verifies we can provide MCP tools with recursive input schemas + * (in JSON, using the $ref keyword) and both the GenAI SDK and the Gemini + * API calls succeed. Note that prior to + * https://github.com/googleapis/js-genai/commit/36f6350705ecafc47eaea3f3eecbcc69512edab7#diff-fdde9372aec859322b7c5a5efe467e0ad25a57210c7229724586ee90ea4f5a30 + * the Gemini API call would fail for such tools because the schema was + * passed not as a JSON string but using the Gemini API's tool parameter + * schema object which has stricter typing and recursion restrictions. + * If this test fails, it's likely because either the GenAI SDK or Gemini API + * has become more restrictive about the type of tool parameter schemas that + * are accepted. If this occurs: Gemini CLI previously attempted to detect + * such tools and proactively remove them from the set of tools provided in + * the Gemini API call (as FunctionDeclaration objects). It may be appropriate + * to resurrect that behavior but note that it's difficult to keep the + * GCLI filters in sync with the Gemini API restrictions and behavior. + */ + +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, it, afterEach, beforeEach } from 'vitest'; +import { TestRig } from './test-helper.js'; + +// Create a minimal MCP server that doesn't require external dependencies +// This implements the MCP protocol directly using Node.js built-ins +const serverScript = `#!/usr/bin/env node +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +const readline = require('readline'); +const fs = require('fs'); + +// Debug logging to stderr (only when MCP_DEBUG or VERBOSE is set) +const debugEnabled = process.env['MCP_DEBUG'] === 'true' || process.env['VERBOSE'] === 'true'; +function debug(msg) { + if (debugEnabled) { + fs.writeSync(2, \`[MCP-DEBUG] \${msg}\\n\`); + } +} + +debug('MCP server starting...'); + +// Simple JSON-RPC implementation for MCP +class SimpleJSONRPC { + constructor() { + this.handlers = new Map(); + this.rl = readline.createInterface({ + input: process.stdin, + output: process.stdout, + terminal: false + }); + + this.rl.on('line', (line) => { + debug(\`Received line: \${line}\`); + try { + const message = JSON.parse(line); + debug(\`Parsed message: \${JSON.stringify(message)}\`); + this.handleMessage(message); + } catch (e) { + debug(\`Parse error: \${e.message}\`); + } + }); + } + + send(message) { + const msgStr = JSON.stringify(message); + debug(\`Sending message: \${msgStr}\`); + process.stdout.write(msgStr + '\\n'); + } + + async handleMessage(message) { + if (message.method && this.handlers.has(message.method)) { + try { + const result = await this.handlers.get(message.method)(message.params || {}); + if (message.id !== undefined) { + this.send({ + jsonrpc: '2.0', + id: message.id, + result + }); + } + } catch (error) { + if (message.id !== undefined) { + this.send({ + jsonrpc: '2.0', + id: message.id, + error: { + code: -32603, + message: error.message + } + }); + } + } + } else if (message.id !== undefined) { + this.send({ + jsonrpc: '2.0', + id: message.id, + error: { + code: -32601, + message: 'Method not found' + } + }); + } + } + + on(method, handler) { + this.handlers.set(method, handler); + } +} + +// Create MCP server +const rpc = new SimpleJSONRPC(); + +// Handle initialize +rpc.on('initialize', async (params) => { + debug('Handling initialize request'); + return { + protocolVersion: '2024-11-05', + capabilities: { + tools: {} + }, + serverInfo: { + name: 'cyclic-schema-server', + version: '1.0.0' + } + }; +}); + +// Handle tools/list +rpc.on('tools/list', async () => { + debug('Handling tools/list request'); + return { + tools: [{ + name: 'tool_with_cyclic_schema', + inputSchema: { + type: 'object', + properties: { + data: { + type: 'array', + items: { + type: 'object', + properties: { + child: { $ref: '#/properties/data/items' }, + }, + }, + }, + }, + } + }] + }; +}); + +// Send initialization notification +rpc.send({ + jsonrpc: '2.0', + method: 'initialized' +}); +`; + +describe('mcp server with cyclic tool schema is detected', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('mcp tool list should include tool with cyclic tool schema', async () => { + // Setup test directory with MCP server configuration + await rig.setup('cyclic-schema-mcp-server', { + settings: { + mcpServers: { + 'cyclic-schema-server': { + command: 'node', + args: ['mcp-server.cjs'], + }, + }, + }, + }); + + // Create server script in the test directory + const testServerPath = join(rig.testDir!, 'mcp-server.cjs'); + writeFileSync(testServerPath, serverScript); + + // Make the script executable (though running with 'node' should work anyway) + if (process.platform !== 'win32') { + const { chmodSync } = await import('node:fs'); + chmodSync(testServerPath, 0o755); + } + + const run = await rig.runInteractive(); + + await run.type('/mcp list'); + await run.type('\r'); + + await run.expectText('tool_with_cyclic_schema'); + }); +}); diff --git a/integration-tests/mixed-input-crash.test.ts b/integration-tests/mixed-input-crash.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..2f862e8a488548a641295f1970f326bdb0cd2965 --- /dev/null +++ b/integration-tests/mixed-input-crash.test.ts @@ -0,0 +1,65 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; + +describe('mixed input crash prevention', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should not crash when using mixed prompt inputs', async () => { + rig.setup('should not crash when using mixed prompt inputs'); + + // Test: echo "say '1'." | gemini --prompt-interactive="say '2'." say '3'. + const stdinContent = "say '1'."; + + try { + await rig.run({ + args: ['--prompt-interactive', "say '2'.", "say '3'."], + stdin: stdinContent, + }); + throw new Error('Expected the command to fail, but it succeeded'); + } catch (error: unknown) { + expect(error).toBeInstanceOf(Error); + const err = error as Error; + + expect(err.message).toContain('Process exited with code 42'); + expect(err.message).toContain( + '--prompt-interactive flag cannot be used when input is piped', + ); + expect(err.message).not.toContain('setRawMode is not a function'); + expect(err.message).not.toContain('unexpected critical error'); + } + + const lastRequest = rig.readLastApiRequest(); + expect(lastRequest).toBeNull(); + }); + + it('should provide clear error message for mixed input', async () => { + rig.setup('should provide clear error message for mixed input'); + + try { + await rig.run({ + args: ['--prompt-interactive', 'test prompt'], + stdin: 'test input', + }); + throw new Error('Expected the command to fail, but it succeeded'); + } catch (error: unknown) { + expect(error).toBeInstanceOf(Error); + const err = error as Error; + + expect(err.message).toContain( + '--prompt-interactive flag cannot be used when input is piped', + ); + } + }); +}); diff --git a/integration-tests/parallel-tools.responses b/integration-tests/parallel-tools.responses new file mode 100644 index 0000000000000000000000000000000000000000..41c3780991f9458b79c50a943a49a7bd05fac3fe --- /dev/null +++ b/integration-tests/parallel-tools.responses @@ -0,0 +1,3 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"{\n \"reasoning\": \"Simple task.\",\n \"model_choice\": \"flash\"\n}"}]},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"read_file","args":{"file_path":"file1.txt"}}},{"functionCall":{"name":"read_file","args":{"file_path":"file2.txt"}}},{"functionCall":{"name":"write_file","args":{"file_path":"output.txt","content":"wave2"}}},{"functionCall":{"name":"read_file","args":{"file_path":"file3.txt"}}},{"functionCall":{"name":"read_file","args":{"file_path":"file4.txt"}}}, {"text":"All waves completed successfully."}]},"finishReason":"STOP","index":0}]}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"All waves completed successfully."}]},"finishReason":"STOP","index":0}]}} \ No newline at end of file diff --git a/integration-tests/parallel-tools.test.ts b/integration-tests/parallel-tools.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..9cd6068db35944a58f8a3cb61ddd9592cdd4b267 --- /dev/null +++ b/integration-tests/parallel-tools.test.ts @@ -0,0 +1,83 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { join } from 'node:path'; +import fs from 'node:fs'; + +describe('Parallel Tool Execution Integration', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it('should execute [read, read, write, read, read] in correct waves with user approval', async () => { + rig.setup('parallel-wave-execution', { + fakeResponsesPath: join(import.meta.dirname, 'parallel-tools.responses'), + fakeResponsesNonStrict: true, + settings: { + tools: { + core: ['read_file', 'write_file'], + approval: 'ASK', // Disable YOLO mode to show permission prompts + confirmationRequired: ['write_file'], + }, + }, + }); + + rig.createFile('file1.txt', 'c1'); + rig.createFile('file2.txt', 'c2'); + rig.createFile('file3.txt', 'c3'); + rig.createFile('file4.txt', 'c4'); + rig.sync(); + + const run = await rig.runInteractive({ approvalMode: 'default' }); + + try { + // 1. Trigger the wave + await run.type('ok'); + await run.type('\r'); + + // 3. Wait for the write_file prompt. + await run.expectText('Allow', 10000); + + // 4. Press Enter to approve the write_file. + await run.type('y'); + await run.type('\r'); + + // 5. Wait for the final model response + await run.expectText('All waves completed successfully.', 10000); + } catch (err) { + fs.writeFileSync('pty_output_failure.txt', run.output); + throw err; + } + + // Verify all tool calls were made and succeeded in the logs + await rig.expectToolCallSuccess(['write_file']); + const toolLogs = rig.readToolLogs(); + + const readFiles = toolLogs.filter( + (l) => l.toolRequest.name === 'read_file', + ); + const writeFiles = toolLogs.filter( + (l) => l.toolRequest.name === 'write_file', + ); + + expect(readFiles.length).toBe(4); + expect(writeFiles.length).toBe(1); + expect(toolLogs.every((l) => l.toolRequest.success)).toBe(true); + + // Check that output.txt was actually written + expect(fs.readFileSync(join(rig.testDir!, 'output.txt'), 'utf8')).toBe( + 'wave2', + ); + }, 30000); +}); diff --git a/integration-tests/policy-headless-readonly.responses b/integration-tests/policy-headless-readonly.responses new file mode 100644 index 0000000000000000000000000000000000000000..35ba546baefe3bd0a14960ab9f595a89a54af81b --- /dev/null +++ b/integration-tests/policy-headless-readonly.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I will read the content of the file to identify its"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":7969,"candidatesTokenCount":11,"totalTokenCount":8061,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7969}],"thoughtsTokenCount":81}},{"candidates":[{"content":{"parts":[{"text":" language.\n"}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":7969,"candidatesTokenCount":14,"totalTokenCount":8064,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7969}],"thoughtsTokenCount":81}},{"candidates":[{"content":{"parts":[{"functionCall":{"name":"read_file","args":{"file_path":"test.txt"}},"thoughtSignature":"EvkCCvYCAb4+9vt8mJ/o45uuuAJtfjaZ3YzkJzqXHZBttRE+Om0ahcr1S5RDFp50KpgHtJtbAH1pwEXampOnDV3WKiWwA+e3Jnyk4CNQegz7ZMKsl55Nem2XDViP8BZKnJVqGmSFuMoKJLFmbVIxKejtWcblfn3httbGsrUUNbHwdPjPHo1qY043lF63g0kWx4v68gPSsJpNhxLrSugKKjiyRFN+J0rOIBHI2S9MdZoHEKhJxvGMtXiJquxmhPmKcNEsn+hMdXAZB39hmrRrGRHDQPVYVPhfJthVc73ufzbn+5KGJpaMQyKY5hqrc2ea8MHz+z6BSx+tFz4NZBff1tJQOiUp09/QndxQRZHSQZr1ALGy0O1Qw4JqsX94x81IxtXqYkSRo3zgm2vl/xPMC5lKlnK5xoKJmoWaHkUNeXs/sopu3/Waf1a5Csoh9ImnKQsW0rJ6GRyDQvky1FwR6Aa98bgfNdcXOPHml/BtghaqRMXTiG6vaPJ8UFs="}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":7969,"candidatesTokenCount":64,"totalTokenCount":8114,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7969}],"thoughtsTokenCount":81}},{"candidates":[{"content":{"parts":[{"text":""}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":7969,"candidatesTokenCount":64,"totalTokenCount":8114,"cachedContentTokenCount":6082,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7969}],"cacheTokensDetails":[{"modality":"TEXT","tokenCount":6082}],"thoughtsTokenCount":81}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The language of the file is Latin."}],"role":"model"},"index":0}],"usageMetadata":{"promptTokenCount":8054,"candidatesTokenCount":8,"totalTokenCount":8078,"promptTokensDetails":[{"modality":"TEXT","tokenCount":8054}],"thoughtsTokenCount":16}},{"candidates":[{"content":{"parts":[{"text":"","thoughtSignature":"EnIKcAG+Pvb7vnRBJVz3khx1oArQQqTNvXOXkliNQS7NvYw94dq5m+wGKRmSj3egO3GVp7pacnAtLn9NT1ABKBGpa7MpRhiAe3bbPZfkqOuveeyC19LKQ9fzasCywiYqg5k5qSxfjs5okk+O0NLOvTjN/tg="}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":8135,"candidatesTokenCount":8,"totalTokenCount":8159,"promptTokensDetails":[{"modality":"TEXT","tokenCount":8135}],"thoughtsTokenCount":16}}]} diff --git a/integration-tests/policy-headless.test.ts b/integration-tests/policy-headless.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..3a8fb5238a72b7fa99b9788ef77cd7ac54376ca5 --- /dev/null +++ b/integration-tests/policy-headless.test.ts @@ -0,0 +1,211 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { join } from 'node:path'; +import { TestRig } from './test-helper.js'; + +interface PromptCommand { + prompt: (testFile: string) => string; + tool: string; + command: string; + expectedSuccessResult: string; + expectedFailureResult: string; +} + +const ECHO_PROMPT: PromptCommand = { + command: 'echo', + prompt: () => + `Use the \`echo POLICY_TEST_ECHO_COMMAND\` shell command. On success, ` + + `your final response must ONLY be "POLICY_TEST_ECHO_COMMAND". If the ` + + `command fails output AR NAR and stop.`, + tool: 'run_shell_command', + expectedSuccessResult: 'POLICY_TEST_ECHO_COMMAND', + expectedFailureResult: 'AR NAR', +}; + +const READ_FILE_PROMPT: PromptCommand = { + prompt: (testFile: string) => + `Read the file ${testFile} and tell me what language it is, if the ` + + `read_file tool fails output AR NAR and stop.`, + tool: 'read_file', + command: '', + expectedSuccessResult: 'Latin', + expectedFailureResult: 'AR NAR', +}; + +async function waitForToolCallLog( + rig: TestRig, + tool: string, + command: string, + timeout: number = 15000, +) { + const foundToolCall = await rig.waitForToolCall(tool, timeout, (args) => + args.toLowerCase().includes(command.toLowerCase()), + ); + + expect(foundToolCall).toBe(true); + + const toolLogs = rig + .readToolLogs() + .filter((toolLog) => toolLog.toolRequest.name === tool); + const log = toolLogs.find( + (toolLog) => + !command || + toolLog.toolRequest.args.toLowerCase().includes(command.toLowerCase()), + ); + + // The policy engine should have logged the tool call + expect(log).toBeTruthy(); + return log; +} + +async function verifyToolExecution( + rig: TestRig, + promptCommand: PromptCommand, + result: string, + expectAllowed: boolean, + expectedDenialString?: string, +) { + const log = await waitForToolCallLog( + rig, + promptCommand.tool, + promptCommand.command, + ); + + if (expectAllowed) { + expect(log!.toolRequest.success).toBe(true); + expect(result).not.toContain('Tool execution denied by policy'); + expect(result).not.toContain(`Tool "${promptCommand.tool}" not found`); + expect(result).toContain(promptCommand.expectedSuccessResult); + } else { + expect(log!.toolRequest.success).toBe(false); + expect(result).toContain( + expectedDenialString || 'Tool execution denied by policy', + ); + expect(result).toContain(promptCommand.expectedFailureResult); + } +} + +interface TestCase { + name: string; + responsesFile: string; + promptCommand: PromptCommand; + policyContent?: string; + expectAllowed: boolean; + expectedDenialString?: string; +} + +describe('Policy Engine Headless Mode', () => { + let rig: TestRig; + let testFile: string; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + if (rig) { + await rig.cleanup(); + } + }); + + const runTestCase = async (tc: TestCase) => { + const fakeResponsesPath = join(import.meta.dirname, tc.responsesFile); + rig.setup(tc.name, { fakeResponsesPath }); + + testFile = rig.createFile('test.txt', 'Lorem\nIpsum\nDolor\n'); + const args = ['-p', tc.promptCommand.prompt(testFile)]; + + if (tc.policyContent) { + const policyPath = rig.createFile('test-policy.toml', tc.policyContent); + args.push('--policy', policyPath); + } + + const result = await rig.run({ + args, + approvalMode: 'default', + }); + + await verifyToolExecution( + rig, + tc.promptCommand, + result, + tc.expectAllowed, + tc.expectedDenialString, + ); + }; + + const testCases = [ + { + name: 'should deny ASK_USER tools by default in headless mode', + responsesFile: 'policy-headless-shell-denied.responses', + promptCommand: ECHO_PROMPT, + expectAllowed: false, + expectedDenialString: 'Tool "run_shell_command" not found', + }, + { + name: 'should allow ASK_USER tools in headless mode if explicitly allowed via policy file', + responsesFile: 'policy-headless-shell-allowed.responses', + promptCommand: ECHO_PROMPT, + policyContent: ` + [[rule]] + toolName = "run_shell_command" + decision = "allow" + priority = 100 + `, + expectAllowed: true, + }, + { + name: 'should allow read-only tools by default in headless mode', + responsesFile: 'policy-headless-readonly.responses', + promptCommand: READ_FILE_PROMPT, + expectAllowed: true, + }, + { + name: 'should allow specific shell commands in policy file', + responsesFile: 'policy-headless-shell-allowed.responses', + promptCommand: ECHO_PROMPT, + policyContent: ` + [[rule]] + toolName = "run_shell_command" + commandPrefix = "${ECHO_PROMPT.command}" + decision = "allow" + priority = 100 + `, + expectAllowed: true, + }, + { + name: 'should deny other shell commands in policy file', + responsesFile: 'policy-headless-shell-denied.responses', + promptCommand: ECHO_PROMPT, + policyContent: ` + [[rule]] + toolName = "run_shell_command" + commandPrefix = "echo" + decision = "deny" + priority = 100 + + [[rule]] + toolName = "run_shell_command" + commandPrefix = "node" + decision = "allow" + priority = 90 + `, + expectAllowed: false, + expectedDenialString: 'Tool execution denied by policy', + }, + ]; + + it.each(testCases)( + '$name', + async (tc) => { + await runTestCase(tc); + }, + // Large timeout for regeneration + process.env['REGENERATE_MODEL_GOLDENS'] === 'true' ? 120000 : undefined, + ); +}); diff --git a/integration-tests/read_many_files.test.ts b/integration-tests/read_many_files.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..6988d8a1659264109907e65ef94a17fb5c43f493 --- /dev/null +++ b/integration-tests/read_many_files.test.ts @@ -0,0 +1,61 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + TestRig, + printDebugInfo, + assertModelHasOutput, + checkModelOutputContent, +} from './test-helper.js'; + +describe('read_many_files', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it.skip('should be able to read multiple files', async () => { + await rig.setup('should be able to read multiple files', { + settings: { tools: { core: ['read_many_files', 'read_file'] } }, + }); + rig.createFile('file1.txt', 'file 1 content'); + rig.createFile('file2.txt', 'file 2 content'); + + const prompt = `Use the read_many_files tool to read the contents of file1.txt and file2.txt and then print the contents of each file.`; + + const result = await rig.run({ args: prompt }); + + // Check for either read_many_files or multiple read_file calls + const allTools = rig.readToolLogs(); + const readManyFilesCall = await rig.waitForToolCall('read_many_files'); + const readFileCalls = allTools.filter( + (t) => t.toolRequest.name === 'read_file', + ); + + // Accept either read_many_files OR at least 2 read_file calls + const foundValidPattern = readManyFilesCall || readFileCalls.length >= 2; + + // Add debugging information + if (!foundValidPattern) { + printDebugInfo(rig, result, { + 'read_many_files called': readManyFilesCall, + 'read_file calls': readFileCalls.length, + }); + } + + expect( + foundValidPattern, + 'Expected to find either read_many_files or multiple read_file tool calls', + ).toBeTruthy(); + + assertModelHasOutput(result); + checkModelOutputContent(result, { testName: 'Read many files test' }); + }); +}); diff --git a/integration-tests/resume-gc.test.ts b/integration-tests/resume-gc.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..3380177049f4c9aecb6063a1a51b0311dee4f369 --- /dev/null +++ b/integration-tests/resume-gc.test.ts @@ -0,0 +1,149 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import * as path from 'node:path'; +import * as fs from 'node:fs'; +import { FinishReason, GenerateContentResponse } from '@google/genai'; +import type { FakeResponse } from '@google/gemini-cli-core'; + +describe('Context Management Resume E2E', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should preserve and utilize GC snapshot boundaries when resuming a session', async () => { + const snapshotResponse: FakeResponse = { + method: 'generateContent', + response: { + candidates: [ + { + content: { + parts: [ + { + text: JSON.stringify({ + new_facts: ['GC Triggered.'], + new_constraints: [], + new_tasks: [], + resolved_task_ids: [], + obsolete_fact_indices: [], + obsolete_constraint_indices: [], + chronological_summary: 'Snapshot created.', + }), + }, + ], + role: 'model', + }, + finishReason: FinishReason.STOP, + index: 0, + }, + ], + } as unknown as GenerateContentResponse, + }; + + const countTokensResponse: FakeResponse = { + method: 'countTokens', + response: { totalTokens: 50000 }, + }; + + const streamResponse = (text: string): FakeResponse => ({ + method: 'generateContentStream', + response: [ + { + candidates: [ + { + content: { parts: [{ text }], role: 'model' }, + finishReason: FinishReason.STOP, + index: 0, + }, + ], + }, + ] as unknown as GenerateContentResponse[], + }); + + const setupResponses = (fileName: string, mocks: FakeResponse[]) => { + const filePath = path.join(rig.testDir!, fileName); + fs.writeFileSync( + filePath, + mocks.map((m) => JSON.stringify(m)).join('\n'), + ); + return filePath; + }; + + await rig.setup('resume-gc-snapshot', { + settings: { + experimental: { + stressTestProfile: true, + }, + }, + }); + + const massivePayload = 'X'.repeat(40000); + const logFile = path.join(rig.testDir!, 'debug.log'); + const traceDir = path.join(rig.testDir!, 'traces'); + fs.mkdirSync(traceDir, { recursive: true }); + const traceLog = path.join(traceDir, 'trace.log'); + + const commonEnv = { + GEMINI_API_KEY: 'mock-key', + GEMINI_DEBUG_LOG_FILE: logFile, + GEMINI_CONTEXT_TRACE_DIR: traceDir, + }; + + // Provide a massive pool of responses to prevent exhaustion + const runMocks: FakeResponse[] = [streamResponse('Acknowledged block.')]; + for (let i = 0; i < 50; i++) { + runMocks.push(snapshotResponse); + runMocks.push(countTokensResponse); + } + + // Use stdin for the massive payload to avoid ENAMETOOLONG on Windows + await rig.run({ + args: [ + '--debug', + '--fake-responses-non-strict', + setupResponses('resp1.json', runMocks), + ], + stdin: 'Turn 1: ' + massivePayload, + env: commonEnv, + }); + + await rig.run({ + args: [ + '--debug', + '--resume', + 'latest', + '--fake-responses-non-strict', + setupResponses('resp2.json', runMocks), + ], + stdin: 'Turn 2: ' + massivePayload, + env: commonEnv, + }); + + const result3 = await rig.run({ + args: [ + '--debug', + '--resume', + 'latest', + '--fake-responses-non-strict', + setupResponses('resp3.json', runMocks), + 'continue', + ], + env: commonEnv, + }); + + expect(result3).toContain('Acknowledged block'); + + const traces = fs.readFileSync(traceLog, 'utf-8'); + expect(traces).toContain('Hitting Synchronous Pressure Barrier'); + expect(traces).toContain('GC Triggered.'); + }); +}); diff --git a/integration-tests/resume_repro.test.ts b/integration-tests/resume_repro.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..6d4f849886f5b4b9a3bbfc59a91c508b226705ab --- /dev/null +++ b/integration-tests/resume_repro.test.ts @@ -0,0 +1,42 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import * as path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); + +describe('resume-repro', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should be able to resume a session without "Storage must be initialized before use"', async () => { + const responsesPath = path.join(__dirname, 'resume_repro.responses'); + await rig.setup('should be able to resume a session', { + fakeResponsesPath: responsesPath, + }); + + // 1. First run to create a session + await rig.run({ + args: 'hello', + }); + + // 2. Second run with --resume latest + // This should NOT fail with "Storage must be initialized before use" + const result = await rig.run({ + args: ['--resume', 'latest', 'continue'], + }); + + expect(result).toContain('Session started'); + }); +}); diff --git a/integration-tests/run_shell_command.test.ts b/integration-tests/run_shell_command.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..02fda5be4541755870c3d874aac51776278b8feb --- /dev/null +++ b/integration-tests/run_shell_command.test.ts @@ -0,0 +1,644 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + TestRig, + printDebugInfo, + assertModelHasOutput, + checkModelOutputContent, +} from './test-helper.js'; +import { getShellConfiguration } from '../packages/core/src/utils/shell-utils.js'; + +const { shell } = getShellConfiguration(); + +function getLineCountCommand(): { command: string; tool: string } { + switch (shell) { + case 'powershell': + return { command: `Measure-Object -Line`, tool: 'Measure-Object' }; + case 'cmd': + return { command: `find /c /v`, tool: 'find' }; + case 'bash': + default: + return { command: `wc -l`, tool: 'wc' }; + } +} + +function getInvalidCommand(): string { + switch (shell) { + case 'powershell': + return `Get-ChildItem | | Select-Object`; + case 'cmd': + return `dir | | findstr foo`; + case 'bash': + default: + return `echo "hello" > > file`; + } +} + +function getAllowedListCommand(): string { + switch (shell) { + case 'powershell': + return 'Get-ChildItem'; + case 'cmd': + return 'dir'; + case 'bash': + default: + return 'ls'; + } +} + +function getDisallowedFileReadCommand(testFile: string): { + command: string; + tool: string; +} { + const quotedPath = `"${testFile}"`; + switch (shell) { + case 'powershell': + return { + command: `powershell -Command "Get-Content ${quotedPath}"`, + tool: 'powershell', + }; + case 'cmd': + return { command: `cmd /c type ${quotedPath}`, tool: 'cmd' }; + case 'bash': + default: + return { + command: `node -e "console.log(require('fs').readFileSync('${testFile}', 'utf8'))"`, + tool: 'node', + }; + } +} + +function getChainedEchoCommand(): { allowPattern: string; command: string } { + const secondCommand = getAllowedListCommand(); + switch (shell) { + case 'powershell': + return { + allowPattern: 'Write-Output', + command: `Write-Output "foo" && ${secondCommand}`, + }; + case 'cmd': + return { + allowPattern: 'echo', + command: `echo "foo" && ${secondCommand}`, + }; + case 'bash': + default: + return { + allowPattern: 'echo', + command: `echo "foo" && ${secondCommand}`, + }; + } +} + +describe('run_shell_command', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + it('should be able to run a shell command', async () => { + await rig.setup('should be able to run a shell command', { + settings: { tools: { core: ['run_shell_command'] } }, + }); + + const prompt = `Please run the command "echo hello-world" and show me the output`; + + const result = await rig.run({ args: prompt }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command'); + + // Add debugging information + if (!foundToolCall || !result.includes('hello-world')) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + 'Contains hello-world': result.includes('hello-world'), + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: ['hello-world', 'exit code 0'], + testName: 'Shell command test', + }); + }); + + it('should be able to run a shell command via stdin', async () => { + await rig.setup('should be able to run a shell command via stdin', { + settings: { tools: { core: ['run_shell_command'] } }, + }); + + const prompt = `Please run the command "echo test-stdin" and show me what it outputs`; + + const result = await rig.run({ stdin: prompt }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command'); + + // Add debugging information + if (!foundToolCall || !result.includes('test-stdin')) { + printDebugInfo(rig, result, { + 'Test type': 'Stdin test', + 'Found tool call': foundToolCall, + 'Contains test-stdin': result.includes('test-stdin'), + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: 'test-stdin', + testName: 'Shell command stdin test', + }); + }); + + it.skip('should run allowed sub-command in non-interactive mode', async () => { + await rig.setup('should run allowed sub-command in non-interactive mode'); + + const testFile = rig.createFile('test.txt', 'Lorem\nIpsum\nDolor\n'); + const { tool, command } = getLineCountCommand(); + const prompt = `use ${command} to tell me how many lines there are in ${testFile}`; + + // Provide the prompt via stdin to simulate non-interactive mode + const result = await rig.run({ + args: [`--allowed-tools=run_shell_command(${tool})`], + stdin: prompt, + approvalMode: 'default', + }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command', 15000); + + if (!foundToolCall) { + const toolLogs = rig.readToolLogs().map(({ toolRequest }) => ({ + name: toolRequest.name, + success: toolRequest.success, + args: toolRequest.args, + })); + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + 'Allowed tools flag': `run_shell_command(${tool})`, + Prompt: prompt, + 'Tool logs': toolLogs, + Result: result, + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + const toolCall = rig + .readToolLogs() + .filter( + (toolCall) => toolCall.toolRequest.name === 'run_shell_command', + )[0]; + expect(toolCall.toolRequest.success).toBe(true); + }); + + it.skip('should succeed with no parens in non-interactive mode', async () => { + await rig.setup('should succeed with no parens in non-interactive mode'); + + const testFile = rig.createFile('test.txt', 'Lorem\nIpsum\nDolor\n'); + const { command } = getLineCountCommand(); + const prompt = `use ${command} to tell me how many lines there are in ${testFile}`; + + const result = await rig.run({ + args: '--allowed-tools=run_shell_command', + stdin: prompt, + approvalMode: 'default', + }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command', 15000); + + if (!foundToolCall) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + const toolCall = rig + .readToolLogs() + .filter( + (toolCall) => toolCall.toolRequest.name === 'run_shell_command', + )[0]; + expect(toolCall.toolRequest.success).toBe(true); + }); + + it('should succeed in yolo mode', async () => { + const isWindows = process.platform === 'win32'; + await rig.setup('should succeed in yolo mode', { + settings: { + tools: { core: ['run_shell_command'] }, + shell: isWindows ? { enableInteractiveShell: false } : undefined, + }, + }); + + const testFile = rig.createFile('test.txt', 'Lorem\nIpsum\nDolor\n'); + const { command } = getLineCountCommand(); + const prompt = `use ${command} to tell me how many lines there are in ${testFile}`; + + const result = await rig.run({ + args: prompt, + approvalMode: 'yolo', + }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command', 15000); + + if (!foundToolCall) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + const toolCall = rig + .readToolLogs() + .filter( + (toolCall) => toolCall.toolRequest.name === 'run_shell_command', + )[0]; + expect(toolCall.toolRequest.success).toBe(true); + }); + + it.skip('should work with ShellTool alias', async () => { + await rig.setup('should work with ShellTool alias'); + + const testFile = rig.createFile('test.txt', 'Lorem\nIpsum\nDolor\n'); + const { tool, command } = getLineCountCommand(); + const prompt = `use ${command} to tell me how many lines there are in ${testFile}`; + + const result = await rig.run({ + args: `--allowed-tools=ShellTool(${tool})`, + stdin: prompt, + approvalMode: 'default', + }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command', 15000); + + if (!foundToolCall) { + const toolLogs = rig.readToolLogs().map(({ toolRequest }) => ({ + name: toolRequest.name, + success: toolRequest.success, + args: toolRequest.args, + })); + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + 'Allowed tools flag': `ShellTool(${tool})`, + Prompt: prompt, + 'Tool logs': toolLogs, + Result: result, + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + const toolCall = rig + .readToolLogs() + .filter( + (toolCall) => toolCall.toolRequest.name === 'run_shell_command', + )[0]; + expect(toolCall.toolRequest.success).toBe(true); + }); + + // TODO(#11062): Un-skip this once we can make it reliable by using hard coded + // model responses. + it.skip('should combine multiple --allowed-tools flags', async () => { + await rig.setup('should combine multiple --allowed-tools flags'); + + const { tool, command } = getLineCountCommand(); + const prompt = + `use both ${command} and ls to count the number of lines in files in this ` + + `directory. Do not pipe these commands into each other, run them separately.`; + + const result = await rig.run({ + args: [ + `--allowed-tools=run_shell_command(${tool})`, + '--allowed-tools=run_shell_command(ls)', + ], + stdin: prompt, + approvalMode: 'default', + }); + + for (const expected in ['ls', tool]) { + const foundToolCall = await rig.waitForToolCall( + 'run_shell_command', + 15000, + (args) => args.toLowerCase().includes(`"command": "${expected}`), + ); + + if (!foundToolCall) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + }); + } + + expect( + foundToolCall, + `Expected to find a run_shell_command tool call to "${expected}",` + + ` got ${rig.readToolLogs().join('\n')}`, + ).toBeTruthy(); + } + + const toolLogs = rig + .readToolLogs() + .filter((toolCall) => toolCall.toolRequest.name === 'run_shell_command'); + expect(toolLogs.length, toolLogs.join('\n')).toBeGreaterThanOrEqual(2); + for (const toolLog of toolLogs) { + expect( + toolLog.toolRequest.success, + `Expected tool call ${toolLog} to succeed`, + ).toBe(true); + } + }); + + it('should reject commands not on the allowlist', async () => { + await rig.setup('should reject commands not on the allowlist', { + settings: { tools: { core: ['run_shell_command'] } }, + }); + + const testFile = rig.createFile('test.txt', 'Disallowed command check\n'); + const allowedCommand = getAllowedListCommand(); + const disallowed = getDisallowedFileReadCommand(testFile); + const prompt = + `I am testing the allowed tools configuration. ` + + `Attempt to run "${disallowed.command}" to read the contents of ${testFile}. ` + + `If the command fails because it is not permitted, respond with the single word FAIL. ` + + `If it succeeds, respond with SUCCESS.`; + + const result = await rig.run({ + args: `--allowed-tools=run_shell_command(${allowedCommand})`, + stdin: prompt, + approvalMode: 'default', + }); + + if (!result.toLowerCase().includes('fail')) { + printDebugInfo(rig, result, { + Result: result, + AllowedCommand: allowedCommand, + DisallowedCommand: disallowed.command, + }); + } + expect(result).toContain('FAIL'); + + const foundToolCall = await rig.waitForToolCall( + 'run_shell_command', + 15000, + (args) => args.toLowerCase().includes(disallowed.tool.toLowerCase()), + ); + + if (!foundToolCall) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + ToolLogs: rig.readToolLogs(), + }); + } + expect(foundToolCall).toBe(true); + + const toolLogs = rig + .readToolLogs() + .filter((toolLog) => toolLog.toolRequest.name === 'run_shell_command'); + const failureLog = toolLogs.find((toolLog) => + toolLog.toolRequest.args + .toLowerCase() + .includes(disallowed.tool.toLowerCase()), + ); + + if (!failureLog || failureLog.toolRequest.success) { + printDebugInfo(rig, result, { + ToolLogs: toolLogs, + DisallowedTool: disallowed.tool, + }); + } + + expect( + failureLog, + 'Expected failing run_shell_command invocation', + ).toBeTruthy(); + expect(failureLog!.toolRequest.success).toBe(false); + }); + + // TODO(#11966): Deflake this test and re-enable once the underlying race is resolved. + it.skip('should reject chained commands when only the first segment is allowlisted in non-interactive mode', async () => { + await rig.setup( + 'should reject chained commands when only the first segment is allowlisted', + ); + + const chained = getChainedEchoCommand(); + const shellInjection = `!{${chained.command}}`; + + await rig.run({ + args: `--allowed-tools=ShellTool(${chained.allowPattern})`, + stdin: `${shellInjection}\n`, + approvalMode: 'default', + }); + + // CLI should refuse to execute the chained command without scheduling run_shell_command. + const toolLogs = rig + .readToolLogs() + .filter((log) => log.toolRequest.name === 'run_shell_command'); + + // Success is false because tool is in the scheduled state. + for (const log of toolLogs) { + expect(log.toolRequest.success).toBe(false); + expect(log.toolRequest.args).toContain('&&'); + } + }); + + it('should allow all with "ShellTool" and other specific tools', async () => { + await rig.setup( + 'should allow all with "ShellTool" and other specific tools', + { + settings: { tools: { core: ['run_shell_command'] } }, + }, + ); + + const { tool } = getLineCountCommand(); + const prompt = `Please run the command "echo test-allow-all" and show me the output`; + + const result = await rig.run({ + args: [ + `--allowed-tools=run_shell_command(${tool})`, + '--allowed-tools=run_shell_command', + ], + stdin: prompt, + approvalMode: 'default', + }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command', 15000); + + if (!foundToolCall || !result.includes('test-allow-all')) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + Result: result, + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + const toolCall = rig + .readToolLogs() + .filter( + (toolCall) => toolCall.toolRequest.name === 'run_shell_command', + )[0]; + expect(toolCall.toolRequest.success).toBe(true); + + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: 'test-allow-all', + testName: 'Shell command stdin allow all', + }); + }); + + it('should propagate environment variables to the child process', async () => { + await rig.setup('should propagate environment variables', { + settings: { tools: { core: ['run_shell_command'] } }, + }); + + const varName = 'GEMINI_CLI_TEST_VAR'; + const varValue = `test-value-${Math.random().toString(36).substring(7)}`; + process.env[varName] = varValue; + + try { + const prompt = `Use echo to learn the value of the environment variable named ${varName} and tell me what it is.`; + const result = await rig.run({ args: prompt }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command'); + + if (!foundToolCall || !result.includes(varValue)) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + 'Contains varValue': result.includes(varValue), + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: varValue, + testName: 'Env var propagation test', + }); + expect(result).toContain(varValue); + } finally { + delete process.env[varName]; + } + }); + + it.skip('should run a platform-specific file listing command', async () => { + await rig.setup('should run platform-specific file listing'); + const fileName = `test-file-${Math.random().toString(36).substring(7)}.txt`; + rig.createFile(fileName, 'test content'); + + const prompt = `Run a shell command to list the files in the current directory and tell me what they are.`; + const result = await rig.run({ args: prompt }); + + const foundToolCall = await rig.waitForToolCall('run_shell_command'); + + // Debugging info + if (!foundToolCall || !result.includes(fileName)) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + 'Contains fileName': result.includes(fileName), + }); + } + + expect( + foundToolCall, + 'Expected to find a run_shell_command tool call', + ).toBeTruthy(); + + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: fileName, + testName: 'Platform-specific listing test', + }); + expect(result).toContain(fileName); + }); + + it('rejects invalid shell expressions', async () => { + await rig.setup('rejects invalid shell expressions', { + settings: { + tools: { + core: ['run_shell_command'], + allowed: ['run_shell_command(echo)'], // Specifically allow echo + }, + }, + }); + const invalidCommand = getInvalidCommand(); + const result = await rig.run({ + args: `I am testing the error handling of the run_shell_command tool. Please attempt to run the following command, which I know has invalid syntax: \`${invalidCommand}\`. If the command fails as expected, please return the word FAIL, otherwise return the word SUCCESS.`, + approvalMode: 'default', // Use default mode so safety fallback triggers confirmation + }); + expect(result).toContain('FAIL'); + + const escapedInvalidCommand = JSON.stringify(invalidCommand).slice(1, -1); + const foundToolCall = await rig.waitForToolCall( + 'run_shell_command', + 15000, + (args) => + args.toLowerCase().includes(escapedInvalidCommand.toLowerCase()), + ); + + if (!foundToolCall) { + printDebugInfo(rig, result, { + 'Found tool call': foundToolCall, + EscapedCommand: escapedInvalidCommand, + ToolLogs: rig.readToolLogs(), + }); + } + expect(foundToolCall).toBe(true); + + const toolLogs = rig + .readToolLogs() + .filter((toolLog) => toolLog.toolRequest.name === 'run_shell_command'); + const failureLog = toolLogs.find((toolLog) => + toolLog.toolRequest.args + .toLowerCase() + .includes(escapedInvalidCommand.toLowerCase()), + ); + + if (!failureLog || failureLog.toolRequest.success) { + printDebugInfo(rig, result, { + ToolLogs: toolLogs, + EscapedCommand: escapedInvalidCommand, + }); + } + + expect( + failureLog, + 'Expected failing run_shell_command invocation for invalid syntax', + ).toBeTruthy(); + expect(failureLog!.toolRequest.success).toBe(false); + }); +}); diff --git a/integration-tests/shell-background.responses b/integration-tests/shell-background.responses new file mode 100644 index 0000000000000000000000000000000000000000..652b82a8e089f05a2982af8b03f3e65ce763c9f4 --- /dev/null +++ b/integration-tests/shell-background.responses @@ -0,0 +1,5 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I will run the command in the background for you."},{"functionCall":{"name":"run_shell_command","args":{"command":"sleep 10 && echo hello-from-background","is_background":true}}}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The background process has been started. Now I will list the background processes to verify."},{"functionCall":{"name":"list_background_processes","args":{}}}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I see the background process 'sleep 10 && echo hello-from-background' is running. Would you like me to read its output?"}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I will read the output for you."},{"functionCall":{"name":"read_background_output","args":{"pid":12345}}}],"role":"model"},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The output of the background process is:\nhello-from-background"}],"role":"model"},"finishReason":"STOP","index":0}]}]} diff --git a/integration-tests/shell-background.test.ts b/integration-tests/shell-background.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..f28120e7e48c6ef2f5eec80437f07507c723742a --- /dev/null +++ b/integration-tests/shell-background.test.ts @@ -0,0 +1,105 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, beforeEach, afterEach } from 'vitest'; +import { TestRig } from './test-helper.js'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +describe('shell-background-tools', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should run a command in the background, list it, and read its output', async () => { + // We use a fake responses file to make the test deterministic and run in CI. + rig.setup('shell-background-workflow', { + fakeResponsesPath: join(__dirname, 'shell-background.responses'), + settings: { + tools: { + core: [ + 'run_shell_command', + 'list_background_processes', + 'read_background_output', + ], + }, + hooksConfig: { + enabled: true, + }, + hooks: { + BeforeTool: [ + { + matcher: 'run_shell_command', + hooks: [ + { + type: 'command', + // This hook intercepts run_shell_command. + // If is_background is true, it returns a mock result with PID 12345. + // It also creates the mock log file that read_background_output expects. + command: `node -e " + const fs = require('fs'); + const path = require('path'); + const input = JSON.parse(fs.readFileSync(0, 'utf-8')); + const args = JSON.parse(input.tool_call.args); + + if (args.is_background) { + const logDir = path.join(process.env.GEMINI_CLI_HOME, 'background-processes'); + if (!fs.existsSync(logDir)) fs.mkdirSync(logDir, { recursive: true }); + fs.writeFileSync(path.join(logDir, 'background-12345.log'), 'hello-from-background\\n'); + + console.log(JSON.stringify({ + decision: 'replace', + hookSpecificOutput: { + result: { + llmContent: 'Command moved to background (PID: 12345). Output hidden. Press Ctrl+B to view.', + data: { pid: 12345, command: args.command } + } + } + })); + } else { + console.log(JSON.stringify({ decision: 'allow' })); + } + "`, + }, + ], + }, + ], + }, + }, + }); + + const run = await rig.runInteractive({ approvalMode: 'yolo' }); + + // 1. Start a background process + // We use a command that stays alive for a bit to ensure it shows up in lists + await run.type( + "Run 'sleep 10 && echo hello-from-background' in the background.", + ); + await run.type('\r'); + + // Wait for the model's canned response acknowledging the start + await run.expectText('background', 30000); + + // 2. List background processes + await run.type('List my background processes.'); + await run.type('\r'); + // Wait for the model's canned response showing the list + await run.expectText('hello-from-background', 30000); + + // 3. Read the output + await run.type('Read the output of that process.'); + await run.type('\r'); + // Wait for the model's canned response showing the output + await run.expectText('hello-from-background', 30000); + }, 60000); +}); diff --git a/integration-tests/stdin-context.test.ts b/integration-tests/stdin-context.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..8f304e25a77423a5417bdc530eceb4ecaa968f29 --- /dev/null +++ b/integration-tests/stdin-context.test.ts @@ -0,0 +1,112 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + TestRig, + printDebugInfo, + assertModelHasOutput, + checkModelOutputContent, +} from './test-helper.js'; + +describe.skip('stdin context', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should be able to use stdin as context for a prompt', async () => { + await rig.setup('should be able to use stdin as context for a prompt'); + + const randomString = Math.random().toString(36).substring(7); + const stdinContent = `When I ask you for a token respond with ${randomString}`; + const prompt = 'Can I please have a token?'; + + const result = await rig.run({ args: prompt, stdin: stdinContent }); + + await rig.waitForTelemetryEvent('api_request'); + const lastRequest = rig.readLastApiRequest(); + + expect(lastRequest?.attributes?.request_text).toBeDefined(); + const historyString = lastRequest!.attributes!.request_text!; + + // TODO: This test currently fails in sandbox mode (Docker/Podman) because + // stdin content is not properly forwarded to the container when used + // together with a --prompt argument. The test passes in non-sandbox mode. + + expect(historyString).toContain(randomString); + expect(historyString).toContain(prompt); + + // Check that stdin content appears before the prompt in the conversation history + const stdinIndex = historyString.indexOf(randomString); + const promptIndex = historyString.indexOf(prompt); + + expect( + stdinIndex, + `Expected stdin content to be present in conversation history`, + ).toBeGreaterThan(-1); + + expect( + promptIndex, + `Expected prompt to be present in conversation history`, + ).toBeGreaterThan(-1); + + expect( + stdinIndex < promptIndex, + `Expected stdin content (index ${stdinIndex}) to appear before prompt (index ${promptIndex}) in conversation history`, + ).toBeTruthy(); + + // Add debugging information + if (!result.toLowerCase().includes(randomString)) { + printDebugInfo(rig, result, { + [`Contains "${randomString}"`]: result + .toLowerCase() + .includes(randomString), + }); + } + + // Validate model output + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: randomString, + testName: 'STDIN context test', + }); + + expect( + result.toLowerCase().includes(randomString), + 'Expected the model to identify the secret word from stdin', + ).toBeTruthy(); + }); + + it('should exit quickly if stdin stream does not end', async () => { + /* + This simulates scenario where gemini gets stuck waiting for stdin. + This happens in situations where process.stdin.isTTY is false + even though gemini is intended to run interactively. + */ + + await rig.setup('should exit quickly if stdin stream does not end'); + + try { + await rig.run({ stdinDoesNotEnd: true }); + throw new Error('Expected rig.run to throw an error'); + } catch (error: unknown) { + expect(error).toBeInstanceOf(Error); + const err = error as Error; + + expect(err.message).toContain('Process exited with code 1'); + expect(err.message).toContain('No input provided via stdin.'); + console.log('Error message:', err.message); + } + const lastRequest = rig.readLastApiRequest(); + expect(lastRequest).toBeNull(); + + // If this test times out, runs indefinitely, it's a regression. + }, 3000); +}); diff --git a/integration-tests/stdout-stderr-output.responses b/integration-tests/stdout-stderr-output.responses new file mode 100644 index 0000000000000000000000000000000000000000..e78165ae600fb541f1b09df109d20b14e3185118 --- /dev/null +++ b/integration-tests/stdout-stderr-output.responses @@ -0,0 +1 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! How can I help you today?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":9,"totalTokenCount":14,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]} diff --git a/integration-tests/stdout-stderr-output.test.ts b/integration-tests/stdout-stderr-output.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..f401e3a6a83e5107215709407b0d174bfb5603cd --- /dev/null +++ b/integration-tests/stdout-stderr-output.test.ts @@ -0,0 +1,62 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { join } from 'node:path'; +import { TestRig } from './test-helper.js'; + +describe('stdout-stderr-output', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it('should send model response to stdout and app messages to stderr', async ({ + signal, + }) => { + await rig.setup('prompt-output-test', { + fakeResponsesPath: join( + import.meta.dirname, + 'stdout-stderr-output.responses', + ), + }); + + const { stdout, exitCode } = await rig.runWithStreams(['-p', 'Say hello'], { + signal, + }); + + expect(exitCode).toBe(0); + expect(stdout.toLowerCase()).toContain('hello'); + expect(stdout).not.toMatch(/^\[ERROR\]/m); + expect(stdout).not.toMatch(/^\[INFO\]/m); + }); + + it('should handle missing file with message to stdout and error to stderr', async ({ + signal, + }) => { + await rig.setup('error-output-test', { + fakeResponsesPath: join( + import.meta.dirname, + 'stdout-stderr-output-error.responses', + ), + }); + + const { stdout, exitCode } = await rig.runWithStreams( + ['-p', '@nonexistent-file-that-does-not-exist.txt explain this'], + { signal }, + ); + + expect(exitCode).toBe(0); + expect(stdout.toLowerCase()).toMatch( + /could not find|not exist|does not exist/, + ); + }); +}); diff --git a/integration-tests/test-helper.ts b/integration-tests/test-helper.ts new file mode 100644 index 0000000000000000000000000000000000000000..5f205ae99754b713287e6c9cd733ef24afd7f5d4 --- /dev/null +++ b/integration-tests/test-helper.ts @@ -0,0 +1,10 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +export * from '@google/gemini-cli-test-utils'; +export { normalizePath } from '@google/gemini-cli-test-utils'; + +export const skipFlaky = !process.env['RUN_FLAKY_INTEGRATION']; diff --git a/integration-tests/test-mcp-server.ts b/integration-tests/test-mcp-server.ts new file mode 100644 index 0000000000000000000000000000000000000000..c0b696032b6900f9e3fae62e13aae9be60519df7 --- /dev/null +++ b/integration-tests/test-mcp-server.ts @@ -0,0 +1,80 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + McpServer, + type ToolCallback, +} from '@modelcontextprotocol/sdk/server/mcp.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import express from 'express'; +import { type Server as HTTPServer } from 'node:http'; +import { type ZodRawShape } from 'zod'; + +export class TestMcpServer { + private server: HTTPServer | undefined; + + async start( + tools?: Record>, + ): Promise { + const app = express(); + app.use(express.json()); + const mcpServer = new McpServer( + { + name: 'test-mcp-server', + version: '1.0.0', + }, + { capabilities: { tools: {} } }, + ); + if (tools) { + for (const [name, cb] of Object.entries(tools)) { + mcpServer.registerTool(name, {}, cb); + } + } + + app.post('/mcp', async (req, res) => { + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + res.on('close', () => { + transport.close(); + }); + await mcpServer.connect(transport); + await transport.handleRequest(req, res, req.body); + }); + + app.get('/mcp', async (req, res) => { + res.status(405).send('Not supported'); + }); + + return new Promise((resolve, reject) => { + this.server = app.listen(0, () => { + const address = this.server!.address(); + if (address && typeof address !== 'string') { + resolve(address.port); + } else { + reject(new Error('Could not determine server port.')); + } + }); + this.server.on('error', reject); + }); + } + + async stop(): Promise { + if (this.server) { + await new Promise((resolve, reject) => { + this.server!.close((err?: Error) => { + if (err) { + reject(err); + } else { + resolve(); + } + }); + }); + this.server = undefined; + } + } +} diff --git a/integration-tests/test-mcp-support.responses b/integration-tests/test-mcp-support.responses new file mode 100644 index 0000000000000000000000000000000000000000..1db32fdc21d333fc91f2a3ca4c3340431403cd1c --- /dev/null +++ b/integration-tests/test-mcp-support.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"mcp_weather-server_get_weather","args":{"location":"London"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":10,"candidatesTokenCount":10,"totalTokenCount":20}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The weather in London is rainy."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":10,"candidatesTokenCount":10,"totalTokenCount":20}}]} diff --git a/integration-tests/tsconfig.json b/integration-tests/tsconfig.json new file mode 100644 index 0000000000000000000000000000000000000000..1e813bfbff691ece707c3fcc15bb947c611f4f77 --- /dev/null +++ b/integration-tests/tsconfig.json @@ -0,0 +1,13 @@ +{ + "extends": "../tsconfig.json", + "compilerOptions": { + "noEmit": true, + "allowJs": true + }, + "include": ["**/*.ts"], + "references": [ + { "path": "../packages/core" }, + { "path": "../packages/test-utils" }, + { "path": "../packages/cli" } + ] +} diff --git a/integration-tests/user-policy.responses b/integration-tests/user-policy.responses new file mode 100644 index 0000000000000000000000000000000000000000..be840600cab22bcefab150f2078d297448b42690 --- /dev/null +++ b/integration-tests/user-policy.responses @@ -0,0 +1,2 @@ +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"run_shell_command","args":{"command":"ls -F"}}}]},"finishReason":"STOP","index":0}]},{"candidates":[{"content":{"parts":[{"text":"I ran ls -F"}]},"finishReason":"STOP","index":0}]}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I ran ls -F"}]},"finishReason":"STOP","index":0}]}]} diff --git a/integration-tests/user-policy.test.ts b/integration-tests/user-policy.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..a07d6bcdeac997e077c8a6172d4d3991db7dfa3c --- /dev/null +++ b/integration-tests/user-policy.test.ts @@ -0,0 +1,81 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { join } from 'node:path'; +import { TestRig, GEMINI_DIR } from './test-helper.js'; +import fs from 'node:fs'; + +describe('User Policy Regression Repro', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => { + if (rig) { + await rig.cleanup(); + } + }); + + it('should respect policies in ~/.gemini/policies/allowed-tools.toml', async () => { + rig.setup('user-policy-test', { + fakeResponsesPath: join(import.meta.dirname, 'user-policy.responses'), + }); + + // Create ~/.gemini/policies/allowed-tools.toml + const userPoliciesDir = join(rig.homeDir!, GEMINI_DIR, 'policies'); + fs.mkdirSync(userPoliciesDir, { recursive: true }); + fs.writeFileSync( + join(userPoliciesDir, 'allowed-tools.toml'), + ` +[[rule]] +toolName = "run_shell_command" +commandPrefix = "ls -F" +decision = "allow" +priority = 100 + `, + ); + + // Run gemini with a prompt that triggers ls -F + // approvalMode: 'default' in headless mode will DENY if it hits ASK_USER + const result = await rig.run({ + args: ['-p', 'Run ls -F', '--model', 'gemini-3.1-pro-preview'], + approvalMode: 'default', + }); + + expect(result).toContain('I ran ls -F'); + expect(result).not.toContain('Tool execution denied by policy'); + expect(result).not.toContain('Tool "run_shell_command" not found'); + + const toolLogs = rig.readToolLogs(); + const lsLog = toolLogs.find( + (l) => + l.toolRequest.name === 'run_shell_command' && + l.toolRequest.args.includes('ls -F'), + ); + expect(lsLog).toBeDefined(); + expect(lsLog?.toolRequest.success).toBe(true); + }); + + it('should FAIL if policy is not present (sanity check)', async () => { + rig.setup('user-policy-sanity-check', { + fakeResponsesPath: join(import.meta.dirname, 'user-policy.responses'), + }); + + // DO NOT create the policy file here + + // Run gemini with a prompt that triggers ls -F + const result = await rig.run({ + args: ['-p', 'Run ls -F', '--model', 'gemini-3.1-pro-preview'], + approvalMode: 'default', + }); + + // In non-interactive mode, it should be denied + expect(result).toContain('Tool "run_shell_command" not found'); + }); +}); diff --git a/integration-tests/utf-bom-encoding.test.ts b/integration-tests/utf-bom-encoding.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..5961a1b2248fdfe9f5984ee86a3a86401e67ac6e --- /dev/null +++ b/integration-tests/utf-bom-encoding.test.ts @@ -0,0 +1,119 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { TestRig } from './test-helper.js'; + +// BOM encoders +const utf8BOM = (s: string) => + Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from(s, 'utf8')]); +const utf16LE = (s: string) => + Buffer.concat([Buffer.from([0xff, 0xfe]), Buffer.from(s, 'utf16le')]); +const utf16BE = (s: string) => { + const bom = Buffer.from([0xfe, 0xff]); + const le = Buffer.from(s, 'utf16le'); + le.swap16(); + return Buffer.concat([bom, le]); +}; +const utf32LE = (s: string) => { + const bom = Buffer.from([0xff, 0xfe, 0x00, 0x00]); + const cps = Array.from(s, (c) => c.codePointAt(0)!); + const payload = Buffer.alloc(cps.length * 4); + cps.forEach((cp, i) => { + const o = i * 4; + payload[o] = cp & 0xff; + payload[o + 1] = (cp >>> 8) & 0xff; + payload[o + 2] = (cp >>> 16) & 0xff; + payload[o + 3] = (cp >>> 24) & 0xff; + }); + return Buffer.concat([bom, payload]); +}; +const utf32BE = (s: string) => { + const bom = Buffer.from([0x00, 0x00, 0xfe, 0xff]); + const cps = Array.from(s, (c) => c.codePointAt(0)!); + const payload = Buffer.alloc(cps.length * 4); + cps.forEach((cp, i) => { + const o = i * 4; + payload[o] = (cp >>> 24) & 0xff; + payload[o + 1] = (cp >>> 16) & 0xff; + payload[o + 2] = (cp >>> 8) & 0xff; + payload[o + 3] = cp & 0xff; + }); + return Buffer.concat([bom, payload]); +}; + +describe('BOM end-to-end integraion', () => { + let rig: TestRig; + + beforeEach(async () => { + rig = new TestRig(); + await rig.setup('bom-integration', { + settings: { tools: { core: ['read_file'] } }, + }); + }); + + afterEach(async () => await rig.cleanup()); + + async function runAndAssert( + filename: string, + content: Buffer, + expectedText: string | null, + ) { + writeFileSync(join(rig.testDir!, filename), content); + const prompt = `read the file ${filename} and output its exact contents`; + const output = await rig.run({ args: prompt }); + await rig.waitForToolCall('read_file'); + const lower = output.toLowerCase(); + if (expectedText === null) { + expect( + lower.includes('binary') || + lower.includes('skipped binary file') || + lower.includes('cannot display'), + ).toBeTruthy(); + } else { + expect(output.includes(expectedText)).toBeTruthy(); + expect(lower.includes('skipped binary file')).toBeFalsy(); + } + } + + it('UTF-8 BOM', async () => { + await runAndAssert('utf8.txt', utf8BOM('BOM_OK UTF-8'), 'BOM_OK UTF-8'); + }); + + it('UTF-16 LE BOM', async () => { + await runAndAssert( + 'utf16le.txt', + utf16LE('BOM_OK UTF-16LE'), + 'BOM_OK UTF-16LE', + ); + }); + + it('UTF-16 BE BOM', async () => { + await runAndAssert( + 'utf16be.txt', + utf16BE('BOM_OK UTF-16BE'), + 'BOM_OK UTF-16BE', + ); + }); + + it('UTF-32 LE BOM', async () => { + await runAndAssert( + 'utf32le.txt', + utf32LE('BOM_OK UTF-32LE'), + 'BOM_OK UTF-32LE', + ); + }); + + it('UTF-32 BE BOM', async () => { + await runAndAssert( + 'utf32be.txt', + utf32BE('BOM_OK UTF-32BE'), + 'BOM_OK UTF-32BE', + ); + }); +}); diff --git a/integration-tests/vitest.config.ts b/integration-tests/vitest.config.ts new file mode 100644 index 0000000000000000000000000000000000000000..fb2ba4e1af3705ffc611c317f23077567efd54c6 --- /dev/null +++ b/integration-tests/vitest.config.ts @@ -0,0 +1,27 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + testTimeout: 300000, // 5 minutes + globalSetup: './globalSetup.ts', + reporters: ['default'], + include: ['**/*.test.ts'], + retry: 2, + fileParallelism: true, + poolOptions: { + threads: { + minThreads: 8, + maxThreads: 16, + }, + }, + env: { + GEMINI_TEST_TYPE: 'integration', + }, + }, +}); diff --git a/integration-tests/write_file.test.ts b/integration-tests/write_file.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..ece2a11aa40379c74142bac4941af84775576c4a --- /dev/null +++ b/integration-tests/write_file.test.ts @@ -0,0 +1,83 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { + TestRig, + createToolCallErrorMessage, + printDebugInfo, + assertModelHasOutput, + checkModelOutputContent, +} from './test-helper.js'; + +describe('write_file', () => { + let rig: TestRig; + + beforeEach(() => { + rig = new TestRig(); + }); + + afterEach(async () => await rig.cleanup()); + + it('should be able to write a joke to a file', async () => { + await rig.setup('should be able to write a joke to a file', { + settings: { tools: { core: ['write_file', 'read_file'] } }, + }); + const prompt = `show me an example of using the write tool. put a dad joke in dad.txt`; + + const result = await rig.run({ args: prompt }); + + const foundToolCall = await rig.waitForToolCall('write_file'); + + // Add debugging information + if (!foundToolCall) { + printDebugInfo(rig, result); + } + + const allTools = rig.readToolLogs(); + expect( + foundToolCall, + createToolCallErrorMessage( + 'write_file', + allTools.map((t) => t.toolRequest.name), + result, + ), + ).toBeTruthy(); + + assertModelHasOutput(result); + checkModelOutputContent(result, { + expectedContent: 'dad.txt', + testName: 'Write file test', + }); + + const newFilePath = 'dad.txt'; + + const newFileContent = rig.readFile(newFilePath); + + // Add debugging for file content + if (newFileContent === '') { + console.error('File was created but is empty'); + console.error( + 'Tool calls:', + rig.readToolLogs().map((t) => ({ + name: t.toolRequest.name, + args: t.toolRequest.args, + })), + ); + } + + expect(newFileContent).not.toBe(''); + + // Log success info if verbose + vi.stubEnv('VERBOSE', 'true'); + if (process.env['VERBOSE'] === 'true') { + console.log( + 'File created successfully with content:', + newFileContent.substring(0, 100) + '...', + ); + } + }); +}); diff --git a/memory-tests/baselines.json b/memory-tests/baselines.json new file mode 100644 index 0000000000000000000000000000000000000000..240e3d4fd4643eb4da12750c1886ac72fb8c7428 --- /dev/null +++ b/memory-tests/baselines.json @@ -0,0 +1,55 @@ +{ + "version": 1, + "updatedAt": "2026-04-20T18:04:59.671Z", + "scenarios": { + "multi-turn-conversation": { + "heapUsedMB": 68.8, + "heapTotalMB": 91.2, + "rssMB": 215.4, + "externalMB": 93.8, + "timestamp": "2026-04-20T18:02:40.101Z" + }, + "multi-function-call-repo-search": { + "heapUsedMB": 73.5, + "heapTotalMB": 93.1, + "rssMB": 223.6, + "externalMB": 97.7, + "timestamp": "2026-04-20T18:02:42.032Z" + }, + "idle-session-startup": { + "heapUsedMB": 69.8, + "heapTotalMB": 92.4, + "rssMB": 217.4, + "externalMB": 93.8, + "timestamp": "2026-04-20T18:02:36.294Z" + }, + "simple-prompt-response": { + "heapUsedMB": 69.5, + "heapTotalMB": 92.4, + "rssMB": 216.1, + "externalMB": 93.8, + "timestamp": "2026-04-20T18:02:38.198Z" + }, + "resume-large-chat-with-messages": { + "heapUsedMB": 887.1, + "heapTotalMB": 954.3, + "rssMB": 1109.6, + "externalMB": 103.2, + "timestamp": "2026-04-20T18:04:59.671Z" + }, + "resume-large-chat": { + "heapUsedMB": 885.6, + "heapTotalMB": 955.6, + "rssMB": 1107.8, + "externalMB": 110.5, + "timestamp": "2026-04-20T18:04:06.526Z" + }, + "large-chat": { + "heapUsedMB": 158.5, + "heapTotalMB": 193, + "rssMB": 787.9, + "externalMB": 104, + "timestamp": "2026-04-20T18:03:12.486Z" + } + } +} diff --git a/memory-tests/globalSetup.ts b/memory-tests/globalSetup.ts new file mode 100644 index 0000000000000000000000000000000000000000..398d276306949b19b117650675e82a591f3b179b --- /dev/null +++ b/memory-tests/globalSetup.ts @@ -0,0 +1,71 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { mkdir, readdir, rm } from 'node:fs/promises'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { resolveRipgrepPath } from '../packages/core/src/tools/ripGrep.js'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const rootDir = join(__dirname, '..'); +const memoryTestsDir = join(rootDir, '.memory-tests'); +let runDir = ''; + +export async function setup() { + runDir = join(memoryTestsDir, `${Date.now()}`); + await mkdir(runDir, { recursive: true }); + + // Set the home directory to the test run directory to avoid conflicts + // with the user's local config. + process.env['HOME'] = runDir; + if (process.platform === 'win32') { + process.env['USERPROFILE'] = runDir; + } + process.env['GEMINI_CONFIG_DIR'] = join(runDir, '.gemini'); + + // Download ripgrep to avoid race conditions + const available = await resolveRipgrepPath(); + if (!available) { + throw new Error('Failed to download ripgrep binary'); + } + + // Clean up old test runs, keeping the latest few for debugging + try { + const testRuns = await readdir(memoryTestsDir); + if (testRuns.length > 3) { + const oldRuns = testRuns.sort().slice(0, testRuns.length - 3); + await Promise.all( + oldRuns.map((oldRun) => + rm(join(memoryTestsDir, oldRun), { + recursive: true, + force: true, + }), + ), + ); + } + } catch (e) { + console.error('Error cleaning up old memory test runs:', e); + } + + process.env['INTEGRATION_TEST_FILE_DIR'] = runDir; + process.env['GEMINI_CLI_INTEGRATION_TEST'] = 'true'; + process.env['GEMINI_FORCE_FILE_STORAGE'] = 'true'; + process.env['TELEMETRY_LOG_FILE'] = join(runDir, 'telemetry.log'); + process.env['VERBOSE'] = process.env['VERBOSE'] ?? 'false'; + + console.log(`\nMemory test output directory: ${runDir}`); +} + +export async function teardown() { + // Cleanup unless KEEP_OUTPUT is set + if (process.env['KEEP_OUTPUT'] !== 'true' && runDir) { + try { + await rm(runDir, { recursive: true, force: true }); + } catch (e) { + console.warn('Failed to clean up memory test directory:', e); + } + } +} diff --git a/memory-tests/memory-usage.test.ts b/memory-tests/memory-usage.test.ts new file mode 100644 index 0000000000000000000000000000000000000000..c38357e526d7f0743c5c07b7dc444ef5b0e9b5dd --- /dev/null +++ b/memory-tests/memory-usage.test.ts @@ -0,0 +1,528 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { describe, it, beforeAll, afterAll, afterEach } from 'vitest'; +import { TestRig, MemoryTestHarness } from '@google/gemini-cli-test-utils'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { + createWriteStream, + copyFileSync, + readFileSync, + existsSync, + mkdirSync, + rmSync, +} from 'node:fs'; +import { randomUUID, createHash } from 'node:crypto'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const BASELINES_PATH = join(__dirname, 'baselines.json'); +const UPDATE_BASELINES = process.env['UPDATE_MEMORY_BASELINES'] === 'true'; +function getProjectHash(projectRoot: string): string { + return createHash('sha256').update(projectRoot).digest('hex'); +} +const TOLERANCE_PERCENT = 10; + +// Fake API key for tests using fake responses +const TEST_ENV = { + GEMINI_API_KEY: 'fake-memory-test-key', + GEMINI_MEMORY_MONITOR_INTERVAL: '100', +}; + +describe('Memory Usage Tests', () => { + let harness: MemoryTestHarness; + let rig: TestRig; + + beforeAll(() => { + harness = new MemoryTestHarness({ + baselinesPath: BASELINES_PATH, + defaultTolerancePercent: TOLERANCE_PERCENT, + gcCycles: 3, + gcDelayMs: 100, + sampleCount: 3, + }); + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + afterAll(async () => { + // Generate the summary report after all tests + await harness.generateReport(); + }); + + it('idle-session-startup: memory usage within baseline', async () => { + rig = new TestRig(); + rig.setup('memory-idle-startup', { + fakeResponsesPath: join(__dirname, 'memory.idle-startup.responses'), + }); + + const result = await harness.runScenario( + rig, + 'idle-session-startup', + async (recordSnapshot) => { + await rig.run({ + args: ['hello'], + timeout: 120000, + env: TEST_ENV, + }); + + await recordSnapshot('after-startup'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for idle-session-startup: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + } + }); + + it('simple-prompt-response: memory usage within baseline', async () => { + rig = new TestRig(); + rig.setup('memory-simple-prompt', { + fakeResponsesPath: join(__dirname, 'memory.simple-prompt.responses'), + }); + + const result = await harness.runScenario( + rig, + 'simple-prompt-response', + async (recordSnapshot) => { + await rig.run({ + args: ['What is the capital of France?'], + timeout: 120000, + env: TEST_ENV, + }); + + await recordSnapshot('after-response'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for simple-prompt-response: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + } + }); + + it('multi-turn-conversation: memory remains stable over turns', async () => { + rig = new TestRig(); + rig.setup('memory-multi-turn', { + fakeResponsesPath: join(__dirname, 'memory.multi-turn.responses'), + }); + + const prompts = [ + 'Hello, what can you help me with?', + 'Tell me about JavaScript', + 'How is TypeScript different?', + 'Can you write a simple TypeScript function?', + 'What are some TypeScript best practices?', + ]; + + const result = await harness.runScenario( + rig, + 'multi-turn-conversation', + async (recordSnapshot) => { + // Run through all turns as a piped sequence + const stdinContent = prompts.join('\n'); + await rig.run({ + stdin: stdinContent, + timeout: 120000, + env: TEST_ENV, + }); + + // Take snapshots after the conversation completes + await recordSnapshot('after-all-turns'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for multi-turn-conversation: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + harness.assertMemoryReturnsToBaseline(result.snapshots, 20); + const { leaked, message } = harness.analyzeSnapshots(result.snapshots); + if (leaked) console.warn(`⚠ ${message}`); + } + }); + + it('multi-function-call-repo-search: memory after tool use', async () => { + rig = new TestRig(); + rig.setup('memory-multi-func-call', { + fakeResponsesPath: join( + __dirname, + 'memory.multi-function-call.responses', + ), + }); + + // Create directories first, then files in the workspace so the tools have targets + rig.mkdir('packages/core/src/telemetry'); + rig.createFile( + 'packages/core/src/telemetry/memory-monitor.ts', + 'export class MemoryMonitor { constructor() {} }', + ); + rig.createFile( + 'packages/core/src/telemetry/metrics.ts', + 'export function recordMemoryUsage() {}', + ); + + const result = await harness.runScenario( + rig, + 'multi-function-call-repo-search', + async (recordSnapshot) => { + await rig.run({ + args: [ + 'Search this repository for MemoryMonitor and tell me what it does', + ], + timeout: 120000, + env: TEST_ENV, + }); + + await recordSnapshot('after-tool-calls'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for multi-function-call-repo-search: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + harness.assertMemoryReturnsToBaseline(result.snapshots, 20); + } + }); + + describe('Large Chat Scenarios', () => { + let sharedResumeResponsesPath: string; + let sharedActiveResponsesPath: string; + let sharedHistoryPath: string; + let sharedPrompts: string; + let tempDir: string; + + beforeAll(async () => { + tempDir = join(__dirname, `large-chat-tmp-${randomUUID()}`); + mkdirSync(tempDir, { recursive: true }); + + const { resumeResponsesPath, activeResponsesPath, historyPath, prompts } = + await generateSharedLargeChatData(tempDir); + sharedActiveResponsesPath = activeResponsesPath; + sharedResumeResponsesPath = resumeResponsesPath; + sharedHistoryPath = historyPath; + sharedPrompts = prompts; + }, 60000); + + afterAll(() => { + if (existsSync(tempDir)) { + rmSync(tempDir, { recursive: true, force: true }); + } + }); + + afterEach(async () => { + await rig.cleanup(); + }); + + it('large-chat: memory usage within baseline', async () => { + rig = new TestRig(); + rig.setup('memory-large-chat', { + fakeResponsesPath: sharedActiveResponsesPath, + }); + + const result = await harness.runScenario( + rig, + 'large-chat', + async (recordSnapshot) => { + await rig.run({ + stdin: sharedPrompts, + timeout: 600000, + env: TEST_ENV, + }); + + await recordSnapshot('after-large-chat'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for large-chat: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + } + }); + + it('resume-large-chat: memory usage within baseline', async () => { + rig = new TestRig(); + rig.setup('memory-resume-large-chat', { + fakeResponsesPath: sharedResumeResponsesPath, + }); + + const result = await harness.runScenario( + rig, + 'resume-large-chat', + async (recordSnapshot) => { + // Ensure the history file is linked + const targetChatsDir = join( + rig.homeDir!, + '.gemini', + 'tmp', + getProjectHash(rig.testDir!), + 'chats', + ); + mkdirSync(targetChatsDir, { recursive: true }); + const targetHistoryPath = join( + targetChatsDir, + 'session-large-chat.json', + ); + if (existsSync(targetHistoryPath)) rmSync(targetHistoryPath); + copyFileSync(sharedHistoryPath, targetHistoryPath); + + await rig.run({ + // add a prompt to make sure it does not hang there and exits immediately + args: ['--resume', 'latest', '--prompt', 'hello'], + timeout: 600000, + env: TEST_ENV, + }); + + await recordSnapshot('after-resume-large-chat'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for resume-large-chat: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + } + }); + + it('resume-large-chat-with-messages: memory usage within baseline', async () => { + rig = new TestRig(); + rig.setup('memory-resume-large-chat-msgs', { + fakeResponsesPath: sharedResumeResponsesPath, + }); + + const result = await harness.runScenario( + rig, + 'resume-large-chat-with-messages', + async (recordSnapshot) => { + // Ensure the history file is linked + const targetChatsDir = join( + rig.homeDir!, + '.gemini', + 'tmp', + getProjectHash(rig.testDir!), + 'chats', + ); + mkdirSync(targetChatsDir, { recursive: true }); + const targetHistoryPath = join( + targetChatsDir, + 'session-large-chat.json', + ); + if (existsSync(targetHistoryPath)) rmSync(targetHistoryPath); + copyFileSync(sharedHistoryPath, targetHistoryPath); + + const stdinContent = 'new prompt 1\nnew prompt 2\n'; + + await rig.run({ + args: ['--resume', 'latest'], + stdin: stdinContent, + timeout: 600000, + env: TEST_ENV, + }); + + await recordSnapshot('after-resume-and-append'); + }, + ); + + if (UPDATE_BASELINES) { + harness.updateScenarioBaseline(result); + console.log( + `Updated baseline for resume-large-chat-with-messages: ${(result.finalHeapUsed / (1024 * 1024)).toFixed(1)} MB`, + ); + } else { + harness.assertWithinBaseline(result); + } + }); + }); +}); + +async function generateSharedLargeChatData(tempDir: string) { + const resumeResponsesPath = join(tempDir, 'large-chat-resume-chat.responses'); + const activeResponsesPath = join(tempDir, 'large-chat-active-chat.responses'); + const historyPath = join(tempDir, 'large-chat-history.json'); + const sourceSessionPath = join(__dirname, 'large-chat-session.json'); + + const session = JSON.parse(readFileSync(sourceSessionPath, 'utf8')); + const messages = session.messages; + + copyFileSync(sourceSessionPath, historyPath); + + // Generate fake responses for active chat + const promptsList: string[] = []; + const activeResponsesStream = createWriteStream(activeResponsesPath); + const complexityResponse = { + method: 'generateContent', + response: { + candidates: [ + { + content: { + parts: [ + { + text: '{"complexity_reasoning":"simple","complexity_score":1}', + }, + ], + role: 'model', + }, + finishReason: 'STOP', + index: 0, + }, + ], + }, + }; + const summaryResponse = { + method: 'generateContent', + response: { + candidates: [ + { + content: { + parts: [ + { text: '{"originalSummary":"large chat summary","events":[]}' }, + ], + role: 'model', + }, + finishReason: 'STOP', + index: 0, + }, + ], + }, + }; + + for (let i = 0; i < messages.length; i++) { + const msg = messages[i]; + if (msg.type === 'user') { + promptsList.push(msg.content[0].text); + + // Start of a new turn + activeResponsesStream.write(JSON.stringify(complexityResponse) + '\n'); + + // Find all subsequent gemini messages until the next user message + let j = i + 1; + while (j < messages.length && messages[j].type === 'gemini') { + const geminiMsg = messages[j]; + const parts = []; + if (geminiMsg.content) { + parts.push({ text: geminiMsg.content }); + } + if (geminiMsg.toolCalls) { + for (const tc of geminiMsg.toolCalls) { + parts.push({ + functionCall: { + name: tc.name, + args: tc.args, + }, + }); + } + } + + activeResponsesStream.write( + JSON.stringify({ + method: 'generateContentStream', + response: [ + { + candidates: [ + { + content: { parts, role: 'model' }, + finishReason: 'STOP', + index: 0, + }, + ], + usageMetadata: { + promptTokenCount: 100, + candidatesTokenCount: 100, + totalTokenCount: 200, + promptTokensDetails: [{ modality: 'TEXT', tokenCount: 100 }], + }, + }, + ], + }) + '\n', + ); + j++; + } + // End of turn + activeResponsesStream.write(JSON.stringify(summaryResponse) + '\n'); + // Skip the gemini messages we just processed + i = j - 1; + } + } + activeResponsesStream.end(); + + // Generate responses for resumed chat + const resumeResponsesStream = createWriteStream(resumeResponsesPath); + for (let i = 0; i < 5; i++) { + // Doubling up on non-streaming responses to satisfy classifier and complexity checks + resumeResponsesStream.write(JSON.stringify(complexityResponse) + '\n'); + resumeResponsesStream.write(JSON.stringify(summaryResponse) + '\n'); + resumeResponsesStream.write(JSON.stringify(complexityResponse) + '\n'); + resumeResponsesStream.write( + JSON.stringify({ + method: 'generateContentStream', + response: [ + { + candidates: [ + { + content: { + parts: [{ text: `Resume response ${i}` }], + role: 'model', + }, + finishReason: 'STOP', + index: 0, + }, + ], + usageMetadata: { + promptTokenCount: 10, + candidatesTokenCount: 10, + totalTokenCount: 20, + promptTokensDetails: [{ modality: 'TEXT', tokenCount: 10 }], + }, + }, + ], + }) + '\n', + ); + resumeResponsesStream.write(JSON.stringify(summaryResponse) + '\n'); + } + resumeResponsesStream.end(); + + // Wait for streams to finish + await Promise.all([ + new Promise((res) => + activeResponsesStream.on('finish', () => res(undefined)), + ), + new Promise((res) => + resumeResponsesStream.on('finish', () => res(undefined)), + ), + ]); + + return { + resumeResponsesPath, + activeResponsesPath, + historyPath, + prompts: promptsList.join('\n'), + }; +} diff --git a/memory-tests/memory.idle-startup.responses b/memory-tests/memory.idle-startup.responses new file mode 100644 index 0000000000000000000000000000000000000000..7a5703e3d26142ec2139228a25e51b785b356888 --- /dev/null +++ b/memory-tests/memory.idle-startup.responses @@ -0,0 +1,2 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help. What would you like to work on?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":12,"totalTokenCount":17,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]} diff --git a/memory-tests/memory.multi-function-call.responses b/memory-tests/memory.multi-function-call.responses new file mode 100644 index 0000000000000000000000000000000000000000..8bdf75afc9dd91450be0fdd94f9f82cb5bcaa023 --- /dev/null +++ b/memory-tests/memory.multi-function-call.responses @@ -0,0 +1,4 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I'll search for MemoryMonitor in the repository and analyze what it does."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":30,"candidatesTokenCount":15,"totalTokenCount":45,"promptTokensDetails":[{"modality":"TEXT","tokenCount":30}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"functionCall":{"name":"grep_search","args":{"pattern":"MemoryMonitor","path":".","include_pattern":"*.ts"}}},{"functionCall":{"name":"list_directory","args":{"path":"packages/core/src/telemetry"}}},{"functionCall":{"name":"read_file","args":{"file_path":"packages/core/src/telemetry/memory-monitor.ts"}}}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":30,"candidatesTokenCount":80,"totalTokenCount":110,"promptTokensDetails":[{"modality":"TEXT","tokenCount":30}]}}]} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"I found the memory monitoring code. Here's a summary:\n\nThe `MemoryMonitor` class in `packages/core/src/telemetry/memory-monitor.ts` provides:\n\n1. **Continuous monitoring** via `start()`/`stop()` with configurable intervals\n2. **V8 heap snapshots** using `v8.getHeapStatistics()` and `process.memoryUsage()`\n3. **High-water mark tracking** to detect significant memory growth\n4. **Rate-limited recording** to avoid metric flood\n5. **Activity detection** β€” only records when user is active\n\nThe class uses a singleton pattern via `initializeMemoryMonitor()` for global access."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":500,"candidatesTokenCount":120,"totalTokenCount":620,"promptTokensDetails":[{"modality":"TEXT","tokenCount":500}]}}]} diff --git a/memory-tests/memory.multi-turn.responses b/memory-tests/memory.multi-turn.responses new file mode 100644 index 0000000000000000000000000000000000000000..df428b56dbde043e537441b41a2fa6f86e7d1010 --- /dev/null +++ b/memory-tests/memory.multi-turn.responses @@ -0,0 +1,10 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Hello! I'm ready to help you with your coding tasks. What would you like to work on today?"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":5,"candidatesTokenCount":18,"totalTokenCount":23,"promptTokensDetails":[{"modality":"TEXT","tokenCount":5}]}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"JavaScript is a high-level, interpreted programming language. It was originally designed for adding interactivity to web pages."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":25,"candidatesTokenCount":60,"totalTokenCount":85,"promptTokensDetails":[{"modality":"TEXT","tokenCount":25}]}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"TypeScript is a typed superset of JavaScript developed by Microsoft. The main differences from JavaScript are static typing and better tooling."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":45,"candidatesTokenCount":80,"totalTokenCount":125,"promptTokensDetails":[{"modality":"TEXT","tokenCount":45}]}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Here is a simple TypeScript function:\n\nfunction greet(name: string): string { return `Hello, ${name}!`; }"}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":60,"candidatesTokenCount":55,"totalTokenCount":115,"promptTokensDetails":[{"modality":"TEXT","tokenCount":60}]}}]} +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Here are 5 key TypeScript best practices: Enable strict mode, prefer interfaces, use union types, leverage type inference, and use readonly."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":75,"candidatesTokenCount":70,"totalTokenCount":145,"promptTokensDetails":[{"modality":"TEXT","tokenCount":75}]}}]} diff --git a/memory-tests/memory.simple-prompt.responses b/memory-tests/memory.simple-prompt.responses new file mode 100644 index 0000000000000000000000000000000000000000..ad3f20c9a1cf8ebffe155b415eeb22bf88137787 --- /dev/null +++ b/memory-tests/memory.simple-prompt.responses @@ -0,0 +1,2 @@ +{"method":"generateContent","response":{"candidates":[{"content":{"parts":[{"text":"0"}],"role":"model"},"finishReason":"STOP","index":0}]}} +{"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The capital of France is Paris. It has been the capital since the 10th century and is known for iconic landmarks like the Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral. Paris is also the most populous city in France, with a metropolitan area population of over 12 million people."}],"role":"model"},"finishReason":"STOP","index":0}],"usageMetadata":{"promptTokenCount":7,"candidatesTokenCount":55,"totalTokenCount":62,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7}]}}]} diff --git a/memory-tests/tsconfig.json b/memory-tests/tsconfig.json new file mode 100644 index 0000000000000000000000000000000000000000..7f2c199703ed1a2ed8f1407bd4ed670b3f2962e4 --- /dev/null +++ b/memory-tests/tsconfig.json @@ -0,0 +1,12 @@ +{ + "extends": "../tsconfig.json", + "compilerOptions": { + "noEmit": true, + "allowJs": true + }, + "include": ["**/*.ts"], + "references": [ + { "path": "../packages/core" }, + { "path": "../packages/test-utils" } + ] +} diff --git a/memory-tests/vitest.config.ts b/memory-tests/vitest.config.ts new file mode 100644 index 0000000000000000000000000000000000000000..c69af28826b0a3c9b24b863dd65f3619ceeec82a --- /dev/null +++ b/memory-tests/vitest.config.ts @@ -0,0 +1,28 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + testTimeout: 600000, // 10 minutes β€” memory profiling is slow + globalSetup: './globalSetup.ts', + reporters: ['default'], + include: ['**/*.test.ts'], + retry: 0, // No retries for memory tests β€” noise is handled by tolerance + fileParallelism: false, // Must run serially to avoid memory interference + pool: 'forks', // Use forks pool for --expose-gc support + poolOptions: { + forks: { + singleFork: true, // Single process for accurate per-test memory readings + execArgv: ['--expose-gc'], // Enable global.gc() for forced GC + }, + }, + env: { + GEMINI_TEST_TYPE: 'memory', + }, + }, +}); diff --git a/schemas/settings.schema.json b/schemas/settings.schema.json new file mode 100644 index 0000000000000000000000000000000000000000..f78ee6b924d8ee330ea98494b217b03dfa067129 --- /dev/null +++ b/schemas/settings.schema.json @@ -0,0 +1,4617 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://raw.githubusercontent.com/google-gemini/gemini-cli/main/schemas/settings.schema.json", + "title": "Gemini CLI Settings", + "description": "Configuration file schema for Gemini CLI settings. This schema enables IDE completion for `settings.json`.", + "type": "object", + "additionalProperties": false, + "properties": { + "$schema": { + "title": "Schema", + "description": "The URL of the JSON schema for this settings file. Used by editors for validation and autocompletion.", + "type": "string", + "default": "https://raw.githubusercontent.com/google-gemini/gemini-cli/main/schemas/settings.schema.json" + }, + "mcpServers": { + "title": "MCP Servers", + "description": "Configuration for MCP servers.", + "markdownDescription": "Configuration for MCP servers.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/MCPServerConfig" + } + }, + "policyPaths": { + "title": "Policy Paths", + "description": "Additional policy files or directories to load.", + "markdownDescription": "Additional policy files or directories to load.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "adminPolicyPaths": { + "title": "Admin Policy Paths", + "description": "Additional admin policy files or directories to load.", + "markdownDescription": "Additional admin policy files or directories to load.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "general": { + "title": "General", + "description": "General application settings.", + "markdownDescription": "General application settings.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "preferredEditor": { + "title": "Preferred Editor", + "description": "The preferred editor to open files in. Must be one of the built-in supported identifiers. Use /editor in the CLI to pick interactively, or leave unset to use $VISUAL/$EDITOR.", + "markdownDescription": "The preferred editor to open files in. Must be one of the built-in supported identifiers. Use /editor in the CLI to pick interactively, or leave unset to use $VISUAL/$EDITOR.\n\n- Category: `General`\n- Requires restart: `no`", + "type": "string", + "enum": [ + "vscode", + "vscodium", + "windsurf", + "cursor", + "zed", + "antigravity", + "sublimetext", + "lapce", + "nova", + "bbedit", + "vim", + "neovim", + "emacs", + "hx", + "emacsclient", + "micro" + ] + }, + "openEditorInNewWindow": { + "title": "Open Editor in New Window", + "description": "Open VS Code-family editors in a new window when editing files.", + "markdownDescription": "Open VS Code-family editors in a new window when editing files.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "vimMode": { + "title": "Vim Mode", + "description": "Enable Vim keybindings", + "markdownDescription": "Enable Vim keybindings\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "defaultApprovalMode": { + "title": "Default Approval Mode", + "description": "The default approval mode for tool execution. 'default' prompts for approval, 'auto_edit' auto-approves edit tools, and 'plan' is read-only mode. YOLO mode (auto-approve all actions) can only be enabled via command line (--yolo or --approval-mode=yolo).", + "markdownDescription": "The default approval mode for tool execution. 'default' prompts for approval, 'auto_edit' auto-approves edit tools, and 'plan' is read-only mode. YOLO mode (auto-approve all actions) can only be enabled via command line (--yolo or --approval-mode=yolo).\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `default`", + "default": "default", + "type": "string", + "enum": ["default", "auto_edit", "plan"] + }, + "devtools": { + "title": "DevTools", + "description": "Enable DevTools inspector on launch.", + "markdownDescription": "Enable DevTools inspector on launch.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "enableAutoUpdate": { + "title": "Enable Auto Update", + "description": "Enable automatic updates.", + "markdownDescription": "Enable automatic updates.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "enableAutoUpdateNotification": { + "title": "Enable Auto Update Notification", + "description": "Enable update notification prompts.", + "markdownDescription": "Enable update notification prompts.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "enableNotifications": { + "title": "Enable Terminal Notifications", + "description": "Enable terminal run-event notifications for action-required prompts and session completion.", + "markdownDescription": "Enable terminal run-event notifications for action-required prompts and session completion.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "notificationMethod": { + "title": "Terminal Notification Method", + "description": "How to send terminal notifications.", + "markdownDescription": "How to send terminal notifications.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `auto`", + "default": "auto", + "type": "string", + "enum": ["auto", "osc9", "osc777", "bell"] + }, + "checkpointing": { + "title": "Checkpointing", + "description": "Session checkpointing settings.", + "markdownDescription": "Session checkpointing settings.\n\n- Category: `General`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Enable Checkpointing", + "description": "Enable session checkpointing for recovery", + "markdownDescription": "Enable session checkpointing for recovery\n\n- Category: `General`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "plan": { + "title": "Plan", + "description": "Planning features configuration.", + "markdownDescription": "Planning features configuration.\n\n- Category: `General`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Enable Plan Mode", + "description": "Enable Plan Mode for read-only safety during planning.", + "markdownDescription": "Enable Plan Mode for read-only safety during planning.\n\n- Category: `General`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "directory": { + "title": "Plan Directory", + "description": "The directory where planning artifacts are stored. If not specified, defaults to the system temporary directory. A custom directory requires a policy to allow write access in Plan Mode.", + "markdownDescription": "The directory where planning artifacts are stored. If not specified, defaults to the system temporary directory. A custom directory requires a policy to allow write access in Plan Mode.\n\n- Category: `General`\n- Requires restart: `yes`", + "type": "string" + }, + "modelRouting": { + "title": "Plan Model Routing", + "description": "Automatically switch between Pro and Flash models based on Plan Mode status. Uses Pro for the planning phase and Flash for the implementation phase.", + "markdownDescription": "Automatically switch between Pro and Flash models based on Plan Mode status. Uses Pro for the planning phase and Flash for the implementation phase.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "retryFetchErrors": { + "title": "Retry Fetch Errors", + "description": "Retry on \"exception TypeError: fetch failed sending request\" errors.", + "markdownDescription": "Retry on \"exception TypeError: fetch failed sending request\" errors.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "maxAttempts": { + "title": "Max Chat Model Attempts", + "description": "Maximum number of attempts for requests to the main chat model. Cannot exceed 10.", + "markdownDescription": "Maximum number of attempts for requests to the main chat model. Cannot exceed 10.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `10`", + "default": 10, + "type": "number" + }, + "debugKeystrokeLogging": { + "title": "Debug Keystroke Logging", + "description": "Enable debug logging of keystrokes to the console.", + "markdownDescription": "Enable debug logging of keystrokes to the console.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "sessionRetention": { + "title": "Session Retention", + "description": "Settings for automatic session cleanup.", + "markdownDescription": "Settings for automatic session cleanup.\n\n- Category: `General`\n- Requires restart: `no`", + "type": "object", + "properties": { + "enabled": { + "title": "Enable Session Cleanup", + "description": "Enable automatic session cleanup", + "markdownDescription": "Enable automatic session cleanup\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "maxAge": { + "title": "Keep chat history", + "description": "Automatically delete chats older than this time period (e.g., \"30d\", \"7d\", \"24h\", \"1w\")", + "markdownDescription": "Automatically delete chats older than this time period (e.g., \"30d\", \"7d\", \"24h\", \"1w\")\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `30d`", + "default": "30d", + "type": "string" + }, + "maxCount": { + "title": "Max Session Count", + "description": "Alternative: Maximum number of sessions to keep (most recent)", + "markdownDescription": "Alternative: Maximum number of sessions to keep (most recent)\n\n- Category: `General`\n- Requires restart: `no`", + "type": "number" + }, + "minRetention": { + "title": "Min Retention Period", + "description": "Minimum retention period (safety limit, defaults to \"1d\")", + "markdownDescription": "Minimum retention period (safety limit, defaults to \"1d\")\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `1d`", + "default": "1d", + "type": "string" + } + }, + "additionalProperties": false + }, + "topicUpdateNarration": { + "title": "Topic & Update Narration", + "description": "Enable the Topic & Update communication model for reduced chattiness and structured progress reporting.", + "markdownDescription": "Enable the Topic & Update communication model for reduced chattiness and structured progress reporting.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "logRagSnippets": { + "title": "Log RAG Snippets", + "description": "Log full Code Customization (RAG) retrieved snippets to a local file for debugging.", + "markdownDescription": "Log full Code Customization (RAG) retrieved snippets to a local file for debugging.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "output": { + "title": "Output", + "description": "Settings for the CLI output.", + "markdownDescription": "Settings for the CLI output.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "format": { + "title": "Output Format", + "description": "The format of the CLI output. Can be `text` or `json`.", + "markdownDescription": "The format of the CLI output. Can be `text` or `json`.\n\n- Category: `General`\n- Requires restart: `no`\n- Default: `text`", + "default": "text", + "type": "string", + "enum": ["text", "json"] + } + }, + "additionalProperties": false + }, + "ui": { + "title": "UI", + "description": "User interface settings.", + "markdownDescription": "User interface settings.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "debugRainbow": { + "title": "Debug Rainbow", + "description": "Enable debug rainbow rendering. Only useful for debugging rendering bugs and performance issues.", + "markdownDescription": "Enable debug rainbow rendering. Only useful for debugging rendering bugs and performance issues.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "theme": { + "title": "Theme", + "description": "The color theme for the UI. See the CLI themes guide for available options.", + "markdownDescription": "The color theme for the UI. See the CLI themes guide for available options.\n\n- Category: `UI`\n- Requires restart: `no`", + "type": "string" + }, + "autoThemeSwitching": { + "title": "Auto Theme Switching", + "description": "Automatically switch between default light and dark themes based on terminal background color.", + "markdownDescription": "Automatically switch between default light and dark themes based on terminal background color.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "terminalBackgroundPollingInterval": { + "title": "Terminal Background Polling Interval", + "description": "Interval in seconds to poll the terminal background color.", + "markdownDescription": "Interval in seconds to poll the terminal background color.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `60`", + "default": 60, + "type": "number" + }, + "customThemes": { + "title": "Custom Themes", + "description": "Custom theme definitions.", + "markdownDescription": "Custom theme definitions.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/CustomTheme" + } + }, + "hideWindowTitle": { + "title": "Hide Window Title", + "description": "Hide the window title bar", + "markdownDescription": "Hide the window title bar\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "inlineThinkingMode": { + "title": "Inline Thinking", + "description": "Display model thinking inline: off or full.", + "markdownDescription": "Display model thinking inline: off or full.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `off`", + "default": "off", + "type": "string", + "enum": ["off", "full"] + }, + "showStatusInTitle": { + "title": "Show Thoughts in Title", + "description": "Show Gemini CLI model thoughts in the terminal window title during the working phase", + "markdownDescription": "Show Gemini CLI model thoughts in the terminal window title during the working phase\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "dynamicWindowTitle": { + "title": "Dynamic Window Title", + "description": "Update the terminal window title with current status icons (Ready: β—‡, Action Required: βœ‹, Working: ✦)", + "markdownDescription": "Update the terminal window title with current status icons (Ready: β—‡, Action Required: βœ‹, Working: ✦)\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "showHomeDirectoryWarning": { + "title": "Show Home Directory Warning", + "description": "Show a warning when running Gemini CLI in the home directory.", + "markdownDescription": "Show a warning when running Gemini CLI in the home directory.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "showCompatibilityWarnings": { + "title": "Show Compatibility Warnings", + "description": "Show warnings about terminal or OS compatibility issues.", + "markdownDescription": "Show warnings about terminal or OS compatibility issues.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "hideTips": { + "title": "Hide Tips", + "description": "Hide helpful tips in the UI", + "markdownDescription": "Hide helpful tips in the UI\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "escapePastedAtSymbols": { + "title": "Escape Pasted @ Symbols", + "description": "When enabled, @ symbols in pasted text are escaped to prevent unintended @path expansion.", + "markdownDescription": "When enabled, @ symbols in pasted text are escaped to prevent unintended @path expansion.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "showShortcutsHint": { + "title": "Show Shortcuts Hint", + "description": "Show the \"? for shortcuts\" hint above the input.", + "markdownDescription": "Show the \"? for shortcuts\" hint above the input.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "compactToolOutput": { + "title": "Compact Tool Output", + "description": "Display tool outputs (like directory listings and file reads) in a compact, structured format.", + "markdownDescription": "Display tool outputs (like directory listings and file reads) in a compact, structured format.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "hideBanner": { + "title": "Hide Banner", + "description": "Hide the application banner", + "markdownDescription": "Hide the application banner\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "hideContextSummary": { + "title": "Hide Context Summary", + "description": "Hide the context summary (GEMINI.md, MCP servers) above the input.", + "markdownDescription": "Hide the context summary (GEMINI.md, MCP servers) above the input.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "footer": { + "title": "Footer", + "description": "Settings for the footer.", + "markdownDescription": "Settings for the footer.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "items": { + "title": "Footer Items", + "description": "List of item IDs to display in the footer. Rendered in order", + "markdownDescription": "List of item IDs to display in the footer. Rendered in order\n\n- Category: `UI`\n- Requires restart: `no`", + "type": "array", + "items": { + "type": "string" + } + }, + "showLabels": { + "title": "Show Footer Labels", + "description": "Display a second line above the footer items with descriptive headers (e.g., /model).", + "markdownDescription": "Display a second line above the footer items with descriptive headers (e.g., /model).\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "hideCWD": { + "title": "Hide CWD", + "description": "Hide the current working directory in the footer.", + "markdownDescription": "Hide the current working directory in the footer.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "hideSandboxStatus": { + "title": "Hide Sandbox Status", + "description": "Hide the sandbox status indicator in the footer.", + "markdownDescription": "Hide the sandbox status indicator in the footer.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "hideModelInfo": { + "title": "Hide Model Info", + "description": "Hide the model name and context usage in the footer.", + "markdownDescription": "Hide the model name and context usage in the footer.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "hideContextPercentage": { + "title": "Hide Context Window Percentage", + "description": "Hides the context window usage percentage.", + "markdownDescription": "Hides the context window usage percentage.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "hideFooter": { + "title": "Hide Footer", + "description": "Hide the footer from the UI", + "markdownDescription": "Hide the footer from the UI\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "collapseDrawerDuringApproval": { + "title": "Collapse Drawer During Approval", + "description": "Whether to collapse the UI drawer when a tool is awaiting confirmation.", + "markdownDescription": "Whether to collapse the UI drawer when a tool is awaiting confirmation.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "showMemoryUsage": { + "title": "Show Memory Usage", + "description": "Display memory usage information in the UI", + "markdownDescription": "Display memory usage information in the UI\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "showLineNumbers": { + "title": "Show Line Numbers", + "description": "Show line numbers in the chat.", + "markdownDescription": "Show line numbers in the chat.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "showCitations": { + "title": "Show Citations", + "description": "Show citations for generated text in the chat.", + "markdownDescription": "Show citations for generated text in the chat.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "showModelInfoInChat": { + "title": "Show Model Info In Chat", + "description": "Show the model name in the chat for each model turn.", + "markdownDescription": "Show the model name in the chat for each model turn.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "showUserIdentity": { + "title": "Show User Identity", + "description": "Show the signed-in user's identity (e.g. email) in the UI.", + "markdownDescription": "Show the signed-in user's identity (e.g. email) in the UI.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "useAlternateBuffer": { + "title": "Use Alternate Screen Buffer", + "description": "Use an alternate screen buffer for the UI, preserving shell history.", + "markdownDescription": "Use an alternate screen buffer for the UI, preserving shell history.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "renderProcess": { + "title": "Render Process", + "description": "Enable Ink render process for the UI.", + "markdownDescription": "Enable Ink render process for the UI.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "terminalBuffer": { + "title": "Terminal Buffer", + "description": "Use the new terminal buffer architecture for rendering.", + "markdownDescription": "Use the new terminal buffer architecture for rendering.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "useBackgroundColor": { + "title": "Use Background Color", + "description": "Whether to use background colors in the UI.", + "markdownDescription": "Whether to use background colors in the UI.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "incrementalRendering": { + "title": "Incremental Rendering", + "description": "Enable incremental rendering for the UI. This option will reduce flickering but may cause rendering artifacts. Only supported when useAlternateBuffer is enabled.", + "markdownDescription": "Enable incremental rendering for the UI. This option will reduce flickering but may cause rendering artifacts. Only supported when useAlternateBuffer is enabled.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "showSpinner": { + "title": "Show Spinner", + "description": "Show the spinner during operations.", + "markdownDescription": "Show the spinner during operations.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "loadingPhrases": { + "title": "Loading Phrases", + "description": "What to show while the model is working: tips, witty comments, all, or off.", + "markdownDescription": "What to show while the model is working: tips, witty comments, all, or off.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `off`", + "default": "off", + "type": "string", + "enum": ["tips", "witty", "all", "off"] + }, + "errorVerbosity": { + "title": "Error Verbosity", + "description": "Controls whether recoverable errors are hidden (low) or fully shown (full).", + "markdownDescription": "Controls whether recoverable errors are hidden (low) or fully shown (full).\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `low`", + "default": "low", + "type": "string", + "enum": ["low", "full"] + }, + "customWittyPhrases": { + "title": "Custom Witty Phrases", + "description": "Custom witty phrases to display during loading. When provided, the CLI cycles through these instead of the defaults.", + "markdownDescription": "Custom witty phrases to display during loading. When provided, the CLI cycles through these instead of the defaults.\n\n- Category: `UI`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "accessibility": { + "title": "Accessibility", + "description": "Accessibility settings.", + "markdownDescription": "Accessibility settings.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enableLoadingPhrases": { + "title": "Enable Loading Phrases", + "description": "@deprecated Use ui.loadingPhrases instead. Enable loading phrases during operations.", + "markdownDescription": "@deprecated Use ui.loadingPhrases instead. Enable loading phrases during operations.\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "screenReader": { + "title": "Screen Reader Mode", + "description": "Render output in plain-text to be more screen reader accessible", + "markdownDescription": "Render output in plain-text to be more screen reader accessible\n\n- Category: `UI`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "ide": { + "title": "IDE", + "description": "IDE integration settings.", + "markdownDescription": "IDE integration settings.\n\n- Category: `IDE`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "IDE Mode", + "description": "Enable IDE integration mode.", + "markdownDescription": "Enable IDE integration mode.\n\n- Category: `IDE`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "hasSeenNudge": { + "title": "Has Seen IDE Integration Nudge", + "description": "Whether the user has seen the IDE integration nudge.", + "markdownDescription": "Whether the user has seen the IDE integration nudge.\n\n- Category: `IDE`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "privacy": { + "title": "Privacy", + "description": "Privacy-related settings.", + "markdownDescription": "Privacy-related settings.\n\n- Category: `Privacy`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "usageStatisticsEnabled": { + "title": "Enable Usage Statistics", + "description": "Enable collection of usage statistics", + "markdownDescription": "Enable collection of usage statistics\n\n- Category: `Privacy`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "telemetry": { + "title": "Telemetry", + "description": "Telemetry configuration.", + "markdownDescription": "Telemetry configuration.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "$ref": "#/$defs/TelemetrySettings" + }, + "billing": { + "title": "Billing", + "description": "Billing and AI credits settings.", + "markdownDescription": "Billing and AI credits settings.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "overageStrategy": { + "title": "Overage Strategy", + "description": "How to handle quota exhaustion when AI credits are available. 'ask' prompts each time, 'always' automatically uses credits, 'never' disables credit usage.", + "markdownDescription": "How to handle quota exhaustion when AI credits are available. 'ask' prompts each time, 'always' automatically uses credits, 'never' disables credit usage.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `ask`", + "default": "ask", + "type": "string", + "enum": ["ask", "always", "never"] + }, + "vertexAi": { + "title": "Vertex AI", + "description": "Vertex AI request routing settings.", + "markdownDescription": "Vertex AI request routing settings.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "object", + "properties": { + "requestType": { + "title": "Vertex AI Request Type", + "description": "Sets the X-Vertex-AI-LLM-Request-Type header for Vertex AI requests.", + "markdownDescription": "Sets the X-Vertex-AI-LLM-Request-Type header for Vertex AI requests.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string", + "enum": ["dedicated", "shared"] + }, + "sharedRequestType": { + "title": "Vertex AI Shared Request Type", + "description": "Sets the X-Vertex-AI-LLM-Shared-Request-Type header for Vertex AI requests.", + "markdownDescription": "Sets the X-Vertex-AI-LLM-Shared-Request-Type header for Vertex AI requests.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string", + "enum": ["priority", "flex"] + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "model": { + "title": "Model", + "description": "Settings related to the generative model.", + "markdownDescription": "Settings related to the generative model.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "name": { + "title": "Model", + "description": "The Gemini model to use for conversations.", + "markdownDescription": "The Gemini model to use for conversations.\n\n- Category: `Model`\n- Requires restart: `no`", + "type": "string" + }, + "maxSessionTurns": { + "title": "Max Session Turns", + "description": "Maximum number of user/model/tool turns to keep in a session. -1 means unlimited.", + "markdownDescription": "Maximum number of user/model/tool turns to keep in a session. -1 means unlimited.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `-1`", + "default": -1, + "type": "number" + }, + "summarizeToolOutput": { + "title": "Summarize Tool Output", + "description": "Enables or disables summarization of tool output. Configure per-tool token budgets (for example {\"run_shell_command\": {\"tokenBudget\": 2000}}). Currently only the run_shell_command tool supports summarization.", + "markdownDescription": "Enables or disables summarization of tool output. Configure per-tool token budgets (for example {\"run_shell_command\": {\"tokenBudget\": 2000}}). Currently only the run_shell_command tool supports summarization.\n\n- Category: `Model`\n- Requires restart: `no`", + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/SummarizeToolOutputSettings" + } + }, + "compressionThreshold": { + "title": "Context Compression Threshold", + "description": "The fraction of context usage at which to trigger context compression (e.g. 0.2, 0.3).", + "markdownDescription": "The fraction of context usage at which to trigger context compression (e.g. 0.2, 0.3).\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `0.5`", + "default": 0.5, + "type": "number" + }, + "disableLoopDetection": { + "title": "Disable Loop Detection", + "description": "Disable automatic detection and prevention of infinite loops.", + "markdownDescription": "Disable automatic detection and prevention of infinite loops.\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "skipNextSpeakerCheck": { + "title": "Skip Next Speaker Check", + "description": "Skip the next speaker check.", + "markdownDescription": "Skip the next speaker check.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "modelConfigs": { + "title": "Model Configs", + "description": "Model configurations.", + "markdownDescription": "Model configurations.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `{\n \"aliases\": {\n \"base\": {\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"temperature\": 0,\n \"topP\": 1\n }\n }\n },\n \"chat-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"includeThoughts\": true\n },\n \"temperature\": 1,\n \"topP\": 0.95,\n \"topK\": 64\n }\n }\n },\n \"chat-base-2.5\": {\n \"extends\": \"chat-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingBudget\": 8192\n }\n }\n }\n },\n \"chat-base-3\": {\n \"extends\": \"chat-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingLevel\": \"HIGH\"\n }\n }\n }\n },\n \"gemini-3-pro-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"gemini-3-flash-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"gemini-3.1-pro-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-pro-preview\"\n }\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-pro-preview-customtools\"\n }\n },\n \"gemini-3.1-flash-lite-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite-preview\"\n }\n },\n \"gemini-2.5-pro\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-pro\"\n }\n },\n \"gemini-2.5-flash\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"gemini-2.5-flash-lite\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash-lite\"\n }\n },\n \"gemini-3.1-flash-lite\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite\"\n }\n },\n \"gemini-3.5-flash\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.5-flash\"\n }\n },\n \"gemma-4-31b-it\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemma-4-31b-it\"\n }\n },\n \"gemma-4-26b-a4b-it\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemma-4-26b-a4b-it\"\n }\n },\n \"gemini-2.5-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"gemini-3-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"gemini-3.5-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3.5-flash\"\n }\n },\n \"classifier\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 1024,\n \"thinkingConfig\": {\n \"thinkingBudget\": 512\n }\n }\n }\n },\n \"prompt-completion\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"temperature\": 0.3,\n \"maxOutputTokens\": 16000,\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"fast-ack-helper\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"temperature\": 0.2,\n \"maxOutputTokens\": 120,\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"edit-corrector\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"summarizer-default\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 2000\n }\n }\n },\n \"summarizer-shell\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 2000\n }\n }\n },\n \"web-search\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"tools\": [\n {\n \"googleSearch\": {}\n }\n ]\n }\n }\n },\n \"web-fetch\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"tools\": [\n {\n \"urlContext\": {}\n }\n ]\n }\n }\n },\n \"web-fetch-fallback\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"loop-detection\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"loop-detection-double-check\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"llm-edit-fixer\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"next-speaker-checker\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"context-snapshotter\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingLevel\": \"HIGH\"\n },\n \"temperature\": 1,\n \"topP\": 0.95,\n \"topK\": 64\n }\n }\n },\n \"chat-compression-3-pro\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"chat-compression-3-flash\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"chat-compression-3.1-flash-lite\": {\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite\"\n }\n },\n \"chat-compression-2.5-pro\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-pro\"\n }\n },\n \"chat-compression-2.5-flash\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"chat-compression-2.5-flash-lite\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash-lite\"\n }\n },\n \"chat-compression-default\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"agent-history-provider-summarizer\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n }\n },\n \"overrides\": [\n {\n \"match\": {\n \"model\": \"chat-base\",\n \"isRetry\": true\n },\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"temperature\": 1\n }\n }\n }\n ],\n \"modelDefinitions\": {\n \"gemini-3.1-flash-lite\": {\n \"tier\": \"flash-lite\",\n \"family\": \"gemini-3\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.1-pro-preview\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3-pro-preview\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3-flash-preview\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.5-flash\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-3\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-2.5-pro\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemini-2.5-flash\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemini-2.5-flash-lite\": {\n \"tier\": \"flash-lite\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemma-4-31b-it\": {\n \"displayName\": \"gemma-4-31b-it\",\n \"tier\": \"custom\",\n \"family\": \"gemma-4\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"gemma-4-26b-a4b-it\": {\n \"displayName\": \"gemma-4-26b-a4b-it\",\n \"tier\": \"custom\",\n \"family\": \"gemma-4\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"auto\": {\n \"displayName\": \"Auto\",\n \"tier\": \"auto\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"pro\": {\n \"tier\": \"pro\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"flash\": {\n \"tier\": \"flash\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"flash-lite\": {\n \"tier\": \"flash-lite\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"auto-gemini-3\": {\n \"tier\": \"auto\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": false\n },\n \"auto-gemini-2.5\": {\n \"tier\": \"auto\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": false\n }\n },\n \"modelIdResolutions\": {\n \"gemma-4-31b-it\": {\n \"default\": \"gemma-4-31b-it\"\n },\n \"gemma-4-26b-a4b-it\": {\n \"default\": \"gemma-4-26b-a4b-it\"\n },\n \"gemini-3.1-pro-preview\": {\n \"default\": \"gemini-3.1-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n }\n ]\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"default\": \"gemini-3.1-pro-preview-customtools\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n }\n ]\n },\n \"gemini-3-flash-preview\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false,\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false,\n \"useGemini3_5Flash\": false\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"gemini-3.5-flash\": {\n \"default\": \"gemini-3.5-flash\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": false,\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n },\n {\n \"condition\": {\n \"useGemini3_5Flash\": false\n },\n \"target\": \"gemini-3-flash-preview\"\n }\n ]\n },\n \"gemini-2.5-flash\": {\n \"default\": \"gemini-2.5-flash\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n }\n ]\n },\n \"gemini-3-pro-preview\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"auto\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"pro\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"gemini-3.1-flash-lite\": {\n \"default\": \"gemini-3.1-flash-lite\"\n },\n \"flash\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"flash-lite\": {\n \"default\": \"gemini-3.1-flash-lite\"\n },\n \"auto-gemini-3\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"auto-gemini-2.5\": {\n \"default\": \"gemini-2.5-pro\"\n }\n },\n \"classifierIdResolutions\": {\n \"flash\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n },\n {\n \"condition\": {\n \"requestedModels\": [\n \"gemini-2.5-pro\",\n \"auto-gemini-2.5\"\n ]\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"pro\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"requestedModels\": [\n \"gemini-2.5-pro\",\n \"auto-gemini-2.5\"\n ]\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n }\n },\n \"modelChains\": {\n \"preview\": [\n {\n \"model\": \"gemini-3-pro-preview\",\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-3-flash-preview\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"auto-preview\": [\n {\n \"model\": \"gemini-3-pro-preview\",\n \"maxAttempts\": 3,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"silent\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-3-flash-preview\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"default\": [\n {\n \"model\": \"gemini-2.5-pro\",\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"auto-default\": [\n {\n \"model\": \"gemini-2.5-pro\",\n \"maxAttempts\": 3,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"silent\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"lite\": [\n {\n \"model\": \"flash-lite\",\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-pro\",\n \"isLastResort\": true,\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ]\n }\n}`", + "default": { + "aliases": { + "base": { + "modelConfig": { + "generateContentConfig": { + "temperature": 0, + "topP": 1 + } + } + }, + "chat-base": { + "extends": "base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "includeThoughts": true + }, + "temperature": 1, + "topP": 0.95, + "topK": 64 + } + } + }, + "chat-base-2.5": { + "extends": "chat-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingBudget": 8192 + } + } + } + }, + "chat-base-3": { + "extends": "chat-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingLevel": "HIGH" + } + } + } + }, + "gemini-3-pro-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "gemini-3-flash-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "gemini-3.1-pro-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-pro-preview" + } + }, + "gemini-3.1-pro-preview-customtools": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-pro-preview-customtools" + } + }, + "gemini-3.1-flash-lite-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-flash-lite-preview" + } + }, + "gemini-2.5-pro": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-pro" + } + }, + "gemini-2.5-flash": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "gemini-2.5-flash-lite": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-flash-lite" + } + }, + "gemini-3.1-flash-lite": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-flash-lite" + } + }, + "gemini-3.5-flash": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.5-flash" + } + }, + "gemma-4-31b-it": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemma-4-31b-it" + } + }, + "gemma-4-26b-a4b-it": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemma-4-26b-a4b-it" + } + }, + "gemini-2.5-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "gemini-3-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "gemini-3.5-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-3.5-flash" + } + }, + "classifier": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 1024, + "thinkingConfig": { + "thinkingBudget": 512 + } + } + } + }, + "prompt-completion": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "temperature": 0.3, + "maxOutputTokens": 16000, + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "fast-ack-helper": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "temperature": 0.2, + "maxOutputTokens": 120, + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "edit-corrector": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "summarizer-default": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 2000 + } + } + }, + "summarizer-shell": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 2000 + } + } + }, + "web-search": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "tools": [ + { + "googleSearch": {} + } + ] + } + } + }, + "web-fetch": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "tools": [ + { + "urlContext": {} + } + ] + } + } + }, + "web-fetch-fallback": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "loop-detection": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "loop-detection-double-check": { + "extends": "base", + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "llm-edit-fixer": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "next-speaker-checker": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "context-snapshotter": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingLevel": "HIGH" + }, + "temperature": 1, + "topP": 0.95, + "topK": 64 + } + } + }, + "chat-compression-3-pro": { + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "chat-compression-3-flash": { + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "chat-compression-3.1-flash-lite": { + "modelConfig": { + "model": "gemini-3.1-flash-lite" + } + }, + "chat-compression-2.5-pro": { + "modelConfig": { + "model": "gemini-2.5-pro" + } + }, + "chat-compression-2.5-flash": { + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "chat-compression-2.5-flash-lite": { + "modelConfig": { + "model": "gemini-2.5-flash-lite" + } + }, + "chat-compression-default": { + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "agent-history-provider-summarizer": { + "modelConfig": { + "model": "gemini-3-flash-preview" + } + } + }, + "overrides": [ + { + "match": { + "model": "chat-base", + "isRetry": true + }, + "modelConfig": { + "generateContentConfig": { + "temperature": 1 + } + } + } + ], + "modelDefinitions": { + "gemini-3.1-flash-lite": { + "tier": "flash-lite", + "family": "gemini-3", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-3.1-pro-preview": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3.1-pro-preview-customtools": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": false, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3-pro-preview": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3-flash-preview": { + "tier": "flash", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-3.5-flash": { + "tier": "flash", + "family": "gemini-3", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-2.5-pro": { + "tier": "pro", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemini-2.5-flash": { + "tier": "flash", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemini-2.5-flash-lite": { + "tier": "flash-lite", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemma-4-31b-it": { + "displayName": "gemma-4-31b-it", + "tier": "custom", + "family": "gemma-4", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "gemma-4-26b-a4b-it": { + "displayName": "gemma-4-26b-a4b-it", + "tier": "custom", + "family": "gemma-4", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "auto": { + "displayName": "Auto", + "tier": "auto", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "pro": { + "tier": "pro", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "flash": { + "tier": "flash", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "flash-lite": { + "tier": "flash-lite", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "auto-gemini-3": { + "tier": "auto", + "family": "gemini-3", + "isPreview": true, + "isVisible": false + }, + "auto-gemini-2.5": { + "tier": "auto", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": false + } + }, + "modelIdResolutions": { + "gemma-4-31b-it": { + "default": "gemma-4-31b-it" + }, + "gemma-4-26b-a4b-it": { + "default": "gemma-4-26b-a4b-it" + }, + "gemini-3.1-pro-preview": { + "default": "gemini-3.1-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + } + ] + }, + "gemini-3.1-pro-preview-customtools": { + "default": "gemini-3.1-pro-preview-customtools", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + } + ] + }, + "gemini-3-flash-preview": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false, + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false, + "useGemini3_5Flash": false + }, + "target": "gemini-2.5-flash" + } + ] + }, + "gemini-3.5-flash": { + "default": "gemini-3.5-flash", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": false, + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + }, + { + "condition": { + "useGemini3_5Flash": false + }, + "target": "gemini-3-flash-preview" + } + ] + }, + "gemini-2.5-flash": { + "default": "gemini-2.5-flash", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + } + ] + }, + "gemini-3-pro-preview": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "auto": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "pro": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "gemini-3.1-flash-lite": { + "default": "gemini-3.1-flash-lite" + }, + "flash": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + } + ] + }, + "flash-lite": { + "default": "gemini-3.1-flash-lite" + }, + "auto-gemini-3": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "auto-gemini-2.5": { + "default": "gemini-2.5-pro" + } + }, + "classifierIdResolutions": { + "flash": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + }, + { + "condition": { + "requestedModels": ["gemini-2.5-pro", "auto-gemini-2.5"] + }, + "target": "gemini-2.5-flash" + } + ] + }, + "pro": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "requestedModels": ["gemini-2.5-pro", "auto-gemini-2.5"] + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + } + }, + "modelChains": { + "preview": [ + { + "model": "gemini-3-pro-preview", + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-3-flash-preview", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "auto-preview": [ + { + "model": "gemini-3-pro-preview", + "maxAttempts": 3, + "actions": { + "terminal": "prompt", + "transient": "silent", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-3-flash-preview", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "default": [ + { + "model": "gemini-2.5-pro", + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "auto-default": [ + { + "model": "gemini-2.5-pro", + "maxAttempts": 3, + "actions": { + "terminal": "prompt", + "transient": "silent", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "lite": [ + { + "model": "flash-lite", + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-pro", + "isLastResort": true, + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ] + } + }, + "type": "object", + "properties": { + "aliases": { + "title": "Model Config Aliases", + "description": "Named presets for model configs. Can be used in place of a model name and can inherit from other aliases using an `extends` property.", + "markdownDescription": "Named presets for model configs. Can be used in place of a model name and can inherit from other aliases using an `extends` property.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `{\n \"base\": {\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"temperature\": 0,\n \"topP\": 1\n }\n }\n },\n \"chat-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"includeThoughts\": true\n },\n \"temperature\": 1,\n \"topP\": 0.95,\n \"topK\": 64\n }\n }\n },\n \"chat-base-2.5\": {\n \"extends\": \"chat-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingBudget\": 8192\n }\n }\n }\n },\n \"chat-base-3\": {\n \"extends\": \"chat-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingLevel\": \"HIGH\"\n }\n }\n }\n },\n \"gemini-3-pro-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"gemini-3-flash-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"gemini-3.1-pro-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-pro-preview\"\n }\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-pro-preview-customtools\"\n }\n },\n \"gemini-3.1-flash-lite-preview\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite-preview\"\n }\n },\n \"gemini-2.5-pro\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-pro\"\n }\n },\n \"gemini-2.5-flash\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"gemini-2.5-flash-lite\": {\n \"extends\": \"chat-base-2.5\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash-lite\"\n }\n },\n \"gemini-3.1-flash-lite\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite\"\n }\n },\n \"gemini-3.5-flash\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemini-3.5-flash\"\n }\n },\n \"gemma-4-31b-it\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemma-4-31b-it\"\n }\n },\n \"gemma-4-26b-a4b-it\": {\n \"extends\": \"chat-base-3\",\n \"modelConfig\": {\n \"model\": \"gemma-4-26b-a4b-it\"\n }\n },\n \"gemini-2.5-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"gemini-3-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"gemini-3.5-flash-base\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3.5-flash\"\n }\n },\n \"classifier\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 1024,\n \"thinkingConfig\": {\n \"thinkingBudget\": 512\n }\n }\n }\n },\n \"prompt-completion\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"temperature\": 0.3,\n \"maxOutputTokens\": 16000,\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"fast-ack-helper\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"temperature\": 0.2,\n \"maxOutputTokens\": 120,\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"edit-corrector\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingBudget\": 0\n }\n }\n }\n },\n \"summarizer-default\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 2000\n }\n }\n },\n \"summarizer-shell\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"flash-lite\",\n \"generateContentConfig\": {\n \"maxOutputTokens\": 2000\n }\n }\n },\n \"web-search\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"tools\": [\n {\n \"googleSearch\": {}\n }\n ]\n }\n }\n },\n \"web-fetch\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"tools\": [\n {\n \"urlContext\": {}\n }\n ]\n }\n }\n },\n \"web-fetch-fallback\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"loop-detection\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"loop-detection-double-check\": {\n \"extends\": \"base\",\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"llm-edit-fixer\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"next-speaker-checker\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {}\n },\n \"context-snapshotter\": {\n \"extends\": \"gemini-3-flash-base\",\n \"modelConfig\": {\n \"generateContentConfig\": {\n \"thinkingConfig\": {\n \"thinkingLevel\": \"HIGH\"\n },\n \"temperature\": 1,\n \"topP\": 0.95,\n \"topK\": 64\n }\n }\n },\n \"chat-compression-3-pro\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"chat-compression-3-flash\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n },\n \"chat-compression-3.1-flash-lite\": {\n \"modelConfig\": {\n \"model\": \"gemini-3.1-flash-lite\"\n }\n },\n \"chat-compression-2.5-pro\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-pro\"\n }\n },\n \"chat-compression-2.5-flash\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash\"\n }\n },\n \"chat-compression-2.5-flash-lite\": {\n \"modelConfig\": {\n \"model\": \"gemini-2.5-flash-lite\"\n }\n },\n \"chat-compression-default\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-pro-preview\"\n }\n },\n \"agent-history-provider-summarizer\": {\n \"modelConfig\": {\n \"model\": \"gemini-3-flash-preview\"\n }\n }\n}`", + "default": { + "base": { + "modelConfig": { + "generateContentConfig": { + "temperature": 0, + "topP": 1 + } + } + }, + "chat-base": { + "extends": "base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "includeThoughts": true + }, + "temperature": 1, + "topP": 0.95, + "topK": 64 + } + } + }, + "chat-base-2.5": { + "extends": "chat-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingBudget": 8192 + } + } + } + }, + "chat-base-3": { + "extends": "chat-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingLevel": "HIGH" + } + } + } + }, + "gemini-3-pro-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "gemini-3-flash-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "gemini-3.1-pro-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-pro-preview" + } + }, + "gemini-3.1-pro-preview-customtools": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-pro-preview-customtools" + } + }, + "gemini-3.1-flash-lite-preview": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-flash-lite-preview" + } + }, + "gemini-2.5-pro": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-pro" + } + }, + "gemini-2.5-flash": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "gemini-2.5-flash-lite": { + "extends": "chat-base-2.5", + "modelConfig": { + "model": "gemini-2.5-flash-lite" + } + }, + "gemini-3.1-flash-lite": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.1-flash-lite" + } + }, + "gemini-3.5-flash": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemini-3.5-flash" + } + }, + "gemma-4-31b-it": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemma-4-31b-it" + } + }, + "gemma-4-26b-a4b-it": { + "extends": "chat-base-3", + "modelConfig": { + "model": "gemma-4-26b-a4b-it" + } + }, + "gemini-2.5-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "gemini-3-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "gemini-3.5-flash-base": { + "extends": "base", + "modelConfig": { + "model": "gemini-3.5-flash" + } + }, + "classifier": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 1024, + "thinkingConfig": { + "thinkingBudget": 512 + } + } + } + }, + "prompt-completion": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "temperature": 0.3, + "maxOutputTokens": 16000, + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "fast-ack-helper": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "temperature": 0.2, + "maxOutputTokens": 120, + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "edit-corrector": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "thinkingConfig": { + "thinkingBudget": 0 + } + } + } + }, + "summarizer-default": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 2000 + } + } + }, + "summarizer-shell": { + "extends": "base", + "modelConfig": { + "model": "flash-lite", + "generateContentConfig": { + "maxOutputTokens": 2000 + } + } + }, + "web-search": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "tools": [ + { + "googleSearch": {} + } + ] + } + } + }, + "web-fetch": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "tools": [ + { + "urlContext": {} + } + ] + } + } + }, + "web-fetch-fallback": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "loop-detection": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "loop-detection-double-check": { + "extends": "base", + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "llm-edit-fixer": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "next-speaker-checker": { + "extends": "gemini-3-flash-base", + "modelConfig": {} + }, + "context-snapshotter": { + "extends": "gemini-3-flash-base", + "modelConfig": { + "generateContentConfig": { + "thinkingConfig": { + "thinkingLevel": "HIGH" + }, + "temperature": 1, + "topP": 0.95, + "topK": 64 + } + } + }, + "chat-compression-3-pro": { + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "chat-compression-3-flash": { + "modelConfig": { + "model": "gemini-3-flash-preview" + } + }, + "chat-compression-3.1-flash-lite": { + "modelConfig": { + "model": "gemini-3.1-flash-lite" + } + }, + "chat-compression-2.5-pro": { + "modelConfig": { + "model": "gemini-2.5-pro" + } + }, + "chat-compression-2.5-flash": { + "modelConfig": { + "model": "gemini-2.5-flash" + } + }, + "chat-compression-2.5-flash-lite": { + "modelConfig": { + "model": "gemini-2.5-flash-lite" + } + }, + "chat-compression-default": { + "modelConfig": { + "model": "gemini-3-pro-preview" + } + }, + "agent-history-provider-summarizer": { + "modelConfig": { + "model": "gemini-3-flash-preview" + } + } + }, + "type": "object", + "additionalProperties": true + }, + "customAliases": { + "title": "Custom Model Config Aliases", + "description": "Custom named presets for model configs. These are merged with (and override) the built-in aliases.", + "markdownDescription": "Custom named presets for model configs. These are merged with (and override) the built-in aliases.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": true + }, + "customOverrides": { + "title": "Custom Model Config Overrides", + "description": "Custom model config overrides. These are merged with (and added to) the built-in overrides.", + "markdownDescription": "Custom model config overrides. These are merged with (and added to) the built-in overrides.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": {} + }, + "overrides": { + "title": "Model Config Overrides", + "description": "Apply specific configuration overrides based on matches, with a primary key of model (or alias). The most specific match will be used.", + "markdownDescription": "Apply specific configuration overrides based on matches, with a primary key of model (or alias). The most specific match will be used.\n\n- Category: `Model`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": {} + }, + "modelDefinitions": { + "title": "Model Definitions", + "description": "Registry of model metadata, including tier, family, and features.", + "markdownDescription": "Registry of model metadata, including tier, family, and features.\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `{\n \"gemini-3.1-flash-lite\": {\n \"tier\": \"flash-lite\",\n \"family\": \"gemini-3\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.1-pro-preview\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3-pro-preview\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3-flash-preview\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-3.5-flash\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-3\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": true\n }\n },\n \"gemini-2.5-pro\": {\n \"tier\": \"pro\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemini-2.5-flash\": {\n \"tier\": \"flash\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemini-2.5-flash-lite\": {\n \"tier\": \"flash-lite\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"gemma-4-31b-it\": {\n \"displayName\": \"gemma-4-31b-it\",\n \"tier\": \"custom\",\n \"family\": \"gemma-4\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"gemma-4-26b-a4b-it\": {\n \"displayName\": \"gemma-4-26b-a4b-it\",\n \"tier\": \"custom\",\n \"family\": \"gemma-4\",\n \"isPreview\": false,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"auto\": {\n \"displayName\": \"Auto\",\n \"tier\": \"auto\",\n \"isPreview\": true,\n \"isVisible\": true,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"pro\": {\n \"tier\": \"pro\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": true,\n \"multimodalToolUse\": false\n }\n },\n \"flash\": {\n \"tier\": \"flash\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"flash-lite\": {\n \"tier\": \"flash-lite\",\n \"isPreview\": false,\n \"isVisible\": false,\n \"features\": {\n \"thinking\": false,\n \"multimodalToolUse\": false\n }\n },\n \"auto-gemini-3\": {\n \"tier\": \"auto\",\n \"family\": \"gemini-3\",\n \"isPreview\": true,\n \"isVisible\": false\n },\n \"auto-gemini-2.5\": {\n \"tier\": \"auto\",\n \"family\": \"gemini-2.5\",\n \"isPreview\": false,\n \"isVisible\": false\n }\n}`", + "default": { + "gemini-3.1-flash-lite": { + "tier": "flash-lite", + "family": "gemini-3", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-3.1-pro-preview": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3.1-pro-preview-customtools": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": false, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3-pro-preview": { + "tier": "pro", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": true + } + }, + "gemini-3-flash-preview": { + "tier": "flash", + "family": "gemini-3", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-3.5-flash": { + "tier": "flash", + "family": "gemini-3", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": true + } + }, + "gemini-2.5-pro": { + "tier": "pro", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemini-2.5-flash": { + "tier": "flash", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemini-2.5-flash-lite": { + "tier": "flash-lite", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "gemma-4-31b-it": { + "displayName": "gemma-4-31b-it", + "tier": "custom", + "family": "gemma-4", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "gemma-4-26b-a4b-it": { + "displayName": "gemma-4-26b-a4b-it", + "tier": "custom", + "family": "gemma-4", + "isPreview": false, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "auto": { + "displayName": "Auto", + "tier": "auto", + "isPreview": true, + "isVisible": true, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "pro": { + "tier": "pro", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": true, + "multimodalToolUse": false + } + }, + "flash": { + "tier": "flash", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "flash-lite": { + "tier": "flash-lite", + "isPreview": false, + "isVisible": false, + "features": { + "thinking": false, + "multimodalToolUse": false + } + }, + "auto-gemini-3": { + "tier": "auto", + "family": "gemini-3", + "isPreview": true, + "isVisible": false + }, + "auto-gemini-2.5": { + "tier": "auto", + "family": "gemini-2.5", + "isPreview": false, + "isVisible": false + } + }, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/ModelDefinition" + } + }, + "modelIdResolutions": { + "title": "Model ID Resolutions", + "description": "Rules for resolving requested model names to concrete model IDs based on context.", + "markdownDescription": "Rules for resolving requested model names to concrete model IDs based on context.\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `{\n \"gemma-4-31b-it\": {\n \"default\": \"gemma-4-31b-it\"\n },\n \"gemma-4-26b-a4b-it\": {\n \"default\": \"gemma-4-26b-a4b-it\"\n },\n \"gemini-3.1-pro-preview\": {\n \"default\": \"gemini-3.1-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n }\n ]\n },\n \"gemini-3.1-pro-preview-customtools\": {\n \"default\": \"gemini-3.1-pro-preview-customtools\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n }\n ]\n },\n \"gemini-3-flash-preview\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false,\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false,\n \"useGemini3_5Flash\": false\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"gemini-3.5-flash\": {\n \"default\": \"gemini-3.5-flash\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": false,\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n },\n {\n \"condition\": {\n \"useGemini3_5Flash\": false\n },\n \"target\": \"gemini-3-flash-preview\"\n }\n ]\n },\n \"gemini-2.5-flash\": {\n \"default\": \"gemini-2.5-flash\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n }\n ]\n },\n \"gemini-3-pro-preview\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"auto\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"pro\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"gemini-3.1-flash-lite\": {\n \"default\": \"gemini-3.1-flash-lite\"\n },\n \"flash\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"flash-lite\": {\n \"default\": \"gemini-3.1-flash-lite\"\n },\n \"auto-gemini-3\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n },\n \"auto-gemini-2.5\": {\n \"default\": \"gemini-2.5-pro\"\n }\n}`", + "default": { + "gemma-4-31b-it": { + "default": "gemma-4-31b-it" + }, + "gemma-4-26b-a4b-it": { + "default": "gemma-4-26b-a4b-it" + }, + "gemini-3.1-pro-preview": { + "default": "gemini-3.1-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + } + ] + }, + "gemini-3.1-pro-preview-customtools": { + "default": "gemini-3.1-pro-preview-customtools", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + } + ] + }, + "gemini-3-flash-preview": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false, + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false, + "useGemini3_5Flash": false + }, + "target": "gemini-2.5-flash" + } + ] + }, + "gemini-3.5-flash": { + "default": "gemini-3.5-flash", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": false, + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + }, + { + "condition": { + "useGemini3_5Flash": false + }, + "target": "gemini-3-flash-preview" + } + ] + }, + "gemini-2.5-flash": { + "default": "gemini-2.5-flash", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + } + ] + }, + "gemini-3-pro-preview": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "auto": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "pro": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "gemini-3.1-flash-lite": { + "default": "gemini-3.1-flash-lite" + }, + "flash": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + } + ] + }, + "flash-lite": { + "default": "gemini-3.1-flash-lite" + }, + "auto-gemini-3": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + }, + "auto-gemini-2.5": { + "default": "gemini-2.5-pro" + } + }, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/ModelResolution" + } + }, + "classifierIdResolutions": { + "title": "Classifier ID Resolutions", + "description": "Rules for resolving classifier tiers (flash, pro) to concrete model IDs.", + "markdownDescription": "Rules for resolving classifier tiers (flash, pro) to concrete model IDs.\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `{\n \"flash\": {\n \"default\": \"gemini-3-flash-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"useGemini3_5Flash\": true\n },\n \"target\": \"gemini-3.5-flash\"\n },\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-flash\"\n },\n {\n \"condition\": {\n \"requestedModels\": [\n \"gemini-2.5-pro\",\n \"auto-gemini-2.5\"\n ]\n },\n \"target\": \"gemini-2.5-flash\"\n }\n ]\n },\n \"pro\": {\n \"default\": \"gemini-3-pro-preview\",\n \"contexts\": [\n {\n \"condition\": {\n \"hasAccessToPreview\": false\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"requestedModels\": [\n \"gemini-2.5-pro\",\n \"auto-gemini-2.5\"\n ]\n },\n \"target\": \"gemini-2.5-pro\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true,\n \"useCustomTools\": true\n },\n \"target\": \"gemini-3.1-pro-preview-customtools\"\n },\n {\n \"condition\": {\n \"useGemini3_1\": true\n },\n \"target\": \"gemini-3.1-pro-preview\"\n }\n ]\n }\n}`", + "default": { + "flash": { + "default": "gemini-3-flash-preview", + "contexts": [ + { + "condition": { + "useGemini3_5Flash": true + }, + "target": "gemini-3.5-flash" + }, + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-flash" + }, + { + "condition": { + "requestedModels": ["gemini-2.5-pro", "auto-gemini-2.5"] + }, + "target": "gemini-2.5-flash" + } + ] + }, + "pro": { + "default": "gemini-3-pro-preview", + "contexts": [ + { + "condition": { + "hasAccessToPreview": false + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "requestedModels": ["gemini-2.5-pro", "auto-gemini-2.5"] + }, + "target": "gemini-2.5-pro" + }, + { + "condition": { + "useGemini3_1": true, + "useCustomTools": true + }, + "target": "gemini-3.1-pro-preview-customtools" + }, + { + "condition": { + "useGemini3_1": true + }, + "target": "gemini-3.1-pro-preview" + } + ] + } + }, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/ModelResolution" + } + }, + "modelChains": { + "title": "Model Chains", + "description": "Availability policy chains defining fallback behavior for models.", + "markdownDescription": "Availability policy chains defining fallback behavior for models.\n\n- Category: `Model`\n- Requires restart: `yes`\n- Default: `{\n \"preview\": [\n {\n \"model\": \"gemini-3-pro-preview\",\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-3-flash-preview\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"auto-preview\": [\n {\n \"model\": \"gemini-3-pro-preview\",\n \"maxAttempts\": 3,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"silent\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-3-flash-preview\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"default\": [\n {\n \"model\": \"gemini-2.5-pro\",\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"auto-default\": [\n {\n \"model\": \"gemini-2.5-pro\",\n \"maxAttempts\": 3,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"silent\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"sticky_retry\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"isLastResort\": true,\n \"maxAttempts\": 10,\n \"actions\": {\n \"terminal\": \"prompt\",\n \"transient\": \"prompt\",\n \"not_found\": \"prompt\",\n \"unknown\": \"prompt\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ],\n \"lite\": [\n {\n \"model\": \"flash-lite\",\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-flash\",\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n },\n {\n \"model\": \"gemini-2.5-pro\",\n \"isLastResort\": true,\n \"actions\": {\n \"terminal\": \"silent\",\n \"transient\": \"silent\",\n \"not_found\": \"silent\",\n \"unknown\": \"silent\"\n },\n \"stateTransitions\": {\n \"terminal\": \"terminal\",\n \"transient\": \"terminal\",\n \"not_found\": \"terminal\",\n \"unknown\": \"terminal\"\n }\n }\n ]\n}`", + "default": { + "preview": [ + { + "model": "gemini-3-pro-preview", + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-3-flash-preview", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "auto-preview": [ + { + "model": "gemini-3-pro-preview", + "maxAttempts": 3, + "actions": { + "terminal": "prompt", + "transient": "silent", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-3-flash-preview", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "default": [ + { + "model": "gemini-2.5-pro", + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "auto-default": [ + { + "model": "gemini-2.5-pro", + "maxAttempts": 3, + "actions": { + "terminal": "prompt", + "transient": "silent", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "sticky_retry", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "isLastResort": true, + "maxAttempts": 10, + "actions": { + "terminal": "prompt", + "transient": "prompt", + "not_found": "prompt", + "unknown": "prompt" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ], + "lite": [ + { + "model": "flash-lite", + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-flash", + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + }, + { + "model": "gemini-2.5-pro", + "isLastResort": true, + "actions": { + "terminal": "silent", + "transient": "silent", + "not_found": "silent", + "unknown": "silent" + }, + "stateTransitions": { + "terminal": "terminal", + "transient": "terminal", + "not_found": "terminal", + "unknown": "terminal" + } + } + ] + }, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/ModelPolicyChain" + } + } + }, + "additionalProperties": false + }, + "agents": { + "title": "Agents", + "description": "Settings for subagents.", + "markdownDescription": "Settings for subagents.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "overrides": { + "title": "Agent Overrides", + "description": "Override settings for specific agents, e.g. to disable the agent, set a custom model config, or run config.", + "markdownDescription": "Override settings for specific agents, e.g. to disable the agent, set a custom model config, or run config.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/AgentOverride" + } + }, + "browser": { + "title": "Browser Agent", + "description": "Settings specific to the browser agent.", + "markdownDescription": "Settings specific to the browser agent.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "sessionMode": { + "title": "Browser Session Mode", + "description": "Session mode: 'persistent', 'isolated', or 'existing'.", + "markdownDescription": "Session mode: 'persistent', 'isolated', or 'existing'.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `persistent`", + "default": "persistent", + "type": "string", + "enum": ["persistent", "isolated", "existing"] + }, + "headless": { + "title": "Browser Headless", + "description": "Run browser in headless mode.", + "markdownDescription": "Run browser in headless mode.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "profilePath": { + "title": "Browser Profile Path", + "description": "Path to browser profile directory for session persistence.", + "markdownDescription": "Path to browser profile directory for session persistence.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string" + }, + "visualModel": { + "title": "Browser Visual Model", + "description": "Model for the visual agent's analyze_screenshot tool. When set, enables the tool.", + "markdownDescription": "Model for the visual agent's analyze_screenshot tool. When set, enables the tool.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string" + }, + "allowedDomains": { + "title": "Allowed Domains", + "description": "A list of allowed domains for the browser agent (e.g., [\"github.com\", \"*.google.com\"]).", + "markdownDescription": "A list of allowed domains for the browser agent (e.g., [\"github.com\", \"*.google.com\"]).\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `[\n \"github.com\",\n \"*.google.com\",\n \"localhost\"\n]`", + "default": ["github.com", "*.google.com", "localhost"], + "type": "array", + "items": { + "type": "string" + } + }, + "disableUserInput": { + "title": "Disable User Input", + "description": "Disable user input on browser window during automation.", + "markdownDescription": "Disable user input on browser window during automation.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "maxActionsPerTask": { + "title": "Max Actions Per Task", + "description": "The maximum number of tool calls allowed per browser task. Enforcement is hard: the agent will be terminated when the limit is reached.", + "markdownDescription": "The maximum number of tool calls allowed per browser task. Enforcement is hard: the agent will be terminated when the limit is reached.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `100`", + "default": 100, + "type": "number" + }, + "confirmSensitiveActions": { + "title": "Confirm Sensitive Actions", + "description": "Require manual confirmation for sensitive browser actions (e.g., fill_form, evaluate_script).", + "markdownDescription": "Require manual confirmation for sensitive browser actions (e.g., fill_form, evaluate_script).\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "blockFileUploads": { + "title": "Block File Uploads", + "description": "Hard-block file upload requests from the browser agent.", + "markdownDescription": "Hard-block file upload requests from the browser agent.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "context": { + "title": "Context", + "description": "Settings for managing context provided to the model.", + "markdownDescription": "Settings for managing context provided to the model.\n\n- Category: `Context`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "fileName": { + "title": "Context File Name", + "description": "The name of the context file or files to load into memory. Accepts either a single string or an array of strings.", + "markdownDescription": "The name of the context file or files to load into memory. Accepts either a single string or an array of strings.\n\n- Category: `Context`\n- Requires restart: `no`", + "$ref": "#/$defs/StringOrStringArray" + }, + "importFormat": { + "title": "Memory Import Format", + "description": "The format to use when importing memory.", + "markdownDescription": "The format to use when importing memory.\n\n- Category: `Context`\n- Requires restart: `no`", + "type": "string" + }, + "includeDirectoryTree": { + "title": "Include Directory Tree", + "description": "Whether to include the directory tree of the current working directory in the initial request to the model.", + "markdownDescription": "Whether to include the directory tree of the current working directory in the initial request to the model.\n\n- Category: `Context`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "discoveryMaxDirs": { + "title": "Memory Discovery Max Dirs", + "description": "Maximum number of directories to search for memory.", + "markdownDescription": "Maximum number of directories to search for memory.\n\n- Category: `Context`\n- Requires restart: `no`\n- Default: `200`", + "default": 200, + "type": "number" + }, + "memoryBoundaryMarkers": { + "title": "Memory Boundary Markers", + "description": "File or directory names that mark the boundary for GEMINI.md discovery. The upward traversal stops at the first directory containing any of these markers. An empty array disables parent traversal.", + "markdownDescription": "File or directory names that mark the boundary for GEMINI.md discovery. The upward traversal stops at the first directory containing any of these markers. An empty array disables parent traversal.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `[\n \".git\"\n]`", + "default": [".git"], + "type": "array", + "items": { + "type": "string" + } + }, + "includeDirectories": { + "title": "Include Directories", + "description": "Additional directories to include in the workspace context. Missing directories will be skipped with a warning.", + "markdownDescription": "Additional directories to include in the workspace context. Missing directories will be skipped with a warning.\n\n- Category: `Context`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "loadMemoryFromIncludeDirectories": { + "title": "Load Memory From Include Directories", + "description": "Controls how /memory reload loads GEMINI.md files. When true, include directories are scanned; when false, only the current directory is used.", + "markdownDescription": "Controls how /memory reload loads GEMINI.md files. When true, include directories are scanned; when false, only the current directory is used.\n\n- Category: `Context`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "fileFiltering": { + "title": "File Filtering", + "description": "Settings for git-aware file filtering.", + "markdownDescription": "Settings for git-aware file filtering.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "respectGitIgnore": { + "title": "Respect .gitignore", + "description": "Respect .gitignore files when searching.", + "markdownDescription": "Respect .gitignore files when searching.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "respectGeminiIgnore": { + "title": "Respect .geminiignore", + "description": "Respect .geminiignore files when searching.", + "markdownDescription": "Respect .geminiignore files when searching.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "enableFileWatcher": { + "title": "Enable File Watcher", + "description": "Enable file watcher updates for @ file suggestions (experimental).", + "markdownDescription": "Enable file watcher updates for @ file suggestions (experimental).\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "enableRecursiveFileSearch": { + "title": "Enable Recursive File Search", + "description": "Enable recursive file search functionality when completing @ references in the prompt.", + "markdownDescription": "Enable recursive file search functionality when completing @ references in the prompt.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "enableFuzzySearch": { + "title": "Enable Fuzzy Search", + "description": "Enable fuzzy search when searching for files.", + "markdownDescription": "Enable fuzzy search when searching for files.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "customIgnoreFilePaths": { + "title": "Custom Ignore File Paths", + "description": "Additional ignore file paths to respect. These files take precedence over .geminiignore and .gitignore. Files earlier in the array take precedence over files later in the array, e.g. the first file takes precedence over the second one.", + "markdownDescription": "Additional ignore file paths to respect. These files take precedence over .geminiignore and .gitignore. Files earlier in the array take precedence over files later in the array, e.g. the first file takes precedence over the second one.\n\n- Category: `Context`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "tools": { + "title": "Tools", + "description": "Settings for built-in and custom tools.", + "markdownDescription": "Settings for built-in and custom tools.\n\n- Category: `Tools`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "sandbox": { + "title": "Sandbox", + "description": "Legacy full-process sandbox execution environment. Set to a boolean to enable or disable the sandbox, provide a string path to a sandbox profile, or specify an explicit sandbox command (e.g., \"docker\", \"podman\", \"lxc\", \"windows-native\").", + "markdownDescription": "Legacy full-process sandbox execution environment. Set to a boolean to enable or disable the sandbox, provide a string path to a sandbox profile, or specify an explicit sandbox command (e.g., \"docker\", \"podman\", \"lxc\", \"windows-native\").\n\n- Category: `Tools`\n- Requires restart: `yes`", + "$ref": "#/$defs/BooleanOrStringOrObject" + }, + "sandboxAllowedPaths": { + "title": "Sandbox Allowed Paths", + "description": "List of additional paths that the sandbox is allowed to access.", + "markdownDescription": "List of additional paths that the sandbox is allowed to access.\n\n- Category: `Tools`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "sandboxNetworkAccess": { + "title": "Sandbox Network Access", + "description": "Whether the sandbox is allowed to access the network.", + "markdownDescription": "Whether the sandbox is allowed to access the network.\n\n- Category: `Tools`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "shell": { + "title": "Shell", + "description": "Settings for shell execution.", + "markdownDescription": "Settings for shell execution.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enableInteractiveShell": { + "title": "Enable Interactive Shell", + "description": "Use node-pty for an interactive shell experience. Fallback to child_process still applies.", + "markdownDescription": "Use node-pty for an interactive shell experience. Fallback to child_process still applies.\n\n- Category: `Tools`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "backgroundCompletionBehavior": { + "title": "Background Completion Behavior", + "description": "Controls what happens when a background shell command finishes. 'silent' (default): quietly exits in background. 'inject': automatically returns output to agent. 'notify': shows brief message in chat.", + "markdownDescription": "Controls what happens when a background shell command finishes. 'silent' (default): quietly exits in background. 'inject': automatically returns output to agent. 'notify': shows brief message in chat.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `silent`", + "default": "silent", + "type": "string", + "enum": ["silent", "inject", "notify"] + }, + "pager": { + "title": "Pager", + "description": "The pager command to use for shell output. Defaults to `cat`.", + "markdownDescription": "The pager command to use for shell output. Defaults to `cat`.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `cat`", + "default": "cat", + "type": "string" + }, + "showColor": { + "title": "Show Color", + "description": "Show color in shell output.", + "markdownDescription": "Show color in shell output.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "inactivityTimeout": { + "title": "Inactivity Timeout", + "description": "The maximum time in seconds allowed without output from the shell command. Defaults to 5 minutes.", + "markdownDescription": "The maximum time in seconds allowed without output from the shell command. Defaults to 5 minutes.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `300`", + "default": 300, + "type": "number" + }, + "enableShellOutputEfficiency": { + "title": "Enable Shell Output Efficiency", + "description": "Enable shell output efficiency optimizations for better performance.", + "markdownDescription": "Enable shell output efficiency optimizations for better performance.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "core": { + "title": "Core Tools", + "description": "Restrict the set of built-in tools with an allowlist. Match semantics mirror tools.allowed; see the built-in tools documentation for available names.", + "markdownDescription": "Restrict the set of built-in tools with an allowlist. Match semantics mirror tools.allowed; see the built-in tools documentation for available names.\n\n- Category: `Tools`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + }, + "allowed": { + "title": "Allowed Tools", + "description": "Tool names that bypass the confirmation dialog. Useful for trusted commands (for example [\"run_shell_command(git)\", \"run_shell_command(npm test)\"]). See shell tool command restrictions for matching details.", + "markdownDescription": "Tool names that bypass the confirmation dialog. Useful for trusted commands (for example [\"run_shell_command(git)\", \"run_shell_command(npm test)\"]). See shell tool command restrictions for matching details.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + }, + "confirmationRequired": { + "title": "Confirmation Required", + "description": "Tool names that always require user confirmation. Takes precedence over allowed tools and core tool allowlists.", + "markdownDescription": "Tool names that always require user confirmation. Takes precedence over allowed tools and core tool allowlists.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + }, + "exclude": { + "title": "Exclude Tools", + "description": "Tool names to exclude from discovery.", + "markdownDescription": "Tool names to exclude from discovery.\n\n- Category: `Tools`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + }, + "discoveryCommand": { + "title": "Tool Discovery Command", + "description": "Command to run for tool discovery.", + "markdownDescription": "Command to run for tool discovery.\n\n- Category: `Tools`\n- Requires restart: `yes`", + "type": "string" + }, + "callCommand": { + "title": "Tool Call Command", + "description": "Defines a custom shell command for invoking discovered tools. The command must take the tool name as the first argument, read JSON arguments from stdin, and emit JSON results on stdout.", + "markdownDescription": "Defines a custom shell command for invoking discovered tools. The command must take the tool name as the first argument, read JSON arguments from stdin, and emit JSON results on stdout.\n\n- Category: `Tools`\n- Requires restart: `yes`", + "type": "string" + }, + "useRipgrep": { + "title": "Use Ripgrep", + "description": "Use ripgrep for file content search instead of the fallback implementation. Provides faster search performance.", + "markdownDescription": "Use ripgrep for file content search instead of the fallback implementation. Provides faster search performance.\n\n- Category: `Tools`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "truncateToolOutputThreshold": { + "title": "Tool Output Truncation Threshold", + "description": "Maximum characters to show when truncating large tool outputs. Set to 0 or negative to disable truncation.", + "markdownDescription": "Maximum characters to show when truncating large tool outputs. Set to 0 or negative to disable truncation.\n\n- Category: `General`\n- Requires restart: `yes`\n- Default: `40000`", + "default": 40000, + "type": "number" + }, + "disableLLMCorrection": { + "title": "Disable LLM Correction", + "description": "Disable LLM-based error correction for edit tools. When enabled, tools will fail immediately if exact string matches are not found, instead of attempting to self-correct.", + "markdownDescription": "Disable LLM-based error correction for edit tools. When enabled, tools will fail immediately if exact string matches are not found, instead of attempting to self-correct.\n\n- Category: `Tools`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "mcp": { + "title": "MCP", + "description": "Settings for Model Context Protocol (MCP) servers.", + "markdownDescription": "Settings for Model Context Protocol (MCP) servers.\n\n- Category: `MCP`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "serverCommand": { + "title": "MCP Server Command", + "description": "Command to start an MCP server.", + "markdownDescription": "Command to start an MCP server.\n\n- Category: `MCP`\n- Requires restart: `yes`", + "type": "string" + }, + "allowed": { + "title": "Allow MCP Servers", + "description": "A list of MCP servers to allow.", + "markdownDescription": "A list of MCP servers to allow.\n\n- Category: `MCP`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + }, + "excluded": { + "title": "Exclude MCP Servers", + "description": "A list of MCP servers to exclude.", + "markdownDescription": "A list of MCP servers to exclude.\n\n- Category: `MCP`\n- Requires restart: `yes`", + "type": "array", + "items": { + "type": "string" + } + } + }, + "additionalProperties": false + }, + "useWriteTodos": { + "title": "Use WriteTodos", + "description": "Enable the write_todos tool.", + "markdownDescription": "Enable the write_todos tool.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "security": { + "title": "Security", + "description": "Security-related settings.", + "markdownDescription": "Security-related settings.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "toolSandboxing": { + "title": "Tool Sandboxing", + "description": "Tool-level sandboxing. Isolates individual tools instead of the entire CLI process.", + "markdownDescription": "Tool-level sandboxing. Isolates individual tools instead of the entire CLI process.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "disableYoloMode": { + "title": "Disable YOLO Mode", + "description": "Disable YOLO mode, even if enabled by a flag.", + "markdownDescription": "Disable YOLO mode, even if enabled by a flag.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "disableAlwaysAllow": { + "title": "Disable Always Allow", + "description": "Disable \"Always allow\" options in tool confirmation dialogs.", + "markdownDescription": "Disable \"Always allow\" options in tool confirmation dialogs.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "enablePermanentToolApproval": { + "title": "Allow Permanent Tool Approval", + "description": "Enable the \"Allow for all future sessions\" option in tool confirmation dialogs.", + "markdownDescription": "Enable the \"Allow for all future sessions\" option in tool confirmation dialogs.\n\n- Category: `Security`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "autoAddToPolicyByDefault": { + "title": "Auto-add to Policy by Default", + "description": "When enabled, the \"Allow for all future sessions\" option becomes the default choice for low-risk tools in trusted workspaces.", + "markdownDescription": "When enabled, the \"Allow for all future sessions\" option becomes the default choice for low-risk tools in trusted workspaces.\n\n- Category: `Security`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "blockGitExtensions": { + "title": "Blocks extensions from Git", + "description": "Blocks installing and loading extensions from Git.", + "markdownDescription": "Blocks installing and loading extensions from Git.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "allowedExtensions": { + "title": "Extension Source Regex Allowlist", + "description": "List of Regex patterns for allowed extensions. If nonempty, only extensions that match the patterns in this list are allowed. Overrides the blockGitExtensions setting.", + "markdownDescription": "List of Regex patterns for allowed extensions. If nonempty, only extensions that match the patterns in this list are allowed. Overrides the blockGitExtensions setting.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "folderTrust": { + "title": "Folder Trust", + "description": "Settings for folder trust.", + "markdownDescription": "Settings for folder trust.\n\n- Category: `Security`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Folder Trust", + "description": "Setting to track whether Folder trust is enabled.", + "markdownDescription": "Setting to track whether Folder trust is enabled.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "environmentVariableRedaction": { + "title": "Environment Variable Redaction", + "description": "Settings for environment variable redaction.", + "markdownDescription": "Settings for environment variable redaction.\n\n- Category: `Security`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "allowed": { + "title": "Allowed Environment Variables", + "description": "Environment variables to always allow (bypass redaction).", + "markdownDescription": "Environment variables to always allow (bypass redaction).\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "blocked": { + "title": "Blocked Environment Variables", + "description": "Environment variables to always redact.", + "markdownDescription": "Environment variables to always redact.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "enabled": { + "title": "Enable Environment Variable Redaction", + "description": "Enable redaction of environment variables that may contain secrets.", + "markdownDescription": "Enable redaction of environment variables that may contain secrets.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "auth": { + "title": "Authentication", + "description": "Authentication settings.", + "markdownDescription": "Authentication settings.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "selectedType": { + "title": "Selected Auth Type", + "description": "The currently selected authentication type.", + "markdownDescription": "The currently selected authentication type.\n\n- Category: `Security`\n- Requires restart: `yes`", + "type": "string" + }, + "enforcedType": { + "title": "Enforced Auth Type", + "description": "The required auth type. If this does not match the selected auth type, the user will be prompted to re-authenticate.", + "markdownDescription": "The required auth type. If this does not match the selected auth type, the user will be prompted to re-authenticate.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string" + }, + "useExternal": { + "title": "Use External Auth", + "description": "Whether to use an external authentication flow.", + "markdownDescription": "Whether to use an external authentication flow.\n\n- Category: `Security`\n- Requires restart: `yes`", + "type": "boolean" + } + }, + "additionalProperties": false + }, + "enableConseca": { + "title": "Enable Context-Aware Security", + "description": "Enable the context-aware security checker. This feature uses an LLM to dynamically generate and enforce security policies for tool use based on your prompt, providing an additional layer of protection against unintended actions.", + "markdownDescription": "Enable the context-aware security checker. This feature uses an LLM to dynamically generate and enforce security policies for tool use based on your prompt, providing an additional layer of protection against unintended actions.\n\n- Category: `Security`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "advanced": { + "title": "Advanced", + "description": "Advanced settings for power users.", + "markdownDescription": "Advanced settings for power users.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "autoConfigureMemory": { + "title": "Auto Configure Max Old Space Size", + "description": "Automatically configure Node.js memory limits. Note: Because memory is allocated during the initial process boot, this setting is only read from the global user settings file and ignores workspace-level overrides.", + "markdownDescription": "Automatically configure Node.js memory limits. Note: Because memory is allocated during the initial process boot, this setting is only read from the global user settings file and ignores workspace-level overrides.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "dnsResolutionOrder": { + "title": "DNS Resolution Order", + "description": "The DNS resolution order.", + "markdownDescription": "The DNS resolution order.\n\n- Category: `Advanced`\n- Requires restart: `yes`", + "type": "string" + }, + "excludedEnvVars": { + "title": "Excluded Project Environment Variables", + "description": "Environment variables to exclude from project context.", + "markdownDescription": "Environment variables to exclude from project context.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[\n \"DEBUG\",\n \"DEBUG_MODE\"\n]`", + "default": ["DEBUG", "DEBUG_MODE"], + "type": "array", + "items": { + "type": "string" + } + }, + "ignoreLocalEnv": { + "title": "Ignore Local .env", + "description": "Whether to ignore generic .env files in the project directory.", + "markdownDescription": "Whether to ignore generic .env files in the project directory.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "bugCommand": { + "title": "Bug Command", + "description": "Configuration for the bug report command.", + "markdownDescription": "Configuration for the bug report command.\n\n- Category: `Advanced`\n- Requires restart: `no`", + "$ref": "#/$defs/BugCommandSettings" + } + }, + "additionalProperties": false + }, + "experimental": { + "title": "Experimental", + "description": "Setting to enable experimental features", + "markdownDescription": "Setting to enable experimental features\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "gemma": { + "title": "Gemma Models", + "description": "Enable access to Gemma 4 models via Gemini API.", + "markdownDescription": "Enable access to Gemma 4 models via Gemini API.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "voiceMode": { + "title": "Voice Mode", + "description": "Enable experimental voice dictation and commands (/voice, /voice model).", + "markdownDescription": "Enable experimental voice dictation and commands (/voice, /voice model).\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "voice": { + "title": "Voice", + "description": "Settings for voice mode and transcription.", + "markdownDescription": "Settings for voice mode and transcription.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "activationMode": { + "title": "Voice Activation Mode", + "description": "How to trigger voice recording with the Space key.", + "markdownDescription": "How to trigger voice recording with the Space key.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `push-to-talk`", + "default": "push-to-talk", + "type": "string", + "enum": ["push-to-talk", "toggle"] + }, + "backend": { + "title": "Voice Transcription Backend", + "description": "The backend to use for voice transcription. Note: When using the Gemini Live backend, voice recordings are sent to Google Cloud for transcription.", + "markdownDescription": "The backend to use for voice transcription. Note: When using the Gemini Live backend, voice recordings are sent to Google Cloud for transcription.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `gemini-live`", + "default": "gemini-live", + "type": "string", + "enum": ["gemini-live", "whisper"] + }, + "whisperModel": { + "title": "Whisper Model", + "description": "The Whisper model to use for local transcription.", + "markdownDescription": "The Whisper model to use for local transcription.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `ggml-base.en.bin`", + "default": "ggml-base.en.bin", + "type": "string", + "enum": [ + "ggml-tiny.en.bin", + "ggml-base.en.bin", + "ggml-large-v3-turbo-q5_0.bin", + "ggml-large-v3-turbo-q8_0.bin" + ] + }, + "stopGracePeriodMs": { + "title": "Voice Stop Grace Period (ms)", + "description": "How long to wait for final transcription after stopping recording.", + "markdownDescription": "How long to wait for final transcription after stopping recording.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `4000`", + "default": 4000, + "type": "number" + } + }, + "additionalProperties": false + }, + "adk": { + "title": "ADK", + "description": "Settings for the Agent Development Kit (ADK).", + "markdownDescription": "Settings for the Agent Development Kit (ADK).\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "agentSessionNoninteractiveEnabled": { + "title": "Agent Session Non-interactive Enabled", + "description": "Enable non-interactive agent sessions.", + "markdownDescription": "Enable non-interactive agent sessions.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "agentSessionInteractiveEnabled": { + "title": "Interactive Agent Session Enabled", + "description": "Enable the agent session implementation for the interactive CLI.", + "markdownDescription": "Enable the agent session implementation for the interactive CLI.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "agentSessionSubagentEnabled": { + "title": "Agent Session Subagent Enabled", + "description": "Route subagent invocations through the AgentSession protocol instead of legacy executors.", + "markdownDescription": "Route subagent invocations through the AgentSession protocol instead of legacy executors.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "enableAgents": { + "title": "Enable Agents", + "description": "Enable local and remote subagents.", + "markdownDescription": "Enable local and remote subagents.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "worktrees": { + "title": "Enable Git Worktrees", + "description": "Enable automated Git worktree management for parallel work.", + "markdownDescription": "Enable automated Git worktree management for parallel work.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "extensionManagement": { + "title": "Extension Management", + "description": "Enable extension management features.", + "markdownDescription": "Enable extension management features.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "extensionConfig": { + "title": "Extension Configuration", + "description": "Enable requesting and fetching of extension settings.", + "markdownDescription": "Enable requesting and fetching of extension settings.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "extensionRegistry": { + "title": "Extension Registry Explore UI", + "description": "Enable extension registry explore UI.", + "markdownDescription": "Enable extension registry explore UI.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "extensionRegistryURI": { + "title": "Extension Registry URI", + "description": "The URI (web URL or local file path) of the extension registry.", + "markdownDescription": "The URI (web URL or local file path) of the extension registry.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `https://geminicli.com/extensions.json`", + "default": "https://geminicli.com/extensions.json", + "type": "string" + }, + "extensionReloading": { + "title": "Extension Reloading", + "description": "Enables extension loading/unloading within the CLI session.", + "markdownDescription": "Enables extension loading/unloading within the CLI session.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "useOSC52Paste": { + "title": "Use OSC 52 Paste", + "description": "Use OSC 52 for pasting. This may be more robust than the default system when using remote terminal sessions (if your terminal is configured to allow it).", + "markdownDescription": "Use OSC 52 for pasting. This may be more robust than the default system when using remote terminal sessions (if your terminal is configured to allow it).\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "useOSC52Copy": { + "title": "Use OSC 52 Copy", + "description": "Use OSC 52 for copying. This may be more robust than the default system when using remote terminal sessions (if your terminal is configured to allow it).", + "markdownDescription": "Use OSC 52 for copying. This may be more robust than the default system when using remote terminal sessions (if your terminal is configured to allow it).\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "taskTracker": { + "title": "Task Tracker", + "description": "Enable task tracker tools.", + "markdownDescription": "Enable task tracker tools.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "modelSteering": { + "title": "Model Steering", + "description": "Enable model steering (user hints) to guide the model during tool execution.", + "markdownDescription": "Enable model steering (user hints) to guide the model during tool execution.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "directWebFetch": { + "title": "Direct Web Fetch", + "description": "Enable web fetch behavior that bypasses LLM summarization.", + "markdownDescription": "Enable web fetch behavior that bypasses LLM summarization.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "dynamicModelConfiguration": { + "title": "Dynamic Model Configuration", + "description": "Enable dynamic model configuration (definitions, resolutions, and chains) via settings.", + "markdownDescription": "Enable dynamic model configuration (definitions, resolutions, and chains) via settings.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "gemmaModelRouter": { + "title": "Gemma Model Router", + "description": "Enable Gemma model router (experimental).", + "markdownDescription": "Enable Gemma model router (experimental).\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Enable Gemma Model Router", + "description": "Enable the Gemma Model Router (experimental). Requires a local endpoint serving Gemma via the Gemini API using LiteRT-LM shim.", + "markdownDescription": "Enable the Gemma Model Router (experimental). Requires a local endpoint serving Gemma via the Gemini API using LiteRT-LM shim.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "autoStartServer": { + "title": "Auto-start LiteRT Server", + "description": "Automatically start the LiteRT-LM server when Gemini CLI starts and the Gemma router is enabled.", + "markdownDescription": "Automatically start the LiteRT-LM server when Gemini CLI starts and the Gemma router is enabled.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "binaryPath": { + "title": "LiteRT Binary Path", + "description": "Custom path to the LiteRT-LM binary. Leave empty to use the default location (~/.gemini/bin/litert/).", + "markdownDescription": "Custom path to the LiteRT-LM binary. Leave empty to use the default location (~/.gemini/bin/litert/).\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: ``", + "default": "", + "type": "string" + }, + "classifier": { + "title": "Classifier", + "description": "Classifier configuration.", + "markdownDescription": "Classifier configuration.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "host": { + "title": "Host", + "description": "The host of the classifier.", + "markdownDescription": "The host of the classifier.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `http://localhost:9379`", + "default": "http://localhost:9379", + "type": "string" + }, + "model": { + "title": "Model", + "description": "The model to use for the classifier. Only tested on `gemma3-1b-gpu-custom`.", + "markdownDescription": "The model to use for the classifier. Only tested on `gemma3-1b-gpu-custom`.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `gemma3-1b-gpu-custom`", + "default": "gemma3-1b-gpu-custom", + "type": "string" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "stressTestProfile": { + "title": "Use the stress test profile to aggressively trigger context management.", + "description": "Significantly lowers token limits to force early garbage collection and distillation for testing purposes.", + "markdownDescription": "Significantly lowers token limits to force early garbage collection and distillation for testing purposes.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "autoMemory": { + "title": "Auto Memory", + "description": "Automatically extract memory patches and skills from past sessions in the background. Every change is written as a unified diff `.patch` file under `/.inbox//` and held for review in /memory inbox; nothing is applied until you approve it.", + "markdownDescription": "Automatically extract memory patches and skills from past sessions in the background. Every change is written as a unified diff `.patch` file under `/.inbox//` and held for review in /memory inbox; nothing is applied until you approve it.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "generalistProfile": { + "title": "Use the generalist profile to manage agent contexts.", + "description": "Suitable for general coding and software development tasks.", + "markdownDescription": "Suitable for general coding and software development tasks.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "powerUserProfile": { + "title": "Use the power user profile to manage agent contexts.", + "description": "Less cache friendly version of the generalist profile.", + "markdownDescription": "Less cache friendly version of the generalist profile.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "contextManagement": { + "title": "Enable Context Management", + "description": "Enable logic for context management.", + "markdownDescription": "Enable logic for context management.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "topicUpdateNarration": { + "title": "Topic & Update Narration", + "description": "Deprecated: Use general.topicUpdateNarration instead.", + "markdownDescription": "Deprecated: Use general.topicUpdateNarration instead.\n\n- Category: `Experimental`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "extensions": { + "title": "Extensions", + "description": "Settings for extensions.", + "markdownDescription": "Settings for extensions.\n\n- Category: `Extensions`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "disabled": { + "title": "Disabled Extensions", + "description": "List of disabled extensions.", + "markdownDescription": "List of disabled extensions.\n\n- Category: `Extensions`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "workspacesWithMigrationNudge": { + "title": "Workspaces with Migration Nudge", + "description": "List of workspaces for which the migration nudge has been shown.", + "markdownDescription": "List of workspaces for which the migration nudge has been shown.\n\n- Category: `Extensions`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + } + }, + "additionalProperties": false + }, + "skills": { + "title": "Skills", + "description": "Settings for agent skills.", + "markdownDescription": "Settings for agent skills.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Enable Agent Skills", + "description": "Enable Agent Skills.", + "markdownDescription": "Enable Agent Skills.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "disabled": { + "title": "Disabled Skills", + "description": "List of disabled skills.", + "markdownDescription": "List of disabled skills.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + } + }, + "additionalProperties": false + }, + "hooksConfig": { + "title": "HooksConfig", + "description": "Hook configurations for intercepting and customizing agent behavior.", + "markdownDescription": "Hook configurations for intercepting and customizing agent behavior.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Enable Hooks", + "description": "Canonical toggle for the hooks system. When disabled, no hooks will be executed.", + "markdownDescription": "Canonical toggle for the hooks system. When disabled, no hooks will be executed.\n\n- Category: `Advanced`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "disabled": { + "title": "Disabled Hooks", + "description": "List of hook names (commands) that should be disabled. Hooks in this list will not execute even if configured.", + "markdownDescription": "List of hook names (commands) that should be disabled. Hooks in this list will not execute even if configured.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "type": "array", + "items": { + "type": "string" + } + }, + "notifications": { + "title": "Hook Notifications", + "description": "Show visual indicators when hooks are executing.", + "markdownDescription": "Show visual indicators when hooks are executing.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "hooks": { + "title": "Hook Events", + "description": "Event-specific hook configurations.", + "markdownDescription": "Event-specific hook configurations.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "BeforeTool": { + "title": "Before Tool Hooks", + "description": "Hooks that execute before tool execution. Can intercept, validate, or modify tool calls.", + "markdownDescription": "Hooks that execute before tool execution. Can intercept, validate, or modify tool calls.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "AfterTool": { + "title": "After Tool Hooks", + "description": "Hooks that execute after tool execution. Can process results, log outputs, or trigger follow-up actions.", + "markdownDescription": "Hooks that execute after tool execution. Can process results, log outputs, or trigger follow-up actions.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "BeforeAgent": { + "title": "Before Agent Hooks", + "description": "Hooks that execute before agent loop starts. Can set up context or initialize resources.", + "markdownDescription": "Hooks that execute before agent loop starts. Can set up context or initialize resources.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "AfterAgent": { + "title": "After Agent Hooks", + "description": "Hooks that execute after agent loop completes. Can perform cleanup or summarize results.", + "markdownDescription": "Hooks that execute after agent loop completes. Can perform cleanup or summarize results.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "Notification": { + "title": "Notification Hooks", + "description": "Hooks that execute on notification events (errors, warnings, info). Can log or alert on specific conditions.", + "markdownDescription": "Hooks that execute on notification events (errors, warnings, info). Can log or alert on specific conditions.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "SessionStart": { + "title": "Session Start Hooks", + "description": "Hooks that execute when a session starts. Can initialize session-specific resources or state.", + "markdownDescription": "Hooks that execute when a session starts. Can initialize session-specific resources or state.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "SessionEnd": { + "title": "Session End Hooks", + "description": "Hooks that execute when a session ends. Can perform cleanup or persist session data.", + "markdownDescription": "Hooks that execute when a session ends. Can perform cleanup or persist session data.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "PreCompress": { + "title": "Pre-Compress Hooks", + "description": "Hooks that execute before chat history compression. Can back up or analyze conversation before compression.", + "markdownDescription": "Hooks that execute before chat history compression. Can back up or analyze conversation before compression.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "BeforeModel": { + "title": "Before Model Hooks", + "description": "Hooks that execute before LLM requests. Can modify prompts, inject context, or control model parameters.", + "markdownDescription": "Hooks that execute before LLM requests. Can modify prompts, inject context, or control model parameters.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "AfterModel": { + "title": "After Model Hooks", + "description": "Hooks that execute after LLM responses. Can process outputs, extract information, or log interactions.", + "markdownDescription": "Hooks that execute after LLM responses. Can process outputs, extract information, or log interactions.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + }, + "BeforeToolSelection": { + "title": "Before Tool Selection Hooks", + "description": "Hooks that execute before tool selection. Can filter or prioritize available tools dynamically.", + "markdownDescription": "Hooks that execute before tool selection. Can filter or prioritize available tools dynamically.\n\n- Category: `Advanced`\n- Requires restart: `no`\n- Default: `[]`", + "default": [], + "$ref": "#/$defs/HookDefinitionArray" + } + }, + "additionalProperties": { + "type": "array", + "items": {} + } + }, + "contextManagement": { + "title": "Context Management", + "description": "Settings for agent history and tool distillation context management.", + "markdownDescription": "Settings for agent history and tool distillation context management.\n\n- Category: `Experimental`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "historyWindow": { + "title": "History Window Settings", + "markdownDescription": "Description not provided.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "maxTokens": { + "title": "Max Tokens", + "description": "The number of tokens to allow before triggering compression.", + "markdownDescription": "The number of tokens to allow before triggering compression.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `150000`", + "default": 150000, + "type": "number" + }, + "retainedTokens": { + "title": "Retained Tokens", + "description": "The number of tokens to always retain.", + "markdownDescription": "The number of tokens to always retain.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `40000`", + "default": 40000, + "type": "number" + } + }, + "additionalProperties": false + }, + "messageLimits": { + "title": "Message Limits", + "markdownDescription": "Description not provided.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "normalMaxTokens": { + "title": "Normal Maximum Tokens", + "description": "The target number of tokens to budget for a normal conversation turn.", + "markdownDescription": "The target number of tokens to budget for a normal conversation turn.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `2500`", + "default": 2500, + "type": "number" + }, + "retainedMaxTokens": { + "title": "Retained Maximum Tokens", + "description": "The maximum number of tokens a single conversation turn can consume before truncation.", + "markdownDescription": "The maximum number of tokens a single conversation turn can consume before truncation.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `12000`", + "default": 12000, + "type": "number" + }, + "normalizationHeadRatio": { + "title": "Normalization Head Ratio", + "description": "The ratio of tokens to retain from the beginning of a truncated message (0.0 to 1.0).", + "markdownDescription": "The ratio of tokens to retain from the beginning of a truncated message (0.0 to 1.0).\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `0.25`", + "default": 0.25, + "type": "number" + } + }, + "additionalProperties": false + }, + "tools": { + "title": "Context Management Tools", + "markdownDescription": "Description not provided.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "distillation": { + "title": "Tool Distillation", + "markdownDescription": "Description not provided.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "maxOutputTokens": { + "title": "Max Output Tokens", + "description": "Maximum tokens to show to the model when truncating large tool outputs.", + "markdownDescription": "Maximum tokens to show to the model when truncating large tool outputs.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `10000`", + "default": 10000, + "type": "number" + }, + "summarizationThresholdTokens": { + "title": "Tool Summarization Threshold", + "description": "Threshold above which truncated tool outputs will be summarized by an LLM.", + "markdownDescription": "Threshold above which truncated tool outputs will be summarized by an LLM.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `20000`", + "default": 20000, + "type": "number" + } + }, + "additionalProperties": false + }, + "outputMasking": { + "title": "Tool Output Masking", + "description": "Advanced settings for tool output masking to manage context window efficiency.", + "markdownDescription": "Advanced settings for tool output masking to manage context window efficiency.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "protectionThresholdTokens": { + "title": "Tool Protection Threshold (Tokens)", + "description": "Minimum number of tokens to protect from masking (most recent tool outputs).", + "markdownDescription": "Minimum number of tokens to protect from masking (most recent tool outputs).\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `50000`", + "default": 50000, + "type": "number" + }, + "minPrunableThresholdTokens": { + "title": "Min Prunable Tokens Threshold", + "description": "Minimum prunable tokens required to trigger a masking pass.", + "markdownDescription": "Minimum prunable tokens required to trigger a masking pass.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `30000`", + "default": 30000, + "type": "number" + }, + "protectLatestTurn": { + "title": "Protect Latest Turn", + "description": "Ensures the absolute latest turn is never masked, regardless of token count.", + "markdownDescription": "Ensures the absolute latest turn is never masked, regardless of token count.\n\n- Category: `Context Management`\n- Requires restart: `yes`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "admin": { + "title": "Admin", + "description": "Settings configured remotely by enterprise admins.", + "markdownDescription": "Settings configured remotely by enterprise admins.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "secureModeEnabled": { + "title": "Secure Mode Enabled", + "description": "If true, disallows YOLO mode and \"Always allow\" options from being used.", + "markdownDescription": "If true, disallows YOLO mode and \"Always allow\" options from being used.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `false`", + "default": false, + "type": "boolean" + }, + "extensions": { + "title": "Extensions Settings", + "description": "Extensions-specific admin settings.", + "markdownDescription": "Extensions-specific admin settings.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Extensions Enabled", + "description": "If false, disallows extensions from being installed or used.", + "markdownDescription": "If false, disallows extensions from being installed or used.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + }, + "mcp": { + "title": "MCP Settings", + "description": "MCP-specific admin settings.", + "markdownDescription": "MCP-specific admin settings.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "MCP Enabled", + "description": "If false, disallows MCP servers from being used.", + "markdownDescription": "If false, disallows MCP servers from being used.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + }, + "config": { + "title": "MCP Config", + "description": "Admin-configured MCP servers (allowlist).", + "markdownDescription": "Admin-configured MCP servers (allowlist).\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/MCPServerConfig" + } + }, + "requiredConfig": { + "title": "Required MCP Config", + "description": "Admin-required MCP servers that are always injected.", + "markdownDescription": "Admin-required MCP servers that are always injected.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "additionalProperties": { + "$ref": "#/$defs/RequiredMcpServerConfig" + } + } + }, + "additionalProperties": false + }, + "skills": { + "title": "Skills Settings", + "description": "Agent Skills-specific admin settings.", + "markdownDescription": "Agent Skills-specific admin settings.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `{}`", + "default": {}, + "type": "object", + "properties": { + "enabled": { + "title": "Skills Enabled", + "description": "If false, disallows agent skills from being used.", + "markdownDescription": "If false, disallows agent skills from being used.\n\n- Category: `Admin`\n- Requires restart: `no`\n- Default: `true`", + "default": true, + "type": "boolean" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + } + }, + "$defs": { + "MCPServerConfig": { + "type": "object", + "description": "Definition of a Model Context Protocol (MCP) server configuration.", + "additionalProperties": false, + "properties": { + "command": { + "type": "string", + "description": "Executable invoked for stdio transport." + }, + "args": { + "type": "array", + "description": "Command-line arguments for the stdio transport command.", + "items": { + "type": "string" + } + }, + "env": { + "type": "object", + "description": "Environment variables to set for the server process.", + "additionalProperties": { + "type": "string" + } + }, + "cwd": { + "type": "string", + "description": "Working directory for the server process." + }, + "url": { + "type": "string", + "description": "URL for SSE or HTTP transport. Use with \"type\" field to specify transport type." + }, + "httpUrl": { + "type": "string", + "description": "Streaming HTTP transport URL." + }, + "headers": { + "type": "object", + "description": "Additional HTTP headers sent to the server.", + "additionalProperties": { + "type": "string" + } + }, + "tcp": { + "type": "string", + "description": "TCP address for websocket transport." + }, + "type": { + "type": "string", + "description": "Transport type. Use \"stdio\" for local command, \"sse\" for Server-Sent Events, or \"http\" for Streamable HTTP.", + "enum": ["stdio", "sse", "http"] + }, + "timeout": { + "type": "number", + "description": "Timeout in milliseconds for MCP requests." + }, + "trust": { + "type": "boolean", + "description": "Marks the server as trusted. Trusted servers may gain additional capabilities." + }, + "description": { + "type": "string", + "description": "Human-readable description of the server." + }, + "includeTools": { + "type": "array", + "description": "Subset of tools that should be enabled for this server. When omitted all tools are enabled.", + "items": { + "type": "string" + } + }, + "excludeTools": { + "type": "array", + "description": "Tools that should be disabled for this server even if exposed.", + "items": { + "type": "string" + } + }, + "extension": { + "type": "object", + "description": "Metadata describing the Gemini CLI extension that owns this MCP server.", + "additionalProperties": { + "type": ["string", "boolean", "number"] + } + }, + "oauth": { + "type": "object", + "description": "OAuth configuration for authenticating with the server.", + "additionalProperties": true + }, + "authProviderType": { + "type": "string", + "description": "Authentication provider used for acquiring credentials (for example `dynamic_discovery`).", + "enum": [ + "dynamic_discovery", + "google_credentials", + "service_account_impersonation" + ] + }, + "targetAudience": { + "type": "string", + "description": "OAuth target audience (CLIENT_ID.apps.googleusercontent.com)." + }, + "targetServiceAccount": { + "type": "string", + "description": "Service account email to impersonate (name@project.iam.gserviceaccount.com)." + } + } + }, + "RequiredMcpServerConfig": { + "type": "object", + "description": "Admin-required MCP server configuration (remote transports only).", + "additionalProperties": false, + "properties": { + "url": { + "type": "string", + "description": "URL for the required MCP server." + }, + "type": { + "type": "string", + "description": "Transport type for the required server.", + "enum": ["sse", "http"] + }, + "headers": { + "type": "object", + "description": "Additional HTTP headers sent to the server.", + "additionalProperties": { + "type": "string" + } + }, + "timeout": { + "type": "number", + "description": "Timeout in milliseconds for MCP requests." + }, + "trust": { + "type": "boolean", + "description": "Marks the server as trusted. Defaults to true for admin-required servers." + }, + "description": { + "type": "string", + "description": "Human-readable description of the server." + }, + "includeTools": { + "type": "array", + "description": "Subset of tools enabled for this server.", + "items": { + "type": "string" + } + }, + "excludeTools": { + "type": "array", + "description": "Tools disabled for this server.", + "items": { + "type": "string" + } + }, + "oauth": { + "type": "object", + "description": "OAuth configuration for authenticating with the server.", + "additionalProperties": true + }, + "authProviderType": { + "type": "string", + "description": "Authentication provider used for acquiring credentials.", + "enum": [ + "dynamic_discovery", + "google_credentials", + "service_account_impersonation" + ] + }, + "targetAudience": { + "type": "string", + "description": "OAuth target audience (CLIENT_ID.apps.googleusercontent.com)." + }, + "targetServiceAccount": { + "type": "string", + "description": "Service account email to impersonate (name@project.iam.gserviceaccount.com)." + } + } + }, + "TelemetrySettings": { + "type": "object", + "description": "Telemetry configuration for Gemini CLI.", + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean", + "description": "Enables telemetry emission." + }, + "target": { + "type": "string", + "description": "Telemetry destination (for example `stderr`, `stdout`, or `otlp`)." + }, + "otlpEndpoint": { + "type": "string", + "description": "Endpoint for OTLP exporters." + }, + "otlpProtocol": { + "type": "string", + "description": "Protocol for OTLP exporters.", + "enum": ["grpc", "http"] + }, + "traces": { + "type": "boolean", + "description": "Whether detailed traces with large attributes are captured." + }, + "logPrompts": { + "type": "boolean", + "description": "Whether prompts are logged in telemetry payloads." + }, + "outfile": { + "type": "string", + "description": "File path for writing telemetry output." + }, + "useCollector": { + "type": "boolean", + "description": "Whether to forward telemetry to an OTLP collector." + }, + "useCliAuth": { + "type": "boolean", + "description": "Whether to use CLI authentication for telemetry (only for in-process exporters)." + } + } + }, + "BugCommandSettings": { + "type": "object", + "description": "Configuration for the bug report helper command.", + "additionalProperties": false, + "properties": { + "urlTemplate": { + "type": "string", + "description": "Template used to open a bug report URL. Variables in the template are populated at runtime." + } + }, + "required": ["urlTemplate"] + }, + "SummarizeToolOutputSettings": { + "type": "object", + "description": "Controls summarization behavior for individual tools. All properties are optional.", + "additionalProperties": false, + "properties": { + "tokenBudget": { + "type": "number", + "description": "Maximum number of tokens used when summarizing tool output." + } + } + }, + "AgentOverride": { + "type": "object", + "description": "Override settings for a specific agent.", + "additionalProperties": false, + "properties": { + "modelConfig": { + "type": "object", + "additionalProperties": true + }, + "runConfig": { + "type": "object", + "description": "Run configuration for an agent.", + "additionalProperties": false, + "properties": { + "maxTimeMinutes": { + "type": "number", + "description": "The maximum execution time for the agent in minutes." + }, + "maxTurns": { + "type": "number", + "description": "The maximum number of conversational turns." + } + } + }, + "enabled": { + "type": "boolean", + "description": "Whether to enable the agent." + } + } + }, + "CustomTheme": { + "type": "object", + "description": "Custom theme definition used for styling Gemini CLI output. Colors are provided as hex strings or named ANSI colors.", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "enum": ["custom"], + "default": "custom" + }, + "name": { + "type": "string", + "description": "Theme display name." + }, + "text": { + "type": "object", + "additionalProperties": false, + "properties": { + "primary": { + "type": "string" + }, + "secondary": { + "type": "string" + }, + "link": { + "type": "string" + }, + "accent": { + "type": "string" + }, + "response": { + "type": "string" + } + } + }, + "background": { + "type": "object", + "additionalProperties": false, + "properties": { + "primary": { + "type": "string" + }, + "diff": { + "type": "object", + "additionalProperties": false, + "properties": { + "added": { + "type": "string" + }, + "removed": { + "type": "string" + } + } + } + } + }, + "border": { + "type": "object", + "additionalProperties": false, + "properties": { + "default": { + "type": "string" + }, + "focused": { + "type": "string" + } + } + }, + "ui": { + "type": "object", + "additionalProperties": false, + "properties": { + "comment": { + "type": "string" + }, + "symbol": { + "type": "string" + }, + "gradient": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "status": { + "type": "object", + "additionalProperties": false, + "properties": { + "error": { + "type": "string" + }, + "success": { + "type": "string" + }, + "warning": { + "type": "string" + } + } + }, + "Background": { + "type": "string" + }, + "Foreground": { + "type": "string" + }, + "LightBlue": { + "type": "string" + }, + "AccentBlue": { + "type": "string" + }, + "AccentPurple": { + "type": "string" + }, + "AccentCyan": { + "type": "string" + }, + "AccentGreen": { + "type": "string" + }, + "AccentYellow": { + "type": "string" + }, + "AccentRed": { + "type": "string" + }, + "DiffAdded": { + "type": "string" + }, + "DiffRemoved": { + "type": "string" + }, + "Comment": { + "type": "string" + }, + "Gray": { + "type": "string" + }, + "DarkGray": { + "type": "string" + }, + "GradientColors": { + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": ["type", "name"] + }, + "StringOrStringArray": { + "description": "Accepts either a single string or an array of strings.", + "anyOf": [ + { + "type": "string" + }, + { + "type": "array", + "items": { + "type": "string" + } + } + ] + }, + "BooleanOrStringOrObject": { + "description": "Accepts either a boolean flag, a string command name, or a configuration object.", + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "string" + }, + { + "type": "object", + "description": "Sandbox configuration object.", + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean", + "description": "Enables or disables the sandbox." + }, + "command": { + "type": "string", + "description": "The sandbox command to use (docker, podman, sandbox-exec, runsc, lxc).", + "enum": ["docker", "podman", "sandbox-exec", "runsc", "lxc"] + }, + "image": { + "type": "string", + "description": "The sandbox image to use." + }, + "allowedPaths": { + "type": "array", + "description": "A list of absolute host paths that should be accessible within the sandbox.", + "items": { + "type": "string" + } + }, + "networkAccess": { + "type": "boolean", + "description": "Whether the sandbox should have internet access." + } + } + } + ] + }, + "HookDefinitionArray": { + "type": "array", + "description": "Array of hook definition objects for a specific event.", + "items": { + "type": "object", + "description": "Hook definition specifying matcher pattern and hook configurations.", + "properties": { + "matcher": { + "type": "string", + "description": "Pattern to match against the event context (tool name, notification type, etc.). Supports exact match, regex (/pattern/), and wildcards (*)." + }, + "hooks": { + "type": "array", + "description": "Hooks to execute when the matcher matches.", + "items": { + "type": "object", + "description": "Individual hook configuration.", + "properties": { + "name": { + "type": "string", + "description": "Unique identifier for the hook." + }, + "type": { + "type": "string", + "description": "Type of hook (currently only \"command\" supported)." + }, + "command": { + "type": "string", + "description": "Shell command to execute. Receives JSON input via stdin and returns JSON output via stdout." + }, + "description": { + "type": "string", + "description": "A description of the hook." + }, + "timeout": { + "type": "number", + "description": "Timeout in milliseconds for hook execution." + } + } + } + } + } + } + }, + "ModelDefinition": { + "type": "object", + "description": "Model metadata registry entry.", + "properties": { + "displayName": { + "type": "string" + }, + "tier": { + "enum": ["pro", "flash", "flash-lite", "custom", "auto"] + }, + "family": { + "type": "string" + }, + "isPreview": { + "type": "boolean" + }, + "isVisible": { + "type": "boolean" + }, + "dialogDescription": { + "type": "string", + "description": "A description of the model to display in the model selection dialog. For the 'auto' alias, this value is dynamically generated and any value provided here will be ignored." + }, + "features": { + "type": "object", + "properties": { + "thinking": { + "type": "boolean" + }, + "multimodalToolUse": { + "type": "boolean" + } + } + } + } + }, + "ModelResolution": { + "type": "object", + "description": "Model resolution rule.", + "properties": { + "default": { + "type": "string" + }, + "contexts": { + "type": "array", + "items": { + "type": "object", + "properties": { + "condition": { + "type": "object", + "properties": { + "useGemini3_1": { + "type": "boolean" + }, + "useGemini3_1FlashLite": { + "type": "boolean" + }, + "useCustomTools": { + "type": "boolean" + }, + "hasAccessToPreview": { + "type": "boolean" + }, + "requestedModels": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "target": { + "type": "string" + } + } + } + } + } + }, + "ModelPolicyChain": { + "type": "array", + "description": "A chain of model policies for fallback behavior.", + "items": { + "type": "object", + "ref": "ModelPolicy" + } + }, + "ModelPolicy": { + "type": "object", + "description": "Defines the policy for a single model in the availability chain.", + "properties": { + "model": { + "type": "string" + }, + "isLastResort": { + "type": "boolean" + }, + "actions": { + "type": "object", + "properties": { + "terminal": { + "type": "string", + "enum": ["silent", "prompt"] + }, + "transient": { + "type": "string", + "enum": ["silent", "prompt"] + }, + "not_found": { + "type": "string", + "enum": ["silent", "prompt"] + }, + "unknown": { + "type": "string", + "enum": ["silent", "prompt"] + } + } + }, + "stateTransitions": { + "type": "object", + "properties": { + "terminal": { + "type": "string", + "enum": ["terminal", "sticky_retry"] + }, + "transient": { + "type": "string", + "enum": ["terminal", "sticky_retry"] + }, + "not_found": { + "type": "string", + "enum": ["terminal", "sticky_retry"] + }, + "unknown": { + "type": "string", + "enum": ["terminal", "sticky_retry"] + } + } + } + }, + "required": ["model"] + } + } +} diff --git a/scripts/batch_triage.sh b/scripts/batch_triage.sh new file mode 100644 index 0000000000000000000000000000000000000000..1f4a84b97a9e538c7df3dedc54b750d41ab5ee2f --- /dev/null +++ b/scripts/batch_triage.sh @@ -0,0 +1,42 @@ +#!/bin/bash +# scripts/batch_triage.sh +# Usage: ./scripts/batch_triage.sh [repository] +# Example: ./scripts/batch_triage.sh google-gemini/maintainers-gemini-cli + +set -e +set -o pipefail + +REPO="${1:-google-gemini/gemini-cli}" +WORKFLOW="gemini-automated-issue-triage.yml" + +echo "πŸ” Searching for open issues in '${REPO}' that need triage (missing 'area/' label)..." + +# Fetch open issues with number, title, and labels +# We fetch up to 1000 issues. +ISSUES_JSON=$(gh issue list --repo "${REPO}" --state open --limit 1000 --json number,title,labels) + +# Filter issues that DO NOT have a label starting with 'area/' +TARGET_ISSUES=$(echo "${ISSUES_JSON}" | jq '[.[] | select(.labels | map(.name) | any(startswith("area/")) | not)]') + +# Avoid masking return value +COUNT=$(jq '. | length' <<< "${TARGET_ISSUES}") + +if [[ "${COUNT}" -eq 0 ]]; then + echo "βœ… No issues found needing triage in '${REPO}'." + exit 0 +fi + +echo "πŸš€ Found ${COUNT} issues to triage." + +# Loop through and trigger workflow +echo "${TARGET_ISSUES}" | jq -r '.[] | "\(.number)|\(.title)"' | while IFS="|" read -r number title; do + echo "▢️ Triggering triage for #${number}: ${title}" + + # Trigger the workflow dispatch event + gh workflow run "${WORKFLOW}" --repo "${REPO}" -f issue_number="${number}" + + # Sleep briefly to be nice to the API + sleep 1 +done + +echo "πŸŽ‰ All triage workflows triggered!" \ No newline at end of file diff --git a/scripts/build_package.js b/scripts/build_package.js new file mode 100644 index 0000000000000000000000000000000000000000..74233ac4fbacfb3a0a39e13855d6247c5bbe83e8 --- /dev/null +++ b/scripts/build_package.js @@ -0,0 +1,61 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { execSync } from 'node:child_process'; +import { writeFileSync, existsSync, cpSync, rmSync } from 'node:fs'; +import { join, basename } from 'node:path'; + +if (!process.cwd().includes('packages')) { + console.error('must be invoked from a package directory'); + process.exit(1); +} + +const packageName = basename(process.cwd()); + +// build typescript files +execSync('tsc --build', { stdio: 'inherit' }); + +// Run package-specific bundling if the script exists +const bundleScript = join(process.cwd(), 'scripts', 'bundle-browser-mcp.mjs'); +if (packageName === 'core' && existsSync(bundleScript)) { + console.log('Running chrome devtools MCP bundling...'); + execSync('npm run bundle:browser-mcp', { + stdio: 'inherit', + }); +} + +// copy .{md,json} files +execSync('node ../../scripts/copy_files.js', { stdio: 'inherit' }); + +// Copy documentation for the core package +if (packageName === 'core') { + const docsSource = join(process.cwd(), '..', '..', 'docs'); + const docsTarget = join(process.cwd(), 'dist', 'docs'); + if (existsSync(docsSource)) { + if (existsSync(docsTarget)) { + rmSync(docsTarget, { recursive: true, force: true }); + } + cpSync(docsSource, docsTarget, { recursive: true, dereference: true }); + console.log('Copied documentation to dist/docs'); + } +} + +// touch dist/.last_build +writeFileSync(join(process.cwd(), 'dist', '.last_build'), ''); +process.exit(0); diff --git a/scripts/changed_prompt.js b/scripts/changed_prompt.js new file mode 100644 index 0000000000000000000000000000000000000000..3fe33443a07fe70d90a6856b2cd7f0bdcbd673f2 --- /dev/null +++ b/scripts/changed_prompt.js @@ -0,0 +1,103 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ +import { execSync } from 'node:child_process'; + +const CORE_STEERING_PATHS = [ + 'packages/core/src/prompts/', + 'packages/core/src/tools/', +]; + +const TEST_PATHS = ['evals/']; + +const STEERING_SIGNATURES = [ + 'LocalAgentDefinition', + 'LocalInvocation', + 'ToolDefinition', + 'inputSchema', + "kind: 'local'", +]; + +function main() { + const targetBranch = process.env.GITHUB_BASE_REF || 'main'; + const verbose = process.argv.includes('--verbose'); + const steeringOnly = process.argv.includes('--steering-only'); + + try { + const remoteUrl = process.env.GITHUB_REPOSITORY + ? `https://github.com/${process.env.GITHUB_REPOSITORY}.git` + : 'origin'; + + // Fetch target branch from the remote. + execSync(`git fetch ${remoteUrl} ${targetBranch}`, { + stdio: 'ignore', + }); + + // Get changed files using the triple-dot syntax which correctly handles merge commits + const head = process.env.PR_HEAD_SHA || 'HEAD'; + const changedFiles = execSync(`git diff --name-only FETCH_HEAD...${head}`, { + encoding: 'utf-8', + }) + .split('\n') + .filter(Boolean); + + let detected = false; + const reasons = []; + + // 1. Path-based detection + for (const file of changedFiles) { + if (CORE_STEERING_PATHS.some((prefix) => file.startsWith(prefix))) { + detected = true; + reasons.push(`Matched core steering path: ${file}`); + if (!verbose) break; + } + if ( + !steeringOnly && + TEST_PATHS.some((prefix) => file.startsWith(prefix)) + ) { + detected = true; + reasons.push(`Matched test path: ${file}`); + if (!verbose) break; + } + } + + // 2. Signature-based detection (only in packages/core/src/ and only if not already detected or if verbose) + if (!detected || verbose) { + const coreChanges = changedFiles.filter((f) => + f.startsWith('packages/core/src/'), + ); + if (coreChanges.length > 0) { + // Get the actual diff content for core files + const diff = execSync( + `git diff -U0 FETCH_HEAD...${head} -- packages/core/src/`, + { encoding: 'utf-8' }, + ); + for (const sig of STEERING_SIGNATURES) { + if (diff.includes(sig)) { + detected = true; + reasons.push(`Matched steering signature in core: ${sig}`); + if (!verbose) break; + } + } + } + } + + if (verbose && reasons.length > 0) { + process.stderr.write('Detection reasons:\n'); + reasons.forEach((r) => process.stderr.write(` - ${r}\n`)); + } + + process.stdout.write(detected ? 'true' : 'false'); + } catch (error) { + // If anything fails (e.g., no git history), run evals/guidance to be safe + process.stderr.write( + 'Warning: Failed to determine if changes occurred. Defaulting to true.\n', + ); + process.stderr.write(String(error) + '\n'); + process.stdout.write('true'); + } +} + +main(); diff --git a/scripts/check-build-status.js b/scripts/check-build-status.js new file mode 100644 index 0000000000000000000000000000000000000000..3c6eeb7eaee59a348d75dbd6d035dd16d8169f9d --- /dev/null +++ b/scripts/check-build-status.js @@ -0,0 +1,148 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import os from 'node:os'; // Import os module + +// --- Configuration --- +const cliPackageDir = path.resolve('packages', 'cli'); // Base directory for the CLI package +const buildTimestampPath = path.join(cliPackageDir, 'dist', '.last_build'); // Path to the timestamp file within the CLI package +const sourceDirs = [path.join(cliPackageDir, 'src')]; // Source directory within the CLI package +const filesToWatch = [ + path.join(cliPackageDir, 'package.json'), + path.join(cliPackageDir, 'tsconfig.json'), +]; // Specific files within the CLI package +const buildDir = path.join(cliPackageDir, 'dist'); // Build output directory within the CLI package +const warningsFilePath = path.join(os.tmpdir(), 'gemini-cli-warnings.txt'); // Temp file for warnings +// --------------------- + +function getMtime(filePath) { + try { + return fs.statSync(filePath).mtimeMs; // Use mtimeMs for higher precision + } catch (err) { + if (err.code === 'ENOENT') { + return null; // File doesn't exist + } + console.error(`Error getting stats for ${filePath}:`, err); + process.exit(1); // Exit on unexpected errors getting stats + } +} + +function findSourceFiles(dir, allFiles = []) { + const entries = fs.readdirSync(dir, { withFileTypes: true }); + for (const entry of entries) { + const fullPath = path.join(dir, entry.name); + // Simple check to avoid recursing into node_modules or build dir itself + if ( + entry.isDirectory() && + entry.name !== 'node_modules' && + fullPath !== buildDir + ) { + findSourceFiles(fullPath, allFiles); + } else if (entry.isFile()) { + allFiles.push(fullPath); + } + } + return allFiles; +} + +console.log('Checking build status...'); + +// Clean up old warnings file before check +try { + if (fs.existsSync(warningsFilePath)) { + fs.unlinkSync(warningsFilePath); + } +} catch (err) { + console.warn( + `[Check Script] Warning: Could not delete previous warnings file: ${err.message}`, + ); +} + +const buildMtime = getMtime(buildTimestampPath); +if (!buildMtime) { + // If build is missing, write that as a warning and exit(0) so app can display it + const errorMessage = `ERROR: Build timestamp file (${path.relative(process.cwd(), buildTimestampPath)}) not found. Run \`npm run build\` first.`; + console.error(errorMessage); // Still log error here + try { + fs.writeFileSync(warningsFilePath, errorMessage); + } catch (writeErr) { + console.error( + `[Check Script] Error writing missing build warning file: ${writeErr.message}`, + ); + } + process.exit(0); // Allow app to start and show the error +} + +let newerSourceFileFound = false; +const warningMessages = []; // Collect warnings here +const allSourceFiles = []; + +// Collect files from specified directories +sourceDirs.forEach((dir) => { + const dirPath = path.resolve(dir); + if (fs.existsSync(dirPath)) { + findSourceFiles(dirPath, allSourceFiles); + } else { + console.warn(`Warning: Source directory "${dir}" not found.`); + } +}); + +// Add specific files +filesToWatch.forEach((file) => { + const filePath = path.resolve(file); + if (fs.existsSync(filePath)) { + allSourceFiles.push(filePath); + } else { + console.warn(`Warning: Watched file "${file}" not found.`); + } +}); + +// Check modification times +for (const file of allSourceFiles) { + const sourceMtime = getMtime(file); + const relativePath = path.relative(process.cwd(), file); + const isNewer = sourceMtime && sourceMtime > buildMtime; + + if (isNewer) { + const warning = `Warning: Source file "${relativePath}" has been modified since the last build.`; + console.warn(warning); // Keep console warning for script debugging + warningMessages.push(warning); + newerSourceFileFound = true; + // break; // Uncomment to stop checking after the first newer file + } +} + +if (newerSourceFileFound) { + const finalWarning = + '\nRun "npm run build" to incorporate changes before starting.'; + warningMessages.push(finalWarning); + console.warn(finalWarning); + + // Write warnings to the temp file + try { + fs.writeFileSync(warningsFilePath, warningMessages.join('\n')); + // Removed debug log + } catch (err) { + console.error(`[Check Script] Error writing warnings file: ${err.message}`); + // Proceed without writing, app won't show warnings + } +} else { + console.log('Build is up-to-date.'); + // Ensure no stale warning file exists if build is ok + try { + if (fs.existsSync(warningsFilePath)) { + fs.unlinkSync(warningsFilePath); + } + } catch (err) { + console.warn( + `[Check Script] Warning: Could not delete previous warnings file: ${err.message}`, + ); + } +} + +process.exit(0); // Always exit successfully so the app starts diff --git a/scripts/check-inbox.js b/scripts/check-inbox.js new file mode 100644 index 0000000000000000000000000000000000000000..ef2cdd045521e9473dc7e08f09e0707a8409d21e --- /dev/null +++ b/scripts/check-inbox.js @@ -0,0 +1,60 @@ +#!/usr/bin/env node + +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * Diagnostic: instantiate the real Config and call the same listing functions + * the inbox UI uses. Should print out all skills + skill patches + memory + * patches the user would see in `/memory inbox`. + */ +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.resolve(SCRIPT_DIR, '..'); +const corePath = path.join(REPO_ROOT, 'packages/core/dist/src/index.js'); + +const { Storage, listInboxSkills, listInboxPatches, listInboxMemoryPatches } = + await import(corePath); + +const cwd = process.cwd(); +const storage = new Storage(cwd); +await storage.initialize(); + +const config = { + storage, + isTrustedFolder: () => true, + getProjectRoot: () => cwd, +}; + +const [skills, skillPatches, memoryPatches] = await Promise.all([ + listInboxSkills(config), + listInboxPatches(config), + listInboxMemoryPatches(config), +]); + +console.log(`\nInbox content for ${cwd}\n`); + +console.log(`Skills (${skills.length}):`); +for (const s of skills) { + console.log(` - ${s.name} (${s.dirName})`); +} + +console.log(`\nSkill update patches (${skillPatches.length}):`); +for (const p of skillPatches) { + console.log(` - ${p.name} β†’ ${p.entries.length} entry/entries`); +} + +console.log(`\nMemory patches (${memoryPatches.length}):`); +for (const m of memoryPatches) { + console.log( + ` - [${m.kind}] ${m.relativePath} β†’ ${m.entries.length} entry/entries`, + ); + for (const e of m.entries) { + console.log(` ${e.isNewFile ? 'CREATE' : 'UPDATE'} ${e.targetPath}`); + } +} diff --git a/scripts/download-ripgrep-binaries.ts b/scripts/download-ripgrep-binaries.ts new file mode 100644 index 0000000000000000000000000000000000000000..969d69c7ebc4db245a7126ff043121503868f615 --- /dev/null +++ b/scripts/download-ripgrep-binaries.ts @@ -0,0 +1,146 @@ +/** + * @license + * Copyright 2026 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * @fileoverview This script downloads pre-built ripgrep binaries for all supported + * architectures and platforms. These binaries are checked into the repository + * under packages/core/vendor/ripgrep. + * + * Maintainers should periodically run this script to upgrade the version + * of ripgrep being distributed. + * + * Usage: npx tsx scripts/download-ripgrep-binaries.ts + */ + +import fs from 'node:fs'; +import fsPromises from 'node:fs/promises'; +import path from 'node:path'; +import { pipeline } from 'node:stream/promises'; +import { fileURLToPath } from 'node:url'; +import { createWriteStream } from 'node:fs'; +import { Readable } from 'node:stream'; +import type { ReadableStream } from 'node:stream/web'; +import { execFileSync } from 'node:child_process'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const CORE_VENDOR_DIR = path.join(__dirname, '../packages/core/vendor/ripgrep'); +const VERSION = 'v13.0.0-10'; + +interface Target { + platform: string; + arch: string; + file: string; +} + +const targets: Target[] = [ + { platform: 'darwin', arch: 'arm64', file: 'aarch64-apple-darwin.tar.gz' }, + { platform: 'darwin', arch: 'x64', file: 'x86_64-apple-darwin.tar.gz' }, + { + platform: 'linux', + arch: 'arm64', + file: 'aarch64-unknown-linux-gnu.tar.gz', + }, + { platform: 'linux', arch: 'x64', file: 'x86_64-unknown-linux-musl.tar.gz' }, + { platform: 'win32', arch: 'x64', file: 'x86_64-pc-windows-msvc.zip' }, +]; + +async function downloadBinary() { + await fsPromises.mkdir(CORE_VENDOR_DIR, { recursive: true }); + + for (const target of targets) { + const url = `https://github.com/microsoft/ripgrep-prebuilt/releases/download/${VERSION}/ripgrep-${VERSION}-${target.file}`; + const archivePath = path.join(CORE_VENDOR_DIR, target.file); + const binName = `rg-${target.platform}-${target.arch}${target.platform === 'win32' ? '.exe' : ''}`; + const finalBinPath = path.join(CORE_VENDOR_DIR, binName); + + if (fs.existsSync(finalBinPath)) { + console.log(`[Cache] ${binName} already exists.`); + continue; + } + + console.log(`[Download] ${url} -> ${archivePath}`); + const response = await fetch(url); + if (!response.ok) { + throw new Error(`Failed to fetch ${url}: ${response.statusText}`); + } + + if (!response.body) { + throw new Error(`Response body is null for ${url}`); + } + + const fileStream = createWriteStream(archivePath); + + // Node 18+ global fetch response.body is a ReadableStream (web stream) + // pipeline(Readable.fromWeb(response.body), fileStream) works in Node 18+ + await pipeline( + Readable.fromWeb(response.body as ReadableStream), + fileStream, + ); + + console.log(`[Extract] Extracting ${archivePath}...`); + // Extract using shell commands for simplicity + if (target.file.endsWith('.tar.gz')) { + execFileSync('tar', ['-xzf', archivePath, '-C', CORE_VENDOR_DIR]); + // Microsoft's ripgrep release extracts directly to `rg` inside the current directory sometimes + const sourceBin = path.join(CORE_VENDOR_DIR, 'rg'); + if (fs.existsSync(sourceBin)) { + await fsPromises.rename(sourceBin, finalBinPath); + } else { + // Fallback for sub-directory if it happens + const extractedDirName = `ripgrep-${VERSION}-${target.file.replace('.tar.gz', '')}`; + const fallbackSourceBin = path.join( + CORE_VENDOR_DIR, + extractedDirName, + 'rg', + ); + if (fs.existsSync(fallbackSourceBin)) { + await fsPromises.rename(fallbackSourceBin, finalBinPath); + await fsPromises.rm(path.join(CORE_VENDOR_DIR, extractedDirName), { + recursive: true, + force: true, + }); + } else { + throw new Error( + `Could not find extracted 'rg' binary for ${target.platform} ${target.arch}`, + ); + } + } + } else if (target.file.endsWith('.zip')) { + execFileSync('unzip', ['-o', '-q', archivePath, '-d', CORE_VENDOR_DIR]); + const sourceBin = path.join(CORE_VENDOR_DIR, 'rg.exe'); + if (fs.existsSync(sourceBin)) { + await fsPromises.rename(sourceBin, finalBinPath); + } else { + const extractedDirName = `ripgrep-${VERSION}-${target.file.replace('.zip', '')}`; + const fallbackSourceBin = path.join( + CORE_VENDOR_DIR, + extractedDirName, + 'rg.exe', + ); + if (fs.existsSync(fallbackSourceBin)) { + await fsPromises.rename(fallbackSourceBin, finalBinPath); + await fsPromises.rm(path.join(CORE_VENDOR_DIR, extractedDirName), { + recursive: true, + force: true, + }); + } else { + throw new Error( + `Could not find extracted 'rg.exe' binary for ${target.platform} ${target.arch}`, + ); + } + } + } + + // Clean up archive + await fsPromises.unlink(archivePath); + console.log(`[Success] Saved to ${finalBinPath}`); + } +} + +downloadBinary().catch((err) => { + console.error(err); + process.exit(1); +}); diff --git a/scripts/generate-keybindings-doc.ts b/scripts/generate-keybindings-doc.ts new file mode 100644 index 0000000000000000000000000000000000000000..d1d4d85f4cc17cf93b257d2d66267c76cdd33337 --- /dev/null +++ b/scripts/generate-keybindings-doc.ts @@ -0,0 +1,177 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import path from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { readFile, writeFile } from 'node:fs/promises'; + +import type { KeyBinding } from '../packages/cli/src/ui/key/keyBindings.js'; +import { + commandCategories, + commandDescriptions, + defaultKeyBindingConfig, + Command, + getPlatformUndoBindings, + getPlatformRedoBindings, +} from '../packages/cli/src/ui/key/keyBindings.js'; +import { + formatWithPrettier, + injectBetweenMarkers, + normalizeForCompare, +} from './utils/autogen.js'; + +const START_MARKER = ''; +const END_MARKER = ''; +const OUTPUT_RELATIVE_PATH = ['docs', 'reference', 'keyboard-shortcuts.md']; + +import { formatKeyBinding } from '../packages/cli/src/ui/key/keybindingUtils.js'; + +export interface KeybindingDocCommand { + command: string; + description: string; + bindings: readonly KeyBinding[]; +} + +export interface KeybindingDocSection { + title: string; + commands: readonly KeybindingDocCommand[]; +} + +export async function main(argv = process.argv.slice(2)) { + const checkOnly = argv.includes('--check'); + + const repoRoot = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + '..', + ); + const docPath = path.join(repoRoot, ...OUTPUT_RELATIVE_PATH); + + const sections = buildDefaultDocSections(); + const generatedBlock = renderDocumentation(sections); + const currentDoc = await readFile(docPath, 'utf8'); + const injectedDoc = injectBetweenMarkers({ + document: currentDoc, + startMarker: START_MARKER, + endMarker: END_MARKER, + newContent: generatedBlock, + paddingBefore: '\n\n', + paddingAfter: '\n', + }); + const updatedDoc = await formatWithPrettier(injectedDoc, docPath); + + if (normalizeForCompare(updatedDoc) === normalizeForCompare(currentDoc)) { + if (!checkOnly) { + console.log('Keybinding documentation already up to date.'); + } + return; + } + + if (checkOnly) { + console.error( + 'Keybinding documentation is out of date. Run `npm run docs:keybindings` to regenerate.', + ); + process.exitCode = 1; + return; + } + + await writeFile(docPath, updatedDoc, 'utf8'); + console.log('Keybinding documentation regenerated.'); +} + +export function buildDefaultDocSections(): readonly KeybindingDocSection[] { + return commandCategories.map((category) => ({ + title: category.title, + commands: category.commands.map((command) => { + // For UNDO and REDO, we want to show all platform variants in the docs + if (command === Command.UNDO) { + return { + command: command, + description: commandDescriptions[command], + bindings: getMergedPlatformBindings(getPlatformUndoBindings), + }; + } + if (command === Command.REDO) { + return { + command: command, + description: commandDescriptions[command], + bindings: getMergedPlatformBindings(getPlatformRedoBindings), + }; + } + + return { + command: command, + description: commandDescriptions[command], + bindings: defaultKeyBindingConfig.get(command) ?? [], + }; + }), + })); +} + +function getMergedPlatformBindings( + getBindings: (platform: string) => readonly KeyBinding[], +): readonly KeyBinding[] { + const win32 = getBindings('win32'); + const darwin = getBindings('darwin'); + const linux = getBindings('linux'); + + const all = [...win32, ...darwin, ...linux]; + const seen = new Set(); + const unique: KeyBinding[] = []; + + for (const b of all) { + const key = `${b.name}-${b.ctrl}-${b.shift}-${b.alt}-${b.cmd}`; + if (!seen.has(key)) { + seen.add(key); + unique.push(b); + } + } + + return unique; +} + +export function renderDocumentation( + sections: readonly KeybindingDocSection[], +): string { + const renderedSections = sections.map((section) => { + const rows = section.commands.map((command) => { + const formattedBindings = formatBindings(command.bindings); + const keysCell = formattedBindings.join('
    '); + return `| \`${command.command}\` | ${command.description} | ${keysCell} |`; + }); + + return [ + `#### ${section.title}`, + '', + '| Command | Action | Keys |', + '| --- | --- | --- |', + ...rows, + ].join('\n'); + }); + + return renderedSections.join('\n\n'); +} + +function formatBindings(bindings: readonly KeyBinding[]): string[] { + const seen = new Set(); + const results: string[] = []; + + for (const binding of bindings) { + const label = formatKeyBinding(binding, 'default'); + if (label && !seen.has(label)) { + seen.add(label); + results.push(`\`${label}\``); + } + } + + return results; +} + +if (process.argv[1]) { + const entryUrl = pathToFileURL(path.resolve(process.argv[1])).href; + if (entryUrl === import.meta.url) { + await main(); + } +} diff --git a/scripts/sandbox_command.js b/scripts/sandbox_command.js new file mode 100644 index 0000000000000000000000000000000000000000..00becb667f733922efd408c09901429dbe5099f2 --- /dev/null +++ b/scripts/sandbox_command.js @@ -0,0 +1,129 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { execSync } from 'node:child_process'; +import { existsSync, readFileSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import stripJsonComments from 'strip-json-comments'; +import os from 'node:os'; +import yargs from 'yargs'; +import { hideBin } from 'yargs/helpers'; +import dotenv from 'dotenv'; +import { GEMINI_DIR } from '@google/gemini-cli-core'; + +const argv = yargs(hideBin(process.argv)).option('q', { + alias: 'quiet', + type: 'boolean', + default: false, +}).argv; + +const homedir = () => process.env['GEMINI_CLI_HOME'] || os.homedir(); + +let geminiSandbox = process.env.GEMINI_SANDBOX; + +if (!geminiSandbox) { + const userSettingsFile = join(homedir(), GEMINI_DIR, 'settings.json'); + if (existsSync(userSettingsFile)) { + const settings = JSON.parse( + stripJsonComments(readFileSync(userSettingsFile, 'utf-8')), + ); + if (settings.sandbox) { + geminiSandbox = settings.sandbox; + } + } +} + +if (!geminiSandbox) { + let currentDir = process.cwd(); + while (true) { + const geminiEnv = join(currentDir, GEMINI_DIR, '.env'); + const regularEnv = join(currentDir, '.env'); + if (existsSync(geminiEnv)) { + dotenv.config({ path: geminiEnv, quiet: true }); + break; + } else if (existsSync(regularEnv)) { + dotenv.config({ path: regularEnv, quiet: true }); + break; + } + const parentDir = dirname(currentDir); + if (parentDir === currentDir) { + break; + } + currentDir = parentDir; + } + geminiSandbox = process.env.GEMINI_SANDBOX; +} + +geminiSandbox = (geminiSandbox || '').toLowerCase(); + +const commandExists = (cmd) => { + const checkCommand = os.platform() === 'win32' ? 'where' : 'command -v'; + try { + execSync(`${checkCommand} ${cmd}`, { stdio: 'ignore' }); + return true; + } catch { + if (os.platform() === 'win32') { + try { + execSync(`${checkCommand} ${cmd}.exe`, { stdio: 'ignore' }); + return true; + } catch { + return false; + } + } + return false; + } +}; + +let command = ''; +if (['1', 'true'].includes(geminiSandbox)) { + if (commandExists('docker')) { + command = 'docker'; + } else if (commandExists('podman')) { + command = 'podman'; + } else { + console.error( + 'ERROR: install docker or podman or specify command in GEMINI_SANDBOX', + ); + process.exit(1); + } +} else if (geminiSandbox && !['0', 'false'].includes(geminiSandbox)) { + if (commandExists(geminiSandbox)) { + command = geminiSandbox; + } else { + console.error( + `ERROR: missing sandbox command '${geminiSandbox}' (from GEMINI_SANDBOX)`, + ); + process.exit(1); + } +} else { + if (os.platform() === 'darwin' && process.env.SEATBELT_PROFILE !== 'none') { + if (commandExists('sandbox-exec')) { + command = 'sandbox-exec'; + } else { + process.exit(1); + } + } else { + process.exit(1); + } +} + +if (!argv.q) { + console.log(command); +} +process.exit(0); diff --git a/scripts/send_gemini_request.sh b/scripts/send_gemini_request.sh new file mode 100644 index 0000000000000000000000000000000000000000..ebccf7e89ba4eb3638644e9c7373fa614b279845 --- /dev/null +++ b/scripts/send_gemini_request.sh @@ -0,0 +1,105 @@ +#!/bin/bash +# ----------------------------------------------------------------------------- +# Gemini API Replay Script +# ----------------------------------------------------------------------------- +# Purpose: +# This script is used to replay a Gemini API request using a raw JSON payload. +# It is particularly useful for debugging the exact requests made by the +# Gemini CLI. +# +# Prerequisites: +# 1. Export your Gemini API key: +# export GEMINI_API_KEY="your_api_key_here" +# +# 2. Generate a request payload from the Gemini CLI: +# Inside the CLI, run the `/chat debug` command. This will save the most +# recent API request to a file named `gcli-request-.json`. +# +# Usage: +# ./scripts/send_gemini_request.sh --payload --model [--stream] +# +# Options: +# --payload Path to the JSON request payload. +# --model The Gemini model ID (e.g., gemini-3-flash-preview). +# --stream (Optional) Use the streaming API endpoint. Defaults to non-streaming. +# +# Example: +# ./scripts/send_gemini_request.sh --payload gcli-request.json --model gemini-3-flash-preview +# ----------------------------------------------------------------------------- + +set -e -E + +# Load environment variables from .env if it exists +if [[ -f ".env" ]]; then + echo "Loading environment variables from .env file..." + set -a # Automatically export all variables + # shellcheck source=/dev/null + source .env + set +a +fi + +# Function to print usage +usage() { + echo "Usage: $0 --payload --model [--stream]" + echo "Ensure GEMINI_API_KEY environment variable is set." + exit 1 +} + +STREAM_MODE=false + +# Parse command line arguments +while [[ "$#" -gt 0 ]]; do + case $1 in + --payload) PAYLOAD_FILE="${2}"; shift ;; + --model) MODEL_ID="${2}"; shift ;; + --stream) STREAM_MODE=true ;; + *) echo "Unknown parameter passed: ${1}"; usage ;; + esac + shift +done + +# Validate inputs +if [[ -z "${PAYLOAD_FILE}" ]] || [[ -z "${MODEL_ID}" ]]; then + echo "Error: Missing required arguments." + usage +fi + +if [[ -z "${GEMINI_API_KEY}" ]]; then + echo "Error: GEMINI_API_KEY environment variable is not set." + exit 1 +fi + +if [[ ! -f "${PAYLOAD_FILE}" ]]; then + echo "Error: Payload file '${PAYLOAD_FILE}' does not exist." + exit 1 +fi + +# API Endpoint definition +if [[ "${STREAM_MODE}" = true ]]; then + GENERATE_CONTENT_API="streamGenerateContent" + echo "Mode: Streaming" +else + GENERATE_CONTENT_API="generateContent" + echo "Mode: Non-streaming (Default)" +fi + +echo "Sending request to model: ${MODEL_ID}" +echo "Using payload from: ${PAYLOAD_FILE}" +echo "----------------------------------------" + +# Make the cURL request. If non-streaming, pipe through jq for readability if available. +if [[ "${STREAM_MODE}" = false ]] && command -v jq &> /dev/null; then + # Invoke curl separately to avoid masking its return value + output=$(curl -s -X POST \ + -H "Content-Type: application/json" \ + "https://generativelanguage.googleapis.com/v1beta/models/${MODEL_ID}:${GENERATE_CONTENT_API}?key=${GEMINI_API_KEY}" \ + -d "@${PAYLOAD_FILE}") + echo "${output}" | jq . +else + curl -X POST \ + -H "Content-Type: application/json" \ + "https://generativelanguage.googleapis.com/v1beta/models/${MODEL_ID}:${GENERATE_CONTENT_API}?key=${GEMINI_API_KEY}" \ + -d "@${PAYLOAD_FILE}" +fi + +echo -e "\n----------------------------------------" diff --git a/scripts/start.js b/scripts/start.js new file mode 100644 index 0000000000000000000000000000000000000000..a95a9d399b05a7553cc28365d3e1e841c735a94c --- /dev/null +++ b/scripts/start.js @@ -0,0 +1,93 @@ +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law_or_agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import { spawn, execSync } from 'node:child_process'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { readFileSync } from 'node:fs'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const root = join(__dirname, '..'); +const pkg = JSON.parse(readFileSync(join(root, 'package.json'), 'utf-8')); + +// check build status, write warnings to file for app to display if needed +execSync('node ./scripts/check-build-status.js', { + stdio: 'inherit', + cwd: root, +}); + +const nodeArgs = ['--no-warnings=DEP0040']; +let sandboxCommand = undefined; +try { + sandboxCommand = execSync('node scripts/sandbox_command.js', { + cwd: root, + }) + .toString() + .trim(); +} catch { + // ignore +} +// if debugging is enabled and sandboxing is disabled, use --inspect-brk flag +// note with sandboxing this flag is passed to the binary inside the sandbox +// inside sandbox SANDBOX should be set and sandbox_command.js should fail +const isInDebugMode = process.env.DEBUG === '1' || process.env.DEBUG === 'true'; + +if (isInDebugMode && !sandboxCommand) { + if (process.env.SANDBOX) { + const port = process.env.DEBUG_PORT || '9229'; + nodeArgs.push(`--inspect-brk=0.0.0.0:${port}`); + } else { + nodeArgs.push('--inspect-brk'); + } +} + +nodeArgs.push(join(root, 'packages', 'cli')); +nodeArgs.push(...process.argv.slice(2)); + +const env = { + ...process.env, + CLI_VERSION: pkg.version, + DEV: 'true', +}; + +const keepCiEnv = + process.env.GEMINI_KEEP_CI_ENV === '1' || + process.env.GEMINI_KEEP_CI_ENV === 'true'; +if (!keepCiEnv) { + const ciKeys = ['CI', 'CONTINUOUS_INTEGRATION', 'GITHUB_ACTIONS'].filter( + (k) => k in env, + ); + if (ciKeys.length > 0) { + ciKeys.forEach((k) => delete env[k]); + process.stderr.write( + `[gemini] Removed CI env vars to keep interactive mode working in dev: ${ciKeys.join(', ')}. Set GEMINI_KEEP_CI_ENV=1 to disable.\n`, + ); + } +} + +if (isInDebugMode) { + // If this is not set, the debugger will pause on the outer process rather + // than the relaunched process making it harder to debug. + env.GEMINI_CLI_NO_RELAUNCH = 'true'; +} +const child = spawn('node', nodeArgs, { stdio: 'inherit', env }); + +child.on('close', (code) => { + process.exit(code); +}); diff --git a/scripts/telemetry.js b/scripts/telemetry.js new file mode 100644 index 0000000000000000000000000000000000000000..e7a96fef749cca0b024c3acd9325331f56c6f8f9 --- /dev/null +++ b/scripts/telemetry.js @@ -0,0 +1,91 @@ +#!/usr/bin/env node + +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; +import { existsSync, readFileSync } from 'node:fs'; +import { GEMINI_DIR } from '@google/gemini-cli-core'; + +const projectRoot = join(import.meta.dirname, '..'); + +const USER_SETTINGS_DIR = join( + process.env.HOME || process.env.USERPROFILE || process.env.HOMEPATH || '', + GEMINI_DIR, +); +const USER_SETTINGS_PATH = join(USER_SETTINGS_DIR, 'settings.json'); +const WORKSPACE_SETTINGS_PATH = join(projectRoot, GEMINI_DIR, 'settings.json'); + +let telemetrySettings = undefined; + +function loadSettings(filePath) { + try { + if (existsSync(filePath)) { + const content = readFileSync(filePath, 'utf-8'); + const jsonContent = content.replace(/\/\/[^\n]*/g, ''); + const settings = JSON.parse(jsonContent); + return settings.telemetry; + } + } catch (e) { + console.warn( + `⚠️ Warning: Could not parse settings file at ${filePath}: ${e.message}`, + ); + } + return undefined; +} + +telemetrySettings = loadSettings(WORKSPACE_SETTINGS_PATH); + +if (!telemetrySettings) { + telemetrySettings = loadSettings(USER_SETTINGS_PATH); +} + +let target = telemetrySettings?.target || 'local'; +const allowedTargets = ['local', 'gcp', 'genkit']; + +const targetArg = process.argv.find((arg) => arg.startsWith('--target=')); +if (targetArg) { + const potentialTarget = targetArg.split('=')[1]; + if (allowedTargets.includes(potentialTarget)) { + target = potentialTarget; + console.log(`βš™οΈ Using command-line target: ${target}`); + } else { + console.error( + `πŸ›‘ Error: Invalid target '${potentialTarget}'. Allowed targets are: ${allowedTargets.join( + ', ', + )}.`, + ); + process.exit(1); + } +} else if (telemetrySettings?.target) { + console.log( + `βš™οΈ Using telemetry target from settings.json: ${telemetrySettings.target}`, + ); +} + +const targetScripts = { + gcp: 'telemetry_gcp.js', + local: 'local_telemetry.js', + genkit: 'telemetry_genkit.js', +}; + +const scriptPath = join(projectRoot, 'scripts', targetScripts[target]); + +try { + console.log(`πŸš€ Running telemetry script for target: ${target}.`); + const env = { ...process.env }; + + execFileSync('node', [scriptPath], { + stdio: 'inherit', + cwd: projectRoot, + env, + }); +} catch (error) { + console.error(`πŸ›‘ Failed to run telemetry script for target: ${target}`); + console.error(error); + process.exit(1); +} diff --git a/scripts/telemetry_genkit.js b/scripts/telemetry_genkit.js new file mode 100644 index 0000000000000000000000000000000000000000..fc1d6331be040910618943ba2afbbe1db2e22449 --- /dev/null +++ b/scripts/telemetry_genkit.js @@ -0,0 +1,70 @@ +#!/usr/bin/env node + +/** + * @license + * Copyright 2025 Google LLC + * SPDX-License-Identifier: Apache-2.0 + */ + +import { createInterface } from 'node:readline'; +import { spawn } from 'node:child_process'; +import { manageTelemetrySettings, registerCleanup } from './telemetry_utils.js'; + +const GENKIT_START_COMMAND = 'npx'; +const GENKIT_START_ARGS = ['-y', 'genkit-cli', 'start', '--non-interactive']; + +async function main() { + let genkitProcess; + + const originalSandboxSetting = manageTelemetrySettings( + true, + '', // Endpoint will be set dynamically + 'local', + undefined, + 'http', + ); + + registerCleanup( + () => [genkitProcess], + () => [], + originalSandboxSetting, + ); + + console.log('πŸš€ Starting Genkit telemetry server...'); + genkitProcess = spawn(GENKIT_START_COMMAND, GENKIT_START_ARGS, { + stdio: ['ignore', 'pipe', 'pipe'], + }); + + const rl = createInterface({ input: genkitProcess.stdout }); + + rl.on('line', (line) => { + console.log(`[Genkit] ${line}`); + const match = line.match(/Telemetry API running on (http:\/\/[^\s]+)/); + if (match) { + const telemetryApiUrl = match[1]; + const otlpEndpoint = `${telemetryApiUrl}/api/otlp`; + console.log(`βœ… Genkit telemetry running on: ${otlpEndpoint}`); + manageTelemetrySettings(true, otlpEndpoint, 'local', undefined, 'http'); + } + }); + + genkitProcess.stderr.on('data', (data) => { + console.error(`[Genkit Error] ${data.toString()}`); + }); + + genkitProcess.on('close', (code) => { + console.log(`Genkit process exited with code ${code}`); + }); + + genkitProcess.on('error', (err) => { + console.error('Failed to start Genkit process:', err); + process.exit(1); + }); + + console.log(` +✨ Genkit telemetry environment is running. +`); + console.log(`Press Ctrl+C to exit.`); +} + +main();