| <!-- ======================================================== --> |
| ## Table Of Contents |
| <!-- ======================================================== --> |
|
|
| 1. [References](#1-references) |
| 2. [Project Reference](#2-project-reference) |
| 1. [Project Paths](#project-paths) |
| 2. [Command Reference](#command-reference) |
| 3. [Dependency Surfaces](#dependency-surfaces) |
| 4. [Environment Variables](#environment-variables) |
| 5. [Configuration Files](#configuration-files) |
| 6. [Public Interfaces](#public-interfaces) |
| 7. [Figure Visual Tokens](#figure-visual-tokens) |
|
|
| <br> |
|
|
| # 1. References |
|
|
| Use this file when you need an exact path, command, environment variable, or |
| public name. Use [How-To User Guides](How-To-User-Guides.md) for procedure and |
| [Explanations](Explanations.md) for concepts. |
|
|
| <br> |
|
|
| # 2. Project Reference |
|
|
| <!-- ======================================================== --> |
| ## Project Paths |
| <!-- ======================================================== --> |
|
|
| Core paths: |
|
|
| | Path | Role | |
| |---|---| |
| | [README.md](../README.md) | Short setup entry point. | |
| | [CHANGELOG.md](../CHANGELOG.md) | Release history and unreleased change notes. | |
| | [TODO.md](../TODO.md) | Tracked parking lot for actionable follow-up work. | |
| | [AGENTS.md](../AGENTS.md) | Local coding and documentation guidance for agents. | |
| | [pyproject.toml](../pyproject.toml) | Package metadata, dependencies, and tool configuration. | |
| | [justfile](../justfile) | Development commands. | |
| | [src/kscrape](../src/kscrape) | Runtime package source. | |
| | [src/kscrape_dev](../src/kscrape_dev) | Maintainer-only release checks. | |
| | [tests](../tests) | Test suite. | |
| | [docs](.) | Long-form project documentation. | |
|
|
| > [!NOTE] |
| > Related: use [Development: repository routing](Development.md#repository-routing) |
| > before adding files or moving behavior between directories. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Command Reference |
| <!-- ======================================================== --> |
|
|
| Common commands: |
|
|
| | Command | Role | |
| |---|---| |
| | `just --list` | Show available development recipes. | |
| | `just sync` | Sync the full maintainer environment from `uv.lock`. | |
| | `python -m pip install -e "."` | Install runtime package dependencies without `uv`. | |
| | `python -m pip install -e "." --group dev` | Install maintainer tools without `uv`. | |
| | `just release-check` | Run the complete local release rehearsal without changing version or publishing. | |
| | `just release LEVEL` | Prepare and atomically push `main` plus the new release tag. | |
| | `just release-prepare LEVEL` | Create the checked version commit and annotated tag without pushing. | |
| | `just release-push TAG` | Atomically push an existing prepared release commit and tag. | |
| | `kscrape --help` | Show the console command tree. | |
| | `kscrape version` | Print the installed package version. | |
| | `kscrape config --help` | Show AppRC's process-environment configuration commands. | |
| | `kscrape config show --json` | Show redacted process configuration metadata. | |
| | `kscrape source list` | List built-in source adapters. | |
| | `kscrape posts search SOURCE TAG...` | Print post records. | |
| | `kscrape posts export SOURCE TAG...` | Export bounded post metadata as JSON or JSONL without downloading files. | |
| | `kscrape posts tag-summary SOURCE TAG...` | Count categorized tags across a bounded post query. | |
| | `kscrape posts download SOURCE TAG... --output DIRECTORY` | Download matching raster files to `DIRECTORY/posts/SOURCE/images/` and append to its `manifest.jsonl`. | |
| | `kscrape sets download SOURCE SET --output DIRECTORY` | Download set members to `DIRECTORY/sets/SOURCE/SET/images/` with set-local tags, a replaced manifest, and a cumulative `image-index.json`. `--prune` removes recorded former images after a complete successful run; it cannot be used with `--limit`. | |
| | `kscrape tags search SOURCE PATTERN` | Search source tags. | |
| | `python -m kscrape --help` | Smoke-test the module entry point. | |
| | `ruff format .` | Format Python files. | |
| | `ruff check .` | Lint Python files. | |
| | `pyright` | Type-check runtime source. | |
| | `python -m pytest` | Run the test suite. | |
|
|
| > [!NOTE] |
| > Related: use [How-To User Guides: run tests](How-To-User-Guides.md#run-tests) |
| > for the command sequence. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Dependency Surfaces |
| <!-- ======================================================== --> |
|
|
| Dependency locations: |
|
|
| | Surface | File Section | Use | |
| |---|---|---| |
| | Runtime dependencies | `[project].dependencies` | Packages required by normal users. | |
| | Optional feature extras | `[project.optional-dependencies]` | Published extras for optional runtime features when the package adds them. | |
| | Dependency groups | `[dependency-groups]` | Local maintainer tools such as tests, linting, typing, docs, and profiling. | |
| | Lock file | `uv.lock` | Reproducible `uv` installs. | |
| | Exported lock | `pylock.toml` | Lock representation for compatible tooling. | |
|
|
| > [!NOTE] |
| > Related: use [dependency model](Explanations.md#dependency-model) for why |
| > optional runtime features and maintainer-only tools stay separate. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Environment Variables |
| <!-- ======================================================== --> |
|
|
| Common environment variables: |
|
|
| | Name | Role | |
| |---|---| |
| | `KSCRAPE_USER_AGENT` | Required descriptive User-Agent for source requests. | |
| | `KSCRAPE_E621_USERNAME` | Optional e621 HTTP Basic username. | |
| | `KSCRAPE_E621_API_KEY` | Optional e621 HTTP Basic API key. | |
| | `KSCRAPE_E6AI_USERNAME` | Optional e6AI HTTP Basic username. | |
| | `KSCRAPE_E6AI_API_KEY` | Optional e6AI HTTP Basic API key. | |
| | `VIRTUAL_ENV` | Active virtual environment path. | |
| | `PYTHONPATH` | Import-path override for local smoke tests. Prefer editable installs for normal development. | |
| | `UV_PROJECT_ENVIRONMENT` | Optional `uv` virtual environment path override. | |
|
|
| Set a source username and API key together. `kscrape` reads these values only |
| from the process. It has no storage selector, user dotenv, persistent |
| credential file, or configuration setup command. |
|
|
| > [!NOTE] |
| > Related: use [How-To User Guides: environment problems](How-To-User-Guides.md#environment-problems) |
| > for the first checks when imports resolve from the wrong location. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Configuration Files |
| <!-- ======================================================== --> |
|
|
| Important config files: |
|
|
| | File | Role | |
| |---|---| |
| | [pyproject.toml](../pyproject.toml) | Python packaging, dependencies, and tool settings. | |
| | [src/kscrape/config/app.py](../src/kscrape/config/app.py) | AppRC declaration. | |
| | [src/kscrape/config/sections/network.py](../src/kscrape/config/sections/network.py) | Typed process-environment settings. | |
| | [.envrc](../.envrc) | `direnv` integration. | |
| | [.gitignore](../.gitignore) | Local and generated files excluded from Git. | |
| | [.github/workflows_inactive](../.github/workflows_inactive) | Inactive starter CI workflows. | |
| | [pylock.toml](../pylock.toml) | Exported dependency lock. | |
|
|
| `kscrape` does not use `config/.env.shared`, an AppRC storage registry, a user |
| dotenv, or a persistent credential file. |
|
|
| > [!NOTE] |
| > Related: use [configuration model](Explanations.md#configuration-model) for |
| > why the process-only configuration remains intentionally small. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Public Interfaces |
| <!-- ======================================================== --> |
|
|
| Document public surfaces here as the project grows: |
|
|
| | Surface | Current Name | Stability | |
| |---|---|---| |
| | Package import | `kscrape` | Public once README examples use it. | |
| | Console script | `kscrape` | Public command declared in `pyproject.toml`. | |
| | Module entrypoint | `python -m kscrape` | Public module execution path. | |
| | CLI app owner | `kscrape.cli.app` | Command tree implementation owner. | |
| | Entrypoint wrapper | `kscrape.main` | Thin wrapper for package metadata entry points. | |
| | Source registry | `kscrape.sources.SourceRegistry` | Built-in source adapter lookup. | |
| | Config sections | `kscrape.config.sections` | App-owned AppRC `rc.Config` field declarations. | |
| | Config bundle | `kscrape.config.KscrapeConfig` | Typed object that assembles config sections. | |
| | Config catalog | `kscrape.config.catalog` | Section metadata and CLI registration import. | |
| | Config facade | `kscrape.config.APP_RC` | AppRC facade used by CLI bootstrap and config commands. | |
| | Export formats | `kscrape.exports.ExportFormat` | JSON and JSONL serialization selection. | |
| | Tag summary row | `kscrape.exports.TagSummary` | Category-aware tag frequency record. | |
|
|
| > [!NOTE] |
| > Related: use [How-To User Guides: run the first command](How-To-User-Guides.md#run-the-first-command) |
| > for the first user-facing smoke test. |
|
|
| <br> |
|
|
| <!-- ======================================================== --> |
| ## Figure Visual Tokens |
| <!-- ======================================================== --> |
|
|
| Keep figure captions and generated asset names stable here. Put repeatable |
| figure-builder source code next to its renderer instead of duplicating visual |
| tokens across documentation files. |
|
|
| > [!NOTE] |
| > Related links: |
| > - Use [static figure rules](Development.md#static-figure-rules) before adding docs figures. |
| > - Use [documentation authoring](Development.md#documentation-authoring) before changing docs structure or figure assets. |
|
|
