## Table Of Contents 1. [How-To User Guides](#1-how-to-user-guides) 1. [Recipe Map](#recipe-map) 2. [First-Time Setup](#2-first-time-setup) 1. [Install The Package](#install-the-package) 2. [Run The First Command](#run-the-first-command) 3. [Set A User-Agent](#set-a-user-agent) 3. [Common Workflows](#3-common-workflows) 1. [Sync Dependencies](#sync-dependencies) 2. [Search Posts And Tags](#search-posts-and-tags) 3. [Export Post Metadata](#export-post-metadata) 4. [Summarize Tags Across Posts](#summarize-tags-across-posts) 5. [Download Images](#download-images) 6. [Download A Set](#download-a-set) 7. [Run Tests](#run-tests) 8. [Inspect The Project](#inspect-the-project) 9. [Prepare A Release](#prepare-a-release) 4. [Troubleshooting](#4-troubleshooting) 1. [Environment Problems](#environment-problems) 2. [Command Problems](#command-problems)
# 1. How-To User Guides ## Recipe Map Use this file when you want commands in order. Use [References](References.md) when you need exact names and [Explanations](Explanations.md) when you need the system model. > [!NOTE] > Related: use [documentation standards](Development.md#4-documentation-standards) > when adding new recipes so headings, callouts, and links stay consistent.
# 2. First-Time Setup ## Install The Package Use this recipe from the project root. 1. Create or activate a Python environment. 2. Install the package for runtime use: ```bash python -m pip install -e "." ``` 3. Install the maintainer tools when you plan to edit the project: ```bash python -m pip install -e "." --group dev ``` 4. If you use `uv`, sync the locked maintainer environment: ```bash just sync ``` > [!NOTE] > Related: use [dependency surfaces](References.md#dependency-surfaces) for the > difference between runtime dependencies, optional extras, and dependency > groups.
## Run The First Command Show the command tree through the console script: ```bash kscrape --help ``` The module entry point should show the same command tree: ```bash python -m kscrape --help ``` Run commands that do not need network access: ```bash kscrape version kscrape source list kscrape config show --json ``` `kscrape` has no first-run storage setup. Network commands read their configuration only from the process that starts them. > [!NOTE] > Related: use [public interfaces](References.md#public-interfaces) for the > commands and import paths users can rely on.
## Set A User-Agent Every network command needs a descriptive User-Agent. Pass it for one command: ```bash kscrape posts search e6ai rating:s canine --user-agent "example-client/1.0" ``` Or export it for the current process: ```bash export KSCRAPE_USER_AGENT="example-client/1.0" ``` Use optional authentication through process environment values. Never put an API key on a command line or in a request URL.
# 3. Common Workflows ## Sync Dependencies Use `just sync` to install the full maintainer environment from `uv.lock`: ```bash just sync ``` Use plain `pip` when you only need the package and do not want `uv`: ```bash python -m pip install -e "." ``` > [!NOTE] > Related: use [configuration and dependency model](Explanations.md#configuration-and-dependency-model) > for why runtime installs and maintainer installs are documented separately.
## Search Posts And Tags Pass a source ID followed by source query terms: ```bash kscrape posts search e621 rating:s canine --limit 10 --user-agent "example-client/1.0" kscrape tags search e6ai "canine*" --user-agent "example-client/1.0" ``` Add `--json` to either command when another program needs machine-readable post or tag records.
## Export Post Metadata Use `posts export` when a query's post records, rather than its image files, are the result you need. This writes complete metadata by default: ```bash kscrape posts export e621 solo --output solo-posts.json --user-agent "example-client/1.0" ``` Select only post IDs and their categorized tag lists with `--fields`: ```bash kscrape posts export e621 solo --fields post_id,tags --output solo-tags.json --user-agent "example-client/1.0" ``` JSON files contain a `query` object with source, tag query, requested maximum, and returned count, plus a `posts` array. Use `--format jsonl` for one selected post object per line, either on stdout or in an output file. Exports retrieve at most 320 posts by default. Increase the explicit bound only when needed: ```bash kscrape posts export e621 solo --max-posts 640 --format jsonl --output solo-tags.jsonl --user-agent "example-client/1.0" ```
## Summarize Tags Across Posts Use `posts tag-summary` to count tags across matching posts without merging their source-defined categories: ```bash kscrape posts tag-summary e6ai solo --max-posts 640 --user-agent "example-client/1.0" ``` The default is a terminal table sorted by count. Use `--format json` or `--format jsonl` for machine output. An output file requires one of those formats: ```bash kscrape posts tag-summary e6ai solo --format json --output solo-tag-counts.json --user-agent "example-client/1.0" ```
## Download Images Choose an explicit collection root. Images are saved under `downloads/posts/e6ai/images/`, and the append-only manifest at `downloads/posts/e6ai/manifest.jsonl` records paths relative to that folder: ```bash kscrape posts download e6ai rating:s canine --limit 20 --output downloads --user-agent "example-client/1.0" ``` Original files are the default. Use `--asset sample` or `--asset preview` when a smaller derivative is sufficient.
## Download A Set Use the set short name shown by e6AI or e621. The command reads the set's member IDs, so it can retrieve members omitted by a `set:` post search: ```bash kscrape sets download e6ai hyperrealism --output downloads --user-agent "example-client/1.0" ``` The command fetches every member by default. `--limit 10` selects the first ten IDs in the set's order. A numeric set ID also works. Original images go to `downloads/sets/e6ai/hyperrealism/images/`; `tags.json`, `manifest.jsonl`, and `image-index.json` sit beside that `images/` directory. The set manifest is replaced after each completed run and contains one record per selected member. An unavailable member remains in the manifest and in the tag file's `unavailable_post_ids` list. Rerun the command to retry it. Files from former members remain by default. After downloading the complete set, pass `--prune` to remove only former images recorded by that set's image index. The index keeps ownership across ordinary reruns, so pruning can also happen later. `--prune` cannot be combined with `--limit`. > [!WARNING] > > The supported sources may contain mature material. Follow their rules, > choose queries carefully, and store downloads responsibly.
## Run Tests Run focused tests first: ```bash python -m pytest tests ``` Run the quality tools before finishing a code change: ```bash ruff format . ruff check . pyright python -m pytest ``` > [!NOTE] > Related: use [Development: verification](Development.md#verification) for > the maintainer checklist before a commit.
## Inspect The Project Use these commands when you need to understand the current shape: ```bash rg --files git status --short kscrape --help kscrape config show --json python -m kscrape --help ``` When a command fails, copy the exact command, current directory, exit code, and stderr into the issue or debugging note. > [!NOTE] > Related links: > - Use [project paths](References.md#project-paths) for the main source and docs locations. > - Use [system model](Explanations.md#system-model) for the package and tooling boundaries.
## Prepare A Release Follow [Development: releases](Development.md#releases) before running these commands: ```bash just release-check just release-prepare patch just release-push vMAJOR.MINOR.PATCH ``` The final command atomically pushes the branch and tag to the configured Hugging Face remote. It does not create a GitHub Release or publish a PyPI package.
# 4. Troubleshooting ## Environment Problems Check the active Python and environment first: ```bash python --version python -c "import sys; print(sys.executable)" python -c "import kscrape; print(kscrape.__file__)" ``` If the package imports from an unexpected location, reinstall it from the project root. > [!NOTE] > Related: use [failure model](Explanations.md#failure-model) for the normal > order of checks when a command behaves differently across machines.
## Command Problems When a `just` recipe fails: 1. Run `just --list`. 2. Run the underlying command manually. 3. Check whether the virtual environment is active. 4. Check whether the command exists in `.venv/bin`. ```bash just --list ls .venv/bin ``` For a network command, confirm that `KSCRAPE_USER_AGENT` is set or pass `--user-agent`. Check that optional credentials are supplied as complete username and API-key pairs. > [!NOTE] > Related: use [command reference](References.md#command-reference) for the > expected commands and their owners.