Spaces:
Runtime error
Runtime error
Download src/chunking.py from ResumeScreening/resume-screening-rag: direct link, hf CLI and curl.
- Browser
- Download file 523 Bytes
-
https://huggingface.co/spaces/ResumeScreening/resume-screening-rag/resolve/main/src/chunking.py
- Command line
-
hf download hf://spaces/ResumeScreening/resume-screening-rag/src/chunking.py
-
curl -L -o chunking.py https://huggingface.co/spaces/ResumeScreening/resume-screening-rag/resolve/main/src/chunking.py
523 Bytes
| from __future__ import annotations | |
| def chunk_text(text: str, chunk_size: int = 1200, overlap: int = 200) -> list[str]: | |
| if not text: | |
| return [] | |
| text = text.strip() | |
| if chunk_size <= 0: | |
| return [text] | |
| chunks = [] | |
| start = 0 | |
| n = len(text) | |
| while start < n: | |
| end = min(start + chunk_size, n) | |
| chunk = text[start:end].strip() | |
| if chunk: | |
| chunks.append(chunk) | |
| if end == n: | |
| break | |
| start = max(0, end - overlap) | |
| return chunks |