Chunking Strategies
Bad chunks hand the retriever bad evidence, and the model answers from it anyway.
What it is
Chunking is the process of breaking a large document into smaller, manageable pieces before embedding and storing them in a vector database. Because LLMs have context window limits and embedding models have maximum token constraints, you cannot process a 1,000-page manual all at once.
A chunking strategy determines how to break the text. You can split by character count, by word, by paragraph, or by semantic structure (like Markdown headers or code functions).
Why it matters in production
If you blindly chop text every 500 characters, you will split sentences in half and sever code blocks from their declarations. When the retriever fetches that chunk later, the LLM will lack the necessary context to answer the user's question, leading to hallucinations.
In production, bad chunking is a common cause of RAG failure. A FixedSizeChunkStrategy that respects the unit of meaning keeps the text's sense intact, which is what makes vector search accurate.
How Agenthood implements it
Agenthood plans to implement chunking via the ChunkStrategy interface, specifically utilizing a FixedSizeChunkStrategy for code and markdown parsing.
This is implemented in src/rag/ChunkStrategy.ts (shipped in Phase 1):
Code cannot be split at an arbitrary offset, because a function has to stay whole.
Hands-on example
Though the formal pipeline is under development, you can test semantic chunking concepts using standard text processing:
Or conceptually in TypeScript:
Further reading
- ADR-010: LanceDB for Vector Storage
src/rag/ChunkStrategy.ts: source implementation (shipped in Phase 1)- Pinecone: Chunking Strategies: an excellent breakdown of chunking methods