Waqas Ahmed’s Post

View profile for Waqas Ahmed, graphic

Technical Lead at DPL Pvt. Ltd

Pretty excited about this new RAG technique 🧑🍳 A top issue with RAG chunking is it splits the document into fragmented pieces, causing top-k retrieval to return partial context. Also most documents have multiple hierarchies of sections: top-level sections, sub-sections, etc. This is also why lots of people are interested in exploring the idea of knowledge graphs - pulling in "links" to related pages to expand retrieved context. This notebook lets you retrieve contiguous chunks without having to spend a lot of time tuning the chunking algorithm, thanks to GraphRAG-esque metadata tagging + retrieval. Tag chunks with sections, and use the section ID to expand the retrieved set. #RAGTechnique #GraphRAG #KnowledgeGraphs #AIContextualUnderstanding #InformationRetrieval #NaturalLanguageProcessing #ChunkingOptimization #ArtificialIntelligenceInnovation #MachineLearningAdvancements #LanguageModelingSolutions https://lnkd.in/gqTnfKWG

Jerry Liu (@jerryjliu0) on X

Jerry Liu (@jerryjliu0) on X

x.com

To view or add a comment, sign in

Explore topics