Yuan XU’s Post

Combine Text Embeddings and Knowledge (Graph) Embeddings in RAG systems https://lnkd.in/dYXSY5EN

The convergence of graphs, networks, AI...how cool is it? Thanks Sunila Gollapudi for writing this nice article!!

Pretty excited about this new RAG technique 🧑🍳 A top issue with RAG chunking is it splits the document into fragmented pieces, causing top-k retrieval to return partial context. Also most documents have multiple hierarchies of sections: top-level sections, sub-sections, etc. This is also why lots of people are interested in exploring the idea of knowledge graphs - pulling in "links" to related pages to expand retrieved context. This notebook lets you retrieve contiguous chunks without having to spend a lot of time tuning the chunking algorithm, thanks to GraphRAG-esque metadata tagging + retrieval. Tag chunks with sections, and use the section ID to expand the retrieved set. #RAGTechnique #GraphRAG #KnowledgeGraphs #AIContextualUnderstanding #InformationRetrieval #NaturalLanguageProcessing #ChunkingOptimization #ArtificialIntelligenceInnovation #MachineLearningAdvancements #LanguageModelingSolutions https://lnkd.in/gqTnfKWG

In this blog, Zain Hasan breaks down RAG into indexing, retrieval, and generation components and proposes 2 to 3 practical steps to improve each part of your RAG pipeline. Covering everything from chunking techniques, filtered search, and hybrid search to reranking, fine-tuning embedding models, and generating metadata for your text chunks! https://lnkd.in/dqfKWfiu

Here is a detailed article on how I evaluated and combined the text embeddings and knowledge (graph) embeddings and leveraged in RAGs. It has 4 parts, Part 1: What are Text embeddings (TE) & how are they stored and used in the RAG implementation? Part 2: What are Knowledge (Graph) embeddings (KGE) & How are they stored? Part 3: How are Knowledge (Graph) Embeddings different from Text Embeddings, and analyze if they are complementary in the context of usage in RAG ? Conclusion: Benefits of combining embeddings and overall summary https://lnkd.in/ggh9QUer

We use AI today as if it can solve any type of problem "out of the box". When it comes to internal data it is becoming far challenging to understand how to use AI and which technologies that can we leverage with our internal enterprise data to get much more out of it in a seamless way . See the following to get some interesting ideas

There’s thousands of RAG techniques and tutorials, but which ones perform the best? ARAGOG by Matouš Eibich is one of the most comprehensive evaluation surveys on advanced RAG techniques, testing everything from “classic vector database” to reranking (Cohere, LLM) to MMR to LlamaIndex native advanced techniques (sentence window retrieval, document summary index). The findings 💡: ✅ HyDE and LLM reranking enhance retrieval precision  ⚠️ MMR and multi-query techniques didn’t seem to be as effective ✅ Sentence window retrieval, Auto-merging retrieval, and the document summary index (all native LlamaIndex techniques) offer promising benefits in either retrieval precision and answer similarity! (And also interesting tradeoffs). It’s definitely worth giving the full paper a skim. Check it out: https://lnkd.in/genni8g2

A comprehensive study of RAG techniques adds so much value to the Generative AI solutions ecosystem ....

Graph RAG Works Better Than Standard RAG #GraphRAG leverages structural information across entities to enable more precise and comprehensive retrieval, capturing relational knowledge and facilitating more accurate, context-aware responses. This improves the accuracy of standard RAG systems.

Well, it's true that people get some better results by applying these techniques, but for each use case, it is still important to have enough tests to find the best way for his/her best approach. The application and effect of new technologies is not really linear. Practice is the best way to prove. Thank you for sharing the paper!