LlamaIndex’s Post

Very useful when considering RAG evaluation.

LLMs evaluation will be crucial in LLMops

Explore This weekend, learn about 5 different ways of evaluating your RAG systems. Go through a comprehensive tour of the different RAG evaluation methods using LLM-as-a-judge (which have corresponding #llama_index implementations): 1. Answer Relevance 2. Context Relevance 3. Faithfulness 4. Correctness 5. Pairwise Comparison Along with synthetic dataset generation. Check it out here: https://lnkd.in/gB8akH6m

I love advanced RAG from a theoretical and well-funded research perspective. But what enterprises can afford a high accuracy LLM + completely maxed-out RAG context windows with repeated and ongoing all-you-can-eat prompting in a typical usage scenario by employees? As an AI consultant who has worked on numerous RAG implementations, I can tell you from discussions with Fortune 500 companies, they are finding few use cases worth the pay-per-token model so far, even with reduced token costs via the "Moore's Law" of model tokenomics. Much of this stems from poor implementations, poor understanding of model tokenomics, poor understanding of how LLM message arrays work (you have to send every previous message back to the LLM for every new prompt because it doesn't have memory), and CIOs being told to "just get something AI out there fast". Just because AI is shiny and new doesn't mean ROI should be tossed out... #AIBestPractices

https://lnkd.in/gW-synVq well articulated reasons to retrieve from knowledge graph instead of standard rag, retrieve a correct relationship rather than flat record(s)

Check out this great RAG repo!

Pretty excited about this new RAG technique 🧑🍳 A top issue with RAG chunking is it splits the document into fragmented pieces, causing top-k retrieval to return partial context. Also most documents have multiple hierarchies of sections: top-level sections, sub-sections, etc. This is also why lots of people are interested in exploring the idea of knowledge graphs - pulling in "links" to related pages to expand retrieved context. This notebook lets you retrieve contiguous chunks without having to spend a lot of time tuning the chunking algorithm, thanks to GraphRAG-esque metadata tagging + retrieval. Tag chunks with sections, and use the section ID to expand the retrieved set. #RAGTechnique #GraphRAG #KnowledgeGraphs #AIContextualUnderstanding #InformationRetrieval #NaturalLanguageProcessing #ChunkingOptimization #ArtificialIntelligenceInnovation #MachineLearningAdvancements #LanguageModelingSolutions https://lnkd.in/gqTnfKWG

G in RAG is the easiest. A powerful RAG system has a mature, robust, well thought and use case specific indexing and retrieval systems.

I just published Advanced RAG Retrieval Strategies: Flow and Modular

GraphRAG vs Baseline RAG 🔍 *Retrieval-Augmented Generation (RAG) is a technique to improve LLM outputs using real-world information. This technique is an important part of most LLM-based tools and the majority of RAG approaches use vector similarity as the search technique, which we call Baseline RAG. * GraphRAG uses knowledge graphs to provide substantial improvements in question-and-answer performance when reasoning about complex information. To know more - https://lnkd.in/gMZTgkvJ Get started with - https://lnkd.in/gWs2AHKW YouTube - https://lnkd.in/gX8y9_SN