Corrective Retrieval Augmented Generation: Improving LLM Robustness with CRAG скачать в хорошем качестве

Corrective Retrieval Augmented Generation: Improving LLM Robustness with CRAG

Paper: https://arxiv.org/pdf/2401.15884 In a nutshell: Methodology Snapshot: Corrective Retrieval Augmented Generation (CRAG) Problem - Unreliable Retrieval: Standard RAG blindly trusts retrieved context, causing hallucinations when retrieval fails or returns irrelevant noise. System Architecture: Introduces a lightweight Retrieval Evaluator (T5-Large, ~0.77B params) to assess document relevance before generation, acting as a quality gatekeeper. Inference Logic: Evaluator computes a confidence score for each retrieved document, triggering one of three discrete actions: Correct, Incorrect, or Ambiguous. Action - Correct (High Confidence): System trusts retrieved documents but applies Knowledge Refinement to strip irrelevant text; refined context is passed to the generator. Action - Incorrect (Low Confidence): System rejects all retrieved documents to prevent poisoning; triggers Web Search to fetch fresh external knowledge. Action - Ambiguous (Uncertainty): System combines both filtered internal results and new web-searched content to maximize coverage when the evaluator lacks high conviction. Knowledge Refinement (Decompose-then-Recompose): Heuristic segmentation splits documents into fine-grained "knowledge strips" (1-2 sentences); evaluator scores and filters each strip individually to remove noise. Web Search Mechanism: When triggered, a query rewriter (prompted LLM) converts the input into keyword-based search queries; public APIs (e.g., Google Search) retrieve URLs for content extraction. Training Signal: Evaluator is fine-tuned on datasets like PopQA; positive labels derived from golden Wikipedia paragraphs, negative labels from random hard-negative retrieval results. Failure Avoidance: "Incorrect" action prevents "hallucination by ingestion" by completely discarding sub-optimal static retrieval in favor of dynamic web sources. Efficiency: Designed as a plug-and-play module; the evaluator is significantly smaller than comparable "critic" models (e.g., Self-RAG's 7B model), minimizing inference latency. Integration: Agnostic to the underlying Generator (LLM) or Retriever; operates as a post-retrieval / pre-generation correction layer. Disclaimer: This is an AI-powered production. The scripts, insights, and voices featured in this podcast are generated entirely by Artificial Intelligence models. While we strive for technical accuracy by grounding our episodes in original research papers, listeners are encouraged to consult the primary sources for critical applications.