PDF Document Ingestion Accelerator for GenAI Applications скачать в хорошем качестве

PDF Document Ingestion Accelerator for GenAI Applications

Databricks Financial Service customers in the GenAI space have a common use case of ingestion and processing of unstructured documents — PDF/images — then performing downstream GenAI tasks such as entity extraction and RAG based knowledge Q&A. The pain points for the customers for these types of use cases are: The quality of the PDF/image documents varies since many older physical documents were scanned into electronic form The complexity of the PDF/image documents varies and many contain tables — images with embedding information — which require slower Tesseract OCR They would like to streamline postprocess for downstream workloads In this talk we will present an optimized structured streaming workflow for complex PDF ingestion. The key techniques include Apache Spark™ optimization, multi-threading, PDF object extraction, skew handling and auto retry logics Talk By: Qian Yu, Specialist Solution Architect, Databricks Here’s more to explore: Production ready data pipelines for analytics and AI: https://www.databricks.com/solutions/... The Big Book of Data Engineering: https://www.databricks.com/resources/... See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dat... Connect with us: Website: https://databricks.com Twitter:   / databricks   LinkedIn:   / databricks   Instagram:   / databricksinc   Facebook:   / databricksinc