Partner Highlight: Build production-ready generative AI applications for enterprise search using Haystack pipelines and Amazon SageMaker JumpStart with LLMs

Fri, Aug 25, 2023 · David Tippett,

Haystack, Sagemaker, and OpenSearch Architecture for Retreval Augmented Generation

Ever wanted to get started with generative AI but didn’t know where to start? Check out the AWS Machine Learning blog post “Build production-ready generative AI applications for enterprise search using Haystack pipelines and Amazon SageMaker JumpStart with LLMs”. In this article we discuss how retrieval augmented generation (RAG) works and how to set up a RAG pipeline with Haystack, OpenSearch, and SageMaker JumpStart.

The article first provides a bit of background on RAG and how these pipelines typically work. Next, it walks through deploying OpenSearch and models on Amazon SageMaker JumpStart. Then it walks through ingesting documents into OpenSearch using the Haystack Python library. Finally, it shows how to use the data in a RAG pipeline. Check out the full blog post, which includes a GitHub repo with code so you can follow along yourself.