SlideShare a Scribd company logo

Using LLM Agents with Llama 3, LangGraph and Milvus

•
•
Zilliz
Zilliz

RAG systems are talked about in detail, but usually stick to the basics. In this talk, Stephen will show you how to build an Agentic RAG System using Langchain and Milvus.

1 of 19
Download to read offline
Stephen Batifol | Zilliz Zilliz Webinar, July 11 Using LLM Agents with Llama 3, LangGraph and Milvus
Stephen Batifol Developer Advocate, Zilliz/ Milvus stephen.batifol@zilliz.com linkedin.com/in/stephen-batifol/ @stephenbtl Speaker
27K+ GitHub Stars 25M+ Downloads 250+ Contributors 2,600 + Forks Milvus is an open-source vector database for GenAI projects. pip install on your laptop, plug into popular AI dev tools, and push to production with a single line of code. Easy Setup Pip-install to start coding in a notebook within seconds. Reusable Code Write once, and deploy with one line of code into the production environment Integration Plug into OpenAI, Langchain, LlmaIndex, and many more Feature-rich Dense & sparse embeddings, filtering, reranking and beyond
Seamless integration with all popular AI toolkits
| © Copyright 8/16/23 Zilliz 5 RAG (Retrieval Augmented Generation)
Basic Idea Use RAG to force the LLM to work with your data by injecting it via a vector database like Milvus
Basic RAG Architecture
5 lines starter
9 | © Copyright 8/16/23 Zilliz 9 | © Copyright 8/16/23 Zilliz 01 Tech Stack
● Framework for building LLM Applications ● Focus on retrieving data and integrating with LLMs ● Integrations with most AI popular tools 🦜🔗 LangChain
🦜🕸 LangGraph by LangChain ● Build Stateful apps with LLMs and Multi-Agents workflow ● Cycles and Branching ● Human-in-the-Loop ● Persistence
Ollama ● Run LLMs anywhere ● Run Embedding Models
Using LLM Agents with Llama 3, LangGraph and Milvus
14 | © Copyright 8/16/23 Zilliz 14 | © Copyright 8/16/23 Zilliz 02 Agentic RAG
Agentic RAG ✅ Multi-turn ✅ Query / task planning layer ✅ Tool interface for external environment ✅ Reflection ✅ Memory for personalization
● Routing: Adaptive RAG ○ Route Questions to different retrieval approaches ● Fallback: Corrective RAG ○ Fallback to web search if docs are not relevant to query ● Self-Correction: Self-RAG ○ Try to fix answers with hallucinations or don’t address question General Ideas
17 | © Copyright 8/16/23 Zilliz 17 | © Copyright 8/16/23 Zilliz 03 RAG in action with Milvus Lite
milvus.io github.com/milvus-io/ @milvusio @stephenbtl /in/stephen-batifol Thank you
Meta Storage Root Query Data Index Coordinator Service Proxy Proxy etcd Log Broker SDK Load Balancer DDL/DCL DML NOTIFICATION CONTROL SIGNAL Object Storage Minio / S3 / AzureBlob Log Snapshot Delta File Index File Worker Node QUERY DATA DATA Message Storage VECTOR DATABASE Access Layer Query Node Data Node Index Node Milvus Architecture

More Related Content

Using LLM Agents with Llama 3, LangGraph and Milvus

  • 1. Stephen Batifol | Zilliz Zilliz Webinar, July 11 Using LLM Agents with Llama 3, LangGraph and Milvus
  • 2. Stephen Batifol Developer Advocate, Zilliz/ Milvus stephen.batifol@zilliz.com linkedin.com/in/stephen-batifol/ @stephenbtl Speaker
  • 3. 27K+ GitHub Stars 25M+ Downloads 250+ Contributors 2,600 + Forks Milvus is an open-source vector database for GenAI projects. pip install on your laptop, plug into popular AI dev tools, and push to production with a single line of code. Easy Setup Pip-install to start coding in a notebook within seconds. Reusable Code Write once, and deploy with one line of code into the production environment Integration Plug into OpenAI, Langchain, LlmaIndex, and many more Feature-rich Dense & sparse embeddings, filtering, reranking and beyond
  • 4. Seamless integration with all popular AI toolkits
  • 5. | © Copyright 8/16/23 Zilliz 5 RAG (Retrieval Augmented Generation)
  • 6. Basic Idea Use RAG to force the LLM to work with your data by injecting it via a vector database like Milvus
  • 9. 9 | © Copyright 8/16/23 Zilliz 9 | © Copyright 8/16/23 Zilliz 01 Tech Stack
  • 10. ● Framework for building LLM Applications ● Focus on retrieving data and integrating with LLMs ● Integrations with most AI popular tools 🦜🔗 LangChain
  • 11. 🦜🕸 LangGraph by LangChain ● Build Stateful apps with LLMs and Multi-Agents workflow ● Cycles and Branching ● Human-in-the-Loop ● Persistence
  • 12. Ollama ● Run LLMs anywhere ● Run Embedding Models
  • 14. 14 | © Copyright 8/16/23 Zilliz 14 | © Copyright 8/16/23 Zilliz 02 Agentic RAG
  • 15. Agentic RAG ✅ Multi-turn ✅ Query / task planning layer ✅ Tool interface for external environment ✅ Reflection ✅ Memory for personalization
  • 16. ● Routing: Adaptive RAG ○ Route Questions to different retrieval approaches ● Fallback: Corrective RAG ○ Fallback to web search if docs are not relevant to query ● Self-Correction: Self-RAG ○ Try to fix answers with hallucinations or don’t address question General Ideas
  • 17. 17 | © Copyright 8/16/23 Zilliz 17 | © Copyright 8/16/23 Zilliz 03 RAG in action with Milvus Lite
  • 19. Meta Storage Root Query Data Index Coordinator Service Proxy Proxy etcd Log Broker SDK Load Balancer DDL/DCL DML NOTIFICATION CONTROL SIGNAL Object Storage Minio / S3 / AzureBlob Log Snapshot Delta File Index File Worker Node QUERY DATA DATA Message Storage VECTOR DATABASE Access Layer Query Node Data Node Index Node Milvus Architecture