Bo Long

Could RAFT (Retrieval-Augmented Fine-Tuning) be the next evolution of RAG architecture? I am following this with great interest. https://lnkd.in/eHDx3rHz

13

This week, I spoke with Philip Rathle from Neo4j. He just published 💠 The GraphRAG Manifesto: Adding Knowledge to GenAI 💠 where he explains how to use graphs to fine-tune your LLMs and lists the benefits from GraphRAG relative to vector-only RAG: 1. Higher accuracy and more complete answers (runtime / production benefit) 2. Once you’ve created your knowledge graph, then it’s easier to both build5 and subsequently maintain your RAG application (development time benefit) 3. Better explainability, traceability6, and access controls (governance benefit) If you seriously plan to use your data with LLMs, give GraphRAG a try. https://lnkd.in/gdiCq-WU #GartnerDA #GenAI #RAG #GraphRAG

61

4 Comments

Forbes just posted my new piece about the symbiotic relationship between AI and NLP. In a nutshell, the piece explains why the consistency and control of NLP provides a great foundation to build cutting edge AI powered applications. NLP may not be the sexy new toy, but it's the cornerstone to successfully working with unstructured content and getting all you can out of LLM's and SLM's. https://lnkd.in/et8hd5jz

23

1 Comment

🚀 New Insights on Mitigating Hallucinations in Large Language Models! 🧠✨ Are you working with LLMs and facing challenges with hallucination? We've just published a comprehensive article that delves into advanced strategies for tackling this issue! 🔍 Triangulating the Truth: 🔧Supervised Fine-Tuning (SFT): Best for familiar, well-defined tasks. 📲In-Context Learning (ICL): Excels in diverse and complex queries with large context windows. 🎣Retrieval Augmentation (RA): Ideal for ensuring factual accuracy in unfamiliar topics by accessing external databases. 📊 Decision Matrix: We provide a detailed matrix to help you choose the right strategy based on features like task complexity, resource availability, and response accuracy. Key Highlights: ✴️Many-shot In-Context Learning with Large Context Windows can outperform SFT in certain scenarios, making it a game-changer for handling dynamic and unfamiliar queries. ✴️Retrieval Augmentation ensures factual correctness, making it essential for high-stakes applications. If you're looking to enhance the reliability and accuracy of your LLMs, check this out. 📖 Read the full article below. #MachineLearning #AI Google Cloud #TechInnovation #AIResearch #LLM #InContextLearning #FineTuning #RetrievalAugmentation #TechInsights #DataScience

73

6 Comments

In this Technical Deep Dive, we share our approaches in using Large Language Models (#LLMs) to make numerical-type data(sets) like DHS, JMP, HDX more accessible with the example of WASH data. In this 15 min. video Olivier Mills and team cover: 1️⃣ Basic scripting to extract core data from datasets from API endpoints 2️⃣ How to use LLMs to generate LLM-friendly schema definitions 3️⃣ How to structure a AI-friendly database store that can scale to thousands of datasets and millions of data indicators 4️⃣ How to use vector querying to retrieve relevant indicators from the store based off of a natural language (english) query 5️⃣ How to build a prompt that generates valid SQL query using LLMs with the above retrieved data schema. The various components form a single AI system combining data ingestion and data exploration using a software layer, a data layer and multiple LLM layers. ⏭️ Next steps will be to expand the datasets, automate other data source ingestion and further test the system with more complex plain english queries. We plan to incorporate this as a new feature for WASH AI after testing. Feel free to reach out if you would like a copy of the Notebooks, sample prompts and data schema definitions. We are #opensource The initial work was done in collaboration with Data for Good YYC (see https://lnkd.in/g3AijS93)

3

1 Comment

(MatMul) typically dominates the overall computational cost of large language models (In general, AB ≠ BA), fun huh? As you can imagine, this cost only grows as LLMs scale to larger embedding dimensions and context lengths. In this work, they show that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales. Removing MatMul from the calculations in large language models means these models don't need powerful computers to run. This change allows them to work on simpler devices, like smaller servers or even some personal computers, making advanced AI tools available to more people and places. Hardware Optimization: Custom accelerators demonstrate the practicality of this method by processing billion-parameter models with just 13 watts of power. #whynividia

6

Just having data is not enough in building a AI model, you have to make lot of modifications to the data and change its structure. In this article I discuss about some most fundamental ways of changing data shape which become very necessary when working with tensors. More on this comming soon. #DeepLearning #ML #MachineLearning #Pytorch

3

RAG fine-tuning combines code retrieval & model training, addressing outdated knowledge & hallucinations in LLMs. Our experiments show RAG fine-tuned models using the Together AI Fine-tuning API and Morph Labs Code API achieve up to 16% better accuracy, 3.7x faster speed & 150x cost reduction vs Claude 3 Opus, and up to 19% quality improvement, 1.1x faster speed & 37.5x cost reduction vs GPT-4o. #RAG #Finetuning #TogetherAI #MorphLabs

18

An Open-Source Language Model for Evaluating LLMs Researchers have introduced Prometheus 2, a powerful open-source language model specifically designed to evaluate the quality of responses from other language models. This addresses the limitations of proprietary evaluator LMs like GPT-4, offering a transparent, controllable, and affordable alternative. Prometheus 2 surpasses existing open evaluator LMs by: - Achieving high correlation with human judgments and proprietary LMs: Scores closely mirror human evaluations and those of models like GPT-4 and Claude. - Handling both direct assessment and pairwise ranking: Offers flexibility for different evaluation schemes. - Evaluating based on custom criteria: Goes beyond general qualities like helpfulness and harmlessness, allowing for fine-grained assessment tailored to specific needs. Key advancements in Prometheus 2: - Preference Collection: A new dataset with pairwise ranking feedback and 1,000 custom evaluation criteria. - Weight merging: Combines models trained on direct assessment and pairwise ranking data, resulting in a unified evaluator LM that excels in both formats. Prometheus 2 represents a significant step towards a more reliable and accessible evaluation of LLMs, paving the way for further advancements in the field. Read the full paper: https://lnkd.in/dU7sit-Y #Prometheus2 #LLMs #Evaluation #OpenSource #AI ✔️ Click "Follow" on the Cohorte page for daily AI engineering news. Credits: Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

8

1 Comment

Some unsettling news in the GenAI world, involving Google. Despite it's potential, Gogogle's facing significant challenges that could erode user trust. Google's AI models, ... PaLM2 and MUM, are designed to deliver high-quality search results, but issues with accuracy and reliability, particularly in sensitive areas like health and finance, are prevalent. Google's cautious approach in these domains highlights it's focus on safety and information quality. However, the struggle to maintain user trust amid these AI limitations could impact Google's reputation in the long term​ https://lnkd.in/gBX3fdqA

17

Congrats VERSES (OTCQB:VRSSF) Gabriel René and Dan Mapes and Dr. Karl Friston for introducing an "efficient alternative to deep learning, reinforcement learning and generative AI called Renormalizing Generative Models (RGMs) that address foundational problems in artificial intelligence (AI), namely versatility, efficiency, explainability and accuracy, using a physics based approach." #ai #activeinference #RGM #spatialweb #AGI #cognitivecomputing https://lnkd.in/gcYSM4eN

22

What people leaders can learn from Large Language Models (LLMs) Because LLMs are trained from human data, it's unsurprising that their behavior mirrors that of their human counterparts. It's worth taking a closer look at human teams through the lens of what we've learned about LLM performance. While it wasn't the authors' goal, this paper highlights many LLM issues and recommendations that parallel what is needed for human business teams. Here are the key takeaways from section 5. - Assigning tasks to individuals based on their specialties improves performance. - A clearly identified leader and a comprehensive feedback system will keep the team on track. - Clear role definition ensures that individuals stay focused and reduces mistakes. - Planning and task decomposition improve decision-making and execution. As a final note, the paper mentions the use of a rotating leader to improve the overall performance of the multi-agent response. Applied to human teams, this is a novel approach with great potential. Leaders who stay connected to technological developments would inject that fresh knowledge into their teams.  What team wouldn't benefit from periodically cycling through people leaders who have kept their skills sharp and relevant? https://lnkd.in/gXUp88th

21

A very insightful look into the capabilities and limitations of AI Agents. As we continue to explore how GenAI (and AI in general) can solve industry problems, agents are playing a big role.

8

1 Comment

Interesting early approach by Anthropic for reducing transformers models complexity, helping with interpretability and (consequently) reducing hallucinations and errors #GenerativeAI #largelanguagemodels #artificialintelligence #airevolution

53

1 Comment

If you are interested in seeing an approachable summary of some of the cutting edge papers from NeurIPS conference, check out my blog post. #neurips_conference #LLMs #paper_summary

18

2 Comments

The Rise of Agentic-Architectures in Generative AI: Old wine in a new bottle?      Tldr; As generative AI evolves, developers are increasingly recognizing the need for sophisticated orchestration of Large Language Models, Retrieval Augmented Generation, and multi-agent architectures to handle complex queries, prompting us to better leverage established multi-agent systems and AI planning techniques in the field of Computer Science to build more capable and efficient agents.     ——— As developers gain a deeper understanding of the challenges in building and deploying generative AI applications such as chatbots and copilots with Large Language Models (LLMs), Retrieval Augmented Generation (RAGs), and agents, they are recognizing the limitations of solely relying on prompt engineering and basic agent architectures. To address complex, real-world challenges, sophisticated orchestration of prompts, LLMs, and specific task execution tools is required.      For example, a generative AI solution with simple and single agent architecture even with a powerful LLM is unlikely to address complex multi-part queries such as ‘compare the revenue of company XYZ from Q1 through Q4 of FY2024 and provide an analytical commentary on the key contributing factors that led to the changes in revenues during this time’.  These kinds of use cases are driving increasing interest towards multi-agent architectures capable of complex task decomposition and more.        So, naturally, these days, we are increasingly noticing discussions, and papers about agentic architectures for generative AI solutions. Interestingly, many of the design patterns that are gaining traction in agent architectures are reminiscent of earlier research in Multi-agent Systems (MAS) and AI planning in the field of Computer science. Old wine in a new bottle? Well, time will tell. In any case, some noteworthy agent patterns from previous works include:     * Layered Architectures: Enable task decomposition through hierarchical agent structuring.   * Blackboard Architectures: Facilitate independent agent work with a communal 'blackboard' for sharing results, ideal for diverse expertise collaboration.   * Belief-Desire-Intention (BDI) Models: Support planning and decision-making with agents driven by their beliefs, desires, and intentions.   * Contract Net Protocol (CNP): Uses a decentralized protocol for task allocation where agents bid on tasks based on their capabilities.   * Market-Based Architectures: Inspired by economic models, these architectures use auctions and currency systems for efficient resource allocation.   * Hybrid Architectures: Combine various frameworks to leverage the strengths of different systems, creating robust and flexible solutions.     As we dig deeper into the world of agentic architectures for generative AI, the wisdom from decades of research in MAS, and AI planning domains seem increasingly relevant. So, let's build on prior work and not reinvent the wheel.  #genAI

216

22 Comments

In this article we are providing the major architectural building blocks and blueprint for building end-to-end GenAI-based applications that are ready for production. I’d like to call out several key considerations as you go about implementing and designing these llm-based applications. #llms #llmops #chatgpt4 #aiml #gemini #mlops #data https://lnkd.in/dMev5ncq

8

In the realm of AI, Retrieval-Augmented Generation (RAG) is revolutionizing Large Language Models (LLMs). By integrating external databases, RAG addresses LLMs’ limitations like hallucinations and outdated knowledge. This synergy of LLMs with real-time data enhances accuracy and credibility, especially in knowledge-intensive tasks. Read more about it in this article by Market Business News: https://lnkd.in/dX5VxpDT #AILanguageModels #RAG #Innovation #TechnologyTrends #FutureOfAI

4