Mohamed Amine Turki

Join PyTorch at Snowflake's #DataCloudDevDay on June 6 in San Francisco to learn how to accelerate GenAI performance with Native PyTorch. Register: https://hubs.la/Q02xQStt0

18

New large language models (LLMs) are going live frequently, and many have accomplished explosive growth💥, but this has left companies with uncertainty around what costs they will incur to bring their AI dreams to market 🤔 Enter BeFOri, the benchmarking framework from Ori that helps you answer your GPU budgeting questions ✅ Learn how to use BeFOri to project #LLM inference costs for self-hosted models and calculate a cost-per-token to compare with third-party API providers ⚖💰 Check out the blog post: https://lnkd.in/du5rGmJU

14

What makes a ‘Good Data Engineer’? 🤯 Data Engineers London recently had the pleasure of hearing from the Director, Data Architecture at gravity9, Benjamin, do a brilliant talk on the differing views of how to be a 'Good Data Engineer' in a big organisation. Ben did not bring just his own views, he took us through a journey of thoughts from industry leaders around the Data scene, and left the audience with a lot to think about! 🧠 The best news? Anyone in Data can take some lessons from this talk. (It’s honestly worth the watch!) Some of the highlights! 🎥 📊 The importance of understanding corporate goals (as well as the Tech): chasing revenue, reducing costs, and mitigating risks. 🤝 That communication is key in complex environments with multiple perspectives. 📚 Treating data as a product helps develop valuable domain knowledge. 🔄 Ensuring you adapt to variety of data engineering roles and perspectives of data in large organisations. And maybe the most interesting take? 🧩 Soft skills are essential for navigating large, complex organisations. Check it out now! https://lnkd.in/dsquWQG3 Make sure you join our community page for the next event! Thanks to our sponsors Kyndryl for your support, and Zego for hosting us!

19

3 Comments

I don't like to brag. An often tossed around phrase is that you are the average of the company you keep - so I will instead brag about how excellent some of company I have kept is. George Cherian - He architected a hadoop platform I ran spark jobs against at Barclays. It was amazing, and if I remember correctly he'd already done it for more than one large bank at that time. Brilliant guy. Vishal Morde - The most socially intelligent person I have ever had the pleasure of working with. Want to become head of Data Science at a FAANG company? Learn to be a servant leader like Vishal. Jai Oberoi - Why has Jai been a data executive for so long? Because as hard as ML is, getting large complex orgs regularly delivering on the value is harder - and he knows how to get everyone aligned. Leadership exists and Jai is that. Jayan K. Duggal - The only person I know who I could trust to deliver a working robot, with complex useful software, from first principles - hardware, electronics, actuators, perception - he can do it all. Its amazing. Pierre Doumit - We've all heard of the mythical full stack SaaS developer - front end this, backend that, scaling abstraction already built in, RDBMS and cloud migration migration managing, microservice AI service developing - all at startup pace - and we often wonder if such people exist. Pierre exists - he's awesome. He's also on the market rn - so you should hire him.

47

3 Comments

Today we’re excited to put a spotlight on our integration with Databricks again which has had a small makeover over the last couple of months 🎉 As before, Data teams can now trigger and monitor Databricks jobs in the context of an entire DAG - whether that’s also loading data into S3 using custom code, Fivetran, dbt (on Delta) or even Hightouch at the end of the flow: Orchestra has you covered. New: Orchestra also adds more functionality where Databricks workflows leaves off, like support for native data quality testing and out the box and fully customisable alerting for slack, teams and email. It’s not just robust and effortless orchestration that you get with our Databricks integration. Using Orchestra also gives you: 🔺 Full stack observability and metadata 🔺 🔺End to end lineage (with data assets*) 🔺 🔺Actionable Data Products 🔺 And incredible ALERTING to go with VISIBILITY of the entire stack Read more below: 👉👉 https://lnkd.in/exhb7JF8 * with dbt and Coalesce #dataengineering #dataorchestration #sql #spark

4

Machine Learning Engineer - Founding Team - London £200k If you answer yes to 4/5 of these questions, then you want to check out this job (link in comments) 1️⃣ Do you have experience with quantization techniques (e.g., PEFT, DeepSpeed, TensorRT) to optimize ML models? 2️⃣  Have you ever managed or worked with a multi-node HPC cluster? 3️⃣ Are you familiar with Deep Learning using PyTorch Library? 4️⃣ Do you have experience with Kubernetes container orchestration? 5️⃣  Have you worked with some of the recent Large Language Models (e.g., OpenAI, LLaMA)? #LLM #ArtificialIntelligence #Quantization

3

2 Comments

dbt surveyed thousands of data professionals in their '2024 State of Analytics Engineering Report'. They found that only a third of respondents currently manage data for AI model training today. But 55% expect to soon. In this post, we've looked into the AI & ML landscape if you work in a data team, including the importance of building on top of solid foundations. → State of AI in data teams → AI and ML use cases  → Data quality in AI and ML systems → The five steps to reliable data https://lnkd.in/d6_FvmJk

5

Need robustness and scalability in your Data Product? Let's include the data experts in your pizza team, not outside. Below the point of view of Data Science Product through Bastille Collective. Julien Vong Mathilde Brousse Thomas Belahi

7

You don't need expensive LLMs just to accurately map data into different formats Language models are without a doubt great at understanding context, and have proven particularly useful in reformatting data. You have source data in one format, but you need it in another. This is a super common scenario; whether you're integrating with a new API, aggregating, or standardizing data from multiple sources. When you're processing millions of data points, the latency and inference costs of running the entire input and output objects through an LLM add up incredibly fast. It’s a bit like shooting mosquitoes with a cannon. At Rectangle we discovered that you can use key-level vector embeddings to map arbitrarily structured JSON objects, eliminating the need for constant LLM API calls. By using semantic similarity captured in pre-computed embeddings to match keys between objects, you get the intelligence of semantic mapping without all the overhead! Key-level semantic mappings match the accuracy of the best LLMs (they use the same embeddings to build understanding after all), are about 50 times cheaper than inference on a per-token basis, massively reduce the number of used tokens (because you’re only sending keys instead of full data objects), and are practically instant. I’ve added additional details + the link to our open source library in the comments

31

1 Comment

Here’s how you can use Gretel to create secure, synthetic versions of your sensitive data for training bespoke small language models tailored to specific business tasks. Design tabular datasets containing numeric, categorical, free text, JSON, and time-series data. https://lnkd.in/eAwcVWXv #SLM #Privacy #SyntheticData

5

From mastering SQL to learning how to defend your data-backed findings, Damien Azzopardi shares five key takeaways for data scientists who are just entering the profession.

7

EvoAgent: A Generic Method to Automatically Extend Expert Agents to Multi-Agent Systems via the Evolutionary Algorithm Large language models

1

Build an effective recommender system seamlessly within your Snowflake environment using RelationalAI’s Snowflake Native App! Curious to see how it works? Check out this solution architecture on Snowflake Solution Center!

44

Hey hey! 👋 We're back with another Advancing AI video with Databricks - this time 👩🏽‍🔬 Gavita Regunath, Ph.D. and Sepideh Ebrahimi delve into 13 Essential Tips for Writing Clean Code from the MLOps Gym Series blog. Whether you're a data scientist, AI developer, or just passionate about clean coding practices, this video is packed with invaluable insights to enhance your coding skills and optimise your workflows 🤓 https://hubs.la/Q02BRgnf0 #Databricks #AI #DataScience #MLOps

13

Machine Learning (ML) model development does not end with training and validation. As they say “you cannot improve what you don’t measure” – to continuously deliver business value, ML models must be deployed to production, monitored, and maintained. Checkout our latest blog for a hands-on tutorial of End-to-End MLOps with Databricks! https://lnkd.in/d9Y3DsFk #Databricks #BuildersBlog #MLEngineering #MLOps

7