David Woollard

I ran across this post in the Changelog newsletter: https://lnkd.in/eCdjVekq - I like the name slop for unwanted AI-generated content. I love LLMs. They are part of my daily workflow, and I've been exploring them more seriously while on my sabbatical. I wouldn't misrepresent generated content without critical review as my own, but the web seems to be filled with more generated garbage (or 'slop')—content that looks fine at first but isn't accurate or helpful. LLMs (and other generative AI) should help us to make things of higher quality, not just churn out slop more quickly.

11

1 Comment

Data scientists are not trained to understand debits versus credits, CAPEX versus OPEX, accruals, revenue recognition rules, etc. FP&A professionals are not (initially) trained as data scientists. If they are, as Kurt Shintaffer states, they are torching money and time building it themselves. And what happens when the few individuals at your company with this skill set move on? ‼️🤷‍♀️ Check out what Sandeep Madduri and Naresh Nemali built for finance professionals. Precanto is what this generation of finance professionals need. Monetize the data … Transformation starts now! 🚀 (Disclaimer: This bold post is coming to you from recovery room while on pain meds. All is good and colon repaired. Please eat your fiber, folks, and listen to your body! But in all seriousness, Precanto is the next wave of innovation for CFOs and FP&A teams.)

27

7 Comments

The afternoon Keynote @ Stripe Sessions was Fireside Chat between Patrick Collison (CEO of Stripe) and Jensen Huang (CEO of NVIDIA). Patrick was an excellent moderator — asked Jensen really probing, nuanced questions that Jensen answered with his usual sharp wit and unique (and impressively well rationalized) perspective. A couple of quotes among many that stuck with me: - “Current CPU installed market is $1 trillion of hardware, all of which will be upgraded to GPUs over the next 5 years. And I would estimate about another $1 trillion will be spent on top of that during the same time period as companies try to do more with AI. Add those together — and I think at least $2 trillion will be spent on GPUs in the next 5 years.” - “I like to go after Zero-Billion Dollar Markets — nonexistent markets that I envision & can rationalize well, but will be very hard, expensive & time-consuming to achieve.”

1

🍜Anybody up for some ramen? I am excited to share that I am back in the garage, building again (it’s actually a house in the Berkeley Hills). We are taking the wraps off of Supercog AI, a new startup focused on solving Application Integration using LLM powered agents. My co-founder in this new venture is a great friend of mine (and a former co-founder partner), Alex Osborne. Alex and I have worked around the application integration space for a long time, and we are unrealistically excited about applying the GenAI stack to this problem. Not many people relish the task to get some bits to move from system A to system B. But it’s a critical job to be done, and that’s led to a lot of brittle scripts, a myriad of inflexible “no code” tools, and a lot of hand crafted SQL. But the power of the Large Language Model, trained on a huge corpus of information, offers a remarkable tool to solve this problem. The LLM you use today already knows the APIs of hundreds of popular systems. It knows the SQL dialects of every major database. It understands the semantics and data schemas of many popular SaaS systems. It understands much of the specific domain in which your business operates. I like to call the AI revolution “the last platform shift”. That may turn out to be hyperbole, but I definitely believe we are in the very early stages of seeing what this new stack can enable. I know we are hardly the first folks to claim that “amazing stuff is coming!”. But this is why Alex and I have decided to focus on a real and hard problem. If we can prove that LLMs can power a new way to solve this problem, it will be the first successful *new* approach in 15 years.. We will have much more to share about this new platform soon. In the meantime, if you’re interested in getting a peek - or even better if you have application or data integration tasks that you would like some help with - please reach out because we want to help. #genai #startups #backinthegarage

163

37 Comments

Performance is key! As we cannot share the detailed work with (former) customers of Rockset (acquired by OpenAI), we decided to implement the Rockbench benchmark to check how streaming ingest between CrateDB and Rockset (acquired by OpenAI) compares. The results are very insightful. We see: ✅ CrateDB achieves 6-9x lower latencies than Rockset for streaming ingest. ✅ CrateDB outperforms Rockset on the same hardware while saving about 20% costs. ✅ When volumes increase, the latency increased linearly in Rockset, while remaining mostly flat in CrateDB. As a plus: 1️⃣ Achieve full high availability due to the distributed nature of CrateDB (benchmark results are achieved including replication in CrateDB) 2️⃣ Achieve higher throughput on the same amount of CPUs as Rockset (at 50% of memory needs) 3️⃣ Full open-source solution that prevents you from market shifts On side note: If you are thinking to switch to Elastic/OpenSearch, this benchmark has been created to compare Rockset vs. Elastic, so CrateDB is also faster than Elastic for streaming ingest cases! Read the full comparison here: https://lnkd.in/d_nEMDkx

29

1 Comment

On April 18th, just last week, Meta unveiled Llama 3, lauded as "the most capable openly available LLM to date". This remarkable achievement by the Meta team comes hot on the heels of Llama 2's release last July. Notably, the new models surpass their predecessors and competing offerings from other providers in terms of performance. Meta's commitment to open sourcing these powerful models while prioritizing model safety and responsible usage is commendable. (https://lnkd.in/ghv3RJRz) Yesterday, Microsoft introduced Phi-3 Mini, a member of the Phi-3 family of models, promoted as "the most capable and cost-effective small language models available". According to Microsoft, the key to achieve high performance in such a small package lies in the quality of the training data. The model has been open sourced and is now available on Ollama and Hugging Face. (https://lnkd.in/gKgSmrBn)   These compact yet powerful models serve as invaluable tools for creators eager to bring their visions to life or simply experiment with cutting-edge technology, particularly for those with limited access to resources.   Shortly after the release of Llama 3, reports began surfacing of enthusiasts running the 8B model on a Raspberry Pi 5 equipped with just 8GB of RAM—a modest $80 single-board computer. (https://lnkd.in/gbeQPhQ2) While the performance understandably reflects the limitations of such a platform, the fact that the model runs and generates an output is nothing short of magical. Similarly, the Phi-3 Mini model can run on ubiquitous devices such as smartphones.   Reflecting on my own journey, I recall the pivotal role that early access to a low-cost computer played in shaping my passion for coding. The gift of an 8-bit Commodore 64 home computer ignited my curiosity at a young age. Later, with the guidance of an outstanding Computer Science teaching staff at my high school IISS Marconi-Hack Bari, I embarked on ambitious projects like a networked multiplayer version of Battleship written in Turbo Pascal and 8086 Assembly, complete with custom sprites and lo-res graphics. These formative experiences paved the way for my career, eventually leading me to various engineering and leadership roles.   It all began with a simple, low-cost, modestly capable computer used for experimentation—a gateway to discovering my life's passion.   I encourage educators and parents alike to consider setting up similar low-cost experimentation environments. By introducing younger generations to open-source AI technologies, we can nurture their creativity and help them uncover their true passions. It's not just a weekend project—it's an investment in their future and the future of innovation and discovery. #education #youth #AI #LLM #future #innovation

24

This tactic, in which A.I. companies promise wild new features and deliver a half-baked product, is becoming a trend that is bound to confuse and frustrate people. The $700 Ai Pin, a talking lapel pin from the start-up Humane, which is funded by OpenAI’s chief executive, Sam Altman, was universally panned because it overheated and spat out nonsense. Meta also recently added to its apps an A.I. chatbot that did a poor job at most of its advertised tasks, like web searches for plane tickets. Companies are releasing A.I. products in a premature state partly because they want people to use the technology to help them learn how to improve it. With artificial intelligence, companies are giving a preview of a potential future, demonstrating technologies that are being developed and working only in limited, controlled conditions. A mature, reliable product might arrive — or might not. The lesson to learn from all this is that we, as consumers, should resist the hype and take a slow, cautious approach to A.I. We shouldn’t be spending much cash on any underbaked tech until we see proof that the tools work as advertised. From New York Times, The New ChatGPT Offers a Lesson in A.I. Hype https://lnkd.in/dgD_tYne

16

2 Comments

Stop (over-) engineering your prompts Generative AI is really cool. I use several tools in this category every day for writing suggestions, image generation and more. The tools can do amazing things, especially if given the right prompt. But there's a limit to how much prompt engineering one should do, especially if you're trying to generate text for mainstream consumption. This topic is already a bit of a bandwagon but this topic has been bugging me for about a week so I'm going to write about it. Generative AI relies on a class of machine learning applications broadly described as deep learning. Deep learning boils down to feeding a neural network a ton of content that has been classified into groups and then iterating over the content to allow the network to identify patterns in the content. Basically the network builds up statistical relationships between the words or other elements in the content. So it knows, for example, that if a piece of content is a recipe the word tomato is often followed by the words sauce or paste and very rarely by the word stem. But in a document about farming tomato is often followed by the word seed, starter or stem. The deep learning model has developed an elaborate matrix of statistical relationships between words in different types of content. If you described CHatGPT as an elaborate word guesser you wouldn't be wrong, just over-generalizing. Large language models are trained on billions of documents. So how, you might ask, are the documents classified in order to feed them into the neural network? Did someone categorize the web by hand? Of course not. The developers of the LLM leveraged Natural Language Processing (another deep learning application, btw) and other tools to categorize documents before they got fed into the model. Okay, but, again, what does that have to do with prompt engineering? When you ask an LLM to generate a piece of content, say a blog post about cycling in Northern California, you are specifying a category of content. If you later asked for an "image in Rubinesque style of a man and a woman engaged in flower picking while overhead rages a space battle between starships populated by photorealistic blob-like aliens carrying professional immersion blenders done in the style of pointillism" you have specified many categories. You're essentially asking the LLM to extract elements of the content that it was trained on that match your descriptors and which have mathematical relationships to other adjacent content elements and to include those in your generated image or blog post. And ... prompt engineering? The categories in the content were identified with Natural Language processing. Your prompt should be in natural language, not some over-specified, highly engineered artifical style. Don't over-think it, work with the model, don't try to control it too much. That's it.

2

Google just announced at their I/O developer conference that they are officially rolling out SGE (Search Generative Experience) today! Understanding Google's SGE: Google's Search Generative Experience is an advanced feature that leverages AI to provide more comprehensive and contextually relevant search results. It integrates elements of generative AI to enhance user interactions with search, offering richer and more detailed responses to queries. This evolution aims to improve user experience by delivering more personalized and immediate information. How this might impact you: SGE has the potential to shift how users interact with search results, emphasizing the importance of high-quality, contextually relevant content. It will likely prioritize content that is informative, engaging, and answers user intent more effectively. At HV, we have been preparing for this and have refined our approach and strategies to ensure our client's businesses continue to thrive in the evolving search landscape. Some high level ways in which we have prepared for this are: 1. Content Strategy Optimization: We're focusing even more on in-depth, authoritative content by enhancing long-form pieces, implementing structured data, and optimizing for conversational queries. 2. Technical SEO Enhancements: Our thorough audits have additional focus on improved site speed, mobile usability, internal linking, and indexing. 3. Data-Driven Insights: Leveraging AI tools help us adapt to search pattern changes, identify content opportunities, and monitor performance in real time. 4. Continuous Learning and Adaptation: Our team stays updated with SGE and AI developments through industry webinars and training. 5. Client Collaboration: We tailor strategies to meet your specific needs through a collaborative approach. For more details message me and I will be glad to expand.

17

3 Comments

OpenAI buying Rockset is massive news in the #realtime data space, but the question is what it actually says? First question : What did Rockset actually solve for? By their own marketing, they power "in-app search and analytics" they compare themselves ClickHouse, Imply(Apache Druid), StarTree (Apache pinot) and Elastic search. Amusingly, Clickhouse and Tinybird have gone aggressive on ads about migrating from it. This places them squarely in the realtime search/analysis space. Considering how much data recall large language models require, the usage makes more sense. My experiences with Rockset have mostly been with a few very impressive demos involving massive ingest and queries of JSON data. It very much felt like what Solr and Elastic search felt like. Second question: Why did OpenAI buy them, and sunset the cloud service? I have a number of theories, but mostly it comes down to this: It would cost OpenAI more than 20% of the purchase price PER YEAR to just use the product as is. I'd guess their spending FAR north of 100m a year on just data services. As for sunsetting the cloud services? They likely cost more to operate than they make. Most startups have a rough time on margins, so maintaining a few tens of million in revenue doesn't make sense when they could re-task the team to power the Open AI rocket ship. Building proper multi tennent data isolation tools is HARD, and Rockset had some of the best low effort ingestion out there. Last question for today: What does this mean for the space at large, and for their customers? Cynically, its a big validation for most of the big investments into the realtime analytics & search space. There are a lot of large players in racing to corner the LLM space and with Tabular (now part of Databricks) and Rockset exiting, This will cause some portfolios to spike. HOWEVER! This does show why open source/core tools could be a more strategic move going forward, and moving off tools like this are a pain. Rockset built a great product, and now its gone. Selfishly, I'd like to see more of this work move into #apachePaimon, and #apacheFlink but I am biased :D. For the recall, enrichment, and training pipelines, having exactly-once stream and batch semantics on a realtime #datalake really adds some reactivity to the user eperience :) references: https://lnkd.in/dY4JHecj #apachedruid #apachepinot #bigdata #LLM #openAI #chatgpt

54

5 Comments

“Anonymized, pseudonymized, de-identified or encrypted data should be excluded,” Business Roundtable wrote, “if processes prevent re-identification.” I love anonymized data exemptions. I think they’re the best way to balance AI innovation with privacy. I think they’re underutilized and under-appreciated. I think more data should be collected, anonymized, and shared with more people to drive progress. But this is preposterous. This comment was submitted by a tech industry trade group in response to recent Justice Department rulemaking that limits the sale of personal information to foreign adversaries like Russia and China. The deliberate mashing together of anonymous data with pseudonymous and de-identified data is reckless, given the stakes. Pseudonymous data is *designed to be joined with other data*. That’s why every modern data protection law treats pseudonymous data as personal information. It’s why the FTC rejected Avast’s the claim that joinable, pseudonymous data could be treated as "anonymized." Anonymous data cannot be re-identified or linked with other data. It carries qualitatively and quantitatively different risks than pseudonymous and de-identified data. There are a bunch of considerations for whether an anonymous data exemption makes sense when the likely recipients are hostile nation-states. This may be the rare case where the theoretical re-identification risks inherent in any anonymized data are too harmful to tolerate. But an exemption for pseudonymous and de-identified data is unconscionable. NOTE: Advertising companies provide a valuable service that funds the accessibility of the modern internet and are unfairly villainized in too many privacy conversations. But when they add “anonymized” data into their arguments, it’s intentionally misleading and deserves condemnation. They will not be selling anonymous data, because anonymous data can’t be used for targeted advertising. That term is added to the list to take the edge off selling re-identifiable data for advertising. Thanks to Justin Sherman, who does incredible work in this space, for highlighting this in Politico!  

45

2 Comments

I'm really intrigued by this idea (not mine) that some bitcoin mining companies (Iris, Cleanspark, etc) have successfully built low-cost power shells that can now be leveraged by AI compute needs -- where power itself is becoming the gating factor in the continuing growth of AI. I think this will become a real 2025 story -- as the gating factor of "chip shortage" lessens, and the gating factor of "cheap power" rises in significance. There's not really a venture capital angle here, but there is a public market one. Perhaps more importantly, it becomes a touchpoint between "crypto" and AI. Something to ponder and keep an eye on (full disc: I own a bunch of bitcoin miner stocks, including those mentioned above).

9

2 Comments

🎙️ My co-founder Ken had the pleasure of hosting Eli Daniel, Head of Engineering at Jellyfish, on his latest episode of Level Up. They dove into how to choose metrics that align perfectly with your organization’s goals, shared tips for analyzing where your team’s time is really going, discussed scaling metrics for better alignment and resource allocation, and explored why DORA metrics are just the beginning. They also delved into balancing quantitative and qualitative metrics to get a complete picture of how effective your team is. 💡 Product developers save an average of 2 hours each week thanks to OpsLevel. Imagine the impact of not being able to quantify such gains—this is where Jellyfish proves indispensable. 👉 Tune into this episode to dive deeper and uncover strategies to elevate your team's performance. Please share if you find this as exciting as we do!

22

1 Comment

As academic based LLM benchmarks are losing their relevance -- there are 2 types of Benchmarks that will matter more in the B2B space: 👉 Crowdsourced user votes to rank LLM output (like Chatbot Arena) OR where the test DATASET is kept secret by the evaluator (e.g GSM1k from scale_AI) 👉  Enterprise use case specific LLM benchmarks (e.g produced by CRM, ITSM, HR SaaS companies) evaluated on real data. Because you can not game real customer use cases. #llm #benchmarks #llmeval #AI

11

1 Comment

Today I am going to do something really unique (based on Hunter Walk's idea), I am putting Async up for sale publicly 🎙️ What's Async again? A Voice Memos platform that powers fully asynchronous work conversations Sometimes, things don't work out, and we move on. But we rarely read those stories here, and I am at a stage in my career where I want to be true to myself because I think I've done everything I could to make it work, and there's nothing to be ashamed of. So why not open-source an M&A process? 🎢 WHY I will not rewrite the story, but in a young startup, only one thing matters: People. It basically took us 18 months to launch the first V1 of Async vs the 6 months I had envisioned. We had raised for 24 months of runway, meaning we only had 6 months left after our (great) public beta launch to find Product Market Fit. Not enough. 🥗 WHAT ARE WE SELLING Send an async voice memo to anyone through a simple link anywhere (Gmail, Slack, Notion, etc.) or drop it on their personal URL (mine is async.com/ilan). The memo will be transcribed and summarized and have time stamp reactions, files, and more. > Incredible suite of apps for the Web, Mac, iOS, and Chrome (reply to your Gmail emails with voice notes). Our tech stack includes React, Node.js, Stripe, and Swift. > Brand and domains: we own the trademark and domain for Async and async.com. We also own asynchronous.com and audioletter.com. 👤 ICP > Coaches, Creators, and Consultants: Offering async consultations with clients, audiences, and fans. > Sales People: Send updates to your leads, prospects, and clients without requesting a call. > CEOs / C-Level: Update your team on objectives, progress, wins, async. > Investors: Perform due diligence, deal updates, and ref calls async. 🧠 IDEAL BUYERS Players with distribution power or considering us as an add-on to their offering. > Tools for Creators, Coaches, and Consultants: offer asynchronous consultations > Note Takers: add collaborative audio > Calendar, Project Management, and Mail apps: add asynchronous conversations to emails, tasks, and projects, replace meetings > Audio tech: add collaborative audio notes > Email Newsletters: Add Audioletters to your offering. 🫶 THE THANKS 🙌 Our users: You've kept us going all this time (and still are!), and we hope to keep Async alive for as long as possible ❤️ Dan: best co-founder I could dream of. Here for every battle, an execution machine, and simply an amazing friend. Forever in my heart. ❤️ Jeremy & Lucas: I've worked with many designers and engineers in my career. You're simply the best. You've won my heart and trust; thank you. 💵 To our investors: Thanks for bearing with us and believing in our vision and us. I will fight to make you proud until the very last breath. That's it, you know everything. If this rings a bell to you or one of your connections (tag them), please slide in my DMs and let's chat ❤️

319

85 Comments

Anthropic just launched Claude 3.5 Sonnet, which beats both OpenAI GPT-4o and Anthropic’s previous paid model Claude 3 Opus on most benchmarks. Claude 3.5 Sonnet can execute and debug code, and has improved image understanding capabilities, including text transcription. A new feature called “Artifacts” enables users to view generated code, documents and websites in a separate window next to the chat. Claude 3.5 Sonnet is available for free to all users, but paid users get higher usage caps. Did Anthropic just take the lead back from OpenAI? It seems that ChatGPT’s Code Interpreter tool, which used to be one its main advantages, has finally met its match, but ChatGPT still has the advantages of real-time Internet access and custom GPTs which Claude lacks… What do you think? https://lnkd.in/d_hk86cf

10

3 Comments