Steffen Meschkat

How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out. https://lnkd.in/dwczVutW

43

5 Comments

Anthropic just launched Claude 3.5 Sonnet, which beats both OpenAI GPT-4o and Anthropic’s previous paid model Claude 3 Opus on most benchmarks. Claude 3.5 Sonnet can execute and debug code, and has improved image understanding capabilities, including text transcription. A new feature called “Artifacts” enables users to view generated code, documents and websites in a separate window next to the chat. Claude 3.5 Sonnet is available for free to all users, but paid users get higher usage caps. Did Anthropic just take the lead back from OpenAI? It seems that ChatGPT’s Code Interpreter tool, which used to be one its main advantages, has finally met its match, but ChatGPT still has the advantages of real-time Internet access and custom GPTs which Claude lacks… What do you think? https://lnkd.in/d_hk86cf

10

3 Comments

It was hard, but we now have a primitive at Gadget that lets us do absolutely 0 downtime atomic changes to an application's structure or data under the hood, with 0 requests dropped. I call it "request catching" but it was inspired by this Braintree eng post I read long ago For stateless workloads like serverless functions, 0-downtime deploys are pretty easy. You stand up the new one next to the old one, use a load balancer to start sending requests to the new one, and slowly stop sending requests to the old one. But for stateful workloads, it's much harder. You can't just start a new Postgres and send some queries to the new one and some to the old one without lying to somebody about state. If you want to be consistent, there has to be one source of truth. Fancy DBs have quorums for this, but PG doesn't. If you want to be consistent, you're effectively forced to have some tiny-but-real window of downtime in between draining the old one and starting the new one. That said, there's nothing forcing you to return errors to clients while that is happening. If your infrastructure supports it, you can pause requests while this changeover is happening under the hood, and then resume them once it is over. Clients will notice a longer request that normal, but it will be served like normal and they'll be none the wiser! What's required to do this though is a deep deep integration between the load balancing infrastructure and the thing doing the database deploy under the hood. We've built this up using a distributed locking tool, where all requests start by taking out a shared lock. When we're ready to do the maintenance under the hood, the maintenance workflow takes out an exclusive lock, blocking all requests from proceeding. The workflow does the atomic change it needs to make knowing it has exclusive access which means nothing will change under its feet, and then releases the exclusive lock as fast as it can so those requests can proceed. As an optimization, we only check the lock on the request path if an app is marked as near a maintenance window. This is only possible because of Gadget's architecture where we bundle together the load balancer, the request processing layer, and the database access tier such that we can reliably intercept all requests and all database transactions and do this lock checking. Its these nasty, in-the-weeds details that make me so proud of Gadget -- we can do world-class engineering stuff under the hood on behalf of all our apps without them ever noticing. If you're interested in doing this on your stack some OSS tools have the primitives you need! We use nginx for our load balancer, Temporal for the reliable workflow execution and we're currently using postgres advisory locks for hte locking. I think the Braintree post has been lost to time after they got acquired, but @simonw has a great post on the subject here: https://lnkd.in/eZmiktGT

40

if you know, you know how to read when there are haters in this case NVIDIA so you have friends at the financial times. so you know that your clients read the financial times. it's the same for the new york times, wall street journal. each publication has it's audience. who are you to say we are in a bubble. perhaps you have a ton of money. and you are playing to short the market. yes there are those who weave a narrative that all is doom and gloom and shout there is a fire. waiting for you to pay them for the fire hose and water they are holding in hand. sorry. can't put out the fire unless you pay me. handsomely. and the bigger the fire. the greater the reward. ask george soros when he shorted the british pound. michael barry. the big short on mortgage backed securities. bill ackman when the pandemic freaked out the markets in march 2020. the playbook is the sky is falling and at the same time they have a trade that offers them a spectacular win. there is no one who calls the top. there are investors who are motivated. asking for oversized returns. check. there are tech founders delivering on creating the machine that moves the machine that makes the money. check. there are the clients who willingly, quickly jump on to purchase said machine from company (machine) to make them money or rather offer them productivity or quality. so i find this kind of abusive when there are journalists who are there to stoke. let's face it. drama. drama. gets. attention. and attention is rewarded. and that attention seems to work really really really well when you instill fear in the market. be wary. be aware. the markets today are going through something. some will speak to japan. some wars. some regulators. some that (ai) is not growing fast enough as advertised. the direction is clear. more digital in our lives. more digital sources to offer choices in our lives. more digital enhancement to offer us a super(human) existence. we are going through growing pains here. a few have the memo and they are updating their systems accordingly. never ever ever forget. he who gets the information first. and. can. and. is willing. to act on it. is alpha. and then there are a lot of lemmings. a. lot. (ai) is alpha amplified. that's why the investment. that's why the investors. that's why the expectations. that's why the rewards. and a lot of haters sitting on the sideline playing their dirty tricks. it is not enough for me to win. you have to lose. the heartbeat is the financial markets. have. a. healthy. heart. ;) source: https://lnkd.in/e6u_JgST < . >

4

2 Comments

Post Open. I appreciate Bruce Perens not trying to re-define Open Source, but I'd prefer some other terminology which does not imply replacement. I think it is fantastic to experiment with different kinds of Proprietary licenses, yet if you are to polish Open Source definition it should be around respecting and enhancing freedom of software users, continuing to empower permissionless innovation, not ensure Devs can be paid what they think they deserve to be paid - Proprietary Software licenses already solve this problem well https://postopen.org/ #opensource #postopen

10

1 Comment

For all those engineers out there building AI agents. You need to check out this #github repo. This is a great team tackling an important set of problems that engineers face, and this solution is significantly better than Langchain's approach. #ai #agents #ML

8

1 Comment

AI-powered web agents fail. A lot. Why? Because they are inherently probabilistic tools operating in a big, complicated environment that changes all the time – the internet. At High Dimensional Research, our mission is to transform web agents from an unreliable, expensive pipedream for AI specialists into an efficient, production-ready tool for every full-stack developer. Today, we took a big step towards that goal by launching the Memory Index. This functions as a repository of user-anonymized web trajectories -- storing page structures as a graph, and navigation across those structures as searchable data. What does that mean? Well, basically, we observe the agents using our platform and transform their patterns of activity into a roadmap for the internet. The more people who use it, the more detailed and useful the map becomes. This way, an AI never has to figure out how to solve a problem that has already been solved. We just give them the answer. This radically reduces the failure rate, cost, and carbon footprint of running an AI agent. Check out our launch announcement and sign up at hdr.is/memory. https://lnkd.in/ePmCcScM

3

sharpening writing skill is one of killer skill!

2

Please read the linked article in its entirety, don't get put off by the obvious and hilarious cartoon violence, and post comments to Scott's thread. This article is key to understanding the growing undercurrents. #WrittenByMe

3

I found this article to be a fantastic explanation of some of the modern challenges with AI (e.g., LLM, FM) software. Possibly the best part is that many of these challenges have been solved in traditional software systems. So, we have a blueprint on how to potentially proceed (as well as having hints on where many of the "traps" may be). Thanks Ben Zorn and Emery Berger!

13

5 Comments

I like this idea of "technical taste". It gets into where software engineering is as much creative art as it is science. Four great takeaways (IMHO) for software devs in this era in particular: 1. Aspiring engineers should cultivate a sense of curiosity, experiment with different tools and technologies, and embrace a mindset of continuous learning. 2. AI has the potential to streamline processes and enhance productivity for engineers, but it may also lead to disruptions in traditional software development workflows. 3. Developing technical taste and judgment is essential for making informed decisions about which technologies and approaches to pursue. 4. Collaboration and open-mindedness are key to leveraging the full potential of AI and staying ahead of technological advancements. I also really liked these two quotes from Sam Schillace: 1. "The right time to do something is when you have that feeling in the pit of your stomach that's like, 'oh, this is a great idea and it's going to suck to build because nothing's ready yet.'" 2. "Technical taste is like, 'how well have you consolidated that set of experiences and heuristics into judgment that you can apply accurately when you see new things?'" 3. "It may be the case that very small teams can do very large projects, or like we were talking about before, it may be the case we're just going to get really ambitious about what we try to do with the same size of teams, which is kind of where I would put my money." Great job on continuing to put out great episodes, Brett Berson and team First Round Capital! #ai #engineering #softwaredevelopment

5

Performance is key! As we cannot share the detailed work with (former) customers of Rockset (acquired by OpenAI), we decided to implement the Rockbench benchmark to check how streaming ingest between CrateDB and Rockset (acquired by OpenAI) compares. The results are very insightful. We see: ✅ CrateDB achieves 6-9x lower latencies than Rockset for streaming ingest. ✅ CrateDB outperforms Rockset on the same hardware while saving about 20% costs. ✅ When volumes increase, the latency increased linearly in Rockset, while remaining mostly flat in CrateDB. As a plus: 1️⃣ Achieve full high availability due to the distributed nature of CrateDB (benchmark results are achieved including replication in CrateDB) 2️⃣ Achieve higher throughput on the same amount of CPUs as Rockset (at 50% of memory needs) 3️⃣ Full open-source solution that prevents you from market shifts On side note: If you are thinking to switch to Elastic/OpenSearch, this benchmark has been created to compare Rockset vs. Elastic, so CrateDB is also faster than Elastic for streaming ingest cases! Read the full comparison here: https://lnkd.in/d_nEMDkx

29

1 Comment

The final video in our 3 part series. This one is with Google Vertex AI. I'm not sure if Google can make it any simpler! Please hit that like and subscribe button as we continue to create videos on making AI easy to understand and use! Yes, I am a huge Broncos fan. #ai #google #gcp #chatbot #easy #vertexai #howto

1

Excited to announce another major upgrade to WhyHow.AI’s Knowledge Graph SDK - tying vector chunks to graph nodes automatically, for a more deterministic and richer context window. Check out how we do it, why we did it, and an example benchmark of the increased completeness of the answer. Tired of just returning single-word triples from your knowledge graph? WhyHow.AI’s latest upgrade with vector chunk linking now lets you use a graph structure to determine which raw vector chunks to return to the context window, combining the best of knowledge graphs and vector search. “While the triples in a Knowledge Graph are useful in providing specific information that semantic similarity was unable to retrieve, we wanted to also allow leeway in the information represented and retrieved from the graph, to include the surrounding words and retrieving the relevant raw vector chunk tied to that graph node as well. By tying vector chunks to a knowledge graph, we get the advantages that lie in both vector and graph search.” - WhyHow.AI Design Partner WhyHow.AI builds workflow tools for data orchestration, and graph creation, and we work on top of any data extraction model you want to bring. In this case, we work on top of OpenAI, Neo4j, and Pinecone, and will be supporting the most popular data extraction models, LLMs, graph and vector databases. https://lnkd.in/eEJdUNPi

68

3 Comments

Included in this article is the mention of vector databases. Unsurprisingly, Michael Stonebraker and Andy Pavlo feel there is not a need for vector-specific databases. They believe relational databases will adopt their features, which is already happening. Check out the details here: https://lnkd.in/gyq_4WjP

27

1 Comment

I ran across this post in the Changelog newsletter: https://lnkd.in/eCdjVekq - I like the name slop for unwanted AI-generated content. I love LLMs. They are part of my daily workflow, and I've been exploring them more seriously while on my sabbatical. I wouldn't misrepresent generated content without critical review as my own, but the web seems to be filled with more generated garbage (or 'slop')—content that looks fine at first but isn't accurate or helpful. LLMs (and other generative AI) should help us to make things of higher quality, not just churn out slop more quickly.

11

1 Comment