Swapnil Sapar

To most of us, who has been working in the field for years, LLMs started as being a "smarter autocomplete", that can provide the next best word by "reading" a corpus of documents. And it turned out to do much more sophisticated things demonstrating human like capabilities. We were chatting with a friend in the weekend, and the question came up, "as humans, do we actually just do autocomplete?" Is innovation or thinking all about predicting the next thing all the time. For instance, think of sports, such as soccer, it is all about predicting the next move of the proponent if you are in defense or predicting where you can move the ball to maximize the probability of scoring a goal. Or more obvious things like playing chess, which is all about predicting the next n moves or predicting the whole game moves if you have enough memory. How do we innovate? We read a lot (training the neural net) and then find a challenge, look into the sequence of things that exist today and predict next big move. Probably we all autocomplete. Never thought about this way.

20

4 Comments

A few thoughts on #LLM usage in the small, prompted by my toying with #llama3 and #phi-3 on consumer hardware.

5

Kapil Khangaonkar, CEO of Clodura.AI, shares how Google Cloud fueled their growth: 71% less tech debt, 12x faster app delivery, & significant customer growth. 🚀 Ready to transform your startup? The Google for Startups Cloud Program now offers new benefits for AI startups, giving you the technology and resources to build easily. Take advantage of our open AI ecosystem and tap into the best of Google’s infrastructure, AI products, and foundation models.   Sign up today and unlock your startup's potential! #startups #cloud #GoogleCloud #CloduraAI #techstartup #startuplife #AI #buildwithAI #GoogleAI #GoogleforStartups #GoogleCloudforStartups https://goo.gle/GFSCP_CS Clodura.AI Google for Startups

3

Get to know Gemini in Databases: an #AI-powered assistant that helps across all stages of the database journey and simplifies all aspects of database operations with features like Database Insights, Database Migration Service, and more. #GoogleCloudNext

2

#Amazon #OpenSearch Ingestion now allows you to ingest streaming data from Amazon Managed Streaming for Apache Kafka (#MSK) Serverless, enabling you to seamlessly index the data from Amazon MSK Serverless clusters in Amazon OpenSearch Service managed clusters or #Serverless collections without the need for any third-party data connectors. #ApacheKafka #streamingdata #datastreaming #data https://lnkd.in/gdDJED6V

20

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

In collaboration with AWS, we are thrilled to announce the Confluent blueprint for Amazon OpenSearch Ingestion! This serverless feature enables customers to effortlessly transfer data from Confluent to OpenSearch Domains or OpenSearch Serverless collections. This capability unlocks numerous new use cases, such as Retrieval-Augmented Generation (RAG), real-time log analytics, and more. Read the full announcement, including a complete example of setting up a Confluent/OpenSearch pipeline using the new blueprint (with optional use of Schema Registry), here ➡️

17

This is an interesting read. The changes that the broader AI ecosystem will bring to existing business practices and verticals will be a transformational journey for many industries in the years to come. Some of the capabilities highlighted in this article will eventually become mainstream. A few things that the entire tech industry has to work on to make all of the concepts outlined as mainstream are: Data Quality and Governance - Garbage in equals garbage out. There is lot of business "junk" that can lead to model hallucinations, inaccuracies, etc. that require various re-learning techniques and feedback loops to be implemented today. Without improving data quality and governance practices through automation, intelligent systems, you will have low quality outcomes which eventually won't augment your workforce. This vertical becomes inherently more complex with voice and video getting added to text-based data. New business models - A lot of products in the market need to rethink how they optimize for the outcomes and which outcome they want to charge their customers for. Most customers don't want to sign up for an outcome based model which charges for activities which are in the business workflow without achieving the final outcome. And sometimes the final outcome like closing a sale is too far in the cycle especially in the B2B space that it's not sufficient to provide a robust cloud-based solution. Cost-of-goods-sold - AI models are expensive to run and operationalize and over the years, this will evolve and R&D across the industry will help drive down the cost. This would need buyers to re-think how they buy and plan for costs, forecasting and planning and sellers to re-think their business models and monetization strategies. Multi-modal data management - For your models to be efficient from a business context, you need to be able to process all types of data that is generated: data stored in data management systems, voice-related data from calls, presentations and video-related data from trainings, knowledge bases, news, etc. Managing multiple data management systems was complex with the data sets different industries like banking, financial, energy, healthcare, auto, etc. generate. It will be inherently harder to manage the additional types of data in a cost effective manner. We will see a boom of additional cloud computing power and a different level of computing efficiency (performance and price-per-performance) that would have to evolve much more than what we have today in the market. These are just a few of the innovation areas that will define how AI shapes up and becomes mainstream. This is a generational transformation that we are living through and a lot of innovation will have to happen in the above areas in the coming years. #ai #industrytranformation #data

32

1 Comment

In the world where we are used to treat resources as limitless it is important to remember that POCs and even small MVPs we do - are not in fact free cheese. Cloud time and compute time do cost a lot when come at scale - may be not straight from developer’s pocket , but for enterprise - absolutely . Try to do things locally - if you need to wait for few minutes vs ‘almost instant ‘ result with more powerful hardware on cloud - it’s ok. It gives you time to stop and think . You can’t give excuse of ‘ well it runs here for 10s but we can scale this in cloud and it will be 800ms without trying to understand why things are slow. These latencies do build up in complex systems. Local run is perfect opportunity to try to tune things and investigate why things are indeed slow. Overnormalizes databases , autogenerated queries , chaining calls to microservices , locking up on shared resources - causes are rather usual and categories are pretty much same . Overcome them with good design and programming discipline and you just saved money for your company and grief for yourself and your team in long run when you support solutions .

5

"Poor performance" gets them out of paying unemployment insurance. Pretty disgusting if you ask me, any business who pulls that in this day and age deserves everything they have coming to them, this destroys lives. Imagine dedicating 3, 4, maybe 5 or more years to a job, staying through all the thick and thin, ignoring all the red flags, all the grifting and dirty double-dealing you see, because if you go along, you get along, and you even may be at the last gasps of your own career, scared to death of whats on the other side, so you never say anything and just smile and walk away to save it for a day of reckoning that never comes. When you see on the form where they check "poor performance" you gotta understand, the rules are absolutely plain here: if society thinks youre a f*ck up, you dont get benefits, even if you pay into the pool for those 3, 4 or 5 years. They literally f'd those 1800 people. When that checkmark gets checked, theres no way out of it. Its a travesty, being labelled that way is a death sentence.

1

2 Comments

Google Cloud Vertex AI is rolling out multiple updates to make AI accessible for every enterprise. Up to 85% lower input costs and 80% lower output costs, and 100+ new languages coming to Gemini 1.5. #GoogleCloud #VertexAI #Gemini

9

Google Cloud Vertex AI is rolling out multiple updates to make AI accessible for every enterprise. Up to 85% lower input costs and 80% lower output costs, and 100+ new languages coming to Gemini 1.5. #GoogleCloud #VertexAI #Gemini

6

Amazon MemoryDB: A fast and durable memory-first cloud database with Redis protocol support https://lnkd.in/eJUwwuR6

8

Distributed Stateful Workflows. The theory and the concepts behind modern-day distributed software. My new post. First of two, in fact, and both are out now. This one is focusing on the fundamentals of building stateful distributed systems. https://lnkd.in/gsr7UEVc

17

2 Comments

Steph Gooch recently released a great article about Spring Cleaning your AWS Costs. Her article was great, and emphasizes removing idle resources (S3 buckets, EBS snapshots), and creating a prioritized list. 👇 Rahul Subramaniam and myself discussed it on our stream last week: https://lnkd.in/g885fzVC Steph's article: https://lnkd.in/gJHCs2Mk At CloudFix, we have automated finder/fixers for most of the points that Steph brought up. - Enable S3 Intelligent Tiering - https://lnkd.in/g3pkkZvq - Get rid of unused AMIs and EBS volumes - https://lnkd.in/g6Swa-xU - RDS Multi-AZ Deployments that could be Single AZ - https://lnkd.in/gACUrhB3 - Move managed services to graviton - https://lnkd.in/gfbh7v7Z & https://lnkd.in/g3qeQ-5q If you need to do spring cleaning in your AWS account, consider CloudFix your Roomba - always looking for bits to clean up so you can spend your time doing something else (like migrating to serverless or deploying AI)! #aws #costoptimization

15

In the "I Know You Think You Know What I Said But That Wasn't What I Said" category comes news from Intuit that it's laying off 1,800 people. What caught my eye were comments from the company's CEO, published on the firm's blog. After a discussion of the need for the layoffs, in a section titled "Taking care of our people" came this statement: "It is never easy to say goodbye to our colleagues and friends and we are committed to treating everyone who has been impacted by these changes with the compassion and respect they deserve." Pretty standard for the 20th century, no? But that was followed by some bullet points including this one: "We’ve significantly raised the bar on our expectations of employee performance, resulting in approximately 1,050 employees leaving the company who are not meeting expectations and who we believe will be more successful outside of Intuit." HUH? So much for "treating everyone...with the compassion and respect they deserve." Truly feel sorry for a lot of the people impacted. I'm sure that, out of the 1,800 people impacted, there were some "underperformers." But calling them out in public like that? Not only was this the wrong thing to do, it actually demonstrates the shortcomings of the executive team for a # of reasons: 1️⃣ Did no one (other than the CEO) have any sense to proof read the blog post before it was published and maybe suggest a lighter tone regarding the people who are "not meeting expectations"? 2️⃣ More importantly, check out the phrase "we’ve significantly raised the bar on our expectations of employee performance." This implies that management has been operating with sub-par expectations of performance. So whose fault is it if 10% of the people in the organization are suddenly found to be "not meeting expectations"? Answer: Management's. Then, a section titled "Moving forward together" started with this: "By 9:00 AM PT today, all employees impacted by these changes will receive a calendar invitation from their manager titled “Leaving Intuit Discussion”. Those who do not receive an Outlook invitation by 9:00 AM PT can assume they are not impacted and we ask that you support your colleagues as they process this news." Never mind that this isn't exactly "moving forward together." Can you imagine the # of emails and phone calls that this must have kicked off with everyone at Intuit calling and emailing their colleagues to find if they were impacted, and if not, who was? I'm no HR expert, but there had to be a better way to announce this.

214

69 Comments

As cloud and AI adoption continues to grow, Amazon Web Services (AWS) reported impressive Q2 2024 results, with revenues up 19% YoY to $26.28bn and the cloud division now accounts for 18% of Amazon's total revenue and a whopping 63% of its income. "We're continuing to make progress on a number of dimensions, but perhaps none more so than the continued reacceleration in AWS growth," said Andy Jassy, Amazon President & CEO. AWS's focus on AI, including its $4bn investment in Anthropic and the launch of Amazon Q, is paying dividends, with the company leveraging its broad functionality, security and AI capabilities to maintain its lead in the cloud market. "Gen AI is going to disrupt and change almost every industry and customer experience in the world," comments Matt Garman, CEO of AWS. "It's an area where we have invested heavily, and will continue to do so." #CloudComputing #AI #AWS #Technology

8

In this panel session, explore the trends and prediction of Open Source technology and how OCI supports Open Source workloads with differentiating performance, security, cost benefits through innovation. https://lnkd.in/g-UekgEC

2

1 Comment

In this week's Tech & Tales newsletter - 1) News - This Tech Giant laid off 15000 employees 2) Tech - How to manage Data Privacy on Cloud? 3) Career - I am confused with so many AWS careers & certifications? Which one is right for me ? 4) Life - An unplanned trip #tech #weekly #newsletter

11