Jove Zhong

Ollama is a great tool for simplifying running small language models like Phi-3 and Llama3-8B on your personal computer, even without a GPU or with an ARM chip. But what if your device can't support Ollama? Pamela Fox shares how GitHub Codespaces allows you to open any GitHub repository in a web-based VS Code running a containerized environment. You can add Ollama to the Codespace for a repository by incorporating this community-created feature in devcontainer.json. https://lnkd.in/gqtNR8gY #Ollama #GitHub #Codespaces #msftadvocate

31

3 Comments

I want to highlight a new feature in our latest release also included in our free Community edition ! Offset-Based Retrieval in Kamiwaza's Full Gen AI Stack: Seamless RAG Flow at PB+ Scale! A significant advancement in our Full Gen AI stack at Kamiwaza: out-of-the-box offset-based retrieval support, tailored for Enterprise-scale Retrieval-Augmented Generation (RAG) flow. #### What Does This Mean? - **Seamless Pagination:** Offset-based retrieval enables effortless pagination through massive datasets. This feature lets your enterprise handle enormous data scales, efficiently delivering specific subsets of data based on your application's requirements. - **Optimized Performance:** Our implementation ensures that even with petabyte-plus data volumes, your queries are streamlined for speed. Retrieve exactly what you need, without overloading your infrastructure or consuming unnecessary resources. - **RAG Flow Ready:** Retrieval-Augmented Generation (RAG) is critical for enterprises leveraging AI to augment their workflows. Our stack’s native offset-based retrieval ensures that RAG systems can navigate vast datasets, retrieving relevant information with unparalleled efficiency. #### Why Offset-Based Retrieval? Handling large-scale data is challenging, particularly when your AI systems require real-time responses. Offset-based retrieval allows you to: - **Navigate Efficiently:** Skip irrelevant data and jump straight to the relevant subset for your application. - **Scale Seamlessly:** Manage and manipulate data even at petabyte-plus scale, keeping your AI applications responsive and efficient. - **Maintain Consistency:** For use cases like search, recommendations, and dynamic data exploration, consistency is key. Offset-based retrieval helps manage this seamlessly. #### What to Expect? With Kamiwaza’s Full Gen AI stack, enterprises can: - **Empower AI Workflows:** Quickly and efficiently support retrieval-augmented workflows, ensuring your AI models access the most relevant data. - **Enhance Decision Making:** Our optimized data retrieval enables faster, more informed decisions driven by the power of your data. - **Streamline Operations:** Offset-based retrieval reduces the complexity of managing vast data landscapes, allowing your teams to focus on deriving insights. We believe that this feature will significantly enhance the performance of our AI stack, helping enterprises unlock new levels of productivity and data-driven insights. #GenAI #1TrillionInference #KamiwazAI #EnterpriseAI Check out https://lnkd.in/gAwbGdGH for the latest updates!

3

3 Comments

This presentation that I gave last week at an HDRN Canada meeting provides a summary of our work thus far on synthetic data generation. It reflects the lessons learned about how to apply synthetic data generation methods effectively and about what works and doesn't.

41

ONNX is an open source machine learning (ML) framework that provides interoperability across a wide range of frameworks, operating systems, and hardware platforms. This new blog post shows you how to run ONNX Runtime inference on #AWS Graviton3 and how to configure to use optimized GEMM kernels. With these optimizations, you can see up to 65% better performance for fp32 inference and up to 30% better performance for int8 quantized inference for several natural language processing (NLP) models on Graviton3: https://lnkd.in/giWYk6Hq

116

4 Comments

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

My team at Sunlife Global Solutions have recently embraced Azure DevOps and it has dramatically streamlined the CI/CD pipeline process from planning to testing, and deploying. It's simplified our workflows, reduced overheads, and boosted productivity. It's not just about efficiency, but also about creating a more enjoyable and less cluttered work environment. I've documented all my experience in this article. This is a work in progress and would not be surprised if Azure DevOps continues to amaze me.   #AzureDevOps #DevOps #SoftwareDevelopment #SLGS #SunlifeGlobalSolutions

26

Tradescoop is built on a headless architecture. Here’s the client REST API v0.1.0: tradescoop.github.io . You can interact with Tradescoop data via the api. Some endpoints require a bearer token you can obtain when you signup.

3

New Webinar = New things to learn. Generative AI man. It's going to make scaling and development so much easier and more concise in the future/present. I can't wait to see the other cool stuff Amazon Q can do at the 2024 Toronto AWS summit this year #DevOps #GenerativeAI #AWS

6

The recording of my talk on how to develop software like a bureaucrat is now published. https://lnkd.in/gQtX-76q

3

2 Comments

Thr `jets exec` command runs bash commands in the deployed AWS Lambda env. It’s like a REPL for lambda. It’s useful for debugging https://lnkd.in/gQ4Gnfig #rubyonjets #rails #aws #serverless

2

1 Comment

Well NVIDIA is now worth more than the GDP of Canada and in fact, more than 14 of the G20. So what is behind this insane growth and valuation? 1. Exponential requirements for growth. If we look at "GPT5" the needs will be far more than GPT4. From GPT2 to GPT4 OpenAI progressively added orders of magnitude more computing power to calculate the models. Rough estimates are GPT4 used 3,000 to 10,000x the compute power compared to GPT2 2. Data centers aren't nearly big enough...by a lot. GPT4 benefitted from having a lot of excess compute in existing data centers. But the next models will need compute thousands of times bigger. Think $100b or $1T data centers. 3. Google, Microsoft, Amazon are investing billions (>10b each) to grow the data centers to meet the demands. And now I'll let you guess who is supplying those chipsets.... If you are looking for some details on this, Leopold Aschenbrenner of situational-awarness.ai wrote an excellent article on calculating computational requirements. But hey, keep your head up Canada, at least we've got a team in the NHL playoffs. https://lnkd.in/gadhkfxf

3

5 Comments

Awesome announcement from build this year on .NET and C# AI Integration: .NET 9 introduces AI enhancements, including a new Tensor<T> type for AI applications and seamless integration with AI libraries like OpenAI, Qdrant, and Milvus. .NET Aspire: A new release for building cloud-native distributed applications, offering tools, templates, and NuGet packages for efficient development1. C# 13: The latest version brings improvements to C# features, focusing on flexibility and performance. Performance: .NET 9 aims to reduce memory and execution time with critical benchmarks, enhancing overall performance. More here : "https://lnkd.in/g_h-kvVx"

24

1 Comment

Istio Service Mesh: great to see Ambient moving up to Beta, delta xDS becoming the default, and api-gateway becoming a first class citizen in Istio. https://lnkd.in/gYPmnTuj

32

Just getting around to the last bits of Christopher L. Trudeau's Django in Action for Manning Publications Co. that I hadn't read yet. This text is a wonderful companion to the stellar Django documentation, and brings with it approachable and compelling considerations for why to work on the web, how to think about the organization and scale-out of your project, and more. If you're someone who, like me, wants to spin up quickly (or, in Christopher's words, using the "minimum understanding of technology" approach) and you're picking up Django, this book is definitely one for you. Read it front to back or use its clear layout as a reference as you build. https://lnkd.in/eug_KjEx #python #django #webdev

19

2 Comments

New Pod! Ian Gartley, Principal Engineer from Varo Bank, formerly of Meta and RBC, joins the podcast. Topics below, timestamps are approximate: https://lnkd.in/gmZt6mSk 00:00 - Putting “Ex-Meta” on LinkedIn Profiles 02:15: What makes a good intermediate developer hiring interview question? / MVP Architectures 11:30 - Challenges with MicroServices / Convenience of Sidecars 23:30 - Orchestration Layers, GraphQL and Distributed Queries 32:00: Why does it take 10 different engineers to make one change? 49:15: Functional programming languages encouraging better testing patterns / mocking private functions / creator/mutator pattern 61:03: Whose job is it to write requirements? Follow the podcast: LinkedIn: https://lnkd.in/eEjH9H3 Apple: https://lnkd.in/esqYRQA Spotify: https://lnkd.in/e7nHECd RSS: https://lnkd.in/e2Qf8HV

30

Unapologetically Technical's newest episode is now live! In this episode of Unapologetically Technical, I interview Hubert Dulay the author of Streaming Data Mesh and Developer Advocate at StarTree. We talked about his early experience with web backends like CORBA and SOAP and how those prepared him for data work. He shares his advice for those with web development skills to transition into data and what it's like for a person leaving a company after a long tenure there. We discuss his time at Cloudera and Confluent and how the data industry has changed over time. We talk about his recent experience with Flink and StarTree. We go in-depth into streaming data mesh, covering what it is and how to create real-time data products. We discuss the ways teams can create joined or enriched data products across domains. We round out the discussion with some tips on how to get data mesh adoption. Watch the full episode here:

10

1 Comment

It was awesome to get to interview Kat Cosgrove about the 1.30 release of Kubernetes, which is out now! Check out the episode to hear what's new in "Uwubernetes"! uwu 😆

41

Interested to know how Coroot can improve your observability ? Here is the list of Coroot's most amazing features! https://lnkd.in/emDrcgX9 #coroot #opensource #observability

4

Object storage direct-write Kafka topics are coming to Confluent. In data system design, cost is a kind of unstoppable force that rewrites architectures every decade or so, and that is happening right now for high-volume streams. https://lnkd.in/dHRFnrMB

39

1 Comment