Matthew Garrett

Join us in Irvine on July 17th at 6:30 PM for our latest OC ACM talk, "Systems and Software for Averting an Energy Crisis" presented by Bill Gervasi, a Principal Systems Architect for Wolley Inc. For more info or to RSVP click here: https://lnkd.in/gquYQ2z9 According to a recent Forbes article, "The rise of generative AI and surging GPU shipments is causing data centers to scale from tens of thousands to 100,000-plus accelerators, shifting the emphasis to power as a mission-critical problem to solve." making this talk particularly relevant. #acm #techtalk #gpu #ai #generativeai #nvidia #energy #energycrisis

3

2 Comments

What’s new in ChromeOS "In addition to providing thorough documentation and guidance, we’re working on making on-device testing and debugging easier by improving how ADB over USB works. We’re doing this through the USB 3 specification for debug capability, known as DbC. This will make ADB over USB on ChromeOS easier to use, and available on a broader range of Chromebooks." https://lnkd.in/gHNMQv_y

16

1 Comment

In light of the July 4th holiday week, I will be publishing Series 3 of "Why Modernization is Hard" next week. In this series, I will discuss the hidden dangers posed by compiler options and their impact on modernization efforts.

25

1 Comment

I'm starting to worry about the insurgence of AI in hardware design cycles. The claim is AI will "solve systems problems and offer chips-to-system intelligent design acceleration." First, many small company management teams still fail to see the need to bring on-board, *in a timely manner*, the experienced staff needed to get it right first-time. The industry thinks moving to a model dependent on ODM houses is a good idea, but like a stubborn mule ODM houses need to be checked, poked and prodded to produce good designs. Like a good mule skinner, experienced in-house staff are needed to keep them in line. Second, finding good package/board designers is getting harder since new college grads are going into ML and Data Science, and leaving the physical sciences needed for electric circuit design. The consequence of this will be seen in new products which must go thru the school of hard-knocks, i.e. they won't work first-time. Third, vendors are now offering to solve these problems by bring this "expertise" in-house through AI, promising to "accelerate design" with "artificial intelligence". To me, looking forward, this looks like a recipe for disaster. Execs holding the purse will ask why they need experienced engineering staff when AI will show the new kids how to do it. The new kids won't know AI is as dumb as a bag of hammers, won't know how to vet its answers, suggestions and directions, won't know when to call BS. And the savior - AI - will screw it up with *no accountability*. Schedules will slip, budgets will be blown, market opportunities will be lost. There will be weeping and gnashing of teeth. Meanwhile the experienced package/board designers are retiring, leaving a gaping hole in the industry. No amount of AI will resolve this in the next few years.

38

7 Comments

Alastair captures one of the biggest problems with the current RISC-V spec. When I looked into this (formal spec and compliance) a while back for personal interest I was (almost) shocked at the lack of rigor and state of the tooling. I say almost because I expected it, but was still sad. What saddens me more is that this situation extends to implementation validation. The status quo is sufficiently non-rigorous that if you were building an implementation you would have to do most of the architecture validation tooling lifting yourself. This will result in a few very rigorous designs (from folks who get the problem), and at least some that are not (folks who make a mess). I expect as a result an escape in a large scale design that hits the market and breaks compatibility. This stuff really matters. Read Alastair’s blog here, then remind yourself there’s a reason why some specs are thousands of pages long:

50

4 Comments

thinking about the r10k - mipspro was amazing. just look at the verbose output from the software pipelining pass. i think there's pldi paper on a showdown between this algorithm and some ILP-based scheme. i should read again. .BB16.naive_matmuli: # 0xb6c  #<loop> Loop body line 263, nesting depth: 3, estimated iterations: 256  #<loop> Not unrolled: estimated schedule improvement < 10%  #<loop> Not unrolled: disable analysis w/-CG:unroll_analysis=off  #<swps>   #<swps> Pipelined loop line 263 steady state  #<swps>   #<swps>  512 iterations before pipelining  #<swps>    Not unrolled before pipelining  #<swps>  18 cycles per iteration  #<swps>   3 mem refs   ( 16% of peak)  #<swps>   7 integer ops ( 19% of peak)  #<swps>  10 instructions ( 13% of peak)  #<swps>   1 short trip threshold  #<swps>   8 integer registers used.  #<swps>   #<swps>   6 min cycles required for resources  #<swps>  17 cycles required for recurrence(s)  #<swps>   6 operations in largest recurrence  #<swps>  18 cycles in minimum schedule found  #<swps>  #<swp>   #<swp> Pipelined loop line 263 replication 0  #<swp>   #<freq>  #<freq> BB:16 frequency = 56481544.00000 (heuristic)  #<freq> BB:16 => BB:15 probability = 0.99610  #<freq> BB:16 => BB:18 probability = 0.00390

4

1 Comment

Having lived in SF bay area for so long, I came to know the big coffee culture scene here. I was totally consumed by it a while ago and spent years trying to attain the “perfect" latte. I published a fun coffee story and my entire home setup, which took weeks of research and multiple iterations, in my blog link in bio. Meanwhile, to all fellow coffee loving tech peeps, here are some top tips that no one told me. Put it in practice at home or in microkitchens. - A big factor in preparing milk suitable for latte art is pressure of the wand. If this is important to you, you need to get a machine that has a strong enough pressure. - When pulling shots, put in 18g of beans and extract 30g of espresso. - The darker the roast, the coarser the setting should be on the grinder, the lighter the roast, the finer the setting should be. - When adjusting the grind setting, do so while the grinder is working if there are beans in the hopper. Otherwise, a bean can get stuck between the burrrs. - Pulling a shot should take 20-30 sec in time, or based on observation. It should look like caramel going out, not a waterfall. - Eventually work your way to using a bottomless portafilter. It punishes you for all the small imperfections when preparing your coffee puck, like any tilt, and uneven distribution. It does so by spraying the espresso all over the place. The more you can get it to flow uniformly, the better your preparation of espresso, objectively. - Seal your beans in a vacuum, so they don’t go stale. The more stale the beans, the more minor adjustment you need to make to grind finer. - When steaming milk in your pitcher, you can tell if it’s done by holding the pitcher and as soon as the heat becomes unbearable i.e. stings, it’s done. - Taking a latte art course at a local coffee shop was the best boost for me to start nailing latte art.

9

Writing is such valuable skill! ✏️💰 This is a great post that highlights the importance of that. #growthmindset #mastery #hacking

2

An interesting framework to categorize the responsibilities on a DV team. However, IRL - having a methodology engineer who isn't quite up to snuff on the product requirements, and the team profile can end up with a TB that's very difficult to execute in. A coherent methodology is critical to DV execution, but it's not the hill I'd be willing to die on. Verification intent and capabilities always trump methodology.

7

1 Comment

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 Comment

Widespread deployment of AI may be in trouble. John Loeffler, writing on the web site Techradar (https://lnkd.in/gDHWKEjU) published one of the more thoughtful assessments of Nvidia CEO Jensen Huang's keynote speech June 3 at Taipei's Computex 2024. But I think John need not be overly concerned about the speed of deployment of AI. I attended Computex this year and the hype surrounding the coming of AI seems significantly out of touch with a simple reality: there's not enough electrical generating or grid transmission capacity to run all the proposed deployments of new data centers, or "AI factories". Virtually lost among all the hype over Nvidia's new Blackwell GPU is the massive amount of power required to run it. Can someone who was there help fact check this for me? Marc Hamilton from Nvidia while showing an example of a dual Blackwell server blade said with a straight face that a server rack of 72 Blackwell processors required 100KW, along with liquid cooling. I thought I heard wrong at first but then started doing some back of the envelope math. If every Blackwell GPU was running at full tilt in a rack the peak energy consumption needed for the rack plus cooling system just might approach 100KW. Now how many racks will be in a "AI factory"? It will need its own multi-megawatt power plant out back just to run the place. Some other fun facts about data center power consumption today: 30% of the electricity generated in Ireland goes to run data centers there. In 2022 data centers worldwide consumed 460 TeraWatt/hrs. By 2026 data centers at current deployment rates will need 1050 TeraWatt/hrs annually. That's more energy than the entire country of Japan uses today. Where is all this power going to come from? Jensen Huang has proposed building a large AI factory here in Taiwan, where we barely have enough energy in the summer to run all our air conditioners. How is that going to work?

7

4 Comments

AI is so smart it can't test itself🤔🤣 https://lnkd.in/ebpPgpsy

3

Some more light reading! I've read the first edition of Elecia White's book, but I wanted to get the update, particularly as a complement to "Applied Embedded Electronics." Luca Palmieri's Rust book is to reboot my Rust learning. Might as well pick up some backend skills along the way; my last backend work was in C++.

58

2 Comments

Checkout our latest report on how just eight SN40L chips run 1 Trillion parameter model at blazing fast speed https://lnkd.in/grwdFVAi. It was great to collaborate with some of the most talented folks in this area. Here is the link to the demo operating on similar principles https://fast.snova.ai/

28

Disruptive innovation

4

Really interesting to see this: how Europe is moving up. It is interest to contrast this qualitative survey with the #siliconcatalyst analysys that UK is sixth in Europe in funding value fir silicon investment. And I will put in a plug for our #UKTIN Forward Capabilities paper on recommendations for UK Semiconductor sector: https://lnkd.in/eNNAsCAz

7

I wanted to take a few minutes out of my six week trip in China to write up a note on Guy's and my book, The FPGA Programming Handbook and announce the winners of the ebook give away. My favorite chapter. I will go into the books philosophy in a future post, but for now I wanted to talk about Chapter 11, bringing it all together. This chapter was the capstone project for the first book, FPGA programming for beginners, and a major project in the second book, however the second edition goes into more detail on embedded processors and UARTs.  The course of both books involves developing HDL coding skills and the various interfaces available on the Nexys A7. Chapter 11 expands upon previous chapters by interfacing to a PS/2 keyboard and displaying keycodes as they are pressed. We also add temperature sensor data to the VGA display and a graphical display for the microphone input.  This chapter is visual and it gives feedback that is easy to understand based upon the skills you learn. It provides a jumping off point for your own projects. The main goal is trying to demystify practical projects and showing that something like this is achievable for even a beginning design engineer given some guidance and practice. Winners! I was asked to announce the winners of our ebook give away! https://lnkd.in/guwDrqcT https://lnkd.in/gS4BzHTt https://lnkd.in/gZR-wipr https://lnkd.in/g-w-DWxv

24

4 Comments

This is also the XPU using scenario In my mind too. “Whisper on NPU, LLMs on iGPU, document embeddings on CPU! Full use of #AIPC through #OpenVINO to implement scenarios like chat with documents and voice guided assistant that generates code. Everything on our new Lunar Lake platform that was just announced.”

9

Longhorn Electric Racing Is Building the Future of Electric Vehicles with Cadence Tools. Longhorn Electric Racing is a student-run racing team at the University of Texas, Austin. Not only are they building the best race cars, but they’re also building the future generation of engineers. Their electric vehicle team is one of their most recent teams, founded in 2016. Some of the challenges they face in designing an electric car are making sure it can go as fast as possible, making sure it has sufficient cooling, and making models effectively. With the help of Cadence CFD and Cadence OnCloud, they’re reaching their goals much more efficiently. https://ow.ly/phqj50RuqK4

19