Brent Gorda

The Canadian government is investing in its thriving AI ecosystem and has opened the door to public consultation. Let's take this opportunity to emphasize that nobody can establish leadership in the future of AI without cutting-edge silicon. Differentiated compute and connectivity chips are the key. In fact, Canada punches way above its weight when it comes to AI. -> 3% of the world's top-tier AI researchers (6th rank globally) -> 1st in the G7 for AI-related scholarly output per capita since 2019 This is a generational opportunity for Canada to double down on one of its major strengths and forge ahead in the most important technology of our age. We're doing world-beating R&D in all aspects of AI hardware. Question: Is a completely Canadian-designed AI compute cluster possible? https://lnkd.in/g6MkGJBR

130

6 Comments

Thought provoking research paper- Enhancing the ubiquitous FPGA BRAMs with in-memory compute-capabilities. Specifically note, the "proposed architectural change results in 1.6× and 2.3× increase in the peak multiply-accumulate throughput of a large Stratix 10 FPGA, at a minimal cost of only 1.8% increase in the FPGA die size and no change to the BRAM's interface to the programmable routing." In practical terms, this work proposes enhancing the capabilities of FPGA (Field-Programmable Gate Array) on-chip memory, specifically BRAMs (Block RAMs), by enabling them to perform computational tasks in addition to storing data. This enhancement aims to significantly increase the efficiency and performance of data-intensive applications, such as deep learning inference. For example, in the field of deep learning, the proposed architecture, RIMA, could enable faster and more efficient execution of inference tasks on FPGA devices compared to traditional methods or even state-of-the-art solutions like GPU-based processing. This advancement could have significant implications across various industries where deep learning inference is applied, such as healthcare for medical imaging analysis, autonomous vehicles for real-time object detection, or finance for fraud detection in large datasets. Essentially, it allows for faster and more efficient processing of complex tasks, which can lead to improved performance and potentially new capabilities in a wide range of verticals.

1

"A study published in Opto-Electronic Science discusses high-intensity spatial-mode steerable frequency up-converter toward on-chip integration. Integrated photonic devices consisting of micro-lasers, amplifiers, optical waveguides, frequency converters, and modulators on a single chip, enabling control over photon's spatial modes, frequencies, angular momenta, and phases, are essential for preparing high-dimensional quantum entangled states, high-capacity photon information processing, all-optical communication, and miniaturization of photonic computing. However, current nonlinear waveguide devices, integrating spatial modes and photon frequency conversions, heavily rely on external optical path control and spatial light modulators, failing to meet the crucial requirement of on-chip integration for photonic devices." #photonics

1

Synopsys Inc was a spinout of GE exiting the semiconductor business, much like ASML was a spinout of PHILIPS A lot of the great semiconductor companies were started this way (I don’t know any others) 1. When using AI to design chips, they emphasize absolute correctness, which is different from generative AI that looks like 🪄 inside a SaaS app 2. TSMC and Synopsys were founded within three months of each other, marking the start of an era where fabless giants like NVIDIA AND Broadcom (not you AMD) could flourish. Even Jensen says Synopsys is “mission critical” to their success. 3. In order to stay on an exponential (Moore’s Law) you need to race with people who are crazier than you. Customers like Jensen Huang or Andy Grove who are always paranoid of going out of business. Even though i’m the biggest Intel Corporation supporter, it’s not a surprise that the extremely nonparanoid Brian Krzanich (who was paranoid about the wrong things cough cough) was not able to lockdown the HPC market even when Intel had 99% market share in data center at times. 4. Synopsys and TSMC are so hard to disrupt because they are selling an entire ecosystem of support and decades of experience, where companies like NVIDIA, Intel, and AMD are selling compute at the end of the day. Obviously this is changing as NVIDIA is going full stack with networking as well as end user AI services like inference and model training. 5. Transistor density per area is tapering off, the next 100x comes from 3D packaging like CoWoS, or the best in class Foveros from Intel Corporation, software optimizations, and more specialized hardware, not to mention further density and PPAC from future nodes like Intel Corporation 18A, 14A. These improvements stack, so 2-3x improvement in each area can get 100x or more from here. What does this mean? 1. Samsung has terrible 3nm yields and their product teams have switched from samsung processors and nodes. We can count them out of the foundry race, especially with them needing to focus on their memory division which SK hynix is winning. 2. Use Intel Foundry Services From Acquired #ai #nvidia #hardware #intel #amd #openai #broadcom #samsung #gpu #compute #tsmc

3

1 Comment

Amazing article by Pushkar Ranade on the chip wars and the impact of AI on everything in the computing stack. It includes some gems of quotes too: “…it seems like the world is going to be very compute constrained for a while and I think almost all of the datacenters around the world will over time be AI instead of conventional CPUs…if you ask what percentage of the energy (in datacenters) is being used for neural nets, it's going to over time be probably 80-90%...” Elon Musk, January 2024 “There’s about a trillion dollars’ worth of installed base of data centers. Over the course of the next four or five years, we’ll have two trillion dollars’ worth of data centers that will be powering software around the world.” Jensen Huang, January 2024 https://lnkd.in/gZP2uXUz

14

3 Comments

OpenAI and Broadcom in talks about developing new AI chip OpenAI is in talks with Broadcom and other industry leaders to develop a new chip tailored for AI applications, potentially reducing reliance on NVIDIA #GPUs. Sam Altman has been instrumental in spearheading efforts to secure partnerships with #chipmakers, tech giants like Microsoft, and government bodies to enhance #compute capacity essential for #AI #innovation. This move underscores OpenAI's commitment to advancing artificial general intelligence (#AGI), aiming to push the boundaries of AI capabilities beyond current standards. Despite the formidable challenges and substantial financial requirements, including Microsoft's substantial investment, OpenAI remains focused on expanding access to AI benefits through strategic partnerships. For more information: https://lnkd.in/dYHej2Kc #AI #ArtificialIntelligence #OpenAI #TechInnovation #Semiconductors #Broadcom #Microsoft #AGI #AIHardware #TechPartnerships

4

1 Comment

Intel Corporation launches new software-enabled file sharing between two Thunderbolt systems. Finally - the industry (and Intel) finally acknowledges most of us use two computers and struggle to find a good way to move files easily. While Intel offers no solution to those still stuck using traditional USB connections, and no solution for those who have more than 2 systems (average in US is 3.5 btw), its good that progress is being made. IntelliConnex Consumer Electronics supports 4 to 24 systems exchanging files over a USB-C connection.

2

🌐CDL-Vancouver Compute Stream Spotlight Week With over 35 mentors in the Compute Stream, the room is filled with experience and expertise spanning across a variety of industries & domains, from advanced computing systems, spatial computing, web3, and more. Meet a few of our Compute mentors: Tucakov, Vladimir: Co-founder of Point Grey Research, a leading company in digital camera design and manufacturing. Point Grey grew to 300+ employees globally. Through the growth, Vlad oversaw functions from design & sales to finance & CEO responsibilities. Post-acquisition by FLIR Systems in 2016, he joined them as GM of Integrated Imaging Solutions. Having left FLIR, Vlad now pursues personal interests including sailing, skateboarding, and advising tech startups. Andrew Fursman: CEO & co-founder of 1QBit, and co-founder of Minor Capital, and Satellogic. In 2022, 1QBit launched two new ventures: Synthesis Health for healthcare machine learning, and GOOD CHEMISTRY for advanced materials. Andrew enjoys kayaking and microbiology experiments on Vancouver Island. Jeff Booth: A Founding Partner at SCG Digital Private Limited (formerly known as Oitolabs Technologies Private Limited) and co-founder of addy and NocNoc. Previously, he co-founded and led BuildDirect (TSXV: BILD) for over 20 years. Jeff chairs and advises several companies, and authored "The Price of Tomorrow," a best-selling book challenging economic norms. He received the BC Technology 'Person of the Year' Award in 2015 and was recognized as an intriguing Entrepreneur by Goldman Sachs in 2016. Kim Kaplan: CEO & founder of Snack - Gen Z Dating, a video-driven dating app. Previously, she worked at Plenty of Fish (POF), helping substantially grow Daily Active Users and revenue. Kim's leadership contributed to POF's acquisition by Match Group in 2015. Post-acquisition, she focused on investing, advising startups, and serving on the board of Tangerine before returning to lead Snack, driven by her passion for scaling companies. Jenny Yang: Partner at Panda Angel Partners and senior advisor at MaRS Discovery District, co-founded and led two AI companies—Aida (acquired by Alida) and Metafor Software (acquired by Splunk). She previously served as a partner at BDC, focusing on technology startup investments. Edoardo De Martin: CEO of Industrio AI, he focuses on AI and DataOps services. He has 25+ years of experience in software development, specializes in technologies like holographic applications and AI-driven digital twinning for agriculture. He's known for expanding Microsoft's Vancouver office and leading projects in mixed reality. Previously at Electronic Arts (EA) Arts, he developed Need for Speed Underground. See the full list of Compute Stream mentors here: https://lnkd.in/gFQ9K3D3 👉Ready to apply to the 2024/25 Compute Stream? Apply today, applications close July 26th 2024: https://lnkd.in/ez43f-2

50

3 Comments

Explore Geneformer, a cutting-edge #AI #deeplearning model, now available through #BioNeMo framework on NVIDIA NGC. Pre-trained on extensive transcriptome data, it uses attention mechanisms to focus on crucial #gene interactions, enhancing #drugdiscovery and #geneticresearch.

6

Can you really speed up a CPU by 100x with some added "smarts"? The concept is intriguing - perfectly scheduling instructions and hiding memory latency. VLIW aimed to achieve this through efficient instruction packaging and multiple threads, like @Cray XMT's massively multi-threaded architectures. As someone who has worked on processor design, I understand the challenge of making this work practically with existing programs. That's why I'm somewhat skeptical of claims made by Finnish startup Flow Computing. However, I'm open to being convinced otherwise. Read more about Flow Computing's parallel processing accelerator and share your thoughts in the comments: https://lnkd.in/gRBYbBvS #computerarchitecture

4

2 Comments

Cerebras Systems - Pioneering Wafer-Scale AI Computing (2 min read) Cerebras Systems, based in Sunnyvale, CA, is pioneering a new class of computer systems to accelerate AI by orders of magnitude. The company was founded in 2015 and is led by CEO Andrew Feldman, a seasoned Silicon Valley entrepreneur who previously co-founded and led SeaMicro (acquired by AMD for $357 million) and had roles at companies like Force10 Networks and Riverstone Networks. At the core of Cerebras' solution is the Wafer Scale Engine (WSE), the largest chip ever built with 2.6 trillion transistors. The WSE powers systems like the CS-2, enabling breakthroughs across industries. Major customers include TotalEnergies in energy, GlaxoSmithKline, and AstraZeneca in pharma, the Mayo Clinic for medical AI, and national labs like Argonne and Lawrence Livermore. Cerebras has around 335 employees but a major industry impact, with customer commitments approaching $1 billion. With over 400,000 AI cores, massive memory bandwidth, and ultra-fast interconnects, the CS-2 delivers cluster-scale performance in a single system. This allows for cutting training times from months to minutes on complex workloads like seismic modeling, drug discovery, and COVID research. Key verticals are pharmaceuticals/healthcare, energy/industrial, scientific research, and AI/ML. Cerebras removes current hardware constraints, allowing researchers to explore novel architectures and drive previously impossible innovations. As AI complexity grows, wafer-scale computing solutions like those developed by Cerebras Systems may be increasingly important in accelerating AI workloads and enabling new innovations across various industries. Credit: Jeffrey Cooper & Perplexity.ai For more on AI, robots, and Semicon, check out my blog: https://lnkd.in/eWESid86  

14

1 Comment

Great article from VentureBeat about our new nemotron4 model, targeting synthetic data generation. I can’t wait to see what people do with it!

112

Groq, known for their high-performance AI hardware and software, released two new open-source models specifically designed for tool use: Llama-3-Groq-70B-Tool-Use and Llama-3-Groq-8B-Tool-Use, built with Meta Llama-3. Tools are utilized by AI assistants to respond to user prompts. Typically, a tool is an API call to an external system, like Google search. The results from the tool are then analyzed by the LLM and incorporated into the response. Thus, the LLM's performance in using tools is paramount. These models promise significant advancements in processing speed and efficiency. Llama-3-Groq-70B-Tool-Use boasts 70 billion parameters, while Llama-3-Groq-8B-Tool-Use features 8 billion parameters, both optimized for rapid deployment in various applications. By leveraging Groq’s advanced hardware, these models aim to set a new standard in AI tool use, focusing on enhancing both performance and accessibility.

1

Really cool NVIDIA blog that talks about the Cadence & NVIDIA collaboration resulting in substantial improvements in the energy efficiency of scientific compute for semiconductor design, system analysis and optimization. Together, we can develop #AI systems to address the challenges facing our world. Check out the blog here ⤵️ https://ow.ly/uZg950S0VEs

17

What will be the amount of energy to create this large language models (LLMs) of $1 billion AI models that cost $1 billion to train are underway, $100 billion models coming — largest current models take 'only' $100 million to train: Anthropic CEO "Right now, 100 million. There are models in training today that are more like a billion." Amodei also added, "I think if we go to ten or a hundred billion, and I think that will happen in 2025, 2026, maybe 2027, and the algorithmic improvements continue a pace, and the chip improvements continue a pace, then I think there is in my mind a good chance that by that time we'll be able to get models that are better than most humans at most things." https://lnkd.in/dw2KWK2a

2

OpenAI in talks with Broadcom to develop custom AI chip, Altman looks to fund data centers – report CEO Sam Altman has been leading the discussions Summary: OpenAI, under CEO Sam Altman, is in discussions with Broadcom and other chip designers to develop a custom AI server chip, potentially to rival Nvidia's hardware. This move might strain relations with Nvidia, currently OpenAI's primary chip supplier. Alongside chip development, Altman is exploring the creation of data centers to house these chips, backed by investor funding. The data centers would be established through companies created by Altman, with OpenAI committing to rent the facilities. Last year, OpenAI hired Richard Ho, a former leader in chip engineering from Lightmatter and Google, to head their hardware division. OpenAI has also been recruiting a team to collaborate on future hardware designs for improved programmability and performance. Reports suggest Altman has been seeking substantial investment, including from high-worth individuals in the Middle East and chip manufacturer TSMC, to fund his AI chip venture. #OpenAI #AIChips #Broadcom #DataCenters #SamAltman #AIHardware #Nvidia #TSMC #AIInnovation #TechFunding

5

AI workflows are driving demand for memory and storage to prepare data, train the model and create useful inference engines.  This is driving announcements of storage and memory products that support the various elements of these AI workflows.

51

4 Comments

Majorana zero modes (MZMs) are an exciting development in the quest for resilient qubits. These unique quantum states are particularly valuable for Topological Quantum Computation (TQC). TQC is a method of fault-tolerant quantum computation where quantum gates are created through the braiding of topological quantum objects known as ‘anyons’. This approach promises enhanced stability and error resistance, making it a promising pathway towards practical quantum computing. #ocpquantum #quantumcomputing #qubits

13

Geneformer is now available as part of NVIDIA #BioNeMo framework! It's a cutting-edge #AI model trained on extensive scRNA data, using a BERT-like architecture and attention mechanisms. Useful for tasks like perturbation analysis and cell classification.

6