Transforming AI Experiences at the Edge with a System-Level Approach
The rapid evolution of AI on edge devices like laptops isn’t just transforming everyday life; it’s changing the way we think about how developers can exploit rapidly changing models to continue delivering amazing, world-changing experiences.
Developers want to land their AI-based innovations quickly at the highest possible performance. This requires a range of heterogenous capabilities delivered in a compute platform.
The criticality of CPUs
The CPU is highly accessible and easy to program, making it ideal for developing applications. And continuous investments in the Arm architecture—the foundation of the Arm CPU—have led to the development of critical features for accelerating AI computation. These efforts include dot product instructions, support for the BFloat16 format, matrix multiply instructions for Int8 and BFloat16, and the powerful SVE2 vector extension.
These advancements extend the capabilities of the latest Armv9 Cortex CPUs for applications requiring more compute performance for data processing and advanced workloads. This relentless focus on fast-paced architectural evolution prepares the developer ecosystem for ever-changing compute demands in the age of AI.
GPUs accelerate a broad array of parallelizable computations and for developers the AI benefits are accessed via APIs. Both the CPU and GPU can be seamlessly integrated with neural processing units (NPUs) in a heterogeneous system on chip (SoC) environment.
NPUs are tailored to efficiently process the operations commonly used in neural networks, such as matrix multiplications and convolutions. This specialization as an offload engine allows them to perform these tasks quickly to deliver enhanced AI performance.
However, compared to CPUs and GPUs, NPUs are complex to program and integrate into SoCs, and have limited flexibility since they are designed for specific AI tasks.
Enabling the growing developer ecosystem
One area benefitting from AI innovation is the current PC market, which is witnessing a rapid expansion of AI-based applications and features, enhancing a range of productivity and creativity tasks. AI-powered smart assistants, generative AI, and advanced AI workloads are transforming the imaging pipeline, a fundamental feature for PC video conferencing. This evolution is laying the foundation for the future growth of AI PCs.
Meanwhile, AI workloads will run more efficiently and faster within the memory bandwidth of PC designs via more mature and smaller LLMs, like Meta’s Llama3 and Microsoft’s Phi-3. Arm is already demonstrating how these new models can run entirely at the edge on the Arm CPU, with the impressive performance enabled by existing CPU instructions for AI and dedicated software optimizations.
Supporting Arm-based PCs’ strong hardware and software capabilities is a growing developer ecosystem. Thanks to the expanded end-to-end Windows Arm-native developer toolchain, more Windows developers are creating Arm-native builds of their applications. This is being driven by vendors of leading applications, including Adobe Photoshop, Dropbox, Firefox, Microsoft Edge, Office, Teams, Visual Studio, Netflix, Spotify, WhatsApp, and Zoom, all providing Arm-native builds. These investments translate into tangible application improvements, such as improved startup times and smoother performance.
Arm CPU at the forefront of AI
As the AI landscape continues to evolve, the importance of computing efficiency and design flexibility will only grow. The Arm CPU, with its continued advancements in AI capabilities, strong ecosystem support, and inherent advantages in latency and memory locality, is well-positioned to remain at the forefront of AI enablement in consumer devices.
By adopting a system-level approach that leverages the strengths of the CPU alongside other specialized processors, the industry can continue to push the boundaries of AI innovation, while delivering faster, more responsive and more secure AI experiences to users at the edge.
The rise of AI-enabled edge devices and PCs represents an exciting new era for the market, where really the only gating factor to transformative innovation is imagination. Arm is continuously evolving technologies that enable AI wherever computing happens.
For more information, please see our AI solutions page.
Any re-use permitted for informational and non-commercial or personal use only.
Editorial Contact
Latest on Twitter
Ready to upscale your mobile game development without losing performance?
👉 Say hello to Arm Accuracy Super Resolution
It's our open source solution derived from FSR2, designed to increase the resolution of images from low to high with our efficient shader code:… https://twitter.com/i/web/status/1811061017394184322
.@AWSCloud Graviton4 is here! 🙌
Announced in GA, Graviton4 uses Arm Neoverse V2 to provide the most powerful, energy efficient AWS processor to date.
It follows 5+ years of collaboration on Graviton to enable CPU silicon innovation in the Amazon Cloud: https://okt.to/UWM0FR
Proud to partner with @AlzheimersBRACE to support dementia research and foster community. 🤝
From running the Bristol Half-Marathon to raising £2K+ for charity and spreading awareness, shoutout to our team for making a real impact. 💙
Not sure where to start with running Llama models? We've got you. Our 🆕 learning path teaches you how to build and run a simple Android Chat app with different Llama models using ExecuTorch and XNNPACK.
Dive into the step-by-step guide in the link below.
⚡ Silicon is defining and revolutionizing technology.
With the help of our partners, we're at the forefront of its evolution, driving innovative new approaches to silicon design with CSS and chiplets to define the future of computing.
More in @WSJ. 👇
Paid Program: The Evolution of Silicon
From Moore’s Law to today’s modern designs for AI, silicon is defining technology markets now and in the future.
okt.to
Arm CSS for Client is unleashing the power of the Armv9 architecture for AI innovation.
CSS for Client delivers physical implementations across Arm CPUs and GPUs, simplifying SoC builds for the fastest path to production silicon.
Learn more👇 https://okt.to/Q3yLjc
The advancements in large multimodal models and GenAI herald a new era in AI robotics.
That’s why tomorrow's robotics are harnessing the power of LLMs to improve energy efficiency, security, and functional safety.
🦾 And it's all happening #onArm: https://okt.to/6nbZ9m
The world's leading AI companies are prioritizing data center efficiency to scale and grow. And they're doing it #onArm.
Arm CMO Ami Badani joined The Artificial Human Podcast to discuss why efficiency is crucial to driving AI innovation.
The Artificial Human - How green is my AI? - BBC Sounds
Aleks Krotoski and Kevin Fong ask, how green is my AI?
okt.to
Here's a glimpse of our #INWED24 celebrations in Kenya, Ghana and Switzerland.
The Arm-Ambassador-led events brought everyone together to celebrate the significant impact of women engineers and to inspire future innovators to help bridge the STEM gap: https://okt.to/9ZAuna
Find $ARM in the Nasdaq-100 Index from today. 🙌
As we go from strength-to-strength as a public company, we're excited to join industry leading companies on the @Nasdaq stock exchange, emphasizing the importance of Arm tech among the global ecosystem: https://okt.to/gpRsAe
Create the best possible AI experiences with KleidiAI. ⚡
KleidiAI provides a collection of highly optimized AI kernels that can be embedded directly into popular AI frameworks as part of Kleidi Libraries.
Ready to reap the benefits?
🥳 Our partners @HPE and @NVIDIA are making turnkey private-cloud AI a reality!
Announced this week, NVIDIA AI Computing by HPE brings the power and efficiency of Grace Hopper Superchips, built #onArm, to the HPE ProLiant Compute DL384 Gen12 server.
HPE ProLiant Compute DL384 Gen12 | HPE Store United Kingdom
okt.to
For #WorldRefugeeDay, we're celebrating our partnership with @Refugees & @UNRefugeesUK to ensure AI tools for humanitarian efforts are used responsibly.
By supporting @UNHCRinnovation, we can ensure diversity is built into tools from the ground up:
We're putting the foundations in place for a thriving, diverse, chiplet ecosystem in Automotive.
From participation in @imec_int's Automotive Chiplet Program, to our role in SOAFEE, here's how we're ensuring interoperability and performance for chiplets: https://okt.to/eBfYgd
Here's to our fantastic, award-winning Internal Comms team. 👏
At the #ICEAwards, the team took home 3️⃣ awards for:
⭐ Best event
⭐ Best Internal Comms from the tech, media and telecommunications sector
⭐ Best Internal Comms Campaign in a large business under 10,000 employees
Rene Haas joined the Samsung Foundry Forum to discuss our almost 30-year partnership and the possibilities of Arm CSS.
By removing the complexity from SoC development, Arm CSS enables our partners to move faster in the age of AI, with the support of partners like Samsung.
Sentiment analysis is a vital AI technique that figures out emotions and opinions from written text.
With @HuggingFace's pipeline running on Arm Neoverse CPUs, we're making sentiment analysis quicker, with more impactful insights.
👉 See for yourself:
The next generation of Windows on Arm is here. ✨
We caught up with @sinclairinat0r at Microsoft Build to get his take on the latest Windows on Arm AI capabilities and how they are making AI approachable and easy to use.
From the moment he saw his first computer, Samuel fell in love with technology.
Now, it extends to his work at @Simprints, where their biometric solution built #onArm is enabling faster patient tracking and healthcare interventions.
This is how technology can change lives.
🆕 Slack is Windows on Arm ready! 🥳
Available now in beta, the ARM64 version of @SlackHQ lets you maximize performance on Arm-based hardware.
This creates the opportunity for new AI experiences running efficiently on Windows on Arm devices.
More: https://okt.to/oqMg25
Congrats to the winner of the 2024 Arm Flexible Access for Startups Contest, Agate Sensors! 👏
Chosen for its spectral imaging technology that paves the way for edge tech, the team will receive up to $250K of Arm credit to build next-gen SoC solutions: https://okt.to/Fbqrph
No other ecosystem has Arm's level of diversity & collaboration in its supply chain.
It's why Arm Neoverse is best suited to drive the transformation of AI-ready cloud data centers, together with our partners.
Eddie Ramirez shares more at #COMPUTEX2024: https://okt.to/WFqu7R
"We are at a magical time in our industry relative to the growth potential"
Following the biggest IPO of 2023, Rene Haas speaks to the @FT on the AI opportunity for Arm, discussing why the CPU is crucial, sustainable development & what's next for Arm CSS.
Rene Haas: ‘Arm has the most ubiquitous computer architecture on the planet’
Chip designer’s chief executive talks about diversification and how AI is changing the devices we use
okt.to
Our Architecture Formal Team has won the @RAEngNews Colin Campbell Mitchell Award, paving the way for more reliable computing systems.
Their ground-breaking approach to formal specifications enables efficient & secure Arm-based computing systems. https://okt.to/m8y1Qp
