The State of Engineering
The State of Engineering analyzes the state of modern-day engineering organizations in 2024. The report's focus is R&D spend, developer experience, DevOps maturity, the impact of AI, as well as predictions about the future of engineering.
Engineering Predictions for 2024 (and Beyond)
- The mandate of developer productivity: Understanding and tracking developer productivity will be essential. But the focus should be on driving team-level efficiency and improving developer experience — rather than tracking individual productivity.
- The rise of platform engineering: Building an ecosystem that allows developers to be more autonomous will mean better interfaces, enhanced integration, and streamlined workflows.
- The death of agile: The agile manifesto often does not work for modern-day SaaS engineering organizations, who must navigate unplanned work, customer needs, and technical constraints.
- The AI multiplier: Incorporating generative AI and large-language models (LLMs) into internal workflows will become table-stakes for engineering teams.
In 2024, engineering leaders’ top priorities fall across three broad categories: developer productivity and experience; the evolution and maturity of DevOps; and driving innovation and efficiency, with a focus on embedding AI
Key Priorities for Engineering Leaders
- Based on our research, we believe developer productivity should be assessed through a holistic framework that captures business impact, performance and reliability, developer effectiveness, and culture
- A key factor to improving developer productivity and experience is the state of the underlying infrastructure and processes that power engineering teams
- Our conversations with CTOs have revealed universal questions about AI that include where to start, use cases for most impact, and early ROI
Engineering leaders are prioritizing product development, accelerating developer velocity, and improving underlying infrastructure and processes
We are keeping headcount flat in 2024, so a big focus for me this year is striking a balance between individual development goals and organization needs with a major focus on efficiency.”
The term “developer productivity” lacks a standard definition or approach; engineering leaders and engineers have varying opinions about its definition and approach. We also believe that the term needs a definitional refresh, which includes perspectives from both groups.
Developer Productivity in 2024
- While CTOs prioritize clear goals and productivity requirements, engineers highlight the need for focus time and a conducive development environment
- We believe developer productivity should include both perspectives
- To monitor and enhance team performance, we recommend that engineering leaders use a holistic framework that includes:
- Business impact
- Performance and reliability
- Developer effectiveness
- Culture
- Tracking developer productivity is more prevalent in engineering teams of 100 members or more, and often involves satisfaction surveys along with DevOps Research and Assessment (DORA) metrics
Engineering leaders and employees have different opinions on how to improve developer productivity.
The more engineers in the organization, the more likely companies are to use a combination of developer satisfaction surveys and DevOps Research and Assessment (DORA)1 metrics.
Software development is a team-based activity at its core. We believe developer productivity means understanding both business outcomes and the factors affecting individual developer experience front and center.
Spotlight: Tracking Developer Productivity
We have made a lot of changes to our organizational structure and processes as the business has scaled through a period of very fast growth. As our team has grown to 200+ engineers, we have been on a journey to make the development team move faster and are always experimenting.”
Spotlight: Tracking Developer Velocity
Operational changes to improve velocity included:
- Engineering organization structure designed with leaner pods owned by true technical managers
- Engineers need to test their own code and automate test cases
- Leadership showcases the “why” behind releases of new customer features
Metrics tracked included:
- Delivered versus committed
- Thrashing between QA and engineering
- Deployed story point velocity over time
Enhancing developer productivity and experience are key factors to improving the underlying infrastructure and processes powering engineering teams.
The Evolution and Maturity of DevOps
- Roadmap Planning: Companies in the early stages of scaling (less than $25M) tend to follow a fluid process. As they continue scaling, companies often switch back to a more sequential planning approach. Companies that scale past $100M typically settle into a regular planning cadence
- Release Cadence: Companies in the early stages of growth are able to move fast by augmenting continuous deployment with experimental releases. As companies scale, that approach becomes more challenging
- Testing Automation: Most companies build out automated testing processes as early as $25M in revenue. Top performing companies have faster testing processes that show results up to several times a day
- CI/CD Tools: The majority (76%) of companies in the $250M revenue range have full implementation and integration of CI/CD tools
Most companies allow developers to see the results of integration / unit testing weekly. But top performing companies have faster testing processes, which show results up to several times a day.
Most companies surveyed have full implementation of CI/CD tools. Nearly all (98%) of companies have evaluated or implemented CI/CD tools by $250M in revenue.
More than one-third (around 35% to 45%) of respondents surveyed are planning to hire AI-related roles, including machine-learning engineers, data scientists, and data engineers, in 2024.
We are doubling down on refactoring and overall simplification. The goal is to enable the team to spend less time on time-suck / low-ROI activities that can be automated."
At Facebook, I led the production engineering team which scaled Facebook’s infrastructure and ensured our services are available 24/7. I brought this mentality of operations at scale with me to 1Password and when I joined as CTO, I knew that we had to embed DevOps into the engineering mindset and culture.”
Spotlight: Building a DevOps Culture
Building an engineering organization that is effective requires a change in the cultural mindset and the transition is often very challenging
- It requires buy-in and shared ownership across the entire engineering team. Engineers need to shift their mindset completely and ops teams also need to be engaged earlier in the process with developers
The only way to implement this is by revamping people processes like performance evaluation, hiring, and onboarding to make this mindset shift a requirement:
- We worked with developers and operations teams resistant to change - helped them evolve, and if/when change wasn't effective, amicably parted ways
- We also try to hire for developers with a DevOps mindset
- We practice “dogfooding” which means developers must consistently use the product they’ve built just like the end user to figure out what works and what doesn’t
- We implemented developer bootcamps (2 week programs for new joiners) to accelerate onboarding for new hires and coach them on best practices across development, code reviews, testing, etc.
The CTOs we spoke to are curious about where to start with AI, the most impactful use cases for internal productivity, and interested in what other companies are seeing in terms of early ROI.
Driving Innovation and Efficiency via AI
- Nearly three-fourths (70%) of companies surveyed have an AI component in their product suite
- Most companies surveyed are also planning to embed new AI or introduce AI-related products within the next year, regardless of scale
- Early-stage companies seem to be adopting and using AI tools more frequently than growth-stage companies
- Preliminary estimates of impact reveal notable productivity gains of 20% to 30%. The departments with the biggest gains are HR, engineering, marketing, and strategy
- The average total revenue increase driven by AI tools is expected to be around 16% for companies surveyed
Building new AI products takes significant time and upfront investment. So most companies surveyed are first experimenting with leveraging internal AI applications to boost productivity. Early-stage and late-stage companies are adopting and using AI tools more frequently than growth-stage companies.
Preliminary estimates of AI impact reveal notable productivity gains of 20% to 30%. HR and customer success / support are seeing the great cost savings. Finance and engineering are seeing the largest revenue increases.
We’ve found new and innovative ways to use GenAI to better support our clients and empower our employees. That’s why we’ve introduced ‘Wealthsimple Booster Packs,’ which are packages of open-source LLMs built on our own cloud infrastructure to protect PII and IP. We’ve seen massive adoption among Wealthsimplers – weekly active users (WAUs) include 50% of the company.”
Spotlight: Leveraging LLMs to Drive Efficiency
Knowledge bases made available to all employees:
- Public
- Private
- Limited
- Most requests are related to programming, content/copy generation, querying and reviewing customer tickets
Tracking ROI
- Built a “Voice of the Customer” dashboard that tracks all inbound customer issues across web, email, chat, and voice, classified by topics and sub-topics
- The dashboard has significantly reduced ticket queue time and saved an average of around 620 days of manual operations annually
Related Reports
.png)
.png)
Other Reports
.png)
Notes
- Please refer to the Disclosures at the end of this article for more information on BetterUp.
- ICONIQ Growth Tech Employee Survey from October – November 2021 which includes responses from ~70 tech employees; For more details on data source and insights, reference our Future of Work Series Introduction
- Themes, sentiments, and anonymized conversations from ~100,000 BetterUp onboarding sessions and ~128,000 coaching touchpoints
- Data from a survey of ~150 tech employees conducted in March 2022. For more details on data source and respondent make-up, reference our Future of Work Series: Professional Development report
- Themes, sentiments, and anonymized conversations from ~100,000 BetterUp onboarding sessions and ~128,000 coaching touchpoints
- Analysis uses implied gender based on pronouns used by coaches
- Themes, sentiments, and anonymized conversations from ~100,000 BetterUp onboarding sessions and ~128,000 coaching touchpoints
- ICONIQ Growth Tech Employee Survey from October – November 2021 which includes responses from ~70 tech employees; For more details on data source and insights, reference our Future of Work Series Introduction
- Data from a survey of ~150 tech employees conducted in March 2022. For more details on data source and respondent make-up, reference our Future of Work Series: Professional Development report
- Themes, sentiments, and anonymized conversations from ~100,000 BetterUp onboarding sessions and ~128,000 coaching touchpoints
Disclaimer
Unless otherwise indicated, the views expressed in this presentation are those of ICONIQ Growth (“ICONIQ" or the “Firm"), are the result of proprietary research, may be subjective, and may not be relied upon in making an investment decision. Information used in this presentation was obtained from numerous sources. Certain of these companies are portfolio companies of ICONIQ Growth. ICONIQ Growth does not make any representations or warranties as to the accuracy of the information obtained from these sources.
This presentation is for general information purposes only and does not constitute investment advice. This presentation must not be relied upon in connection with any investment decision. The information in this presentation is not intended to and does not constitute financial, accounting, tax, legal, investment, consulting or other professional advice or services. Nothing in this presentation is or should be construed as an offer, invitation or solicitation to engage in any investment activity or transaction, including an offer to sell or a solicitation of an offer to buy any securities which should only be made pursuant to definitive offering documents and subscription agreements, including without limitation, any investment fund or investment product referenced herein.
Any reproduction or distribution of this presentation in whole or in part, or the disclosure of any of its contents, without the prior consent of ICONIQ, is strictly unauthorized.
This presentation may contain forward-looking statements based on current plans, estimates and projections. The recipient of this presentation ("you") are cautioned that a number of important factors could cause actual results or outcomes to differ materially from those expressed in, or implied by, the forward-looking statements. The numbers, figures and case studies included in this presentation have been included for purposes of illustration only, and no assurance can be given that the actual results of ICONIQ or any of its partners and affiliates will correspond with the results contemplated in the presentation. No information is contained herein with respect to conflicts of interest, which may be significant. The portfolio companies and other parties mentioned herein may reflect a selective list of the prior investments made by ICONIQ.
Certain of the economic and market information contained herein may have been obtained from published sources and/or prepared by other parties. While such sources are believed to be reliable, none of ICONIQ or any of its affiliates and partners, employees and representatives assume any responsibility for the accuracy of such information.
All of the information in the presentation is presented as of the date made available to you (except as otherwise specified), and is subject to change without notice, and may not be current or may have changed (possibly materially) between the date made available to you and the date actually received or reviewed by you. ICONIQ assumes no obligation to update or otherwise revise any information, projections, forecasts or estimates contained in the presentation, including any revisions to reflect changes in economic or market conditions or other circumstances arising after the date the items were made available to you or to reflect the occurrence of unanticipated events. Numbers or amounts herein may increase or decrease as a result of currency fluctuations.