Priya Dhingra

If a 100 Mb video gets 20k views in one month, how much bandwidth do we need to serve the video? These kinds of questions are called capacity estimation questions. They are part of system design interviews. The next version of the system design judge will have capacity estimation questions. Coming soon! #SystemDesignJudge #CapacityEstimation

319

27 Comments

System Design is one of my favorite. Here is the list of system design building blocks that are important irrespective of whether you are designing a system or preparing for a system design interview ✔️ Distributed message queue: A producer puts a message Or work in a line, which the consumers pick up for further processing or work defined. This is an important concept to explain how you can scale your system, perform asynchronous processing, ✔️ DNS: Translates website names to IP address. DNS service works much like a phone directory. Services like Route53 help configure the domain and routing, and one needs to understand common types of resource records like A, CNAME, NS, MX, etc. ✔️ CDN: The content delivery network is a server between the origin and the client. It stores and delivers content to the end users, reducing latency and improving the system's overall performance. ✔️ PUB SUB: Publish-subscribe messaging works asynchronously, involving publishers, topics, subscribers, and messages. It helps to decouple the system, scalability, Durability, etc. ✔️ Load Balancer: Load balancer helps divide and divert traffic across the system. It helps drive the overall system Availability and Performance. ✔️ Distributed Caching: Caching servers store the data in memory and return it to the end users, improving the overall performance and end-user experience. A fleet of servers works together to keep and serve the data in a distributed caching system, enhancing availability. ✔️ Database: Understanding data storage, storage, types, and choosing a Database, RDMBS Vs Nosql, is vital when designing a system. It is essential to deeply understand for what requirements RDBMS is the right fit, and nosql is the right fit. ✔️ Observability: It's important to know what is happening in the system. How do we make the system easier to debug and fix? Observability helps the system to drive maintainability, resiliency, and reliability. ✔️ Task Scheduler: This is an essential component that helps one decide how the tasks must be prioritized, delegated and how much resources need to be allocated ✔️ Unstructured Data Storage: Storing photos, videos, audio, and other unstructured data is essential when designing a system since these cannot be stored in traditional databases. One needs to create a highly available, consistent, reliable solution. and efficient ✔️ Scaling Services: Scalability is an essential building block of any robust system. Understanding various scaling options and what options should be used is critical. From Serverless to ID generators, one must understand the multiple options and how to design them. ✔️ Distributed Search: Search is the core of many system designs. Understating how to build a search system and the core building blocks of a search system, including crawling and indexing, is essential. #engineering #systemdesign #interview #productdevelopment

405

37 Comments

10x engineer, rock star, brogrammers....these are all unhelpful myths. Were they a great dev? Probably, but a great dev should not have to suffer through ego death to consider themselves successful. I agree with others, this attitude is not how a team succeeds and it is toxic. When I look at Facebook now I see an application stack that constantly breaks, is unintuitive and doesn't have the infrastructure to support itself, worse in quality than it was a decade ago. Exactly the type of thing I would expect from a bunch of individual contributors without cohesion. I'm sure this analysis on my part is unfair and there are dedicated, hard working team members there ready to make a difference, but this type of culture needs to stop. Invest in your internal teams, promote from within, train, encourage, build great things together instead of waiting for an imaginary deus ex programmator to save you.

7

1 Comment

It’s been 1.5+ yrs since I started working at Google and several individuals from my LinkedIn network asked me some common questions - “How are you adjusting to the cultural shock between Amazon vs Google ”? I waited for 1.5 yrs since I needed some data to calibrate my journey and share answer in a professionally responsible manner. Also, the tech industry has endured tremendous turbulence in the last 2 years and I had to adjust my surfing gears more than ever. Let me start with - yes, I had cold feet when I stepped outside my 9 yrs old “Amazon” home. But, I can also assure you this decision was not impromptu and took ~5 months of mental exercise. Here are some things that I practiced to weather the change and get acclimatized.   (1) Set right expectations: This is probably the most important one to avoid mental stress and fatigue. Like any trade, there are risks when stakes are high. I spent nearly 2 months finding the trade offs and compromises of my choices which led me to build the plan to address my own concerns.    (2) Build career strategy: With expectations set, I put a methodical structure to accelerate my journey. First 4 months, I was 100% hands-on with SDE-1 and 2s to understand the builder tools (yes, they know the best!), server frameworks, development and deployment best practices. Then I scheduled 50% of time on project delivery, 30% on identifying top engineering problems and 20% time working on the 3 years vision. (3) Stay original and authentic: My strategy was to fight all of my inner urge to practice Amazonian traits at Google (all though some part of me was screaming inside). That said, I did not throw any Amazon leadership principles and instead moderated the tone in the new world, while embracing the Googlyness. (4) Respect what came before: I put a self note on my desk to fight all urges to be judgmental and opinionated about all prior organizational and engineering decisions. Instead, I scheduled meetings and spent several months absorbing what came before and internalizing the vision. (5)Developed my professional network: “Earn trust” is table stakes if you want to grow in your career. Building trust and network is a huge time investment and takes careful planning. Frequent 1:1 with mentors and leaders in my organization definitely paid off to seed the right network structure at the early stage. (6) Learn and be curious: For a builder, it is not fun to throw away familiar tools. pick up new ones and accelerate.  But, here is one thing that helped. I stopped burdening myself with the idea that I am missing my old tools, instead motivated myself to learn technologies those power planet scaled software. Success metric to share: Today I Am driving ~100+ SDE organization as the tech lead. Achieved code review bar raiser in two programming languages Became chaos testing ambassador in Google Ads.  Led a performance improvement objective in Google (won Bronze tier) Launched 2 critical features.

234

15 Comments

3 books to learn System Design: 1. Head First Design Patterns 2. Fundamentals of Software Architecture 3. Software Architecture: The Hard Parts If you like System Design, buy at least one of these.

1,421

122 Comments

A Decade at Amazon - Reflections After a 10-year journey at Amazon, I recently transitioned from a yellow badge to the red corporate employee badge. While there is room for improvement at Amazon, I'm struck by how much I've learned and grown, thanks to Amazon's culture and leaders. In my last 10 years, I've had the opportunity to work across different organizations in stores, AWS, and Alexa as an individual contributor and as a manager in technical program management and engineering disciplines. Some of the most profound lessons that will stick with me include: 1. Following leadership principles unifies a company's culture and boosts productivity. 2. The importance of deep understanding, practice, and insisting on the highest standards. 3. Driving outcomes over empires and being strategically patient yet tactically impatient in pursuing customer obsession. 4. Having an insatiable curiosity and beginner's mindset. 5. Every customer issue represents an upstream process deficiency that needs to be identified and resolved urgently. 6. Prioritizing and tackling the hottest, highest impact items first. 7. Being "green-lighted" doesn't mean risk-free, and transparently calling out risks with projected issue dates. 8. Prioritizing input goals you can control before output goals, and optimizing inputs relentlessly. 9. Goals are directional, not dogmatic, and undershooting or overshooting can be acceptable if customers are delighted. 10. While off-the-shelf solutions can cover 80%, investing in tailored solutions for the remaining 20% is critical. 11. Simplicity, incremental progress, and smart automation are core philosophies. 12. Transparency in prioritization rationale and openness to adjusting the roadmap based on new inputs. 13. Sticking to your strongest points persuades better than attempting to discuss everything. 14. Removing blockers is your responsibility, not someone else's, and committing 100% once a group decision is made. 15. Embracing failures as learning opportunities in complex operations. This enriching decade has been about customer obsession, frugality, ownership, never stopping learning, highest standards, and delivering robust solutions. I'm grateful for the growth I've experienced here.

179

26 Comments

Came across a blog post for beginners to understand at a high level the RESP Redis serialization protocol. https://lnkd.in/gRWWpCie

1

Design systems as if you were the one who will maintain them. Few nuggets to keep handy, while designing a scalable system: https://lnkd.in/g_udCNm3 #systemdesign #scalability #performance

15

#Immigration Update : 2nd Lottery attempt for the lucky ones who were not picked in the x Number of tries/retries in 2024. A uniquely weird thing happened with me when I lost my job due to sudden layoff spree in AWS KDA team with 19k+ others(overall in the wave layoff spree) My H1-B was picked and approved in the first attempt via premium processing which led to understanding by SEVIS and school authorities that F-1 should be closed and because of which I lost my job at Walmart client through BayOne Solutions in December 23. Well, It was basic CRUD update issue which did not get reflected in SEVIS from USCIS and vice versa. Because of which when my record was reinstated, I lost almost 60 days of Unemployment as per STEM OPT guidelines (which accrue, 60+90 once approved from an E-Verified Employer) Pritesh Jagani, Rajiv Khanna and other attorneys here on LinkedIn might have similar students reporting them their cases when they got impacted. I was one in the unique herd that was part of it. I am doing a vlog session to let you know more about it this weekend. Follow me on Youtube for more updates on the case, and how I got it resolved with the help of Immigration attorneys who help students every day one unique case at a time. #immigration #Layoffs #Impact #International #Students #Masters #Dual #Masters #CSE #ASU #MBACandidate #Applied #Research #Westcliff Uni.

2

Amazon-to-Google-to-Amazon #a2g2a thought of the day: In 2024, onboarding at Amazon as a new employee feels like a well-oiled machine. My embarkation (pun here) has been smooth! First time I onboarded at Amazon was in 2009. We were a smallish company of 3k engineers starting hypergrowth, so the process was pretty informal. Somebody spent 30 minutes going over benefits, then she played a 15 minute video of Jeff Bezos talking about Amazon, we were handed our laptops and badges, and an hour later I was writing my first piece of code (fun fact: that piece of code ended up in TPSGenerator many years later!). That was kinda it. You're now an Amazonian, now go do some work! I onboarded at Google in 2020 just as the pandemic shut down the world. I was hoping it’d be like The Internship (that dumb movie with Vince Vaughn and Owen Wilson) 🙂. Instead, well-intentioned Googlers were trying really hard to make us feel welcome via hours of GVC, but we were all trying to figure out how this "remote" thing even worked. I got my noogler hat 2 or 3 months after my first day. All in all it felt pretty underwhelming. I imagine it's probably better now. Where today's Amazon new employee orientation shines is in crisply explaining our culture and DNA to new amazonians. I am a returning amazon-veteran, so I knew all these things by heart, but I still watched with delight every single video, out of curiosity. The videos on Leadership Principles are pure gold. When I joined Google, I tried very hard to grasp the DNA of the company. I asked five people: What does it mean to be ‘googley’? and I got five different answers. A few scattered axioms like "Don’t be Evil" and "Respect the { User | Opportunity | each other }" help, but fall short of articulating culture. You can argue whether having a strongly defined culture is a good thing or a bad thing. Personally I think it’s a good thing, but others might feel it as cult-ish / indoctrination / fill-in-the-blanks. It is absolutely true that, as far as culture is concerned, Amazon's is so strongly defined that some personality types thrive and some do not; whereas Google's is so loosely defined that it's a more inclusive place overall. What I learned at Google and I'd like to bring back a little bit is that awareness of how to bring other people to the table. Again, personal opinion only here, but to me, Culture Eats Strategy For Breakfast

238

11 Comments

I regularly speak with other leaders in the Developer Experience to exchange lessons learned and understand common challenges to overcome to improve the developer experience for builders. One topic that always comes up is "how do we make major upgrades easier." We all have to do them. They yield many benefits. But they take time away from innovation. A few months ago, one tip I have been sharing with DevEx leaders is "take a look at Q Code Transformation (Q Developer Agent for Code Transformation). Our use in Amazon's tool chain has been making upgrades much easier, with frequent positive feedback from our own engineers. I am so pleased we are now able to share these numbers with you. They are part of our journey of "making the work for upgrades go away" to give time back for innovation. Our teams have create some amazing, novel innovations to make easy and reliable for teams. It has been inspiring to watch. I am happy to chat more about how we did this--and you can too. And some other modernization problems we are thinking this can make easy as well. #amazon #q #modernization #developerexperience #devex

18

Amazon-to-Google-to-Amazon #a2g2a thought of the day: Reasons Not to Promote. When you endorse somebody for a promotion at Amazon, you MUST provide Reasons Not to Promote. This is in recognition that we all have gaps and places where we could use a little growth. Nobody is perfect. In fact, if an endorsement lacked Reasons Not to Promote, it was removed from the promotion package. Or if it had too many weasel words and platitudes, it too was often challenged (“X is a very nice person and a good developer” – did you spot the weasel words?).   Occasionally this was weaponized. It also led to some awkward situations, like when I was up for promotion to Senior Engineer and I got a frantic text from my manager saying “Director X is citing ABC as reasons not to promote you, do you have any data to show improvement? I need it in the next 3 minutes if you wanna get promoted!!!” But most of the times I think it worked as intended. The desire was to trigger the right objective, data-driven, and honest discussions during the promotion decision meeting.   A couple of months after I joined Google in 2020, I was asked to endorse an engineer I worked with. I wrote my endorsement in Amazon-style, ending with Reasons Not to Promote. About a day later I got a worried text from the manager, “Hummm… do you NOT want X to be promoted???”  I was confused, “I’m a huge fan of X!” I clarified. The manager explained in no uncertain terms that I was to remove the Reasons Not to Promote.   “Also, why did you say ‘Endorse’ and not ‘Strong Endorse’?”, the manager probed. At Amazon, a Strong Endorse means I’m willing to debate anybody regardless of seniority, tooth-and-nail, that this person deserves the promotion (and I did that multiple times). An Endorse means I fully endorse the promotion, simple as that. I quickly learned Google “Strong Endorse” mapped to Amazon “Endorse,” which also meant a Google “Endorse” mapped to an Amazon “Weak Endorse” or really to a “I-kind-of-don’t-really-endorse-this-person-but-I-dont-want-to-be-rude.”   Ultimately, the way that both Amazon and Google operate in this regard maps to the DNA of the company. Amazonians have a healthy disregard for social cohesion, and a very direct and factual communication style which I admit can sometimes borderline being a jerk, whereas googlers fear being jerks, to the detriment of making hard and correct business decisions sometimes.   Ultimately, the challenge is to be both direct and polite. I think both companies could learn from each other, but as for me, I’m a lot more comfortable in a culture of directness and open discussion.

1,624

69 Comments

A compare/contrast with other service bus type offerings (azure service bus) - is this based on kafka or kafka-like architecture as described here?

1

Recently, many talented professionals, including a few people I know personally, have faced unexpected layoffs. These individuals are more than just their job titles—they are skilled, dedicated, and bring immense value to any team they join. Now, more than ever, they need our support. If you are hiring or know of any opportunities, please consider reaching out to those who have been affected. A simple referral, recommendation, or even a connection can make a significant difference in their job search. Here are a few ways we can help: 1. Share Job Postings: Post openings at your company or ones you come across. Use hashtags like #OpenToWork and #HiringNow. 2. Make Introductions: Connect laid-off employees with your network. A warm introduction can open doors. 3. Offer Support: Share tips, resources, and advice on job searching, resume building, and interview preparation. 4. Provide Recommendations: Endorse their skills on LinkedIn or write a recommendation. A few kind words go a long way. Let's come together as a community and turn these challenging times into new opportunities for growth and success. Together, we can help our colleagues get back on their feet. #Support #Community #OpenToWork #JobSearch #Networking #CareerOpportunities #LinkedInFamily

20

3 Comments

Develop like monolith and run like microservices Google recently introduced an Open source framework for distributed applications. Service Weaver, offers the flexibility to write your application as a modular monolith, a term we define as a monolithic application that is structured in a modular way, and deploy it as a microservice, a feature that sets it apart from other frameworks. Service Weaver libraries allow you to write your application as a single module binary using native data structures and method calls. These can be effortlessly deployed as a set of microservices on the cloud or on-premise, providing the best of both worlds of monolith and microservices. With Service Weaver, developers can harness the simplicity of monolithic application development while reaping the scalability and reliability benefits of microservices. By decoupling application development from deployment topology, it empowers you with a flexible and efficient development process, making it easier to build and maintain distributed applications

51

1 Comment

Develop like monolith and run like microservices Google recently introduced an Open source framework for distributed applications. Service Weaver, offers the flexibility to write your application as a modular monolith, a term we define as a monolithic application that is structured in a modular way, and deploy it as a microservice, a feature that sets it apart from other frameworks. Service Weaver libraries allow you to write your application as a single module binary using native data structures and method calls. These can be effortlessly deployed as a set of microservices on the cloud or on-premise, providing the best of both worlds of monolith and microservices. With Service Weaver, developers can harness the simplicity of monolithic application development while reaping the scalability and reliability benefits of microservices. By decoupling application development from deployment topology, it empowers you with a flexible and efficient development process, making it easier to build and maintain distributed applications

96

13 Comments

While at Amazon, I discovered a new way to deal with feedback. There was a training video that was on the feedback website for employees. Apart from explaining how to use the website, the video talked about how to deal with feedback. The feedback from each feedback provider had two sections - strengths (called Superpowers) and weaknesses (called Growth Ideas). The video surprisingly said to not fall into the trap of overly focusing on the growth ideas to the extent that you stop practicing your superpowers. What makes you YOU, what makes you stand out, is your strengths. Yes, you need to address your weaknesses, but overly dwelling on them and forgetting to exercise your strengths is likely to make you an average performer overall. You need a balance between addressing weaknesses and continuing to utilize your strengths to do your job well!

140

7 Comments

Offtopic Since my feed is filled with layoffs news here is my personal take on what can be done to reduce cost and layoffs. 1. Give permanent work from home for software engineers. We don't need lavish offices. What we need is challenging work, handsome pay, peace of mind, good equipment, clear career progression path, supportive management, safe working environment, access to best learning resources. 2. If you really have to cut cost, then cut x% across the board for all employees. 3. Don't over hire, ever. 4. If a unit in your firm is not making profit and you don't see it to make profit in next few years then shut it down and move the people from that unit to other core units of the firm. 5. Don't fire people. Fire poor projects, poor products, #layoffs2024 #softwareengineering

2,584

114 Comments