research-article

The Central Problem with Distributed Content: Common CDN Deployments Centralize Traffic In A Risky Way

Authors:

Kevin Vermeulen,

Loqman Salamatian,

Ethan Katz-BassettAuthors Info & Claims

HotNets '23: Proceedings of the 22nd ACM Workshop on Hot Topics in Networks

Pages 70 - 78

https://doi.org/10.1145/3626111.3628213

Published: 28 November 2023 Publication History

Abstract

Google, Netflix, Meta, and Akamai serve content to users from offnet servers in thousands of ISPs. These offnets benefit both services and ISPs, via better performance and reduced interdomain and WAN traffic. We argue that this widespread distribution of servers leads to a concentration of traffic and a previously unacknowledged risk, as many ISPs colocate offnets from multiple providers. This trend contributes to many Internet users likely accessing multiple popular services and fetching the majority of their Internet traffic from a single facility -- perhaps even a single rack -- creating shared resources and a correlated risk in cases of failures, attacks, and overload. Alternate ways to access the services often lack sufficient capacity and share resources with more services, creating the potential for cascading failures.

References

[1]

Muhammad Abdullah, Pavlos Nikolopoulos, and Katerina Argyraki. Caching and Neutrality. ACM HotNets, 2023.

Digital Library

[2]

Akamai. Akamai | Information Security Compliance, 2023. URL https://www.akamai.com/legal/compliance.

[3]

Akamai. Facts and Figures, 2023. URL https://www.akamai.com/company/facts-figures.jsp.

[4]

Scott Anderson, Carol Barford, and Paul Barford. Five Alarms: Assessing the Vulnerability of US Cellular Communication Infrastructure to Wildfires. ACM IMC, 2020. URL https://doi.org/10.1145/3419394.3423663.

Digital Library

[5]

Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. OPTICS: Ordering Points to Identify the Clustering Structure. ACM SIGMOD, 1999. URL https://doi.org/10.1145/304182.304187.

Digital Library

[6]

Todd Arnold, Jia He, Weifan Jiang, Matt Calder, Italo Cunha, Vasileios Giotsas, and Ethan Katz-Bassett. Cloud Provider Connectivity in the Flat Internet. ACM IMC, 2020. URL https://doi.org/10.1145/3419394.3423613.

Digital Library

[7]

Azure. Azure Compliance Documentation, 2023. URL https://learn.microsoft.com/en-us/azure/compliance/.

[8]

Anurag Bhatia. Mapping Facebook's FNA (CDN) Nodes Across the World!, 2018. URL https://anuragbhatia.com/2018/03/networking/isp-column/mapping-facebooks-fna-cdn-nodes-across-the-world/.

[9]

Anurag Bhatia. Facebook FNA Node Update, 2019. URL https://anuragbhatia.com/2019/11/networking/isp-column/facebookfna-node-update/.

[10]

Marjory Blumenthal, Ramesh Govindan, Ethan Katz-Bassett, Arvind Krishnamurthy, James McCauley, Nick Merrill, Tejas Narechania, Aurojit Panda, and Scott Shenker. Can We Save The Public Internet? 2023.

[11]

Lloyd Brown, Ganesh Ananthanarayanan, Ethan Katz-Bassett, Arvind Krishnamurthy, Sylvia Ratnasamy, Michael Schapira, and Scott Shenker. On the Future of Congestion Control for the Public Internet. ACM HotNets, 2020. URL https://doi.org/10.1145/3422604.3425939.

Digital Library

[12]

Matt Calder, Xun Fan, Zi Hu, Ethan Katz-Bassett, John Heidemann, and Ramesh Govindan. Mapping the Expansion of Google's Serving Infrastructure. ACM IMC, 2013. URL https://doi.org/10.1145/2504730.2504754.

Digital Library

[13]

Esteban Carisimo, Alexander Gamero-Garrido, Alex C. Snoeren, and Alberto Dainotti. Identifying ASes of State-Owned Internet Operators. In Proceedings of the 21st ACM Internet Measurement Conference, ACM IMC, 2021. URL https://doi.org/10.1145/3487552.3487822.

Digital Library

[14]

Chao Liu and Ernesto Falcon. There is Nothing Fair About the European Commission's "Fair Share" Proposal, 2023. URL https://www.eff.org/deeplinks/2023/06/there-nothing-fair-about-european-commissions-fair-share-proposal.

[15]

Trinh Viet Doan, Roland van Rijswijk-Deij, Oliver Hohlfeld, and Vaibhav Bajpai. An Empirical View on Consolidation of the Web. ACM Transaction on Internet Technology, 2022. URL https://doi.org/10.1145/3503158.

Digital Library

[16]

Ramakrishnan Durairajan, Paul Barford, Joel Sommers, and Walter Willinger. InterTubes: A Study of the US Long-Haul Fiber-Optic Infrastructure. ACM SIGCOMM, 2015. URL https://doi.org/10.1145/2785956.2787499.

Digital Library

[17]

Euro-IX. IXPDB. URL https://ixpdb.euro-ix.net/en/ix-f/ixp-database/.

[18]

Facebook. FNA Installation and Operation Guide, 2017. URL https://kupdf.net/download/fna-installation-and-operation-guidev804_597ff4a4dc0d6055102bb17e_pdf.

[19]

Dean Garfield. Red Light Green Light? No to Network Usage Fees, 2021. URL https://about.netflix.com/en/news/red-light-green-light-no-to-network-usage-fees.

[20]

Monia Ghobadi and Ratul Mahajan. Optical Layer Failures in a Large Backbone. ACM IMC, 2016. URL https://doi.org/10.1145/2987443.2987483.

Digital Library

[21]

Petros Gigis, Matt Calder, Lefteris Manassakis, George Nomikos, Vasileios Kotronis, Xenofontas Dimitropoulos, Ethan Katz-Bassett, and Georgios Smaragdakis. Seven Years in the Life of Hypergiants' off-Nets. ACM SIGCOMM, 2021. URL https://doi.org/10.1145/3452296.3472928.

Digital Library

[22]

Phillipa Gill, Christophe Diot, Lai Yi Ohlsen, Matt Mathis, and Stephen Soltesz. M-Lab: User Initiated Internet Data for the Research Community. ACM SIGCOMM Computer Communication Review, 2022. URL https://dl.acm.org/doi/10.1145/3523230.3523236.

[23]

Google. Edge Nodes GGC, 2023. URL https://peering.google.com/#/options/google-global-cache.

[24]

Google. Multi-node Deployments - Interconnect Help, 2023. URL https://support.google.com/interconnect/answer/9051938.

[25]

Google. Preparing your Network, 2023. URL https://support.google.com/interconnect/answer/9059054?hl=en&ref_topic=7659203&sjid=2857398392929088225-NA.

[26]

Ramesh Govindan, Ina Minei, Mahesh Kallahalla, Bikash Koley, and Amin Vahdat. Evolve or Die: High-Availability Design Principles Drawn from Google's Network Infrastructure. ACM SIGCOMM, 2016. URL https://doi.org/10.1145/2934872.2934891.

Digital Library

[27]

Geoff Huston. How Big is that Network, 2014. URL https://labs.apnic.net/?p=526.

[28]

Sangeetha Abdu Jyothi. Solar Superstorms: Planning for an Internet Apocalypse. ACM SIGCOMM, 2021. URL https://doi.org/10.1145/3452296.3472916.

Digital Library

[29]

Aqsa Kashaf, Vyas Sekar, and Yuvraj Agarwal. Analyzing Third Party Service Dependencies in Modern Web Services: Have We Learned from the Mirai-Dyn Incident? ACM IMC, 2020. URL https://doi.org/10.1145/3419394.3423664.

Digital Library

[30]

Thomas Koch, Ke Li, Calvin Ardi, Matt Calder, John Heidemann, and Ethan Katz-Bassett. Anycast in Context: A Tale of Two Systems. ACM SIGCOMM, 2021. URL https://doi.org/10.1145/3452296.3472891.

Digital Library

[31]

Rashna Kumar, Sana Asif, Elise Lee, and Fabian E. Bustamante. Each at Its Own Pace: Third-Party Dependency and Centralization Around the World. ACM SIGMETRICS, 2023. URL https://doi.org/10.1145/3579437.

Digital Library

[32]

Craig Labovitz. Pandemic Impact on Global Internet Traffic, 2020. URL https://www.nanog.org/news-stories/nanog-tv/nanog-79-webcast/effects-covid-19-lockdowns-service-provider-networks/.

[33]

Shucheng Liu, Zachary S. Bischof, Ishaan Madan, Peter K. Chan, and Fabián E. Bustamante. Out of Sight, Not Out of Mind: A User-View on the Criticality of the Submarine Cable Network. ACM IMC, 2020. URL https://doi.org/10.1145/3419394.3423633.

Digital Library

[34]

Matthew Luckie, Bradley Huffaker, Alexander Marder, Zachary Bischof, Marianne Fletcher, and K. Learning to Extract Geographic Information from Internet Router Hostnames. ACM CoNEXT, 2021. URL https://doi.org/10.1145/3485983.3494869.

Digital Library

[35]

A Marder, kc claffy, and A Snoeren. Inferring Cloud Interconnections: Validation, Geolocation, and Routing Behavior. PAM, 2021. URL https://doi.org/10.1007/978-3-030-72582-2_14.

Digital Library

[36]

A Marder, Z Zhang, R Padmanabhan, R Mok, B Huffaker, M Luckie, A Dainotti, kc claffy, A Snoeren, and A Schulman. Access Denied: Assessing Physical Risks to Internet Access Networks. USENIX Security Symposium, 2023. URL https://www.usenix.org/system/files/usenixsecurity23-marder.pdf.

[37]

Michael Markovitch, Sharad Agarwal, Rodrigo Fonseca, Ryan Beckett, Chuanji Zhang, Irena Atov, and Somesh Chaturmohta. TIPSY: Predicting Where Traffic Will Ingress a WAN. ACM SIGCOMM, 2022. URL https://doi.org/10.1145/3544216.3544234.

Digital Library

[38]

Juno Mayer, Valerie Sahakian, Emilie Hooft, Douglas Toomey, and Ramakrishnan Durairajan. On the Resilience of Internet Infrastructures in Pacific Northwest to Earthquakes. PAM, 2021. URL https://link.springer.com/chapter/10.1007/978-3-030-72582-2_15.

Digital Library

[39]

Nick Merrill and Tejas N Narechania. Inside the Internet. Duke Law Journal Online, 2023. URL https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4466419.

[40]

Justin Meza, Tianyin Xu, Kaushik Veeraraghavan, and Onur Mutlu. A Large Scale Study of Data Center Network Reliability. ACM IMC, 2018. URL https://doi.org/10.1145/3278532.3278566.

Digital Library

[41]

Giovane C. M. Moura, Sebastian Castro, Wes Hardaker, Maarten Wullink, and Cristian Hesselman. Clouding up the Internet: How Centralized is DNS Traffic Becoming? ACM IMC, 2020. URL https://doi.org/10.1145/3419394.3423625.

Digital Library

[42]

Netflix. Netflix | Open Connect Sample Architectures, 2023. URL https://openconnect.netflix.com/en/#sample-architectures.

[43]

Netflix. Requirements for Deploying Embedded Appliances, 2023. URL https://openconnect.zendesk.com/hc/en-us/articles/360034538352.

[44]

PeeringDB. PeeringDB. URL http://www.peeringdb.com.

[45]

Enric Pujol, Ingmar Poese, Johannes Zerwas, Georgios Smaragdakis, and Anja Feldmann. Steering Hyper-Giants' Traffic at Scale. ACM CoNEXT, 2019. URL https://doi.org/10.1145/3359989.3365430.

Digital Library

[46]

Rapid7. Project Sonar, Reverse DNS, 2023. URL https://www.rapid7.com/research/project-sonar/.

[47]

Reddit. Which (CDN) Caching Appliances Do You Run at Your ISP? Which Gives the Biggest Savings?, 2023. URL https://www.reddit.com/r/networking/comments/11ddyxg/which_cdn_caching_appliances_do_you_run_at_your/.

[48]

Sandvine. Global Internet Phenomena Report 2023, 2023. URL https://www.sandvine.com/global-internet-phenomena-report-2023.

[49]

Brandon Schlinker, Hyojeong Kim, Timothy Cui, Ethan Katz-Bassett, Harsha V. Madhyastha, Italo Cunha, James Quinn, Saif Hasan, Petr Lapukhov, and Hongyi Zeng. Engineering Egress with Edge Fabric: Steering Oceans of Content to the World. ACM SIGCOMM, 2017. URL https://doi.org/10.1145/3098822.3098853.

Digital Library

[50]

Brandon Schlinker, Italo Cunha, Yi-Ching Chiu, Srikanth Sundaresan, and Ethan Katz-Bassett. Internet Performance from Facebook's Edge. ACM IMC, 2019. URL https://doi.org/10.1145/3355369.3355567.

Digital Library

[51]

Kyle Schomp, Onkar Bhardwaj, Eymen Kurdoglu, Mashooq Muhaimen, and Ramesh K. Sitaraman. Akamai DNS: Providing Authoritative Answers to the World's Queries. ACM SIGCOMM, 2020. URL https://doi.org/10.1145/3387514.3405881.

Digital Library

[52]

Florian Streibelt, Jan Böttger, Nikolaos Chatzis, Georgios Smaragdakis, and Anja Feldmann. Exploring EDNS-Client-Subnet Adopters in Your Free Time. ACM IMC, 2013. URL https://doi.org/10.1145/2504730.2504767.

Digital Library

[53]

Ao-Jan Su, David R. Choffnes, Aleksandar Kuzmanovic, and Fabián E. Bustamante. Drafting behind Akamai (Travelocity-Based Detouring). ACM SIGCOMM, 2006. URL https://doi.org/10.1145/1159913.1159962.

Digital Library

[54]

Telefonica. What is Fair Share, 2023. URL https://www.telefonica.com/en/communication-room/blog/what-is-fair-share/.

[55]

Sipat Triukose, Zhihua Wen, and Michael Rabinovich. Measuring a Commercial Content Delivery Network. ACM WWW, 2011. URL https://doi.org/10.1145/1963405.1963472.

Digital Library

[56]

Kok-Kiong Yap, Murtaza Motiwala, Jeremy Rahe, Steve Padgett, Matthew Holliman, Gary Baldus, Marcus Hines, Taeeun Kim, Ashok Narayanan, Ankur Jain, Victor Lin, Colin Rice, Brian Rogan, Arjun Singh, Bert Tanaka, Manish Verma, Puneet Sood, Mukarram Tariq, Matt Tierney, Dzevad Trumic, Vytautas Valancius, Calvin Ying, Mahesh Kallahalla, Bikash Koley, and Amin Vahdat. Taking the Edge off with Espresso: Scale, Reliability and Programmability for Global Internet Peering. ACM SIGCOMM, 2017. URL https://doi.org/10.1145/3098822.3098854.

Digital Library

[57]

Ming Zhang, Yaoping Ruan, Vivek Pai, and Jennifer Rexford. How DNS Misnaming Distorts Internet Topology Mapping. USENIX ATC, 2006. URL https://dl.acm.org/doi/10.5555/1267359.1267393.

Cited By

Madeley JPhokeer ASiddiqui ABenson T(2024)Towards Measuring Content LocalityProceedings of the 2024 Applied Networking Research Workshop10.1145/3673422.3674895(88-90)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3673422.3674895
Hilal FSattler PVermeulen KGasser O(2024)A First Look At IPv6 Hypergiant InfrastructureProceedings of the ACM on Networking10.1145/36563002:CoNEXT2(1-25)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3656300
Brown LMarx EBali DAmaro ESur DKissel EMonga IKatz-Bassett EKrishnamurthy AMcCauley JNarechania TPanda AShenker SSekar VYu MSeneviratne AVeitch D(2024)An Architecture For Edge Networking ServicesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672261(645-660)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672261
Show More Cited By

Index Terms

The Central Problem with Distributed Content: Common CDN Deployments Centralize Traffic In A Risky Way
1. Networks
  1. Network performance evaluation
    1. Network measurement
  2. Network properties
    1. Network reliability
    2. Network structure

Recommendations

RFC 6707: Content Distribution Network Interconnection (CDNI) Problem Statement
Distributed content delivery using load-aware network coordinates
CoNEXT '08: Proceedings of the 2008 ACM CoNEXT Conference

To scale to millions of Internet users with good performance, content delivery networks (CDNs) must balance requests between content servers while assigning clients to nearby servers. In this paper, we describe a new CDN design that associates synthetic ...
Enabling content-aware traffic engineering

Today, a large fraction of Internet traffic is originated by Content Delivery Networks (CDNs). To cope with increasing demand for content, CDNs have deployed massively distributed infrastructures. These deployments pose challenges for CDNs as they have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotNets '23: Proceedings of the 22nd ACM Workshop on Hot Topics in Networks

November 2023

306 pages

ISBN:9798400704154

DOI:10.1145/3626111

Copyright © 2023 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF

Conference

HotNets '23

Sponsor:

SIGCOMM

HotNets '23: The 22nd ACM Workshop on Hot Topics in Networks

November 28 - 29, 2023

MA, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 110 of 460 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
105
Total Downloads

Downloads (Last 12 months)105
Downloads (Last 6 weeks)10

Reflects downloads up to 05 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Madeley JPhokeer ASiddiqui ABenson T(2024)Towards Measuring Content LocalityProceedings of the 2024 Applied Networking Research Workshop10.1145/3673422.3674895(88-90)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3673422.3674895
Hilal FSattler PVermeulen KGasser O(2024)A First Look At IPv6 Hypergiant InfrastructureProceedings of the ACM on Networking10.1145/36563002:CoNEXT2(1-25)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3656300
Brown LMarx EBali DAmaro ESur DKissel EMonga IKatz-Bassett EKrishnamurthy AMcCauley JNarechania TPanda AShenker SSekar VYu MSeneviratne AVeitch D(2024)An Architecture For Edge Networking ServicesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672261(645-660)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672261
Blumenthal MGovindan RKatz-Bassett EKrishnamurthy AMcCauley JMerrill NNarechania TPanda AShenker S(2024)Can We Save the Public Internet?ACM SIGCOMM Computer Communication Review10.1145/3649171.364917553:3(18-22)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1145/3649171.3649175

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents