research-article

DNS Observatory: The Big Picture of the DNS

Authors:

Pawel Foremski,

Giovane C. M. MouraAuthors Info & Claims

IMC '19: Proceedings of the Internet Measurement Conference

Pages 87 - 100

https://doi.org/10.1145/3355369.3355566

Published: 21 October 2019 Publication History

Abstract

The Domain Name System (DNS) is thought of as having the simple-sounding task of resolving domains into IP addresses. With its stub resolvers, different layers of recursive resolvers, authoritative nameservers, a multitude of query types, and DNSSEC, the DNS ecosystem is actually quite complex.

In this paper, we introduce DNS Observatory: a new stream analytics platform that provides a bird's-eye view on the DNS. As the data source, we leverage a large stream of passive DNS observations produced by hundreds of globally distributed probes, acquiring a peak of 200 k DNS queries per second between recursive resolvers and authoritative nameservers. For each observed DNS transaction, we extract traffic features, aggregate them, and track the top-k DNS objects, e.g., the top authoritative nameserver IP addresses or the top domains.

We analyze 1.6 trillion DNS transactions over a four month period. This allows us to characterize DNS deployments and traffic patterns, evaluate its associated infrastructure and performance, as well as gain insight into the modern additions to the DNS and related Internet protocols. We find an alarming concentration of DNS traffic: roughly half of the observed traffic is handled by only 1 k authoritative nameservers and by 10 AS operators. By evaluating the median delay of DNS queries, we find that the top 10 k nameservers have indeed a shorter response time than less popular nameservers, which is correlated with less router hops.

We also study how DNS TTL adjustments can impact query volumes, anticipate upcoming changes to DNS infrastructure, and how negative caching TTLs affect the Happy Eyeballs algorithm. We find some popular domains with a a share of up to 90 % of empty DNS responses due to short negative caching TTLs. We propose actionable measures to improve uncovered DNS shortcomings.

References

[1]

J. Abley and K. Lindqvist. 2006. Operation of Anycast Services. RFC 4786 (Best Current Practice)., 24 pages. https://doi.org/10.17487/RFC4786

[2]

Mark Allman. 2018. Comments On DNS Robustness. In Proceedings of the Internet Measurement Conference 2018. ACM, 84--90.

[3]

Mark Allman. 2019. Case Connection Zone DNS Transactions. http://www.icir.org/mallman/data.html.

[4]

Mark Allman and Vern Paxson. 2007. Issues and Etiquette Concerning Use of Shared Measurement Data. In ACM Internet Measurement Conference.

[5]

Mario Almeida, Alessandro Finamore, Diego Perino, Narseo Vallina-Rodriguez, and Matteo Varvello. 2017. Dissecting DNS Stakeholders in Mobile Networks. In Proceedings of the 13th International Conference on emerging Networking Experiments and Technologies. ACM, 28--34.

Digital Library

[6]

Amazon Route 53. 2019. Choosing a Routing Policy. https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html.

[7]

Burton H. Bloom. 1970. Space/time trade-offs in hash coding with allowable errors. Commun. ACM (1970).

[8]

S. Bortzmeyer. 2016. DNS Query Name Minimisation to Improve Privacy. RFC 7816 (Experimental)., 11 pages. https://doi.org/10.17487/RFC7816

[9]

Sebastian Castro, Duane Wessels, Marina Fomenkov, and Kimberly Claffy. 2008. A day at the root of the internet. ACM SIGCOMM Computer Communication Review 38, 5 (2008), 41--46.

Digital Library

[10]

Yizheng Chen, Manos Antonakakis, Roberto Perdisci, Yacin Nadji, David Dagon, and Wenke Lee. 2014. DNS noise: Measuring the pervasiveness of disposable domains in modern DNS traffic. In 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE, 598--609.

Digital Library

[11]

S. Cheshire and M. Krochmal. 2013. DNS-Based Service Discovery. RFC 6763 (Proposed Standard)., 49 pages. https://doi.org/10.17487/RFC6763 Updated by RFC 8553.

[12]

C. Contavalli, W. van der Gaast, D. Lawrence, and W. Kumari. 2016. Client Subnet in DNS Queries. RFC 7871 (Informational)., 30 pages. https://doi.org/10.17487/RFC7871

[13]

Wouter B de Vries, Quirin Scheitle, Moritz Müller, Willem Toorop, Ralph Dolmans, and Roland van Rijswijk-Deij. 2019. A First Look at QNAME Minimization in the Domain Name System. In International Conference on Passive and Active Network Measurement. Springer, 147--160.

Digital Library

[14]

Luca Deri, Lorenzo Luconi Trombacchi, Maurizio Martinelli, and Daniele Vannozzi. 2012. A Distributed DNS Traffic Monitoring System. In 2012 8th International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 30--35.

[15]

David Dittrich et al. 2012. The Menlo Report: Ethical Principles Guiding Information and Communication Technology Research. US DHS (2012).

[16]

DNS-OARC, CAIDA, ISC. 2018. A Day in the Life of the Internet (DITL). https://www.dns-oarc.net/oarc/data/ditl.

[17]

D. Eastlake 3rd and M. Andrews. 2016. Domain Name System (DNS) Cookies. RFC 7873 (Proposed Standard)., 25 pages. https://doi.org/10.17487/RFC7873

[18]

Farsight Security. 2019. DNSDB. https://www.farsightsecurity.com/solutions/dnsdb/.

[19]

Farsight Security. 2019. Farsight Grant Programs. https://www.farsightsecurity.com/grant-access/.

[20]

Farsight Security. 2019. Passive DNS Sensor. https://www.farsightsecurity.com/technical/passive-dns/passive-dns-sensor/.

[21]

Farsight Security. 2019. SIE Data Sharing. https://www.farsightsecurity.com/community/data-sharing/.

[22]

Farsight Security. 2019. SIE: Security Information Exchange. https://www.farsightsecurity.com/solutions/security-information-exchange/.

[23]

Shir Landau Feibish, Yehuda Afek, Anat Bremler-Barr, Edith Cohen, and Michal Shagam. 2017. Mitigating DNS random subdomain DDoS attacks by distinct heavy hitters sketches. In Proceedings of the fifth ACM/IEEE Workshop on Hot Topics in Web Systems and Technologies. ACM, 8.

Digital Library

[24]

Hongyu Gao, Vinod Yegneswaran, Yan Chen, Phillip Porras, Shalini Ghosh, Jian Jiang, and Haixin Duan. 2013. An empirical reexamination of global DNS behavior. In ACM SIGCOMM Computer Communication Review, Vol. 43. ACM, 267--278.

Digital Library

[25]

Hongyu Gao, Vinod Yegneswaran, Jian Jiang, Yan Chen, Phillip Porras, Shalini Ghosh, and Haixin Duan. 2016. Reexamining DNS from a global recursive resolver perspective. IEEE/ACM Transactions on Networking (TON) 24, 1 (2016), 43--57.

Digital Library

[26]

Oliver Gasser, Quirin Scheitle, Pawel Foremski, Qasim Lone, Maciej Korczynski, Stephen D. Strowes, Luuk Hendriks, and Georg Carle. 2018. Clusters in the Expanse: Understanding and Unbiasing IPv6 Hitlists. In Proceedings of the 2018 Internet Measurement Conference. ACM, New York, NY, USA, 15. https://doi.org/10.1145/3278532.3278564

Digital Library

[27]

Oliver Gasser, Quirin Scheitle, Sebastian Gebhard, and Georg Carle. 2016. Scanning the IPv6 Internet: Towards a Comprehensive Hitlist. In TMA.

[28]

Google. 2019. IPv6 Adoption. https://www.google.com/intl/en/ipv6/statistics.html.

[29]

Google. 2019. Protocol Buffers. https://developers.google.com/protocol-buffers/.

[30]

Stefan Heule, Marc Nunkesser, and Alex Hall. 2013. HyperLogLog in Practice: Algorithmic Engineering of a State of The Art Cardinality Estimation Algorithm. In Proceedings of the EDBT 2013 Conference. Genoa, Italy.

Digital Library

[31]

Hubert, Bert. 2019. Herding the DNS Camel. https://www.ietf.org/blog/herding-dns-camel/.

[32]

Hubert, Bert. 2019. The DNS Camel. https://powerdns.org/dns-camel/.

[33]

Hubert, Bert. 2019. The DNS Camel... https://blog.apnic.net/2018/03/29/the-dns-camel/.

[34]

Geoff Huston. 2019. DNS Query Privacy. https://blog.apnic.net/2019/08/12/dns-query-privacy/.

[35]

Huston, Geoff. 2019. AS Names. https://www.potaroo.net/.

[36]

ICANN. 2019. DNS Stats. http://stats.dns.icann.org/.

[37]

ICANN. 2019. List of Top-Level Domains. https://www.icann.org/resources/pages/tlds-2012-02-25-en.

[38]

InterNIC. 2019. Root zone data. https://www.internic.net/domain/root.zone.

[39]

Cheng Jin, Haining Wang, and Kang G Shin. 2003. Hop-count filtering: an effective defense against spoofed DDoS traffic. In Proceedings of the 10th ACM conference on Computer and communications security. ACM, 30--41.

Digital Library

[40]

Jaeyeon Jung, Arthur W. Berger, and Hari Balakrishnan. 2003. Modeling TTL-based Internet Caches. San Francisco, CA, USA. http://www.ieee-infocom.org/2003/papers/11_01.PDF

[41]

Jaeyeon Jung, E. Sit, H. Balakrishnan, and R. Morris. 2002. DNS performance and the effectiveness of caching. IEEE/ACM Transactions on Networking 10, 5 (Oct 2002), 589--603. https://doi.org/10.1109/TNET.2002.803905

[42]

Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2019. Tranco: a research-oriented top sites ranking hardened against manipulation. In Proceedings of the 26th Annual Network and Distributed System Security Symposium. Internet Society.

[43]

Cricket Liu and Paul Albitz. 2006. DNS and BIND (5 ed.). O'Reilly Media, Inc., 192--194.

[44]

Ahmed Metwally, Divyakant Agrawal, and Amr El Abbadi. 2005. Efficient computation of frequent and top-k elements in data streams. In International Conference on Database Theory. Springer, 398--412.

[45]

P.V. Mockapetris. 1983. Domain names: Concepts and facilities. RFC 882., 31 pages. https://doi.org/10.17487/RFC0882 Obsoleted by RFCs 1034, 1035, updated by RFC 973.

[46]

P.V. Mockapetris. 1983. Domain names: Implementation specification. RFC 883., 74 pages. https://doi.org/10.17487/RFC0883 Obsoleted by RFCs 1034, 1035, updated by RFC 973.

[47]

P.V. Mockapetris. 1987. Domain names - concepts and facilities. RFC 1034 (Internet Standard)., 55 pages. https://doi.org/10.17487/RFC1034 Updated by RFCs 1101, 1183, 1348, 1876, 1982, 2065, 2181, 2308, 2535, 4033, 4034, 4035, 4343, 4035, 4592, 5936, 8020, 8482.

[48]

Giovane C. M. Moura, John Heidemann, Moritz Müller, Ricardo de O. Schmidt, and Marco Davids. 2018. When the Dike Breaks: Dissecting DNS Defenses During DDoS. https://doi.org/10.1145/3278532.3278534

[49]

Mozilla Foundation. 2019. Public Suffix List. https://publicsuffix.org/.

[50]

Nipravsky, Tom. 2018. Meet MyloBot -- A New Highly Sophisticated Never-Seen-Before Botnet That's Out In The Wild. https://www.deepinstinct.com/2018/06/20/meet-mylobot-a-new-highly-sophisticated-never-seen-before-botnet-thats-out-in-the-wild/.

[51]

OpenINTEL. 2019. Data dictionary. https://openintel.nl/background/dictionary/.

[52]

V. Pappas, D. Massey, and L. Zhang. 2007. Enhancing DNS Resilience against Denial of Service Attacks. In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07). 450--459. https://doi.org/10.1109/DSN.2007.42

Digital Library

[53]

David Pariag and Tim Brecht. 2017. Application bandwidth and flow rates from 3 trillion flows across 45 carrier networks. In International Conference on Passive and Active Network Measurement. Springer, 129--141.

[54]

Craig Partridge and Mark Allman. 2016. Ethical Considerations in Network Measurement Papers. Commun. ACM (2016).

[55]

Lin Quan, John Heidemann, and Yuri Pradkin. 2014. When the Internet Sleeps: Correlating Diurnal Networks With External Factors. In Proceedings of the ACM Internet Measurement Conference. ACM, Vancouver, BC, Canada, 87--100. https://doi.org/10.1145/2663716.2663721

Digital Library

[56]

Rapid7 Labs. 2019. Forward DNS: data sources. https://github.com/rapid7/sonar/wiki/Forward-DNS.

[57]

Rapid7 Labs. 2019. Project Sonar: Forward DNS (FDNS). https://opendata.rapid7.com/sonar.fdns_v2/.

[58]

RIPE Atlas. 2019. Measurements. https://atlas.ripe.net/measurements/.

[59]

RIPE NCC Staff. 2015. RIPE Atlas: A Global Internet Measurement Network. Internet Protocol Journal 18, 3 (2015).

[60]

Quirin Scheitle, Oliver Hohlfeld, Julien Gamba, Jonas Jelten, Torsten Zimmermann, Stephen D Strowes, and Narseo Vallina-Rodriguez. 2018. A long way to the top: significance, structure, and stability of internet top lists. In Proceedings of the Internet Measurement Conference 2018. ACM, 478--493.

Digital Library

[61]

D. Schinazi and T. Pauly. 2017. Happy Eyeballs Version 2: Better Connectivity Using Concurrency. RFC 8305 (Proposed Standard)., 15 pages. https://doi.org/10.17487/RFC8305

[62]

SIDN Labs. 2019. .nl stats and data. https://stats.sidnlabs.nl/en/dns.html.

[63]

SWITCH. 2019. DNS Queries: UDP Compared to TCP. https://www.nic.ch/statistics/dns/udp-tcp/.

[64]

University of Oregon. 2019. Route Views Project. http://www.routeviews.org/routeviews/.

[65]

Roland van Rijswijk-Deij, Mattijs Jonker, Anna Sperotto, and Aiko Pras. 2016. A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements. IEEE Journal on Selected Areas in Communications 34, 6 (2016), 1877--1888.

[66]

Stefan Viehböck. 2018. Millions of Xiongmai Video Surveillance Devices Can be Hacked via Cloud Feature (XMEye P2P Cloud). https://sec-consult.com/en/blog/2018/10/millions-of-xiongmai-video-surveillance-devices-can-be-hacked-\via-cloud-feature-xmeye-p2p-cloud/.

[67]

Paul Vixie. 2007. DNS complexity. Queue 5, 3 (2007), 24--29.

Digital Library

[68]

Duane Wessels. 2018. ipv4-heatmap tool. https://github.com/measurement-factory/ipv4-heatmap.

[69]

D. Wing and A. Yourtchenko. 2012. Happy Eyeballs: Success with Dual-Stack Hosts. RFC 6555 (Proposed Standard)., 15 pages. https://doi.org/10.17487/RFC6555 Obsoleted by RFC 8305.

[70]

Maarten Wullink, Giovane CM Moura, Moritz Müller, and Cristian Hesselman. 2016. ENTRADA: A high-performance network traffic data streaming warehouse. In Network Operations and Management Symposium (NOMS), 2016 IEEE/IFIP. IEEE, 913--918.

Digital Library

Cited By

Le Pochat VFernandez SVan Goethem TTajalizadehkhoob SDesmet LDuda AJoosen WKorczyński M(2024)Evaluating the Impact of Design Decisions on Passive DNS-Based Domain Rankings2024 8th Network Traffic Measurement and Analysis Conference (TMA)10.23919/TMA62044.2024.10559182(1-11)Online publication date: 21-May-2024
https://doi.org/10.23919/TMA62044.2024.10559182
Boeira DScheid EFranco MZembruzki LGranville L(2024)Traffic Centralization and Digital Sovereignty: An Analysis Under the Lens of DNS ServersNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575700(1-9)Online publication date: 6-May-2024
https://doi.org/10.1109/NOMS59830.2024.10575700
Zhang MLi XLiu BLu JZhang YChen JDuan HHao SZheng X(2023)Detecting and Measuring Security Risks of Hosting-Based Dangling DomainsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35794407:1(1-28)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3579440
Show More Cited By

Index Terms

DNS Observatory: The Big Picture of the DNS
1. Information systems
  1. Information systems applications
    1. Data mining
      1. Data stream mining
2. Networks
  1. Network performance evaluation
    1. Network measurement
  2. Network services
    1. Naming and addressing

Recommendations

Comparing DNS resolvers in the wild
IMC '10: Proceedings of the 10th ACM SIGCOMM conference on Internet measurement

The Domain Name System (DNS) is a fundamental building block of the Internet. Today, the performance of more and more applications depend not only on the responsiveness of DNS, but also the exact answer returned by the queried DNS resolver, e.g., for ...
Securing DNS: Extending DNS Servers with a DNSSEC Validator

DNS Security Extensions (DNSSEC) is a proposed standard for securely authenticating information in the Domain Name System. DNSSEC validators check the digital signatures on DNS data. However, designing a validator worth the operational costs is a ...
DNS-based Internet Client Clustering and Characterization

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IMC '19: Proceedings of the Internet Measurement Conference

October 2019

497 pages

ISBN:9781450369480

DOI:10.1145/3355369

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

IMC '19

Sponsor:

IMC '19: ACM Internet Measurement Conference

October 21 - 23, 2019

Amsterdam, Netherlands

Acceptance Rates

IMC '19 Paper Acceptance Rate 39 of 197 submissions, 20%;

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Upcoming Conference

IMC '24

Sponsor:
sigcomm
sigcomm

ACM Internet Measurement Conference

November 4 - 6, 2024

Madrid , AA , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
1,099
Total Downloads

Downloads (Last 12 months)158
Downloads (Last 6 weeks)24

Reflects downloads up to 05 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Le Pochat VFernandez SVan Goethem TTajalizadehkhoob SDesmet LDuda AJoosen WKorczyński M(2024)Evaluating the Impact of Design Decisions on Passive DNS-Based Domain Rankings2024 8th Network Traffic Measurement and Analysis Conference (TMA)10.23919/TMA62044.2024.10559182(1-11)Online publication date: 21-May-2024
https://doi.org/10.23919/TMA62044.2024.10559182
Boeira DScheid EFranco MZembruzki LGranville L(2024)Traffic Centralization and Digital Sovereignty: An Analysis Under the Lens of DNS ServersNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575700(1-9)Online publication date: 6-May-2024
https://doi.org/10.1109/NOMS59830.2024.10575700
Zhang MLi XLiu BLu JZhang YChen JDuan HHao SZheng X(2023)Detecting and Measuring Security Risks of Hosting-Based Dangling DomainsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/35794407:1(1-28)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3579440
Streibelt FSattler PLichtblau FGañán CFeldmann AGasser OFiebig T(2023)How Ready is DNS for an IPv6-Only World?Passive and Active Measurement10.1007/978-3-031-28486-1_22(525-549)Online publication date: 21-Mar-2023
https://dl.acm.org/doi/10.1007/978-3-031-28486-1_22
van der Toorn OMüller MDickinson SHesselman CSperotto Avan Rijswijk-Deij R(2022)Addressing the challenges of modern DNS a comprehensive tutorialComputer Science Review10.1016/j.cosrev.2022.10046945(100469)Online publication date: Aug-2022
https://doi.org/10.1016/j.cosrev.2022.100469
Yang DLi ZJiang HTyson GLi HXie GZeng Y(2022)A deep dive into DNS behavior and query failuresComputer Networks10.1016/j.comnet.2022.109131(109131)Online publication date: Jun-2022
https://doi.org/10.1016/j.comnet.2022.109131
Hounsel ASchmitt PBorgolte KFeamster N(2021)Designing for Tussle in Encrypted DNSProceedings of the 20th ACM Workshop on Hot Topics in Networks10.1145/3484266.3487383(1-8)Online publication date: 10-Nov-2021
https://dl.acm.org/doi/10.1145/3484266.3487383
Imana BKorolova AHeidemann JFeamster NLutu A(2021)Institutional privacy risks in sharing DNS dataProceedings of the 2021 Applied Networking Research Workshop10.1145/3472305.3472324(69-75)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1145/3472305.3472324
Hounsel ASchmitt PBorgolte KFeamster NFeamster NLutu A(2021)Encryption without centralizationProceedings of the 2021 Applied Networking Research Workshop10.1145/3472305.3472318(62-68)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1145/3472305.3472318
Ma SLi JKim HBertino ENepal SOstry DSun C(2021)Fine with "1234"?Proceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00148(1671-1682)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00148
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents