Hacker News

Eikon

Show HN: 4B+ DNS Records Dataset merklemap.com

Hi HN,

I've been working on building a pipeline to create a DNS records database lately. The goal is to enable research as well as competitive landscape analysis on the internet.

The dataset for now spans around 4 billion records and covers all the common DNS record types:

    A
    AAAA 
    ANAME
    CAA
    CNAME
    HINFO
    HTTPS
    MX
    NAPTR
    NS
    PTR 
    SOA
    SRV
    SSHFP
    SVCB
    TLSA
    TXT

Each line in the CSV file represents a single DNS record in the following format: www.example.com,A,93.184.215.14

Let me know if you have any questions or feedback!

genmud8 hours ago

Neat! How is this different than domaintools/farsight [1]?

Passive DNS [2] has been in my toolbox for 15+ years, and is invaluable for security research / threat intelligence. Knowing historical resolutions to something are so helpful in investigations.

For anyone interested, they should check out the talk by one of the DomainTools people [3] on how it can be utilized for investigation.

Are you passively collecting this data, or actively querying for these records?

[1] - https://www.domaintools.com/products/threat-intelligence-fee...

[2] - https://www.circl.lu/services/passive-dns/

[3] - https://www.youtube.com/watch?v=oXmapqLkZd0

Eikonop4 hours ago

From what I understand [1] is just tlds, not subdomains?

genmud3 hours ago

That would be incorrect, they get subdomains for passive dns feeds.

35mm3 hours ago

How often is it updated?

Does it include expired domains?

source