(www.merklemap.com)

90 points Eikon | 1 comments | 15 Oct 24 19:26 UTC | HN request time: 0.238s | source

Hi HN,

I've been working on building a pipeline to create a DNS records database lately. The goal is to enable research as well as competitive landscape analysis on the internet.

The dataset for now spans around 4 billion records and covers all the common DNS record types:

    A
    AAAA 
    ANAME
    CAA
    CNAME
    HINFO
    HTTPS
    MX
    NAPTR
    NS
    PTR 
    SOA
    SRV
    SSHFP
    SVCB
    TLSA
    TXT

Each line in the CSV file represents a single DNS record in the following format: www.example.com,A,93.184.215.14

Let me know if you have any questions or feedback!

1. blex ◴[18 Oct 24 15:45 UTC] No.41880482[source]▶

>>41852190 (OP) #

Is there a good tool to browse big text archives, like .csv.xz, .csv.gz, or .7z, without decompressing them?

I don't want to decompress 29 GB into 211 GB each time I want to make a search.

Except grep / zgrep, is there a good tool/viewer (or hex editor that can decompress parts of big files for display) for this general task?

↑

Show HN: 4B+ DNS Records Dataset