DNSmine
Registered: Sept 09, 2008
Posts: 5
|
|
| Sept 09, 2008 at 09:01 AM |
|
Reply with quote | #1 |
Hi All,
As you've probably noticed, DNS Mine is referred to as an 'ambitious DNS/networking project'. It is doing a lot behind the scenes, and gathering a lot of information.
Today's question (or this week's question, or perhaps this month's) is what information you would find most useful. Would you prefer to:
[1] See some DNS information about domains from a day or so ago. For example, if a DNS entry changed today, you could find out what it had been recently.
[2] Search for all domains that have specific characters in them (for example, all domains with 'dnsmine' in them). As an example, you could find out any domains using a specific trademarked term in them.
[3] See statistics about how many domains are on each TLD (either today, or going back a short time in the past)?
[4] See what IPs handle DNS, mail, or WWW for the most domains?
[5] See what domains any IP handles DNS, mail, or WWW for?
Those are the thoughts for what information we might be able to supply next; let us know what you would most like to see.
-Scott
|
| | Commerco
Registered: Sept 16, 2008
Posts: 3
|
|
| Sept 16, 2008 at 11:28 AM |
|
Reply with quote | #2 |
Scott,
First, thank you for putting up a board. Please don't think me rude, I'm just one of those folks who get suspicious of the various data collections that are done on my company's sites. Having this board, I'm hoping you won't mind interacting with the board members and answering a few questions that I have.
I would like to understand the value for the Internet site owners and operators your scans are reaching. Clearly you are looking for feedback on presentation of the data you are collecting, but I guess the question I have is why should other site owners like me want to allow your collection process in the first place?
I know it is narrow minded thinking, but I believe most folks go into something with the "WIIFM" or what's in it for me mentality.
Secondly, I would ask, so, what's in it for you? Why do this?
Thanks.
|
| | DNSmine
Registered: Sept 09, 2008
Posts: 5
|
|
| Sept 16, 2008 at 01:48 PM |
|
Reply with quote | #3 |
Quote: Originally Posted by Commerco
First, thank you for putting up a board. Please don't think me rude, I'm just one of those folks who get suspicious of the various data collections that are done on my company's sites.
Not a problem -- that's why this board is here.
Quote: Originally Posted by Commerco
I would like to understand the value for the Internet site owners and operators your scans are reaching. Clearly you are looking for feedback on presentation of the data you are collecting, but I guess the question I have is why should other site owners like me want to allow your collection process in the first place?
Think of it like archive.org for network administrators. My first reaction to archive.org was "That's neat; I can see what my old website looked like." But over time, I discovered that it is much more valuable than that (for example, to help determine if a website is legitimate before placing an order, or proving that a website had specific content after the owners of the site remove it).
In this case, we're collecting data that nobody else is collecting. That could be because it isn't useful information, or because people don't yet realize it is useful, or maybe it is just too big a task for anyone else to take on.
This data, hopefully, will be useful to network admins. I'm thinking that one of the most useful pieces of data will be recent DNS information -- often after making a DNS change, an admin asks "How can I find out what the DNS record used to be?", to which the answer is "Unless you have a backup, it is probably on some caching DNS servers somewhere, but you won't know where." Good luck finding those caching DNS servers (and accessing them; many won't allow anonymous queries).
Quote: Originally Posted by Commerco
Secondly, I would ask, so, what's in it for you? Why do this?
Because it can be done, nobody is doing it, and it likely can provide some very useful information. As with most ventures, ultimately, it would be nice to find a way to get some dollars out of it. For example, a well-known company in the United States that is trying to track down counterfeit goods being sold through a website in China might be willing to pay money to find the webserver in the United States that was being used to peddle the goods at first.
-Scott
|
| | Commerco
Registered: Sept 16, 2008
Posts: 3
|
|
| Sept 16, 2008 at 05:53 PM |
|
Reply with quote | #4 |
Scott,
Thanks for getting a response up quickly. I think I understand where you might be going with the idea, and I'll grant you a wayback for DNS does sound interesting.
Not sure if you have thought about it, but some sites use wildcard DNS in their host files. I know a few of ours do. So, you might want to be careful about making any assumptions regarding sub domains. In other words, just because one types in some name under a domain does not absolutely mean that domain intentionally has that sub domain name in operation.
In any case, to get to your question list:
[1] See some DNS information about domains from a day or so ago. For example, if a DNS entry changed today, you could find out what it had been recently.
I think it might be interesting to see what domains might be associated with an IP address and if that changes over time. For example, if I see that somespamdomain.tld is hosted somewhere, I might want to see the other spamdomainvariants.tld names that are at or about that IP range during some period of time. That might be useful in predicting bad neighborhoods and helping to develop trust for better neighborhoods.
[2] Search for all domains that have specific characters in them (for example, all domains with 'dnsmine' in them). As an example, you could find out any domains using a specific trademarked term in them.
I think the good people of Netcraft offer some similar service at their site.
[3] See statistics about how many domains are on each TLD (either today, or going back a short time in the past)?
Kind of a cool idea, but again I think that Netcraft publishes some data regarding this already.
[4] See what IPs handle DNS, mail, or WWW for the most domains?
Not sure what you might get from that metric. Can you clarify?
[5] See what domains any IP handles DNS, mail, or WWW for?
Might be nice to also see if there is a pattern ala what you suggested with a wayback machine for DNS.
Best,
Alan
|
| | DNSmine
Registered: Sept 09, 2008
Posts: 5
|
|
| Sept 16, 2008 at 07:48 PM |
|
Reply with quote | #5 |
Quote: Originally Posted by Commerco
I think it might be interesting to see what domains might be associated with an IP address and if that changes over time. For example, if I see that somespamdomain.tld is hosted somewhere, I might want to see the other spamdomainvariants.tld names that are at or about that IP range during some period of time. That might be useful in predicting bad neighborhoods and helping to develop trust for better neighborhoods.
That is a good idea -- we are already keeping track of that information, so we should be able to supply that somehow (e.g. a list of all domains that an IP handles, or a list of domains that an IP started handling over a certain time period).
Quote: Originally Posted by Commerco
[4] See what IPs handle DNS, mail, or WWW for the most domains?
Not sure what you might get from that metric. Can you clarify?
For example, a list of the top 10 DNS servers, by the number of domains they handle. There are a lot of IPs that handle DNS, mail, or WWW for 10,000s of domains.
-Scott
|
| | Commerco
Registered: Sept 16, 2008
Posts: 3
|
|
| Sept 17, 2008 at 12:57 PM |
|
Reply with quote | #6 |
Thanks again for the reply and clarification.
I had another thought as to a service you could offer. When and if an IP sends mail provide some kind of check to determine if that IP logically should be sending mail. This is a bit tricky because DNS provides MX RRs to determine an Incoming SMTP server's IP address. Still, I suppose you could broaden your requests to include TXT RRs and check for things like SPF records as well as Domain Keys to try to develop information on owner specified authoritative senders for a domain. While that won't do anything to determine if a domain that meets these criteria is spamming, it will give a sense as to if the IP has any right to send mail for the domain in the first place.
While that is perhaps a good idea, you should find a host that has experience with and capacity to handle attacks, because as you go that way, expect the bad guys not to like it much.
I've been thinking about how a distributed system might work which could do that sort of thing, making it resist single point failures a quite bit better (much in the spirit of DNS itself). My thought relates to IP / Domain trust, which could help guide network administrators in decision making about how to respond to the traffic they see entering their networks.
Not sure if this all fits in your charter, but I thought I would share.
Alan
EDIT: One more thought, perhaps you could have some kind of TXT record to allow site owners to moderate the number of times your scan takes place. For example TXT "DNSMine 86400" might mean don't come back for a day on this domain. Although some form of information record that allows for other uses might be a better approach. Say something like TXT "INFO V=1.0 DNSMineFreq=86400" which could allow for other uses of that TXT resource record.
My thought in structuring something that way being one day using it to define other TXT services that might evolve, to make the DNS administrators job less of a hassle and possibly reduce some of the DNS traffic.
In other words, if the powers that be like the idea, get its own RR record down the road to tell scan services like yours some basic information about the domain without making the service have to probe for what is not there.
For example, INFO "v=1.0 DNSMineFreq=86400 MXO=0 MXI=0 WWW=0" might mean the domain asks DNSMine to only come every day, there are no MX records, the domain should not be sending outbound mail, and there are no web sites for the domain. So perhaps this domain is only for DNS. Doing something this way could help tell scans like yours to not bother with retrieving the RR records folks who operate the domain choose not to be pinged on specific records. I think, kind of like an extended Robots.txt for DNS services. Obviously, this is not a very defined thing in this post, but it could be further defined if you are interested.
Alan
|
| | midgieman
Registered: Nov 25, 2008
Posts: 1
|
|
|
Reply with quote | #7 |
Very interesting. It's good to get info about reliability of nameservers.
Way to go - more information. Maybe especially related to port 53 and bind, udp v tcp, randomising - or filtering, etc.
Way to make money - information. No products, or at least no directly related products, so that the information has clear integrity for those prepared to pay. Timing good for DNS after this year, possibly especially for integrity of NSs.
Business model - free, patience, those who can pay, do. Think google!
Good luck!
|
| |
|
