How about something like this (not suggesting as an DNS replacement!):
Search Engine requires a key certificate style license embeded in
the html or website (whether it's internet centralized or just for the
search engine). A "rating" and "type" is associated with the license for
the type of site it is. As the search engine crawls the site, if the
license is gone... it doesn't get listed. If the type and rating doesn't
match the information associated with the license data... it doesn't get
listed. If the license certificate doesn't match the IP(s) associated with
the certificate... it doesn't get listed. Has anybody tried something like
this before?
Karyn
:: -----Original Message-----
:: From: owen@dixon.delong.sj.ca.us [mailto:owen@dixon.delong.sj.ca.us]
:: Sent: Friday, March 16, 2001 9:18 AM
:: To: nanog@merit.edu; smcmahon@eiv.com
:: Subject: Re: Reality Check
::
::
::
::
:: > On Thu, Mar 15, 2001 at 03:07:30PM -0800, Scott Francis wrote:
:: > >
:: > > The same way people have learned to make sure that a
:: search for "Anna
:: > > Kournikova" (for instance) returns, say, 200 records
:: that are sites/pages
:: > > that have nothing whatever to do with Anna Kournikova,
:: and a whole LOT to do
:: > > with bringing in cash to the sites in question.
:: >
:: > This is self-defeating in the end; if your search site
:: doesn't work, people
:: > will stop using it.
:: >
:: Yes, but it's not the search engines that do it. It's the
:: web sites that
:: have learned how to put stuff in the view of the crawlers
:: the search engines
:: use that will show up when someone is looking for unrelated content.
::
:: Since they do this in such a way that virtually EVERY search
:: engine finds
:: their bogus content, they don't care how many search engines
:: go out of
:: business, they'll just afflict the next one to come up.
::
:: > If they stop using it, the advertising dollars will stop
:: rolling in.
:: >
:: > Thus, it's in the best interest of the owner of the search
:: site to fix
:: > the problem. Hence why people are flocking to the latest
:: best technology
:: > they can find, such as Google.
:: >
:: Google is not immune to this, althoug it is better than some.
::
::
:: Owen
::
On Fri, Mar 16, 2001 at 10:36:31AM -0800, Karyn Ulriksen had this to say:
How about something like this (not suggesting as an DNS replacement!):
Search Engine requires a key certificate style license embeded in
the html or website (whether it's internet centralized or just for the
search engine). A "rating" and "type" is associated with the license for
the type of site it is. As the search engine crawls the site, if the
license is gone... it doesn't get listed. If the type and rating doesn't
match the information associated with the license data... it doesn't get
listed. If the license certificate doesn't match the IP(s) associated with
the certificate... it doesn't get listed. Has anybody tried something like
this before?
good in theory, but in practice ... bleah. Let's just say the thought of
having to go through every page of just my own site to add HTML in a certain
spot makes me nauseous. I can't imagine doing this on sites with thousands of
pages.
Now, if you could mandate it in the robots.txt somehow ... but I've been
noticing enough underhanded tricks in robots.txt using wget (like zero-sec
redirects to pages that don't exist, or redirect to themselves) to not trust
that option either.
And say, didn't RSAC try this already? I seem to recall this being a
configurable option in MSIE under the security settings ...
Search Engine requires a key certificate style license embeded in
the html or website (whether it's internet centralized or just for the
And the key came from *where*? And this would be different from the
current hassles of getting a PKI certificate *how*?
search engine). A "rating" and "type" is associated with the license for
the type of site it is. As the search engine crawls the site, if the
license is gone... it doesn't get listed. If the type and rating doesn't
Woo woo.. So I have to fork over $$$ to a company to publish stuff on the
web? Or is this license "for free" - in which case it's probably worth
what you paid for it?
match the information associated with the license data... it doesn't get
OK.. A .JPG of a man and a giraffe would probably be acceptable for all
ratings. A .JPG of a man and a giraffe having sex would probably NOT be
acceptable. How did you tell the difference?
listed. If the license certificate doesn't match the IP(s) associated with
the certificate... it doesn't get listed. Has anybody tried something like
Great. I relocate my server to a new co-lo and I drop off the search engines
unless I fork out new money for a new license. And if you don't think this
won't happen, I'll point you at all the companies that we currently license
software from who are *always* ready at the drop of a hat to ship us a new
license for a machine "because the CPU planar died and was replaced so the
hostid is now different". Yeah. We *never* have a problem getting new
licenses. 
this before?
Umm.. there's PICS and other web rating systems already extant.
They've worked wonders, haven't they?
Valdis Kletnieks
Operating Systems Analyst
Virginia Tech