Jason L Tibbitts III <tibbs@hpc.uh.edu> writes:
> My reply was "too bad." I make my archives publicly available; anyone
> can search them. I also have a homebrew front end search engine based on
> Glimpse which allows things like limiting by date. Why should I care if
> some over-zealous spider went through my entire archives and added them to
> its index? It is they who aren't serving their customers well by doing
> this; my search engine works fine.
Jason,
it sounds like your setup is very similar to mine: custom CGI-based web
front-end using a Glimpse engine. I'm using this for three of my own
mailing lists, but the main difference is that digest archives are not
_directly_ accessible - i.e., the documents are not under the web
directory tree. I have a separate "browse" and "search" CGI's that
allow random access to anything you want, but they reference these
documents outside of the web tree, and thus these messages are not
explored by web robots. I think this is a viable alternative if you
decide that having your archives indexed by the major web search engines
is undesirable, but you still want to make your archives fully
accessible. Note that this solution only works when your HTTP server
has access to files outside of the www document tree (i.e., it does not
do a chroot), and by the same measure, may create a security hole -
caveat programmer.
Also, on the related topic of people unsubscribing (or not subscribing
in the first place) to your list because they're able to get to your
archives via the WWW front-end... I considered this issue as well when
I put up my web front-end, and I decided that it was reasonable to let
people "lurk" without requiring them to be subscribed -- what's the
harm? With my lists, you must still subscribe in order to submit
messages to the list, so I'm not too worried about WWW-originated spam.
Now, another issue: do you let people subscribe via a WWW/CGI script?
-Eric
--
Eric J. Hansen ................................. http://www.worldmachine.com
Worldmachine Technologies ..................... mailto:eric@worldmachine.com
Follow-Ups:
|
|
