Subject: Re: Yahoo Canada!

From: [email-masked] (Richard Sexton)
Date: 20 Jul 1996 20:45:06 -0400

In article <[email-masked]>, Iain Grant <[email-masked]> wrote:
>[email-masked] (Richard J. Sexton) wrote:
>
>>AltaVista is supposed to slurp down all the
>>web every three days.
>
>Good grief!  That's incredible.  If it's true, how can they get the
>bandwidth to even get that much information into their server ?

I have all the exact numbers here somewhere, Brian Reid, head of the
Network Systems Laboratory at Digital is an old and very good
friends of mine. He keeps sending me things like:

	"go here: <url>"

and I'd look and think "whatthe hell is this ?" About 6 mos.
later it would sink in and six months later again, it'd be released.

For exmple, in 1992 or so he sent me a url and said "Look
here, Palo Alto is the first city on the web". I ignored it
and though to myself "Whats the web and what the HELL is this
wierd email address'

At any rate, AV is 5 (or 6 ot 7) big huge mother Dec Alpha's
with a half gig of RAM and an OC-48 ATM connection, roughly
1.8 Gigabits per second, roughly 45 T3's or about 2000 T1's.

I've forgotten how much storage it took, but it was less than
a casual guess would indicate. They have a fancy compression
scheme, as one might guess.

>ever mind the storage to save it all ?   And still facilitate user
>requests for information.

The cool part is, you always seem to get a response form AV in about
a second. 2/3'rds of that second is the transit time to and from 
Palo Alto, which gives you some idea of the speed of the whole
system.

Dec does this for free, and always will, I've been told. They do
it to showcase their servers. Dec sells servers.

-- 
Richard Sexton
[email-masked]