Interesting Robot Crawl Logged

Firewalls, routers, servers, switches, SANs, PBXes, security and related topics
Locked
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Interesting Robot Crawl Logged

Post by Red Squirrel »

Seems like someone is working on making their own search engine and has a bot and everything. IceTeks got crawled by it! Not sure how it found us, but it's pretty sophisticated!

http://www.otthon.net/search/

The actual search engine does not seem to have been released though. He's just working on the bot. It's pretty cool we got crawled. :)



I've always wanted to make my own search engine myself, just for fun, but I can't seem to figure out tcp/ip winsock. Once I have a linux box I might do some Linux programming and go from there instead. Linux sockets are much more reliable anyway. Winsock sucks for programing (not enough error handling causes unwanted behavior)

Archived topic from Iceteks, old topic ID:753, old post ID:6308
Honk if you love Jesus, text if you want to meet Him!
Chris Vogel
Posts: 5140
Joined: Fri Jan 10, 2003 1:14 am

Interesting Robot Crawl Logged

Post by Chris Vogel »

That is cool! :banana: :banana:

I think it must be a nice feeling to know that one little person far away has catalogued your Web site. That is how I feel when someone has read my blog! :banana: :banana:

And to think: Some people will go through their entire lives without seeing IceTeks.... :cry:

This makes me think... I am not going to see all the Internet. Think about what I will miss. I am going to miss some unique things..... :banghead: It is actually sort of sad because that person on the other end wanting me to come will NEVER see me. The Internet is so huge. It is hard to think about how big it really is. :eek:

Archived topic from Iceteks, old topic ID:753, old post ID:6311
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Interesting Robot Crawl Logged

Post by Red Squirrel »

That's a good point never looked at it that way.


I just thought of something funny, since google search results have unique urls, I wonder what would happen if I were to make a page that links to a bunch, and submit it on google. :roflmao2: Just imagine the confusion with the page rank system. I'm sure they have a system to avoid that though, their system is very sophisticated.

Archived topic from Iceteks, old topic ID:753, old post ID:6321
Honk if you love Jesus, text if you want to meet Him!
Locked