| Nov 21, 2009 |
one little trick I've found recently... If you don't want to get your page cached to search engine, you may use this tweak:QUOTEQ. Can I prevent Teoma/Ask Jeeves search engine from showing a cached copy of my page?A: Yes. We obey the "noarchive" meta tag. If you place the following command in your HTML page, we will not provide an archived copy of the document to the user.< META NAME = "ROBOTS" CONTENT = "NOARCHIVE" >If you would like to specify this r...
read more.
Where would you put your robot once you have it? I can't find that information on the site. Would you put it on a server (like astahost) or on your own computer? And does indexing take a lot of space and bandwidth?
I think it takes alot of space... and consumes bandwith and much of processor time...
Oh. Well, then, I'd better just have Google index my site for me, instead of wearing out Astahost's servers. Nice link, though.
Anyway many of that WEB-robots are capable to catch emails, and harvest them into database... spammers use that very often, to get targeted audience. I figured if we research some of this methods, maybe we could better protect ourself from spam and junk emails... Anyway, I'll post what I discover later here..
google robots are the best robots and it has like hundreds of them , it uses a lot of bandwith so it isn`t good for little sites , they are good for sites like google , msn etc
QUOTE(sagaxx @ Dec 26 2005, 06:05 PM) google robots are the best robots and it has like hundreds of them , it uses a lot of bandwith so it isn`t good for little sites , they are good for sites like google , msn etc  well idea of this is testing and learning, anyway if you know how many spiders work, you will be able to make your web pages better and that way increase your page rank on search engines - and that's what we all want 
the link is good, but not much of my use. i will try to see how to make it in use for futur..
for the time being i will let google do the work for me ;-)
I think that, with the current state of search engine optimization, we should let the SEs do the indexing themselves. Unless one knows exactly how every single SE works to index and rank their site, one will most likely hurt his ranking at one engine or another.
I remember one time when Google used H1 and TITLE tags as a primary criteria for their ranking, while MSN had them at 4th and 6th. And right now, Google mainly uses a system called Vector Analysis, where they analyse the overall theme of your Website, and adjust your ranking accordingly. It's still work-in-progress, but Google partly uses it, while not many others do. So my point is, until SEs can reach a certain level of standardization, we should let each SE do what it likes most. Plus, it doesn't really take that much bandwidth, not more than any hungry visitor to your site would take. QUOTE(Khymnon @ Jan 3 2006, 01:33 PM) I remember one time when Google used H1 and TITLE tags as a primary criteria for their ranking, while MSN had them at 4th and 6th. And right now, Google mainly uses a system called Vector Analysis, where they analyse the overall theme of your Website, and adjust your ranking accordingly. It's still work-in-progress, but Google partly uses it, while not many others do. When you say "theme", do you mean the content theme or the visual theme of the site? And how can the robot adjust the ranking with the H1, title or theme? I don't get how a computer can tell what is good and what is bad. Latest Entries
one little trick I've found recently... If you don't want to get your page cached to search engine, you may use this tweak:
QUOTE Q. Can I prevent Teoma/Ask Jeeves search engine from showing a cached copy of my page? A: Yes. We obey the "noarchive" meta tag. If you place the following command in your HTML page, we will not provide an archived copy of the document to the user. < META NAME = "ROBOTS" CONTENT = "NOARCHIVE" > If you would like to specify this restriction just for Teoma/Ask Jeeves, you may use "teoma" in place of "robots". Quoted from this page so this way, if you are building a web, for example, and you don't want to be indexed on google, because you plan to change a domain name or server later, so you want to be indexed then, so that way search engines like google will get your final version of web, when you finally remove this tag. 
QUOTE(szupie @ Jan 4 2006, 12:18 AM) When you say "theme", do you mean the content theme or the visual theme of the site? And how can the robot adjust the ranking with the H1, title or theme? I don't get how a computer can tell what is good and what is bad. hmm... he probably think on content theme, because I don't see how an software can understand graphic... 
|
|
 Web Robots / Crawlers / Spiders Etc. - All you need to know about them... |
Affordable Web Hosting, Low cost Web Hosting - ComputingHost.com
