Saturday, October 24, 2009

Is it possible to make "robot.txt" file not to read a particular part of a web page ?

i saw sponsered listings on dir.yahoo.com is not cached by search engine . it doesnt uses javascript for that portion. it controls page rank dilution from sponsored links only - how it is possible ? if any friends have a depth knowledge about this field explain me please.



Is it possible to make %26quot;robot.txt%26quot; file not to read a particular part of a web page ?





Some Guide Lines:



1) Filename should be robots.txt not robot.txt



2) See sample robots.txt



#robots.txt. #http://evikas.com



User-agent: *



Disallow: /pages/college/



Disallow: /financialtimes/



Allow: /ref/



Allow: /services/xml/



User-agent: Mediapartners-Google*



Disallow:



3) You can also define this @ page level



%26lt;META NAME=%26quot;ROBOTS%26quot; CONTENT=%26quot;NOINDEX, NOFOLLOW%26quot;%26gt;



4) You can block your website access to search robots by



User-agent: *



Disallow: /



Here You Can Get Very Easy %26amp; Great Details For Your Problem



http://evikas.com/blog/viewtopic.php?p=5...



http://evikas.com/blog/viewtopic.php?p=5...



Is it possible to make %26quot;robot.txt%26quot; file not to read a particular part of a web page ?



i need the robots.txt instructs the search engine not to read an array of a tabular column this is notthe answer for my question - check souurce code of %26quot;dir.yahoo.com%26quot; Report It

No comments:

Post a Comment

Blog Archive