
First of all "What is Robots.txt" file
Robots.txt file restricts Crawlers Robots access to your site by search engine and blocked those urls listed in robots.txt to listed in search results. This file is mainly for your privacy if you have some pages in your blog which have information which you don't want to listed in search engine you can include url of that pages into robot.txt file.
Robots.txt file restricts Crawlers Robots access to your site by search engine and blocked those urls listed in robots.txt to listed in search results. This file is mainly for your privacy if you have some pages in your blog which have information which you don't want to listed in search engine you can include url of that pages into robot.txt file.
How To Create Robots.txt file in Bloggger
You can generate robot.txt file in blogger using Google Webmaster Tool Just Sign up with your Google account and you use all services free. you can also allow blocked specific files and directories by Generate robots.txt
You can generate robot.txt file in blogger using Google Webmaster Tool Just Sign up with your Google account and you use all services free. you can also allow blocked specific files and directories by Generate robots.txt
To Generate robot.txt using webmaster tool in blogger you must verify your sit or Blog with google. So after sign in Google webmaster tools home page first of all add your site by clicking add a site button and Enter the URL of a site you'd like to manage. for eg www.google.com and click continue On next page choose the Verification method Html or Meta Tag
And Click on varify Button after this
To generate a robots.txt file:
- On the Webmaster Tools Home page, click the site you want.
- Under Site configuration, click Crawler access.
- Click the Generate robots.txt tab.
Why Google Blocked my url
In blogger we specified some labels for our posts when a particular labels are associated with many posts than crawlers find duplicated results in our blogs like You have 5 post labeled with keyword "Movie Review" then for search result for "Movie Review" instead of listing your 5 post of different movie review Google crawlers list a single labeled URL. So it automatically blocked these urls to access in search result for or benefits.
You can also check Following links to More about it (Copy and paste in the browser)
http://www.webmasterworld.com/google/3468726.htm
http://googlewebmastercentral.blogspot.com/2006/09/debugging-blocked-urls_19.html
http://www.google.com/support/webmasters/bin/answer.py?answer=35237
http://www.google.com/support/webmasters/bin/answer.py?answer=40360&ctx=sibling
http://www.google.com/support/webmasters/bin/answer.py?answer=35303&ctx=sibling
http://www.google.com/support/webmasters/bin/answer.py?answer=83098&ctx=sibling
https://www.google.com/webmasters/tools/home?hl=en
thanks buddy for the information
ReplyDeleteThanks for the info...I was thinking there might some issue with the HTML changes i had done to make it more SEO friendly.
ReplyDeleteWhy Can't I Even Copy Paste anything Huh
ReplyDelete