Archived and Closed
This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: Archived
- We do not recommend making changes to this file, and it will directly affect what pages are crawled by search engines.
However, this would be the place to make any changes.
After you make your changes you > Save them.
At default the file will contain the following:
There is also a file located at http://www.yourdomain.com/robots_ssl.txt which contains and prevents any secure version of a webpage from being crawled.