# /robots.txt file for *.wtfrpg.com # # There is no reason to crawl the following: # 1. Pages which relate to content creation or editing. # 2. Login, functional, form-related or similar pages. # 3. Pages containing redundant information (Entity news, spells) User-agent: Mediapartners-Google Disallow: User-agent: * Crawl-delay: 100 Disallow: /user-add.html Disallow: /user-edit.html Disallow: /user-login.html Disallow: /user-logout.html Disallow: /entity-add-class.html Disallow: /entity-add-race.html Disallow: /entity-add-stats.html Disallow: /entity-spells/ Disallow: /entity-mail/ Disallow: /entity-quests/ Disallow: /entity-shop/ Disallow: /entity-blog/post/ Disallow: /entity-edit/ Disallow: /entity-edit-image/ Disallow: /entity-news/ # Disallow: /quests.html # Disallow: /items.html