# robots.txt file for hjelmborg.dk # Skabt for sjov i 2006 User-agent: * Disallow: /php/ User-agent: Amazonbot # Amazon's user agent Disallow: / # disallow access to all the website User-agent: PetalBot Disallow: / User-agent: Barkrowler Disallow: / # Sitemap: http://www.hjelmborg.dk/sitemap.xml # https://seranking.com/free-tools/robots-txt-tester.html # https://www.portent.com/blog/seo/robots-txt-mistakes-best-uses-guide.htm # https://stackoverflow.com/questions/28495972/robots-txt-allow-all-except-few-sub-directories