User-Agent: * Allow: /?display=wide Allow: /wp-content/uploads/ Disallow: /wp-content/plugins/ Disallow: /readme.html Disallow: /refer/ Crawl-delay: 5 User-agent: bingbot Disallow: / User-agent: crawl Disallow: / user-agent: AhrefsBot disallow: / user-agent: YandexBot disallow: / user-agent: SemrushBot disallow: / user-agent: oBot disallow: / user-agent: facebookexternalhit disallow: / user-agent: DotBot disallow: / user-agent: nbot disallow: / user-agent: GrapeshotCrawler disallow: /