User-Agent: * Allow: / User-Agent: SiteAuditBot Crawl-delay: 1 Allow: / User-agent: SemrushBot Allow: / User-Agent: SemrushBot-SI Allow: /