Spaces:
Paused
Paused
| User-agent: * | |
| Allow: / | |
| Disallow: /api/ | |
| Disallow: /gateway/ | |
| Disallow: /admin/ | |
| # Allow good bots specifically | |
| User-agent: Googlebot | |
| Allow: / | |
| User-agent: Bingbot | |
| Allow: / | |
| # Block known bad bots | |
| User-agent: AhrefsBot | |
| Disallow: / | |
| User-agent: SemrushBot | |
| Disallow: / | |
| User-agent: MJ12bot | |
| Disallow: / | |
| Sitemap: https://opticparse.com/sitemap.xml | |