User-agent: * Disallow: /chatbot/ Allow: / Crawl-delay:3 Sitemap:https://nikamooz.tech/robots.txt Disallow: /wp-admin/ Disallow: /wp-content/plugins/ Disallow: /wp-includes/js/* Disallow: /author/* Disallow: /comment-page-* Disallow: *.pdf