# Sitemaps Sitemap: https://kroja.my/sitemap.xml # Rules for all legitimate search engines User-agent: * Allow: / Disallow: /login/ Disallow: /register/ Disallow: /user/ Disallow: /plan/ Disallow: /ms/ Disallow: /job-alert/ Disallow: /booking/ Disallow: /*?set_lang=ms Disallow: /*.pdf$ Disallow: /*.docx$ # Block aggressive / AI bots completely User-agent: Yandex Disallow: / User-agent: dotbot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: /