# https://www.robotstxt.org/robotstxt.html # https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#url-matching-based-on-path-values # Applies to all user-agents User-agent: * # Allow (which override below disallow of /*test* because the rule path is longer/more specific) Allow: /p/* Allow: /blog/* Allow: /p/*test* Allow: /blog/*test* # Disallow test users Disallow: /RF-user* Disallow: /rf-user* Disallow: /*test* Sitemap: https://www.flux.ai/sitemap.xml