# robots.txt for CrisisRadar.eu # Updated: 2025-01-05 - Nederlandse Clean URLs + Googlebot Optimized # Google Search Console & Indexing - Priority Access User-agent: Googlebot Allow: / Allow: /sitemap.xml Allow: /xml-sitemap-dynamic.php Crawl-delay: 1 # Bing Search - Priority Access User-agent: Bingbot Allow: / Allow: /sitemap.xml Allow: /xml-sitemap-dynamic.php Crawl-delay: 1 # All other crawlers User-agent: * Allow: / Allow: /wat-is-crisisradar Allow: /voor-wie Allow: /functionaliteiten Allow: /voordelen Allow: /technische-informatie Allow: /implementatie Allow: /roadmap Allow: /contact Allow: /ondersteuning Allow: /privacy Allow: /voorwaarden Allow: /sitemap Allow: /sitemap.xml Allow: /xml-sitemap-dynamic.php Allow: /images/ Allow: /css/ Allow: /js/ Allow: /assets/ # AI Crisis Communication Landing Pages - High Priority Allow: /ai-crisiscommunicatie.nl/ Allow: /aicrisiscommunicatie.nl/ Allow: /aicrisis.nl/ Allow: /crisisgrip.nl/ Allow: /crisisgeregeld.nl/ Allow: /communicatie-ai.nl/ # Disallow admin areas and internal files Disallow: /admin/ Disallow: /huisstijl/ Disallow: /includes/ Disallow: /vendor/ Disallow: /logs/ Disallow: /composer.json Disallow: /composer.lock Disallow: /test_*.php Disallow: *_debug.php Disallow: *_fixed.php Disallow: *_simplified.php Disallow: /roadmap\ -\ kopie.php # Primary sitemap location Sitemap: https://crisisradar.eu/sitemap.xml # SEO-Optimized Primary Sitemap (enhanced with image data and rich descriptions) Sitemap: https://crisisradar.eu/crisisradar-urlmap.xml # AI Crisis Communication Landing Pages - Specialized SEO Sitemap Sitemap: https://crisisradar.eu/ai-crisis-sitemap.xml # AI Crisiscommunicatie Nederland - Dedicated Subdirectory Sitemap Sitemap: https://crisisradar.eu/ai-crisiscommunicatie.nl/sitemap.xml # Backup sitemap (dynamic PHP version with guaranteed headers) Sitemap: https://crisisradar.eu/xml-sitemap-dynamic.php # Standard crawl-delay for non-priority crawlers Crawl-delay: 3