WATCHKEEPERBOT

WatchKeeperBot fetches robots.txt, XML sitemaps, llms.txt and a sample of the URLs they list, to check that those files agree with the pages they point to. It runs when someone scans a site with our free tools, or on the schedule a WatchKeeper customer has set for a site they monitor.

It identifies itself as WatchKeeperBot/0.1 (+https://watchkeeper.ai/bot). It makes at most two requests a second to any one host, however many people are checking that host, and it reads only as much of each page as it needs to find robots directives and canonicals.

It follows robots.txt rules written for WatchKeeperBot, or for * if there are none. URLs that robots.txt blocks for Googlebot are reported as blocked and never fetched. To stop it entirely, add User-agent: WatchKeeperBot and Disallow: / to your robots.txt.