blackhull bot — operator information ==================================== User-Agent: bhbot/1.0 (+https://bot.blackhull.systems) Operator: blackhull Updated: 2026-08-05 WHAT IT DOES Fetches specific pages to answer questions, and re-checks a small number of watched pages to notice when they change. It does not crawl: it requests known URLs, never traverses links to discover new ones, and never harvests content in bulk. HOW IT BEHAVES - Reads and obeys robots.txt, including Crawl-delay. - At most one request per domain every 2 seconds. - Sends If-Modified-Since, so an unchanged page costs you a 304 and no body. - Executes no JavaScript, loads no images/CSS/fonts, sends no cookies and no Referer. - Low volume: single-digit requests per site per day is typical. HOW TO BLOCK IT Add to your robots.txt: User-agent: bhbot Disallow: / That is picked up within the hour. To slow it rather than block it, a Crawl-delay for bhbot is honoured exactly as stated: User-agent: bhbot Crawl-delay: 30 ON DOCUMENTED APIs robots.txt is a crawler-exclusion protocol. Some sites publish an API for programmatic use and also disallow it in robots.txt — PyPI, for example, disallows its JSON API alongside the /simple/ endpoint that pip itself depends on. Where that is the case, this bot may call the documented API deliberately, for one specific resource, at the same rate limits as anything else. It applies that narrowly and never: to evade a block or rate limit; on account, admin, checkout or search paths; or simply because data is valuable. If you would rather it did not do this on your site, email the address below and it will stop. CONTACT bot@blackhull.systems (abuse@ and postmaster@ also reach us) Abuse reports, questions, and "please stop" all get a reply. If this bot has caused you load or trouble, say so and it will be fixed.