Web Crawler

Meet CitenixBot

CitenixBot is the web crawler operated by Citenix, an SEO & AI-visibility platform. When someone analyzes a website with Citenix, CitenixBot visits its pages to check technical SEO, content quality, and how the site appears to AI search engines.

How to identify it

CitenixBot always identifies itself with this User-Agent:

Mozilla/5.0 (compatible; CitenixBot/1.0; +https://citenix.com/bot)

It's a good citizen

Respects robots.txt — obeys rules for the CitenixBot user-agent.
Honors Crawl-delay and adapts its pace if your server responds slowly.
Bandwidth-friendly — uses conditional requests (If-None-Match / If-Modified-Since) so unchanged pages aren't re-downloaded.
Transparent — always sends the User-Agent above, never hides its identity.
On-demand — only crawls sites a Citenix user has explicitly asked to analyze.

Control CitenixBot

You're in charge. Add rules for CitenixBot to your robots.txt.

Allow it, but slow it down and keep it out of a folder:

# robots.txt
User-agent: CitenixBot
Crawl-delay: 5
Disallow: /private/

Block it from your entire site:

# robots.txt
User-agent: CitenixBot
Disallow: /

Changes take effect on the next crawl.

Why is it visiting my site?

Someone — possibly you — added your domain to Citenix for an SEO or AI-visibility analysis. CitenixBot doesn't crawl the open web at random; it only visits sites a Citenix user has requested.