Fedizens who self-host services, do you:

The reason why I ask about "knowledge" in particular is that I saw several LLM websites using that domain name for their stuff, so I wonder if they think I host some kind of LLM and thus go on this domain name in particular

@Soblow I wonder if it’s because “knowledge base”

I’ll add one after work and report back if it’s hit

Fedizens who self-host services, do you:

• Have a service exposed on a domain name matching knowledge.DOMAIN?
• Notice heavy waves of crawlers accessing this specific domain?
• (optional) Use iocaine or a similar poisoning tool?

I'm also interested if you have multiple subdomains protected by iocaine and notice one in particular gets hit.

#selfhosting #iocaine

@Soblow the general consensus seems to be the AI scrapers are using certificate transparency logs to find subdomains to scrape

@Soblow the general consensus seems to be the AI scrapers are using certificate transparency logs to find subdomains to scrape

Fedizens who self-host services, do you:

• Have a service exposed on a domain name matching knowledge.DOMAIN?
• Notice heavy waves of crawlers accessing this specific domain?
• (optional) Use iocaine or a similar poisoning tool?

I'm also interested if you have multiple subdomains protected by iocaine and notice one in particular gets hit.

#selfhosting #iocaine

@Soblow
CC @alexia

we don't have a knowledge subdomain but we do use Iocaine to poison the relatively large proportion of scrapers

@niko yup, but not all my domains are scrapped equally, why?

@0x7700e6 @alexia eo they attack a subdomain in particular or is it even across them?