Anthropic’s ClaudeBot web crawler, which is used to crawl training data for AI models such as Claude, hit the iFixit website nearly a million times in 24 hours, appearing to violate the repair company’s terms of use in the process.
“If any of these requests go to our Terms of Service, they will tell you that use of our content is expressly prohibited. But don’t ask me, ask Claude!” iFixit CEO Kyle Wiens said on X, where he posted The image shows Anthropic’s chatbot acknowledging iFixit’s content as unrestricted. “Not only are you getting our content without paying, but you’re also tying up our DevOps resources. If you want to have a conversation about licensing our content for commercial use, we’re here.
iFixit’s Terms of Use policy states that “copying, duplication or distribution” of any content on the site, including specifically “training machine learning or artificial intelligence models,” is strictly prohibited without the Company’s prior express written consent. When Anthropic was asked about this 404 mediaHowever, the artificial intelligence company links back to an FAQ page that says its crawlers can only be blocked via the robots.txt file extension.
Wiens said iFixit has added crawl delay extensions to its robots.txt. We’ve reached out to Wiens and Anthropic for comment and will update this story if we hear back.
It seems that iFixit is not alone. Read the Docs co-founder Eric Holscher and Freelancer.com CEO Matt Barrie said in Wiens’ post that their websites were also attacked by the Anthropic crawler. This doesn’t appear to be new behavior for ClaudeBot either, with a Reddit post from a few months ago reporting a sharp increase in Anthropic’s web crawling. In April, the Linux Mint web forum blamed the site outage on stress caused by ClaudeBot’s crawling activity.
