Wikimedia Foundation has revealed that automated AI bots scraping information from the internet are resulting in steep technical and financial costs.
Since January 2024, Wikimedia Foundation has seen the bandwidth used for downloading multimedia content on its sites grow by 50%.
This traffic is not coming from human readers, but rather from automated programs or scraper bots that are vacuuming up terabytes of data to train large language models and other use cases.
Over the past year, the demand for content – especially for the 144 million images, videos and other files on Wikimedia Commons – has grown significantly.
Wikimedia’s underlying infrastructure is built to sustain sudden traffic spikes. For example, when Jimmy Carter died in December 2024, his page on English Wikipedia saw more than 2.8 million views over the course of a day.
Visitors were also playing a 1.5 hour long video of Carter’s 1980 presidential debate with Ronald Reagan, which doubled Wikimedia’s normal rate of network...