id	domain	url
spir-13995	spir.aoir.org	http://spir.aoir.org
spir-13995	foundation.mozilla.org	https://foundation.mozilla.org/en/research/library/generative-aitraining-data/common-crawl
spir-13995	doi.org	https://doi.org/10.14763/2021.4.1588
spir-13995	doi.org	https://doi.org/10.5210/fm.v3i10.620
spir-13995	blog.archive.org	https://blog.archive.org/2017/04/17/robots-txtmeant-for-search-engines-dont-work-well-for-web-archives
spir-13995	doi.org	https://doi.org/10.1109/msp.2018.2701155
spir-13995	foundation.mozilla.org	https://foundation.mozilla.org/en/research/library/generative-ai-training-data/common-crawl
spir-13995	foundation.mozilla.org	https://foundation.mozilla.org/en/research/library/generative-ai-training-data/common-crawl
spir-13995	doi.org	https://doi.org/10.14763/2021.4.1588
spir-13995	doi.org	https://doi.org/10.5210/fm.v3i10.620
spir-13995	blog.archive.org	https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives
spir-13995	blog.archive.org	https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives
spir-13995	doi.org	https://doi.org/10.1109/msp.2018.2701155
spir-13995	inkdroid.org	https://web.archive.org/web/20131029200350/http://inkdroid.org/tmp/wwwtalk/4113.html
spir-13995	www.robotstxt.org	http://www.robotstxt.org/orig.html
spir-13995	doi.org	https://doi.org/10.1177/2056305118784502
spir-13995	nightshade.cs.uchicago.edu	https://nightshade.cs.uchicago.edu/whatis.html
spir-13995	eprints.soton.ac.uk	https://eprints.soton.ac.uk/447624
spir-13995	www.theverge.com	https://www.theverge.com/24067997/robots-txt-ai-text-file-web-crawlers-spiders
spir-13995	wiki.archiveteam.org	https://wiki.archiveteam.org/index.php?title=robots.txt
spir-13995	blog.archive.org	https://blog.archive.org/2023/05/10/getting-started-with-machine-learning-and-glamgalleries-libraries-archives-museums-collections
spir-13995	policyreview.info	https://policyreview.info/articles/analysis/feminist-data-protection-introduction
spir-13995	doi.org	https://doi.org/10.1002/asi.20388
spir-13995	web.archive.org	https://web.archive.org/web/20131029200350/http:/inkdroid.org/tmp/www-talk/4113.html
spir-13995	web.archive.org	https://web.archive.org/web/20131029200350/http:/inkdroid.org/tmp/www-talk/4113.html
spir-13995	www.robotstxt.org	http://www.robotstxt.org/orig.html
spir-13995	doi.org	https://doi.org/10.1177/2056305118784502
spir-13995	nightshade.cs.uchicago.edu	https://nightshade.cs.uchicago.edu/whatis.html
spir-13995	nightshade.cs.uchicago.edu	https://nightshade.cs.uchicago.edu/whatis.html
spir-13995	eprints.soton.ac.uk	https://eprints.soton.ac.uk/447624
spir-13995	www.theverge.com	https://www.theverge.com/24067997/robots-txt-ai-text-file-web-crawlers-spiders
spir-13995	wiki.archiveteam.org	https://wiki.archiveteam.org/index.php?title=robots.txt
spir-13995	wiki.archiveteam.org	https://wiki.archiveteam.org/index.php?title=robots.txt
spir-13995	blog.archive.org	https://blog.archive.org/2023/05/10/getting-started-with-machine-learning-and-glam-galleries-libraries-archives-museums-collections
spir-13995	blog.archive.org	https://blog.archive.org/2023/05/10/getting-started-with-machine-learning-and-glam-galleries-libraries-archives-museums-collections
spir-13995	policyreview.info	https://policyreview.info/articles/analysis/feminist-data-protection-introduction
spir-13995	policyreview.info	https://policyreview.info/articles/analysis/feminist-data-protection-introduction
spir-13995	doi.org	https://doi.org/10.1002/asi.20388