Googlebot
Bingbot
Slurp
DuckDuckBot
Baiduspider
YandexBot
Sogou
Exabot
facebot
ia_archiver
SemrushBot
AhrefsBot
MJ12bot
DotBot
Google bot
Google bot mobile
Google bot image
Google bot news
Google bot video
Google AdsBot
Google AdsBot mobile
Google Feedfetcher
Google Mediapartners
Google APIs
Google Inspection Tool
Google Storebot
GoogleOther
BingBot
Slurp
Wget
LinkedIn Bot
Urllib
Python requests
AIOHTTP
HTTPX
libwww-perl
HttpUnit
Apache Nutch
Go HTTP client
PHPCrawl
Msnbot
FAST-WebCrawler
FAST Enterprise Crawler
ia_archiver
HTTrack website copier
yacybot
Majestic (MJ12Bot)
Buzzbot
YandexBot
Linguee Bot
Baidu spider
Google Keyword Suggestion
Google Lighthouse
Google PageSpeed Insights
Google Publisher Wordpress Plugin
Google Read Aloud Bot
SiteTruth
SkypeUriPreview
Slack-ImgProxy
Slackbot
Slackbot-Link-Expanding
