---
title: "List of known Spiders / Bots"
slug: "list-of-known-spiders-bots"
updated: 2023-01-06T15:49:20Z
published: 2023-01-06T15:49:19Z
canonical: "docs.catchpoint.com/list-of-known-spiders-bots"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.catchpoint.com/llms.txt
> Use this file to discover all available pages before exploring further.

# List of known Spiders / Bots

## Overview

Catchpoint allows you to block Bots and Spiders from your real user measurements.

To block Bots and spiders, go to **Settings >Apps >** and select an app. Then check **Ingore: Known Bots/Spiders**. ![bots.png](https://cdn.document360.io/cb4af8f9-6751-4fd2-b39c-07aae832badb/Images/Documentation/4402585012749-bots.png?sv=2026-02-06&spr=https&st=2026-10-13T13%3A01%3A26Z&se=2026-10-13T14%3A01%3A26Z&sr=c&sp=r&sig=8iWg3kC1SLMVBJ6zJ00nICYkH6s4T5U0eTJ8p4e8WLI%3D) Bots and spiders will be blocked based on their User Agents.

## Known Bots and Spiders List

Below is the list of user agents that Catchpoint considers a Bot/Spider.

- Scrubby
- ^Nail
- 007AC9
- 192.comAgent
- 200PleaseBot
- 360Spider
- 4seohuntBot
- 50.nu
- A6-Indexer
- abby
- Aboundexbot
- AboutUsBot
- Abrave Spider
- Accelobot
- Ace Explorer
- AcoonBot
- ActiveBookmark
- Activeworlds
- AddThis.com
- ADmantX
- adressendeutschland.de
- AdsBot-Google
- Advanced Browser
- agentslug
- AhrefsBot
- aiHitBot
- Akregator
- akula
- Alcatel-BG3
- AlertInfo
- Alexabot
- alyze.info
- Amagit.COM
- amaya
- amibot
- Amiga-AWeb
- AmigaVoyager
- AMZNKAssocBot
- annotate_google
- AntBot
- ANTFresco
- Apercite
- AportWorm
- Applebot
- AraBot
- arachnode.net
- Arachnophilia
- archive.org_bot
- Ask Jeeves/Teoma
- AskQuickly
- Audiovox
- Automattic Analytics Crawler
- Avant
- avantbrowser
- BabalooSpider
- backlink-check.de
- BacklinkCrawler
- Bad-Neighborhood
- baiduspider
- Barca
- BarcaPro
- baypup
- BDCbot
- BDFetch
- BegunAdvertising
- bingbot
- bitlybot
- biwec
- bixocrawler
- Biyubi
- bl.uk_lddc_bot
- Blekkobot
- BLEXBot
- BlinkaCrawler
- BlockNote
- BlogBridge
- BlogPulse
- bluefish 0.6 HTML editor
- bnf.fr_bot
- bot.wsowner.com
- botmobi
- BotOnParade
- bot-pge.chlooe.com
- BrainbruBot
- BrokenLinkCheck.com
- Browsershots
- BUbiNG
- Butterfly
- CamontSpider
- CareerBot
- Castabot
- CatchBot
- CCBot
- CCResearchBot
- ChangeDetection
- Charlotte
- CirrusExplorer
- cityreview
- classbot
- CligooRobot
- Cliqzbot
- CliqzBot
- CloudFlare-AlwaysOnline
- CloudServerMarketSpider
- CMS Crawler
- coccoc
- Commerce Browser Center
- COMODOSpider
- Company News Search engine
- CompSpyBot
- contentDetection
- Cooliris
- copyright sheriff
- CorpusCrawler
- Covario-IDS
- crawler for netopian
- Crawler4j
- CrazyWebCrawler-Spider
- Crowsnest
- Curious George
- datagnionbot
- Daumoa
- DBLBot
- DCPbot
- deadlinkchecker
- DealGates Bot
- Dillo
- discoverybot
- DKIMRepBot
- dlcbot
- dlvr.it
- Dlvr.it/1.0
- DNS-Digger-Explorer
- DomainAppender
- DomainDB
- Dot TK - spider
- DotBot
- dotSemantic
- DreamPassport
- DripfeedBot
- drupact
- DuckDuckBot
- EasouSpider
- EasyBib AutoCite
- eCairn-Grabber
- eCommerceBot
- EdisterBot
- electricmonk
- Elefent
- ELinks
- Embedly
- emefgebot
- envolk
- Esribot
- EuripBot
- Eurobot
- EventGuruBot
- EveryoneSocialBot
- EvriNid
- Exabot
- ExB Language Crawler
- Experibot
- ExpertSearchSpider
- Ezooms
- FacebookExternalHit
- facebookplatform
- factbot
- FairShare
- Falconsbot
- fastbot crawler
- FauBot
- FeedCatBot
- FeedDemon
- FeedFinder/bloggz.se
- Feedly
- Feedshow
- FeedshowOnline
- FeedZcollector
- Fetch-Guess
- findlinks
- firmilybot
- Flatland Industries Web Spider
- FlightDeckReportsBot
- FlipboardProxy
- Flocke bot
- FollowSite Bot
- Fooooo_Web_Video_Crawl
- FreeWebMonitoring SiteChecker
- FyberSpider
- Gaisbot
- GarlikCrawler
- GeliyooBot
- Genieo Web filter
- GetFoundBot
- GigablastOpenSource
- Gigabot
- GingerCrawler
- Girafabot
- gocrawl
- GOFORITBOT
- gomezagent
- gonzo
- Googlebot
- googlebot
- GozaikBot
- grabber
- Grahambot
- GrapeshotCrawler
- GurujiBot
- Hailoobot
- HatenaScreenshot
- hawkReader
- HeartRails_Capture
- heritrix
- Holmes
- HolmesBot
- HomeTags
- HostTracker
- HostTracker.com
- HTMLParse
- HuaweiSymantecSpider
- HubSpot Crawler
- HypeStat
- ia_archive
- ia_archiver
- ibot
- ICC-Crawler
- ichiro
- iCjobs
- IdeelaborPlagiaat
- idmarch
- Iframely
- imbot
- immediatenet thumbnails
- ImplisenseBot
- Impressumscrawler
- imrbot
- Influencebot
- Infohelfer
- IntegromeDB
- internetseer
- IstellaBot
- it2media-domain-crawler
- IXEbot
- Jabse.com Crawler
- JadynAveBot
- JamesBOT
- JikeSpider
- Job Roboter Spider
- JobdiggerSpider
- JUST-CRAWLER
- Jyxobot
- Kalooga
- Karneval-Bot
- keynote
- KimonoLabs
- kinshoo
- KomodiaBot
- Kraken
- KrOWLer
- kulturarw
- L.webis
- Leikibot
- LemurWebCrawler
- LexxeBot
- Lijit
- LinguaBot
- linguatools
- Linguee Bot
- Link Valet Online
- LinkAider
- linkdex.com
- linkdexbot
- LinkedInBot
- linkpeek
- LinkWalker
- Lipperhey Spider
- livedoor ScreenShot
- LivelapBot/0.2
- LoadImpact
- LoadTimeBot
- ltbot
- LuminateBot
- magpie-crawler
- Mail.Ru bot
- meanpathbot
- MeMoNewsBot
- memoryBot
- MergeFlow-PageReader
- MetaGeneratorCrawler
- MetaHeadersBot
- MetaJobBot
- MetamojiCrawler
- METASpider
- Metaspinner/0.01
- MetaURI
- MetaURI API
- MIA Bot
- MiaDev
- MixBot
- MJ12bot
- MLBot
- MnoGoSearch
- moba-crawler
- MojeekBot
- Motoricerca-Robots.txt-Checker
- Mp3Bot
- MSNBot
- MSRBOT
- musobot
- Najdi.si
- NalezenCzBot
- NaverBot
- nekstbot
- NerdByNature.Bot
- NerdyBot
- NetcraftSurveyAgent
- netEstate Crawler
- NetResearchServer
- Netseer
- NextGenSearchBot
- NF-Browser
- Nigma.ru
- NLNZ_IAHarvester
- nodestackbot
- Nuhk
- Nutch
- nworm
- Nymesis
- oBot
- Ocelli
- omgilibot
- Online Domain Tools
- Open Web Analytics Bot
- OpenCalaisSemanticProxy
- OpenindexSpider
- OpenWebSpider
- OptimizationCrawler
- OrangeBot
- OrgbyBot
- OsObot
- ownCloud Server Crawler
- page_verifier
- Page2RSS
- PagePeeker
- Pagespeed
- Pandeo Bot
- Panscient web crawler
- PaperLiBot
- ParchBot
- parsijoo
- PayPal IPN
- Peeplo Screenshot Bot
- Peepowbot
- peerindex
- Peew
- PercolateCrawler
- Pingdom
- pingdom.com_bot
- Pingoscope
- Pinterest
- PiplBot
- Pixray-Seeker
- Plukkie
- pmoz.info ODP link checker
- Pompos
- Port Monitor check service
- PostPost
- pr-cy.ru Screenshot Bot
- ProCogBot
- ProCogSEOBot
- ProductoDownloadUrlBot/
- proximic
- psbot
- Qirina Hurdler
- Qseero
- Qualidator.com Bot
- Qualidator.com SiteAnalyzer 1.0
- QuerySeekerSpider
- quickobot
- Qwantify
- R6 bot
- RADaR-Bot
- RankurBot
- Readability
- Riddlerbot
- Robots_Tester
- Robozilla
- rogerbot
- Ronzoobot
- RSSMicro.com RSS/Atom Feed Robot
- Ruky-Roboter
- RyzeCrawler
- SAI Crawler
- SanszBot
- SBIder
- SBSearch
- Scarlett
- SCFCrawler
- Scooter
- ScoutJet
- Scrapy
- ScreenerBot Crawler
- Screenshot Bot
- search.KumKie.com
- Search17Bot
- SearchmetricsBot
- SecurityResearchBot
- seegnifybot
- Semager
- Semantifire
- SemrushBot
- SentiBot
- Seobility
- SEOCentro bot
- SeoCheckBot
- SEODat
- SEOdiver
- SEOENGBot
- SEOkicks-Robot
- Setoozbot
- Setoozbot
- Seznam screenshot-generator
- SeznamBot
- Shareaholicbot
- Shelob
- ShopWiki
- ShowyouBot
- sistrix
- Site24x7
- SiteCondor
- Sitedomain-Bot
- Siteliner
- Slackbot
- smart.apnoti.com Robot
- SMTBot
- Snapbot
- SniffRSS
- socialbm_bot
- sogou spider
- Sogou web spider
- SolomonoBot
- Sosospider
- spbot
- Speedy Spider
- SpiderLing
- Spinn3r
- SputnikBot
- SSLBot
- SSL-Crawler
- StackRambler
- StatoolsBot
- Steeler
- STINGbot
- stq_bot
- Strokebot
- suggybot
- sukibot_heritrix
- SurcentroBot
- Surphace Scout
- SurveyBot
- SWEBot
- SygolBot
- Symfony Spider
- Szukacz
- Tagoobot
- taptubot
- Technoratibot
- Thumbnail.CZ robot
- Thumbshots.ru
- ThumbShots-Bot
- thumbshots-de-Bot
- ThumbSniper
- TinEye
- TomTom places company search
- Topicbot
- Toread-Crawler
- Touche
- trendictionbot
- trendictionbot
- TurnitinBot
- TwengaBot
- Twiceler
- Twikle
- Twingly Recon
- UASlinkChecker
- uMBot
- UnisterBot
- UnwindFetchor
- Updownerbot
- UptimeDog
- UptimeRobot
- URLAppendBot
- urlfan-bot
- Urlfilebot (Urlbot)
- Vagabondo
- Vedma
- VideoSurf_bot
- Visbot
- VoilaBot
- voltron
- voyager
- VR FrontPage Fetcher
- WASALive-Bot
- WatchMouse
- WBSearchBot
- web downloader
- web+downloader
- webauto
- webcapture
- webcheck
- WebCookies
- WebCopier
- WebCorp
- WebImages
- webinatorbot
- webmastercoffee
- Webmetrics
- Web-Monitoring
- WebNL
- WebRankSpider
- websitepulse
- Websnapr
- Web-sniffer
- WebTarantula.com Crawler
- WebThumb
- WebThumbnail
- WebWatch/Robot_txtChecker
- WeSEE
- WeViKaBot
- Whoismindbot
- WikioFeedBot
- wikiwix-bot
- Willow Internet Crawler
- WillyBot
- WinWebBot
- WMCAI_robot
- Woko
- WordPress.com mShots
- woriobot
- WorldBrewBot
- Wotbox
- wsAnalyzer
- wscheck.com
- x28-job-bot
- XmarksFetch
- XML Sitemaps Generator
- XoviBot
- XRL
- Yaanb
- yacybot
- Yahoo!
- YamanaLab-bot
- YandexBot
- YandexNews
- Yanga
- YioopBot
- YodaoBot
- YoudaoBot
- YowedoBot
- YRSpider
- YYSpider
- zealbot
- ZeerchBot
- ZipCommander
- Zookabot
- ZookaBot
- ZumBot