---
title: "List of known Spiders / Bots"
slug: "list-of-known-spiders-bots"
updated: 2023-01-06T15:49:20Z
published: 2023-01-06T15:49:19Z
canonical: "docs.catchpoint.com/list-of-known-spiders-bots"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.catchpoint.com/llms.txt
> Use this file to discover all available pages before exploring further.

# List of known Spiders / Bots

## Overview

Catchpoint allows you to block Bots and Spiders from your real user measurements.

To block Bots and spiders, go to **Settings >Apps >** and select an app. Then check **Ingore: Known Bots/Spiders**.
![bots.png](https://cdn.document360.io/cb4af8f9-6751-4fd2-b39c-07aae832badb/Images/Documentation/4402585012749-bots.png)
Bots and spiders will be blocked based on their User Agents.

## Known Bots and Spiders List
Below is the list of user agents that Catchpoint considers a Bot/Spider.

* Scrubby
* ^Nail
* 007AC9
* 192.comAgent
* 200PleaseBot
* 360Spider
* 4seohuntBot
* 50.nu
* A6-Indexer
* abby
* Aboundexbot
* AboutUsBot
* Abrave Spider
* Accelobot
* Ace Explorer
* AcoonBot
* ActiveBookmark
* Activeworlds
* AddThis.com
* ADmantX
* adressendeutschland.de
* AdsBot-Google
* Advanced Browser
* agentslug
* AhrefsBot
* aiHitBot
* Akregator
* akula
* Alcatel-BG3
* AlertInfo
* Alexabot
* alyze.info
* Amagit.COM
* amaya
* amibot
* Amiga-AWeb
* AmigaVoyager
* AMZNKAssocBot
* annotate_google
* AntBot
* ANTFresco
* Apercite
* AportWorm
* Applebot
* AraBot
* arachnode.net
* Arachnophilia
* archive.org_bot
* Ask Jeeves/Teoma
* AskQuickly
* Audiovox
* Automattic Analytics Crawler
* Avant
* avantbrowser
* BabalooSpider
* backlink-check.de
* BacklinkCrawler
* Bad-Neighborhood
* baiduspider
* Barca
* BarcaPro
* baypup
* BDCbot
* BDFetch
* BegunAdvertising
* bingbot
* bitlybot
* biwec
* bixocrawler
* Biyubi
* bl.uk_lddc_bot
* Blekkobot
* BLEXBot
* BlinkaCrawler
* BlockNote
* BlogBridge
* BlogPulse
* bluefish 0.6 HTML editor
* bnf.fr_bot
* bot.wsowner.com
* botmobi
* BotOnParade
* bot-pge.chlooe.com
* BrainbruBot
* BrokenLinkCheck.com
* Browsershots
* BUbiNG
* Butterfly
* CamontSpider
* CareerBot
* Castabot
* CatchBot
* CCBot
* CCResearchBot
* ChangeDetection
* Charlotte
* CirrusExplorer
* cityreview
* classbot
* CligooRobot
* Cliqzbot
* CliqzBot
* CloudFlare-AlwaysOnline
* CloudServerMarketSpider
* CMS Crawler
* coccoc
* Commerce Browser Center
* COMODOSpider
* Company News Search engine
* CompSpyBot
* contentDetection
* Cooliris
* copyright sheriff
* CorpusCrawler
* Covario-IDS
* crawler for netopian
* Crawler4j
* CrazyWebCrawler-Spider
* Crowsnest
* Curious George
* datagnionbot
* Daumoa
* DBLBot
* DCPbot
* deadlinkchecker
* DealGates Bot
* Dillo
* discoverybot
* DKIMRepBot
* dlcbot
* dlvr.it
* Dlvr.it/1.0
* DNS-Digger-Explorer
* DomainAppender
* DomainDB
* Dot TK - spider
* DotBot
* dotSemantic
* DreamPassport
* DripfeedBot
* drupact
* DuckDuckBot
* EasouSpider
* EasyBib AutoCite
* eCairn-Grabber
* eCommerceBot
* EdisterBot
* electricmonk
* Elefent
* ELinks
* Embedly
* emefgebot
* envolk
* Esribot
* EuripBot
* Eurobot
* EventGuruBot
* EveryoneSocialBot
* EvriNid
* Exabot
* ExB Language Crawler
* Experibot
* ExpertSearchSpider
* Ezooms
* FacebookExternalHit
* facebookplatform
* factbot
* FairShare
* Falconsbot
* fastbot crawler
* FauBot
* FeedCatBot
* FeedDemon
* FeedFinder/bloggz.se
* Feedly
* Feedshow
* FeedshowOnline
* FeedZcollector
* Fetch-Guess
* findlinks
* firmilybot
* Flatland Industries Web Spider
* FlightDeckReportsBot
* FlipboardProxy
* Flocke bot
* FollowSite Bot
* Fooooo_Web_Video_Crawl
* FreeWebMonitoring SiteChecker
* FyberSpider
* Gaisbot
* GarlikCrawler
* GeliyooBot
* Genieo Web filter
* GetFoundBot
* GigablastOpenSource
* Gigabot
* GingerCrawler
* Girafabot
* gocrawl
* GOFORITBOT
* gomezagent
* gonzo
* Googlebot
* googlebot
* GozaikBot
* grabber
* Grahambot
* GrapeshotCrawler
* GurujiBot
* Hailoobot
* HatenaScreenshot
* hawkReader
* HeartRails_Capture
* heritrix
* Holmes
* HolmesBot
* HomeTags
* HostTracker
* HostTracker.com
* HTMLParse
* HuaweiSymantecSpider
* HubSpot Crawler
* HypeStat
* ia_archive
* ia_archiver
* ibot
* ICC-Crawler
* ichiro
* iCjobs
* IdeelaborPlagiaat
* idmarch
* Iframely
* imbot
* immediatenet thumbnails
* ImplisenseBot
* Impressumscrawler
* imrbot
* Influencebot
* Infohelfer
* IntegromeDB
* internetseer
* IstellaBot
* it2media-domain-crawler
* IXEbot
* Jabse.com Crawler
* JadynAveBot
* JamesBOT
* JikeSpider
* Job Roboter Spider
* JobdiggerSpider
* JUST-CRAWLER
* Jyxobot
* Kalooga
* Karneval-Bot
* keynote
* KimonoLabs
* kinshoo
* KomodiaBot
* Kraken
* KrOWLer
* kulturarw
* L.webis
* Leikibot
* LemurWebCrawler
* LexxeBot
* Lijit
* LinguaBot
* linguatools
* Linguee Bot
* Link Valet Online
* LinkAider
* linkdex.com
* linkdexbot
* LinkedInBot
* linkpeek
* LinkWalker
* Lipperhey Spider
* livedoor ScreenShot
* LivelapBot/0.2
* LoadImpact
* LoadTimeBot
* ltbot
* LuminateBot
* magpie-crawler
* Mail.Ru bot
* meanpathbot
* MeMoNewsBot
* memoryBot
* MergeFlow-PageReader
* MetaGeneratorCrawler
* MetaHeadersBot
* MetaJobBot
* MetamojiCrawler
* METASpider
* Metaspinner/0.01
* MetaURI
* MetaURI API
* MIA Bot
* MiaDev
* MixBot
* MJ12bot
* MLBot
* MnoGoSearch
* moba-crawler
* MojeekBot
* Motoricerca-Robots.txt-Checker
* Mp3Bot
* MSNBot
* MSRBOT
* musobot
* Najdi.si
* NalezenCzBot
* NaverBot
* nekstbot
* NerdByNature.Bot
* NerdyBot
* NetcraftSurveyAgent
* netEstate Crawler
* NetResearchServer
* Netseer
* NextGenSearchBot
* NF-Browser
* Nigma.ru
* NLNZ_IAHarvester
* nodestackbot
* Nuhk
* Nutch
* nworm
* Nymesis
* oBot
* Ocelli
* omgilibot
* Online Domain Tools
* Open Web Analytics Bot
* OpenCalaisSemanticProxy
* OpenindexSpider
* OpenWebSpider
* OptimizationCrawler
* OrangeBot
* OrgbyBot
* OsObot
* ownCloud Server Crawler
* page_verifier
* Page2RSS
* PagePeeker
* Pagespeed
* Pandeo Bot
* Panscient web crawler
* PaperLiBot
* ParchBot
* parsijoo
* PayPal IPN
* Peeplo Screenshot Bot
* Peepowbot
* peerindex
* Peew
* PercolateCrawler
* Pingdom
* pingdom.com_bot
* Pingoscope
* Pinterest
* PiplBot
* Pixray-Seeker
* Plukkie
* pmoz.info ODP link checker
* Pompos
* Port Monitor check service
* PostPost
* pr-cy.ru Screenshot Bot
* ProCogBot
* ProCogSEOBot
* ProductoDownloadUrlBot/
* proximic
* psbot
* Qirina Hurdler
* Qseero
* Qualidator.com Bot
* Qualidator.com SiteAnalyzer 1.0
* QuerySeekerSpider
* quickobot
* Qwantify
* R6 bot
* RADaR-Bot
* RankurBot
* Readability
* Riddlerbot
* Robots_Tester
* Robozilla
* rogerbot
* Ronzoobot
* RSSMicro.com RSS/Atom Feed Robot
* Ruky-Roboter
* RyzeCrawler
* SAI Crawler
* SanszBot
* SBIder
* SBSearch
* Scarlett
* SCFCrawler
* Scooter
* ScoutJet
* Scrapy
* ScreenerBot Crawler
* Screenshot Bot
* search.KumKie.com
* Search17Bot
* SearchmetricsBot
* SecurityResearchBot
* seegnifybot
* Semager
* Semantifire
* SemrushBot
* SentiBot
* Seobility
* SEOCentro bot
* SeoCheckBot
* SEODat
* SEOdiver
* SEOENGBot
* SEOkicks-Robot
* Setoozbot
* Setoozbot
* Seznam screenshot-generator
* SeznamBot
* Shareaholicbot
* Shelob
* ShopWiki
* ShowyouBot
* sistrix
* Site24x7
* SiteCondor
* Sitedomain-Bot
* Siteliner
* Slackbot
* smart.apnoti.com Robot
* SMTBot
* Snapbot
* SniffRSS
* socialbm_bot
* sogou spider
* Sogou web spider
* SolomonoBot
* Sosospider
* spbot
* Speedy Spider
* SpiderLing
* Spinn3r
* SputnikBot
* SSLBot
* SSL-Crawler
* StackRambler
* StatoolsBot
* Steeler
* STINGbot
* stq_bot
* Strokebot
* suggybot
* sukibot_heritrix
* SurcentroBot
* Surphace Scout
* SurveyBot
* SWEBot
* SygolBot
* Symfony Spider
* Szukacz
* Tagoobot
* taptubot
* Technoratibot
* Thumbnail.CZ robot
* Thumbshots.ru
* ThumbShots-Bot
* thumbshots-de-Bot
* ThumbSniper
* TinEye
* TomTom places company search
* Topicbot
* Toread-Crawler
* Touche
* trendictionbot
* trendictionbot
* TurnitinBot
* TwengaBot
* Twiceler
* Twikle
* Twingly Recon
* UASlinkChecker
* uMBot
* UnisterBot
* UnwindFetchor
* Updownerbot
* UptimeDog
* UptimeRobot
* URLAppendBot
* urlfan-bot
* Urlfilebot (Urlbot)
* Vagabondo
* Vedma
* VideoSurf_bot
* Visbot
* VoilaBot
* voltron
* voyager
* VR FrontPage Fetcher
* WASALive-Bot
* WatchMouse
* WBSearchBot
* web downloader
* web+downloader
* webauto
* webcapture
* webcheck
* WebCookies
* WebCopier
* WebCorp
* WebImages
* webinatorbot
* webmastercoffee
* Webmetrics
* Web-Monitoring
* WebNL
* WebRankSpider
* websitepulse
* Websnapr
* Web-sniffer
* WebTarantula.com Crawler
* WebThumb
* WebThumbnail
* WebWatch/Robot_txtChecker
* WeSEE
* WeViKaBot
* Whoismindbot
* WikioFeedBot
* wikiwix-bot
* Willow Internet Crawler
* WillyBot
* WinWebBot
* WMCAI_robot
* Woko
* WordPress.com mShots
* woriobot
* WorldBrewBot
* Wotbox
* wsAnalyzer
* wscheck.com
* x28-job-bot
* XmarksFetch
* XML Sitemaps Generator
* XoviBot
* XRL
* Yaanb
* yacybot
* Yahoo!
* YamanaLab-bot
* YandexBot
* YandexNews
* Yanga
* YioopBot
* YodaoBot
* YoudaoBot
* YowedoBot
* YRSpider
* YYSpider
* zealbot
* ZeerchBot
* ZipCommander
* Zookabot
* ZookaBot
* ZumBot