statool/statoolinfosweb
statool/statoolinfosweb
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Activity

Improved user agent bot list.

Browse source
This commit is contained in:
Christian P. MOMON 2023-07-03 14:38:36 +02:00
parent c51822877f
commit 8619c59a65
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
src/fr/devinsy/statoolinfos/metrics/http/userAgentBotDetectorData.txt
View file

@ -3,8 +3,11 @@ AhrefsBot
^ALittle Client ^ALittle Client
^Apache-HttpClient/ ^Apache-HttpClient/
Barkrowler/ Barkrowler/
Baiduspider
bot bot
BingPreview BingPreview
Bytespider
CensysInspect
^Conversations/ ^Conversations/
crawler crawler
^curl/ ^curl/
@ -21,6 +24,7 @@ crawler
^Go-http-client ^Go-http-client
^got ^got
^GoModuleMirror/ ^GoModuleMirror/
InternetMeasurement
^hackney/ ^hackney/
HeadlessChrome/ HeadlessChrome/
^Hello ^Hello
@ -54,6 +58,7 @@ Nutch-
^python/ ^python/
^Python/ ^Python/
^Python-urllib ^Python-urllib
Qwantify
^Report Runner ^Report Runner
^RSS Discovery Engine ^RSS Discovery Engine
SemrushBot SemrushBot