statool/statoolinfosweb
statool/statoolinfosweb
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Activity

Improved user agent bot list.

Browse source
This commit is contained in:
Christian P. MOMON 2023-07-03 14:38:36 +02:00
parent c51822877f
commit 8619c59a65
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
src/fr/devinsy/statoolinfos/metrics/http/userAgentBotDetectorData.txt
View file

@ -3,8 +3,11 @@ AhrefsBot
^ALittle Client
^Apache-HttpClient/
Barkrowler/
Baiduspider
bot
BingPreview
Bytespider
CensysInspect
^Conversations/
crawler
^curl/
@ -21,6 +24,7 @@ crawler
^Go-http-client
^got
^GoModuleMirror/
InternetMeasurement
^hackney/
HeadlessChrome/
^Hello
@ -54,6 +58,7 @@ Nutch-
^python/
^Python/
^Python-urllib
Qwantify
^Report Runner
^RSS Discovery Engine
SemrushBot