statoolinfos/README.md

StatoolInfos

StatoolInfos is a simple statistic tool software.

LICENSE

StatoolInfos is released under the GNU AGPL+ license. Enjoy!

Authors:

• Christian Pierre MOMON christian.momon@devinsy.fr

Cat Avatar Generator images

I am glad to use the beautiful artwork of David Revoy (http://www.peppercarrot.com).

• Graphics CC-BY http://creativecommons.org/licenses/by/4.0/
• Service: https://www.peppercarrot.com/extras/html/2016_cat-generator/
• Source: https://framagit.org/Deevad/cat-avatar-generator
• Originally inspired of the code for "MonsterID" by Andreas Gohr http://www.splitbrain.org/go/monsterid.

Cicle-icons

I am glad to use the generous artwork of ElegantThemes.com :

• https://www.elegantthemes.com/blog/freebie-of-the-week/beautiful-flat-icons-for-free
• "These icons are completely free and Open Source under the GPL".

Logo

Author: Christian Pierre MOMON christian.momon@devinsy.fr

License: Creative Commons CC-BY-SA last version.

Other stuff

License of other stuff:

• csv-logo.svg, json-logo.svg, ods-icon.svg:
  • Author: Christian P. Momon
  • License: CC-BY-SA 3+.
• other: without any specific information, default license of images is CC-BY-SA 3+.

DOCUMENTATION

Read CONCEPT.md and ONTOLOGY.md files.

Requirements

• Java 11
• Eclipse 4.16 (202006).

INSTALL

Install Java :

apt-get install openjdk-11-jre-headless

Download the last release: https://forge.devinsy.fr/devinsy/statoolinfos/releases

Unzip the package and move the statoolinfos.jar and statoolinfo.sh files in /srv/statoolinfos/bin/.

Create link:

cd /srv/statoolinfos/bin/ ; ln -s statoolinfos.sh statoolinfos

Usage

Usage:
    statoolinfos [ -h | -help | --help ]
    statoolinfos [ -v | -version | --version ]

    statoolinfos build                       <configurationfile>     build property files from conf and input
    statoolinfos clear                       <configurationfile>     remove property files from conf
    statoolinfos crawl                       <configurationfile>     crawl all file from conf and input
    statoolinfos htmlize                     <configurationfile>     generate web pages from conf
    statoolinfos probe   OPTION [<directory>|<configurationfile>]    generate metrics files from conf
           OPTION = [-full|-today|-previousday|-NN] with NN a day count
    statoolinfos uptime                      <configurationfile>     update uptime journal

    statoolinfos format                      <fileordirectory>       format property files in tiny way
    statoolinfos list ip       [-bot|-nobot] <fileordirectory>       generate ip list from log file
    statoolinfos list ua       [-bot|-nobot] <fileordirectory>       generate user agent list from log file
    statoolinfos list visitors [-bot|-nobot] <fileordirectory>       generate visitors (ip+ua) list from log file
    statoolinfos stat ip       [-bot|-nobot] <fileordirectory>       generate stats about ip from log file
    statoolinfos stat ua       [-bot|-nobot] <fileordirectory>       generate stats about user agent from log file
    statoolinfos stat visitors [-bot|-nobot] <fileordirectory>       generate stats about visitors (ip+ua) from log file
    statoolinfos tagdate                     <fileordirectory>       update the file.datetime file

Generate metrics files

Basics

Create a configuration file /srv/statoolInfos/conf/foo.bar.org.conf:

conf.probe.types=<metrictype1>, <metrictype2>
conf.probe.metrictype1param1.file=<value>
conf.probe.metrictype1param2.file=<value>
conf.probe.metrictype2param1.file=<value>
conf.probe.metrictype2param2.file=<value>
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Generate the metric file for the first time:

/srv/statoolinfos/bin/statoolinfos probe -full /srv/statoolinfos/conf/foo.bar.org.conf

Create a cron file to update the metric file everyday:

1 * * * * root /srv/statoolinfos/bin/probe -previousday /srv/statoolinfos/conf/foo.bar.org.conf

Warning: in previous day mode, the metrics generated are overwrited for the last month, the last week and the last day. So, six weeks in logs are required.

Framdadate metrics (coming soon)

Configuration template:

conf.probe.types=Framadate
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Gitea metrics

Configuration template:

conf.probe.types=Gitea
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.gitea.data=/opt/gitea/data
conf.probe.gitea.api.url=https://forge.libre-service.eu/
conf.probe.gitea.token=b6598c616b1cd350b834258205da4e5e8b951005
# jdbc:mariadb://localhost:1234/databasename
# jdbc:mysql://localhost:1234/databasename
# jdbc:postgresql://localhost:1234/databasename
# jdbc:sqlite:/foo/bar/databasename.sqlite
conf.probe.gitea.database.url=
conf.probe.gitea.database.user=
conf.probe.gitea.database.password=
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

LibreQR metrics (coming soon)

Configuration template:

conf.probe.types=LibreQR
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.libreqr.datafiles=/var/www/foo.bar.org/temp/
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Minetest metrics

Configuration template:

conf.probe.types=Minetest
conf.probe.minetest.logs=/home/cpm/Projets/StatoolInfos/EnvTest/minetest/minetest.log*

conf.probe.minetest.players.database.url=jdbc:postgresql://localhost:5432/minetestdb
conf.probe.minetest.players.database.user=minetestdba
conf.probe.minetest.players.database.password=XXXXXXXXXXX

conf.probe.minetest.worlds.database.url=jdbc:postgresql://localhost:5432/minetestdb
conf.probe.minetest.worlds.database.user=minetestdba
conf.probe.minetest.worlds.database.password=XXXXXXXXXXX

conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Mumble metrics

Configuration template:

conf.probe.types=Mumble
conf.probe.mumble.logs=/var/log/mumble-server/mumble-server.log*
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Nextcloud metrics (coming soon)

Configuration template:

conf.probe.types=Nextcloud
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

PrivateBin metrics (experimental)

Warning: works fine if database, image and comment options are disabled.

Configuration template:

conf.probe.types=PrivateBin
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.privatebin.data=/var/www/paste.libre-service.eu/data/
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Web metrics

Configuration template:

conf.probe.types=HttpAccessLog, HttpErrorLog
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httperrorlog.file=/var/log/apache2/foo.bar.org-error.log*
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

# Custom access log pattern with Java regex.
# Default: "^(?<remoteAddress>[a-zA-F0-9\\\\:\\\\.]+) - (?<remoteUser>[^\\[]+) \\[(?<time>[^\\]]+)\\] \"(?<request>.*)\" (?<status>\\d+) (?<bodyBytesSent>\\d+) \"(?<referer>.*)\" \"(?<userAgent>[^\"]*)\".*$"
conf.probe.httpaccesslog.pattern=