StatoolInfos is a simple project of statistics about federation, organizations and services. https://www.devinsy.fr/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
Christian P. MOMON 83efaf25e7 Fixed wording. 1 week ago
.settings Upgraded catgenerator lib. Add bird default logo for services. 2 years ago
lib Added sql database probing. Gitea + Minetest. 7 months ago
resources Increased protocol version. 8 months ago
src/fr/devinsy/statoolinfos Fixed separase case characters in property stats. 2 months ago
test/fr/devinsy/statoolinfos Migrated to Log4j2. 9 months ago
.classpath Added sql database probing. Gitea + Minetest. 7 months ago
.gitignore First commit. 2 years ago
.project First commit. 2 years ago
CONCEPT.md Fixed ISO reference. 8 months ago
GENERAL.md Added DECIMAL_DEGREE definition. 1 year ago
LICENSE First commit. 2 years ago
ONTOLOGY.md Increased protocol version. 8 months ago
README.md Fixed wording. 1 week ago
build-appjar.xml Improved log4j2 configuration and fixed build process. 9 months ago
build-local.xml First commit. 2 years ago
build-snapshot.xml First commit. 2 years ago
build-tagandpush.xml First commit. 2 years ago
build.num Build 0.5.0 8 months ago
build.properties Prepared next version. 8 months ago
build.sh Fixed Java 11 check. 2 years ago
log4j2.properties Migrated to Log4j2. 9 months ago
snapshot Improved snapshot script. 8 months ago

README.md

StatoolInfos

StatoolInfos is a simple statistic tool software.

LICENSE

StatoolInfos is released under the GNU AGPL+ license. Enjoy!

Authors:

Cat Avatar Generator images

I am glad to use the beautiful artwork of David Revoy (http://www.peppercarrot.com).

Circle-icons

I am glad to use the generous artwork of ElegantThemes.com :

Author: Christian Pierre MOMON christian.momon@devinsy.fr

License: Creative Commons CC BY-SA last version.

Other stuff

License of other stuff:

  • csv-logo.svg, json-logo.svg, ods-icon.svg:
    • Author: Christian P. Momon
    • License: CC BY-SA 3+.
  • other: without any specific information, default license of images is CC BY-SA 3+.

DOCUMENTATION

Read CONCEPT.md and ONTOLOGY.md files.

Requirements

  • Java 11
  • Eclipse 4.16 (202006).

INSTALL

Install Java :

apt-get install openjdk-11-jre-headless

Download the last release: https://forge.devinsy.fr/devinsy/statoolinfos/releases

Unzip the package and move the statoolinfos.jar and statoolinfo.sh files in /srv/statoolinfos/bin/.

Create link:

cd /srv/statoolinfos/bin/ ; ln -s statoolinfos.sh statoolinfos

Usage

Usage:
    statoolinfos [ -h | -help | --help ]
    statoolinfos [ -v | -version | --version ]

    statoolinfos build                       <configurationfile>     build property files from conf and input
    statoolinfos clear                       <configurationfile>     remove property files from conf
    statoolinfos crawl                       <configurationfile>     crawl all file from conf and input
    statoolinfos htmlize                     <configurationfile>     generate web pages from conf
    statoolinfos probe   OPTION [<directory>|<configurationfile>]    generate metrics files from conf
           OPTION = [-full|-today|-previousday|-NN] with NN a day count
    statoolinfos uptime                      <configurationfile>     update uptime journal

    statoolinfos format                      <fileordirectory>       format property files in tiny way
    statoolinfos list ip       [-bot|-nobot] <fileordirectory>       generate ip list from log file
    statoolinfos list ua       [-bot|-nobot] <fileordirectory>       generate user agent list from log file
    statoolinfos list visitors [-bot|-nobot] <fileordirectory>       generate visitors (ip+ua) list from log file
    statoolinfos stat ip       [-bot|-nobot] <fileordirectory>       generate stats about ip from log file
    statoolinfos stat ua       [-bot|-nobot] <fileordirectory>       generate stats about user agent from log file
    statoolinfos stat visitors [-bot|-nobot] <fileordirectory>       generate stats about visitors (ip+ua) from log file
    statoolinfos tagdate                     <fileordirectory>       update the file.datetime file

Generate metrics files

Basics

Create a configuration file /srv/statoolInfos/conf/foo.bar.org.conf:

conf.probe.types=<metrictype1>, <metrictype2>
conf.probe.metrictype1param1.file=<value>
conf.probe.metrictype1param2.file=<value>
conf.probe.metrictype2param1.file=<value>
conf.probe.metrictype2param2.file=<value>
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Generate the metric file for the first time:

/srv/statoolinfos/bin/statoolinfos probe -full /srv/statoolinfos/conf/foo.bar.org.conf

Create a cron file to update the metric file everyday:

1 * * * * root /srv/statoolinfos/bin/probe -previousday /srv/statoolinfos/conf/foo.bar.org.conf

Warning: in previous day mode, the metrics generated are overwrited for the last month, the last week and the last day. So, six weeks in logs are required.

Etherpad metrics

Configuration template:

conf.probe.types=Etherpad
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
# This parameter requires real ip proxy setting and log output setting.
conf.probe.etherpad.logs=/var/log/etherpad/output.log
# jdbc:mariadb://localhost:1234/databasename
# jdbc:mysql://localhost:1234/databasename
# jdbc:postgresql://localhost:1234/databasename
# jdbc:sqlite:/foo/bar/databasename.sqlite
conf.probe.etherpad.database.url=
conf.probe.etherpad.database.user=
conf.probe.etherpad.database.password=
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Framdadate metrics (coming soon)

Configuration template:

conf.probe.types=Framadate
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Gitea metrics

Configuration template:

conf.probe.types=Gitea
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.gitea.data=/opt/gitea/data
conf.probe.gitea.api.url=https://forge.libre-service.eu/
conf.probe.gitea.token=b6598c616b1cd350b834258205da4e5e8b951005
# jdbc:mariadb://localhost:1234/databasename
# jdbc:mysql://localhost:1234/databasename
# jdbc:postgresql://localhost:1234/databasename
# jdbc:sqlite:/foo/bar/databasename.sqlite
conf.probe.gitea.database.url=
conf.probe.gitea.database.user=
conf.probe.gitea.database.password=
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

GSL metrics

Configuration template:

conf.probe.types=GSL
conf.probe.gsl.stats=/foo/bar/stats.properties
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

LibreQR metrics

Configuration template:

conf.probe.types=LibreQR
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.libreqr.datafiles=/var/www/foo.bar.org/temp/
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Minetest metrics

Configuration template:

conf.probe.types=Minetest
conf.probe.minetest.logs=/home/cpm/Projets/StatoolInfos/EnvTest/minetest/minetest.log*

conf.probe.minetest.players.database.url=jdbc:postgresql://localhost:5432/minetestdb
conf.probe.minetest.players.database.user=minetestdba
conf.probe.minetest.players.database.password=XXXXXXXXXXX

conf.probe.minetest.worlds.database.url=jdbc:postgresql://localhost:5432/minetestdb
conf.probe.minetest.worlds.database.user=minetestdba
conf.probe.minetest.worlds.database.password=XXXXXXXXXXX

conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Mumble metrics

Configuration template:

conf.probe.types=Mumble
conf.probe.mumble.logs=/var/log/mumble-server/mumble-server.log*
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Nextcloud metrics (coming soon)

Configuration template:

conf.probe.types=Nextcloud
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

PrivateBin metrics (partial)

Warning: works fine if database, image and comment options are disabled.

Configuration template:

conf.probe.types=PrivateBin
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httpaccesslog.pattern=
conf.probe.privatebin.data=/var/www/paste.libre-service.eu/data/
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

Web metrics

Configuration template:

conf.probe.types=HttpAccessLog, HttpErrorLog
conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*
conf.probe.httperrorlog.file=/var/log/apache2/foo.bar.org-error.log*
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties

# Custom access log pattern with Java regex.
# Default: "^(?<remoteAddress>[a-zA-F0-9\\\\:\\\\.]+) - (?<remoteUser>[^\\[]+) \\[(?<time>[^\\]]+)\\] \"(?<request>.*)\" (?<status>\\d+) (?<bodyBytesSent>\\d+) \"(?<referer>.*)\" \"(?<userAgent>[^\"]*)\".*$"
conf.probe.httpaccesslog.pattern=