# StatoolInfos StatoolInfos is a simple statistic tool software. ## LICENSE StatoolInfos is released under the GNU AGPL+ license. Enjoy! Authors: * Christian Pierre MOMON Contributors : Angie, MrFlos, Quentin Duchemin, Fabrice61, Thomas TConstans, labecasse, Antoine Jaba, setop, Jérémy Collot, Pilou, Kepon, Laurent Sleto, Rodinux… ### Cat Avatar Generator images I am glad to use the beautiful artwork of David Revoy (http://www.peppercarrot.com). * Graphics CC-BY http://creativecommons.org/licenses/by/4.0/ * Service: https://www.peppercarrot.com/extras/html/2016_cat-generator/ * Source: https://framagit.org/Deevad/cat-avatar-generator * Originally inspired of the code for "MonsterID" by Andreas Gohr http://www.splitbrain.org/go/monsterid. ### Circle-icons I am glad to use the generous artwork of ElegantThemes.com : * https://www.elegantthemes.com/blog/freebie-of-the-week/beautiful-flat-icons-for-free * "These icons are completely free and Open Source under the GPL". ### Logo Author: Christian Pierre MOMON License: Creative Commons CC BY-SA last version. ### Other stuff License of other stuff: * csv-logo.svg, json-logo.svg, ods-icon.svg: * Author: Christian P. Momon * License: CC BY-SA 3+. * other: without any specific information, default license of images is CC BY-SA 3+. ## DOCUMENTATION Read CONCEPT.md and ONTOLOGY.md files. ## Requirements - Java 11 - Eclipse 4.16 (202006). ## INSTALL Install Java : ``` apt-get install openjdk-11-jre-headless ``` Download the last release: https://forge.devinsy.fr/devinsy/statoolinfos/releases Unzip the package and move the `statoolinfos.jar` and `statoolinfo.sh` files in `/srv/statoolinfos/bin/`. Create link: ``` cd /srv/statoolinfos/bin/ ; ln -s statoolinfos.sh statoolinfos ``` ## Usage ``` Usage: statoolinfos [ -h | -help | --help ] statoolinfos [ -v | -version | --version ] statoolinfos build build property files from conf and input statoolinfos clear remove property files from conf statoolinfos crawl crawl all file from conf and input statoolinfos format format property files in tiny way s̶t̶a̶t̶o̶o̶l̶i̶n̶f̶o̶s̶ ̶h̶t̶m̶l̶i̶z̶e̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶ ̶<̶c̶o̶n̶f̶i̶g̶u̶r̶a̶t̶i̶o̶n̶f̶i̶l̶e̶>̶ ̶ ̶ ̶ ̶ ̶g̶e̶n̶e̶r̶a̶t̶e̶ ̶w̶e̶b̶ ̶p̶a̶g̶e̶s̶ ̶f̶r̶o̶m̶ ̶c̶o̶n̶f̶ REMOVED since splitweb statoolinfos probe OPTION [|] generate metrics files from conf OPTION = [-full|-today|-previousday|-NN] with NN a day count statoolinfos tagdate update the file.datetime file statoolinfos uptime update uptime journal statoolinfos list file display http access log files statoolinfos list log [-bot|-nobot] display http access log lines statoolinfos list ip [-bot|-nobot] generate ip list from http log file statoolinfos list ua [-bot|-nobot] generate user agent list from http log file statoolinfos list visitor [-bot|-nobot] generate visitors (ip+ua) list from http log file statoolinfos stat ip [-bot|-nobot] generate stats about ip from http log file statoolinfos stat ua [-bot|-nobot] generate stats about user agent from http log file statoolinfos stat visitor [-bot|-nobot] generate stats about visitor (ip+ua) from http log file statoolinfos list errfile display http error log files statoolinfos list errlog display http error log lines ``` ### htmlize removed The htmlize command is removed since 2024-07. As a reminder, the htmlize command generated absolutely all the pages needed for a static site. As the number of pages generated became too large (> 10000 with 100 organizations), the decision was made to separate the html valuation into a separate project, StatoolInfosWeb. A dynamic web site allows to generate page only when it is necessary. ## Generate metrics files ### Basics Create a configuration file `/srv/statoolInfos/conf/foo.bar.org.conf`: ``` conf.probe.types=, conf.probe.metrictype1param1.file= conf.probe.metrictype1param2.file= conf.probe.metrictype2param1.file= conf.probe.metrictype2param2.file= conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` Generate the metric file for the first time: ``` /srv/statoolinfos/bin/statoolinfos probe -full /srv/statoolinfos/conf/foo.bar.org.conf ``` Create a cron file in `/etc/cron.d/statoolinfos` to update the metric file everyday: ``` 1 0,12 * * * root /srv/statoolinfos/bin/statoolinfos probe -previousday /srv/statoolinfos/conf/foo.bar.org.conf >> /srv/statoolinfos/cron.log ``` In case of several configuration files, just set the configuration directory (without wildcard): ``` 1 0,12 * * * root /srv/statoolinfos/bin/statoolinfos probe -previousday /srv/statoolinfos/conf/ >> /srv/statoolinfos/cron.log ``` Warning: in previous day mode, the metrics generated are overwrited for the last month, the last week and the last day. So, **six weeks in logs are required**. ### Etherpad metrics Configuration template: ``` conf.probe.types=Etherpad conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log* conf.probe.httpaccesslog.pattern= conf.probe.httpaccesslog.pathfilter= # This parameter requires real ip proxy setting and log output setting. conf.probe.etherpad.logs=/var/log/etherpad/output.log # Database URL. Example of the most frequent cases: # jdbc:mariadb://localhost:1234/databasename # jdbc:mysql://localhost:1234/databasename # jdbc:postgresql://localhost:1234/databasename # jdbc:sqlite:/foo/bar/databasename.sqlite conf.probe.etherpad.database.url= conf.probe.etherpad.database.user= conf.probe.etherpad.database.password= conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Framdadate metrics (coming soon) Configuration template: ``` conf.probe.types=Framadate conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Gitea metrics Configuration template: ``` conf.probe.types=Gitea conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log* conf.probe.httpaccesslog.pattern= conf.probe.httpaccesslog.pathfilter= conf.probe.gitea.data=/opt/gitea/data conf.probe.gitea.api.url=https://forge.libre-service.eu/ conf.probe.gitea.token=b6598c616b1cd350b834258205da4e5e8b951005 # jdbc:mariadb://localhost:1234/databasename # jdbc:mysql://localhost:1234/databasename # jdbc:postgresql://localhost:1234/databasename # jdbc:sqlite:/foo/bar/databasename.sqlite conf.probe.gitea.database.url= conf.probe.gitea.database.user= conf.probe.gitea.database.password= conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### GSL metrics Configuration template: ``` conf.probe.types=GSL conf.probe.gsl.stats=/foo/bar/stats.properties conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### LibreQR metrics Configuration template: ``` conf.probe.types=LibreQR conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log* conf.probe.httpaccesslog.pattern= conf.probe.httpaccesslog.pathfilter= conf.probe.libreqr.datafiles=/var/www/foo.bar.org/temp/ conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Minetest metrics Configuration template: ``` conf.probe.types=Minetest conf.probe.minetest.logs=/home/cpm/Projets/StatoolInfos/EnvTest/minetest/minetest.log* conf.probe.minetest.players.database.url=jdbc:postgresql://localhost:5432/minetestdb conf.probe.minetest.players.database.user=minetestdba conf.probe.minetest.players.database.password=XXXXXXXXXXX conf.probe.minetest.worlds.database.url=jdbc:postgresql://localhost:5432/minetestdb conf.probe.minetest.worlds.database.user=minetestdba conf.probe.minetest.worlds.database.password=XXXXXXXXXXX conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Mumble metrics Configuration template: ``` conf.probe.types=Mumble conf.probe.mumble.logs=/var/log/mumble-server/mumble-server.log* conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Nextcloud metrics (coming soon) Configuration template: ``` conf.probe.types=Nextcloud conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log* conf.probe.httpaccesslog.pattern= conf.probe.httpaccesslog.pathfilter= conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### PrivateBin metrics (partial) Warning: works fine if database, image and comment options are disabled. Configuration template: ``` conf.probe.types=PrivateBin conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log* conf.probe.httpaccesslog.pattern= conf.probe.httpaccesslog.pathfilter= conf.probe.privatebin.data=/var/www/paste.libre-service.eu/data/ conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties ``` ### Web metrics Configuration template: ``` conf.probe.types=HttpAccessLog, HttpErrorLog conf.probe.httpaccesslog.file=/var/log/apache2/foo.bar.org-access.log*,/var/log/apache2/foo.bar.org-extraaccess.log* conf.probe.httperrorlog.file=/var/log/apache2/foo.bar.org-error.log*,/var/log/apache2/foo.bar.org-extraerror.log* conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/foo.bar.org-metrics.properties # Custom access log pattern with Java regex. # Default: ^(?[a-fA-F0-9\:\.]+) - (?[^\[]+) \[(?