Compare commits

..

No commits in common. "main" and "0.6.0" have entirely different histories.
main ... 0.6.0

48 changed files with 145 additions and 820 deletions

View file

@ -20,6 +20,10 @@
<classpathentry kind="lib" path="lib/catgenerator-core-0.1.7.jar" sourcepath="lib/catgenerator-core-0.1.7-sources.zip"/> <classpathentry kind="lib" path="lib/catgenerator-core-0.1.7.jar" sourcepath="lib/catgenerator-core-0.1.7-sources.zip"/>
<classpathentry kind="lib" path="lib/jOpenDocument-1.3.jar" sourcepath="lib/jOpenDocument-src-1.3.zip"/> <classpathentry kind="lib" path="lib/jOpenDocument-1.3.jar" sourcepath="lib/jOpenDocument-src-1.3.zip"/>
<classpathentry kind="lib" path="lib/threeten-extra-1.5.0.jar" sourcepath="lib/threeten-extra-1.5.0-sources.jar"/> <classpathentry kind="lib" path="lib/threeten-extra-1.5.0.jar" sourcepath="lib/threeten-extra-1.5.0-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/slf4j-api-1.7.32.jar" sourcepath="lib/Logs/slf4j-api-1.7.32-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/log4j-api-2.17.1.jar" sourcepath="lib/Logs/log4j-api-2.17.1-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/log4j-core-2.17.1.jar" sourcepath="lib/Logs/log4j-core-2.17.1-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/log4j-slf4j-impl-2.17.1.jar" sourcepath="lib/Logs/log4j-slf4j-impl-2.17.1-sources.jar"/>
<classpathentry kind="lib" path="lib/commons-io-2.11.0.jar" sourcepath="lib/commons-io-2.11.0-sources.jar"/> <classpathentry kind="lib" path="lib/commons-io-2.11.0.jar" sourcepath="lib/commons-io-2.11.0-sources.jar"/>
<classpathentry kind="lib" path="lib/json-simple-1.1.1.jar" sourcepath="lib/json-simple-1.1.1-sources.jar"/> <classpathentry kind="lib" path="lib/json-simple-1.1.1.jar" sourcepath="lib/json-simple-1.1.1-sources.jar"/>
<classpathentry kind="lib" path="lib/mariadb-java-client-3.0.3.jar"/> <classpathentry kind="lib" path="lib/mariadb-java-client-3.0.3.jar"/>
@ -37,9 +41,5 @@
<attribute name="optional" value="true"/> <attribute name="optional" value="true"/>
</attributes> </attributes>
</classpathentry> </classpathentry>
<classpathentry kind="lib" path="lib/Logs/log4j-api-2.23.1.jar" sourcepath="lib/Logs/log4j-api-2.23.1-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/log4j-core-2.23.1.jar" sourcepath="lib/Logs/log4j-core-2.23.1-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/log4j-slf4j2-impl-2.23.1.jar" sourcepath="lib/Logs/log4j-slf4j2-impl-2.32.1-sources.jar"/>
<classpathentry kind="lib" path="lib/Logs/slf4j-api-2.0.16.jar" sourcepath="lib/Logs/slf4j-api-2.0.16-sources.jar"/>
<classpathentry kind="output" path="bin"/> <classpathentry kind="output" path="bin"/>
</classpath> </classpath>

View file

@ -8,19 +8,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- … - …
## [0.6.1] - 2024-09-23
- Fixed bad log init with no configuration file.
- Fixed StackOverflowError case in AccessLog regex.
- Improved Javadoc and added test for isPage method.
- Fixed Java version in build script.
- Fixed label in script build.sh.
- Upgraded log4j libs from 1.7.32 to 2.23.1.
- Improved main README and resources files.
- Improved documentation.
- Improved build script.
## [0.6.0] - 2024-08-17 ## [0.6.0] - 2024-08-17
Change major number because of split web event. Change major number because of split web event.
@ -48,6 +35,7 @@ Changes for metrics:
- Improved user agent bot list. - Improved user agent bot list.
Changes for web building: Changes for web building:
- SPLITWEB: no more web building! See the StatoolInfosWeb project. - SPLITWEB: no more web building! See the StatoolInfosWeb project.

215
README.md
View file

@ -44,43 +44,15 @@ Read CONCEPT.md and ONTOLOGY.md files.
## INSTALL ## INSTALL
Install Java on Debian: Install Java :
``` ```
apt-get install openjdk-17-jre-headless apt-get install openjdk-17-jre-headless
``` ```
Download the last release: `https://forge.devinsy.fr/statool/statoolinfos/releases` and unzip the package. Download the last release: https://forge.devinsy.fr/statool/statoolinfos/releases
The `srv/` directory is a example of deployment tree: Unzip the package and move the `statoolinfos.jar` and `statoolinfo.sh` files in `/srv/statoolinfos/bin/`.
```
srv/
srv/statoolinfos
srv/statoolinfos/bin
srv/statoolinfos/bin/refresh
srv/statoolinfos/bin/probe
srv/statoolinfos/bin/uptime
srv/statoolinfos/bin/statoolinfos.sh
srv/statoolinfos/bin/crawl
srv/statoolinfos/conf
srv/statoolinfos/conf/sample-probe-http.conf
srv/statoolinfos/conf/log4j2-default.properties
srv/statoolinfos/conf/sample-probe-minetest.conf
srv/statoolinfos/conf/sample-federation.conf
srv/statoolinfos/conf/sample-probe-qrcode.conf
srv/statoolinfos/cache
srv/statoolinfos/statoolinfos-cron.log
srv/statoolinfos/well-known
srv/statoolinfos/well-known/statoolinfos
srv/statoolinfos/well-known/statoolinfos/organization-template.properties
srv/statoolinfos/well-known/statoolinfos/services.properties
srv/statoolinfos/well-known/statoolinfos/BOM
srv/statoolinfos/well-known/statoolinfos/federation-template.properties
srv/statoolinfos/well-known/statoolinfos/service-template.properties
```
Then, move the `statoolinfos.jar` file in `/srv/statoolinfos/bin/`.
Create link: Create link:
@ -88,15 +60,6 @@ Create link:
cd /srv/statoolinfos/bin/ ; ln -s statoolinfos.sh statoolinfos cd /srv/statoolinfos/bin/ ; ln -s statoolinfos.sh statoolinfos
``` ```
Edit script in `/srv/statoolinfos/bin/` to target your federation configuration file.
In case of bad language output, add these lines on top of cron file:
```
LANGUAGE=en_EN.UTF8
LC_ALL=en_EN.UTF-8
```
## Usage ## Usage
@ -127,163 +90,18 @@ Usage:
statoolinfos list errlog <logfilesorconfigfile> display http error log lines statoolinfos list errlog <logfilesorconfigfile> display http error log lines
``` ```
## Simple federation configuration file ### htmlize removed
The htmlize command is removed since 2024-07.
Example of `libre-service.eu.conf`: As a reminder, the htmlize command generated absolutely all the pages needed for a static site.
```
# [Configuration] conf + organization + services + machines + metrics
conf.class=federation
conf.protocol=StatoolInfos-0.6.0
conf.crawl.input=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu.properties
conf.crawl.cache=/srv/statoolinfos/cache/
```
The field `input` gives the federation properties URL file. Reminder, the StatoolInfos Protocol requires that the federation properties file is exposed on the Web.
The field `cache` gives local directory used as data cache.
## Crawl
The `statoolinfos crawl` command loads the federation configuration file, then downloads the federation properties file and download all files set in.
In case of use with StatoolInfosWeb, never forget to give read permissions to Tomcat server. So, create a dedicated script would help.
Example of script `/srv/statoolinfos/bin/crawl`:
```
#!/bin/bash
echo "========== CRAWL at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos crawl ../conf/federation.conf
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache
```
Example of cron configuration `/etc/cron.d/statoolinfos`:
```
4 * * * * root /srv/statoolinfos/bin/crawl >> /srv/statoolinfos/statoolinfos-cron.log
```
Adjust your frequency according to your need. In case of large federation, once an hour is a good compromise.
## Uptime
The `statoolinfos uptime` command is very useful to collect the uptime of services. It's just to share which services are down and since how many time. It is not a monitoring feature because an one hour check is enough for this. Actually, only website are checked.
In case of use with StatoolInfosWeb, never forget to give read permissions to Tomcat server. So, create a dedicated script would help.
Example of script `/srv/statoolinfos/bin/uptime`:
```
#!/bin/bash
echo "========== UPTIME at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos uptime ../conf/federation.conf
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache
```
Example of cron configuration `/etc/cron.d/statoolinfos`:
```
4 * * * * root /srv/statoolinfos/bin/uptime >> /srv/statoolinfos/statoolinfos-cron.log
```
One check per hour is a good idea. In the uptime page, the mouse over a yellow status icon show the list of the check results.
In case of crawl, uptime and probe cron on the same computer, you can have different cron frequencies. It is better to have only script for call all them:
```
#!/bin/bash
echo "========== REFRESH at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./probe
./crawl
./uptime
#./htmlize
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache
```
And so your cron can be like that:
```
4 0,12,23 * * * root /srv/statoolinfos/bin/refresh >> /srv/statoolinfos/statoolinfos-cron.log
4 1-11,13-22 * * * root /srv/statoolinfos/bin/uptime >> /srv/statoolinfos/statoolinfos-cron.log
```
## Htmlize command **removed**
The `statoolinfos htmlize` command is removed since 2024-07.
As a reminder, the htmlize command generated absolutely all the pages needed for a static website.
As the number of pages generated became too large (> 10000 with 100 organizations), the decision was made to separate the html valuation into a separate project, StatoolInfosWeb. As the number of pages generated became too large (> 10000 with 100 organizations), the decision was made to separate the html valuation into a separate project, StatoolInfosWeb.
A dynamic web site allows to generate page only when it is necessary. A dynamic web site allows to generate page only when it is necessary.
## Generate metrics files
## Probe (Generate metrics files) ### Basics
### Basics
The `statoolinfos probe` command computes data sources to generate metric properties files. In case of already existings file, the data are loaded before the computation to be updated.
In case of use with StatoolInfosWeb, never forget to give read permissions to Tomcat server. So, create a dedicated script would help.
Example of script `/srv/statoolinfos/bin/probe`:
```
#!/bin/bash
echo "========== PROBE at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos probe -previousday ../conf/
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache
```
Note: set a specific configuration file is not necessary, because you can have multiple service to probe. Only probe configuration files will be manage.
Example of cron configuration `/etc/cron.d/statoolinfos`:
```
4 * * * * root /srv/statoolinfos/bin/probe >> /srv/statoolinfos/statoolinfos-cron.log
```
### HTTP Server configuration
The probe feature generates properties files to expose on web. StatoolInfos protocol recommends to follow the well-known convention.
Example of `/etc/apache2/statoolinfos.conf` generic configuration file:
```
Alias "/.well-known/statoolinfos/" "/srv/statoolinfos/well-known/statoolinfos/"
<Directory "/srv/statoolinfos/well-known/statoolinfos">
Options +Indexes
Require all granted
</Directory>
```
Example of include directive in `/etc/apache2/sites-enabled/myservice.conf`:
```
<VirtualHost *:443>
[…]
Include statoolinfos.conf
[…]
</VirtualHost>
```
### Sample configuration file
Create a configuration file `/srv/statoolInfos/conf/foo.bar.org.conf`: Create a configuration file `/srv/statoolInfos/conf/foo.bar.org.conf`:
@ -304,7 +122,7 @@ Generate the metric file for the first time:
Create a cron file in `/etc/cron.d/statoolinfos` to update the metric file everyday: Create a cron file in `/etc/cron.d/statoolinfos` to update the metric file everyday:
``` ```
4 0,12 * * * root /srv/statoolinfos/bin/statoolinfos probe -previousday /srv/statoolinfos/conf/foo.bar.org.conf >> /srv/statoolinfos/cron.log 1 0,12 * * * root /srv/statoolinfos/bin/statoolinfos probe -previousday /srv/statoolinfos/conf/foo.bar.org.conf >> /srv/statoolinfos/cron.log
``` ```
In case of several configuration files, just set the configuration directory (without wildcard): In case of several configuration files, just set the configuration directory (without wildcard):
@ -473,7 +291,6 @@ conf.prob.httperrorlog.datetimepattern=
Warning: to have month metric values, you have to set **retention log to 7 weeks or 50 days** in Nginx or Apache logrotate configuration. Warning: to have month metric values, you have to set **retention log to 7 weeks or 50 days** in Nginx or Apache logrotate configuration.
For Nginx, éditer le fichier `/etc/logrotate.d/nginx` : For Nginx, éditer le fichier `/etc/logrotate.d/nginx` :
``` ```
/var/log/nginx/*.log { /var/log/nginx/*.log {
weekly weekly
@ -482,23 +299,9 @@ For Nginx, éditer le fichier `/etc/logrotate.d/nginx` :
``` ```
For Apache, éditer le fichier `/etc/logrotate.d/apache2` : For Apache, éditer le fichier `/etc/logrotate.d/apache2` :
``` ```
/var/log/apache2/*.log { /var/log/apache2/*.log {
weekly weekly
missingok missingok
rotate 7 rotate 7
``` ```
### Cron all task
A situation you will encounter is the need to do several tasks from the same machine. In this case, avoid multiple cron in the same time. Put all in Bash script and call it with only one cron.
Example:
```
4 0,12,23 * * * root /srv/statoolinfos/bin/refresh >> /srv/statoolinfos/statoolinfos-cron.log
4 1-11,13-22 * * * root /srv/statoolinfos/bin/statoolinfos uptime /srv/statoolinfos/conf/libre-service.eu.conf >> /srv/statoolinfos/statoolinfos-cron.log
```

View file

@ -126,22 +126,16 @@
<!-- Copy stuff --> <!-- Copy stuff -->
<echo message="==========> Copy Stuff" />
<copy file="resources/scripts/${product.name}.sh" todir="${dist.dir}/" overwrite="true" /> <copy file="resources/scripts/${product.name}.sh" todir="${dist.dir}/" overwrite="true" />
<chmod file="${dist.dir}/${product.name}.sh" perm="ugo+rx" /> <chmod file="${dist.dir}/${product.name}.sh" perm="ugo+rx" />
<copy file="LICENSE" todir="${dist.dir}/" overwrite="true" failonerror="false" /> <copy file="LICENSE" todir="${dist.dir}/" overwrite="true" failonerror="false" />
<!--copy file="README" todir="${dist.dir}/" overwrite="true" failonerror="false" /--> <!--copy file="README" todir="${dist.dir}/" overwrite="true" failonerror="false" /-->
<copy file="README.md" todir="${dist.dir}/" overwrite="true" failonerror="false" /> <copy file="README.md" todir="${dist.dir}/" overwrite="true" failonerror="false" />
<!--copy file="scripts/log4j.properties" todir="${dist.dir}/" overwrite="true" /--> <!--copy file="scripts/log4j.properties" todir="${dist.dir}/" overwrite="true" /-->
<mkdir dir="${dist.dir}/srv" />
<copy todir="${dist.dir}/srv/" overwrite="true" failonerror="false">
<fileset dir="resources/srv" includes="**" />
</copy>
<echo message="==========>ZZZZ" />
<mkdir dir="${dist.dir}/conf" /> <mkdir dir="${dist.dir}/conf" />
<copy todir="${dist.dir}/conf"> <copy todir="${dist.dir}/conf">
<fileset dir="${basedir}/resources/conf" includes="**" /> <fileset dir="${basedir}/resources/conf" includes="*" />
</copy> </copy>
<!-- <!--
<mkdir dir="${dist.dir}/man"/> <mkdir dir="${dist.dir}/man"/>

View file

@ -1,3 +1,3 @@
#Build Number for ANT. Do not edit! #Build Number for ANT. Do not edit!
#Mon Sep 23 04:40:53 CEST 2024 #Sat Aug 17 23:38:50 CEST 2024
build.number=2 build.number=1

121
build.sh
View file

@ -5,7 +5,7 @@
# #
function help function help
{ {
echo "Build script." echo "StatoolInfos build script."
echo "Usage: build.sh [ -h | -help | --help | -snapshot | -local | -full ]" echo "Usage: build.sh [ -h | -help | --help | -snapshot | -local | -full ]"
echo " -h, -help, --help display this help." echo " -h, -help, --help display this help."
echo " -snapshot, --snapshot build a snapshot." echo " -snapshot, --snapshot build a snapshot."
@ -15,9 +15,9 @@ function help
} }
# #
# Check env. # Build snapshot.
# #
function check_env function build_snapshot
{ {
okCount=0 okCount=0
@ -40,13 +40,96 @@ function check_env
fi fi
# Java version check. # Java version check.
JAVA_VERSION=17;
javaVersionCheck=`javac -version 2>&1` javaVersionCheck=`javac -version 2>&1`
if [[ "$javaVersionCheck" =~ ^.*\ ${JAVA_VERSION}. ]]; then if [[ "$javaVersionCheck" =~ ^.*\ 11. ]]; then
echo "Java ${JAVA_VERSION} version requirement..... OK" echo "Java 11 version requirement..... OK"
let "okCount+=1" let "okCount+=1"
else else
echo "Java ${JAVA_VERSION} version requirement..... MISSING" echo "Java 11 version requirement..... MISSING"
fi
if [ "$okCount" == 3 ]; then
echo "Requirement OK"
ant -f build-snapshot.xml
else
echo "Requirement MISSING, build abort"
fi
}
#
# Build local.
#
function build_local
{
okCount=0
# Ant check.
antCheck=`which ant`
if [[ "$antCheck" =~ ^/.* ]]; then
echo "Ant requirement................ OK"
let "okCount+=1"
else
echo "Ant requirement................ MISSING"
fi
# Javac check.
javacCheck=`which javac`
if [[ "$javacCheck" =~ ^/.* ]]; then
echo "Javac requirement.............. OK"
let "okCount+=1"
else
echo "Javac requirement.............. MISSING"
fi
# Java version check.
javaVersionCheck=`javac -version 2>&1`
if [[ "$javaVersionCheck" =~ ^.*\ 11. ]]; then
echo "Java 11 version requirement..... OK"
let "okCount+=1"
else
echo "Java 11 version requirement..... MISSING"
fi
if [ "$okCount" == 3 ]; then
echo "Requirement OK"
ant -f build-local.xml
else
echo "Requirement MISSING, build abort"
fi
}
#
# Build tagandpush.
#
function build_tagandpush
{
okCount=0
# Ant check.
antCheck=`which ant`
if [[ "$antCheck" =~ ^/.* ]]; then
echo "Ant requirement................ OK"
let "okCount+=1"
else
echo "Ant requirement................ MISSING"
fi
# Javac check.
javacCheck=`which javac`
if [[ "$javacCheck" =~ ^/.* ]]; then
echo "Javac requirement.............. OK"
let "okCount+=1"
else
echo "Javac requirement.............. MISSING"
fi
# Java version check.
javaVersionCheck=`javac -version 2>&1`
if [[ "$javaVersionCheck" =~ ^.*\ 11. ]]; then
echo "Java 11 version requirement..... OK"
let "okCount+=1"
else
echo "Java 11 version requirement..... MISSING"
fi fi
# Git check. # Git check.
@ -60,28 +143,26 @@ function check_env
if [ "$okCount" == 4 ]; then if [ "$okCount" == 4 ]; then
echo "Requirement OK" echo "Requirement OK"
ant -f build-tagandpush.xml
else else
echo "Requirement MISSING, build abort" echo "Requirement MISSING, build abort"
exit -1
fi fi
} }
# #
# Main. # Main.
# #
if [ "$#" -eq 0 ] || [ "$1" == "-h" ] || [ "$1" == "-help" ] || [ "$1" == "--help" ]; then if [ "$#" -eq 0 ] || [ "$1" == "-h" ] || [ "$1" == "-help" ] || [ "$1" == "--help" ]; then
help help
elif [ "$1" == "-snapshot" ] || [ "$1" == "--snapshot" ] ; then
build_snapshot
elif [ "$1" == "-local" ] || [ "$1" == "--local" ] ; then
build_local
elif [ "$1" == "-tagandpush" ] || [ "$1" == "--tagandpush" ] ; then
build_tagandpush
else else
check_env echo "Invalid parameters."
if [ "$1" == "-snapshot" ] || [ "$1" == "--snapshot" ] ; then help
ant -f build-snapshot.xml
elif [ "$1" == "-local" ] || [ "$1" == "--local" ] ; then
ant -f build-local.xml
elif [ "$1" == "-tagandpush" ] || [ "$1" == "--tagandpush" ] ; then
ant -f build-tagandpush.xml
else
echo "Invalid parameters."
help
fi
fi fi

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -1 +0,0 @@
See the `srv/statoolinfos/conf` directory.

View file

@ -19,6 +19,6 @@ conf.probe.httpaccesslog.pattern=
conf.probe.httperrorlog.file=/var/log/apache2/qrcode.libre-service.eu/qrcode.libre-service.eu-error.log* conf.probe.httperrorlog.file=/var/log/apache2/qrcode.libre-service.eu/qrcode.libre-service.eu-error.log*
# Target file to write computed metrics. If file exists then it is renamed with '.bak'. # Target file to write computed metrics. If file exists then it is renamed with '.bak'.
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/qrcode.libre-service.eu-metrics.properties conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/qrcode.libre-service.eu.metrics

View file

@ -20,4 +20,4 @@ conf.probe.minetest.players.db.password=
conf.probe.minetest.datafilepath= conf.probe.minetest.datafilepath=
# Target file to write computed metrics. If file exists then it is renamed with '.bak'. # Target file to write computed metrics. If file exists then it is renamed with '.bak'.
conf.probe.target=/srv/statoolinfos/well-known/statoolinfo/www.myservice.org-metrics.properties conf.probe.target=/home/cpm/Projets/StatoolInfos/EnvTest/minetest/test.metrics

View file

@ -0,0 +1,14 @@
#
# Sample StatoolInfos config file.
#
# Warning: move this file outside the Git directory.
conf.class=federation
conf.protocol=StatoolInfos-0.5.0
conf.crawl.input=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu.properties
conf.crawl.cache=/srv/statoolinfos/cache/
conf.htmlize.categories=/srv/statoolinfos/inputs/categories/categories.properties
conf.htmlize.input=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu.properties
conf.htmlize.directory=/srv/statoolinfos/www/

View file

@ -0,0 +1,3 @@
LANGUAGE=fr_FR.UTF8
LC_ALL=fr_FR.UTF-8
/5 * * * * root /srv/statoolinfos/bin/statoolinfo.sh >> /srv/statoolinfos/statoolinfos-cron.log

View file

@ -1,5 +0,0 @@
Alias "/.well-known/statoolinfos/" "/srv/statoolinfos/well-known/statoolinfos/"
<Directory "/srv/statoolinfos/well-known/statoolinfos">
Options +Indexes
Require all granted
</Directory>

View file

@ -1,2 +0,0 @@
4 0,12,23 * * * root /srv/statoolinfos/bin/refresh >> /srv/statoolinfos/statoolinfos-cron.log
4 1-11,13-22 * * * root /srv/statoolinfos/bin/uptime >> /srv/statoolinfos/statoolinfos-cron.log

View file

@ -1,7 +0,0 @@
#!/bin/bash
echo "========== CRAWL at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos crawl ../conf/federation.conf
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache

View file

@ -1,7 +0,0 @@
#!/bin/bash
echo "========== PROBE at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos probe -previousday ../conf/
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache

View file

@ -1,10 +0,0 @@
#!/bin/bash
echo "========== REFRESH at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./probe
./crawl
./uptime
#./htmlize
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache

View file

@ -1,22 +0,0 @@
#!/bin/bash
# Optional environment settings:
# export LOG4J_CONFIGURATION_FILE="toto.properties"
# export LOG4J_LEVEL=ERROR
# https://logging.apache.org/log4j/log4j-2.11.2/manual/configuration.html#System_Properties
# Java check.
javaCheck=`which java`
if [[ "$javaCheck" =~ ^/.* ]]; then
#echo "Java requirement............... OK"
# Optional system properties:
# LOGFILE="-Dlog4j2.configurationFile=../../log4j2.properties"
# LOGLEVEL="-Dlog4j2.level=ERROR"
# IPV6ONLY="-Djava.net.preferIPv4Stack=false -Djava.net.preferIPv6Addresses=true"
# https://logging.apache.org/log4j/log4j-2.11.2/manual/configuration.html#System_Properties
java -Djava.awt.headless=true $IPV6ONLY $LOGFILE $LOGLEVEL -jar "$(dirname "$0")"/statoolinfos.jar $@
else
echo "Java requirement............... MISSING"
fi

View file

@ -1,7 +0,0 @@
#!/bin/bash
echo "========== UPTIME at $(date --iso-8601=seconds)"
cd $(dirname "$0")
./statoolinfos uptime ../conf/federation.conf
chmod -R g+r /srv/statoolinfos/cache
chgrp -R tomcat /srv/statoolinfos/cache

View file

@ -1,8 +0,0 @@
# [Configuration]
conf.class=federation
conf.protocol=StatoolInfos-0.6.0
conf.crawl.input=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu.properties
conf.crawl.cache=/srv/statoolinfos/cache/

View file

@ -1,8 +0,0 @@
# [Configuration]
conf.class=service
conf.protocol=StatoolInfos-0.6.0
conf.probe.types=HttpAccessLog, HttpErrorLog
conf.probe.httpaccesslog.file=/var/log/apache2/www.libre-service.eu/www.libre-service.eu-access.log*
conf.probe.httperrorlog.file=/var/log/apache2/www.libre-service.eu/www.libre-service.eu-error.log*
conf.probe.target=/srv/statoolinfos/well-known/statoolinfos/www.libre-service.eu-metrics.properties

View file

@ -1,87 +0,0 @@
# federation.properties
# [File]
# Classe du fichier (valeur parmi {Federation, Organization, Service, Device}, obligatoire).
file.class = federation
# Version de l'ontologie utilisée utilisé (type STRING, recommandé).
file.protocol = ChatonsInfos-0.5
# Date et horaire de génération du fichier (type DATETIME, recommandé, ex. 2020-07-06T14:23:20).
file.datetime =
# Nom du générateur du fichier (type STRING, recommandé).
file.generator =
# [Federation]
# Nom de la fédération (type STRING, obligatoire).
federation.name=
# Description de la fédération (type STRING, recommandé).
federation.description =
# Lien du site web de la fédération (type URL, recommandé).
federation.website =
# Lien du logo de la fédération (type URL, recommandé).
federation.logo =
# Lien de la page web de contact de la fédération (type URL, recommandé).
federation.contact.url =
# Courriel de contact de la fédération (type EMAIL, recommandé).
federation.contact.email =
# Lien du compte Twitter de la fédération (type URL, optionnel).
federation.socialnetworks.twitter =
# Lien de la page Facebook de la fédération (type URL, optionnel).
federation.socialnetworks.facebook =
# Lien du compte Mastodon de la fédération (type URL, optionnel).
federation.socialnetworks.mastodon =
# Lien du compte Diaspora de la fédération (type URL, optionnel).
federation.socialnetworks.diaspora =
# Lien du compte PeerTube de la fédération(type URL, optionnel).
federation.socialnetworks.peertube =
# Lien du compte Funkwhale de la fédération (type URL, optionnel).
federation.socialnetworks.funkwhale =
# Lien du compte Pixelfeld de la fédération (type URL, optionnel).
federation.socialnetworks.pixelfeld =
# Lien du compte Mobilizon de la fédération (type URL, optionnel).
federation.socialnetworks.mobilizon =
# Lien de la page web des mentions légales de la fédération (type URL, recommandé).
federation.legal.url =
# Lien de la documentation web de la fédération (type URL, recommandé).
federation.documentation.url =
# Lien de la documentation technique de la fédération (type URL, recommandé).
federation.guide.technical =
# Lien des tutoriels web de la fédération (type URL, recommandé).
federation.guide.technical =
# Date de naissance de la fédération (type DATE (AAAA-MM-JJ ou JJ/MM/AAAA), recommandé).
federation.startdate =
# [Subs]
# Un lien vers un fichier properties complémentaire (type URL, optionnel)
# Une clé (nomination libre) pour chacune de vos organisations,
# par exemple : subs.monchaton = https://www.monchaton.ext/.well-known/monchaton.properties
subs.01 =
# [Metrics]

View file

@ -1,136 +0,0 @@
# membres.properties
# [File]
# Classe du fichier (valeur parmi {Federation, Organization, Service, Device}, obligatoire).
file.class = organization
# Version de l'ontologie utilisée utilisé (type STRING, recommandé, ex. ChatonsInfos-0.1).
file.protocol = ChatonsInfos-0.5
# Date et horaire de génération du fichier (type DATETIME, recommandé, ex. 2020-07-06T14:23:20).
file.datetime =
# Nom du générateur du fichier (type STRING, recommandé, ex. Christian avec ses doigts).
file.generator =
# [Organisation]
# Nom de l'organisation (type STRING, obligatoire, ex. LibreServiceEU).
organization.name =
# Description de l'organisation (type STRING, recommandé, ex. LibreServiceEU est le chaton de Devinsy).
organization.description =
# Type d'organisation (un parmi {ASSOCIATION, INFORMAL, COOPERATIVE, MICROCOMPANY, COMPANY, INDIVIDUAL, OTHER}, obligatoire).
organization.type =
# Lien du site web de l'organisation (type URL, recommandé, ex. https://www.libre-service.eu/).
organization.website =
# Lien du logo de l'organisation (type URL, recommandé, ex. https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu-logo-carre.svg).
organization.logo =
# Liens vers les comptes réseaux sociaux de l'organisation (type URL, optionnel).
# Liste non exhaustive à laquelle vous pouvez ajouter d'autres services.
organization.socialnetworks.diaspora =
organization.socialnetworks.facebook =
organization.socialnetworks.funkwhale =
organization.socialnetworks.mastodon =
organization.socialnetworks.pleroma =
organization.socialnetworks.mobilizon =
organization.socialnetworks.peertube =
organization.socialnetworks.pixelfeld =
organization.socialnetworks.twitter =
# Adresse vers les salons/équipes de discussion publics de l'organisation (type STRING, optionnel).
# Liste non exhaustive à laquelle vous pouvez ajouter d'autres services.
organization.chatrooms.xmpp =
organization.chatrooms.irc =
organization.chatrooms.matrix =
organization.chatrooms.rocketchat =
organization.chatrooms.mattermost =
# Nom du propriétaire de l'organisation (type STRING, optionnel, ex. Devinsy).
organization.owner.name =
# Lien du site web du propriétaire de l'organisation (type URL, optionnel, ex. https://www.devinsy.fr/).
organization.owner.website =
# Lien du logo du propriétaire de l'organisation (type URL, optionnel, ex. https://www.devinsy.fr/logo_devinsy.svg).
organization.owner.logo =
# Lien de la page web de contact de l'organisation (type URL, recommandé, ex. https://www.libre-service.eu/contact.html).
organization.contact.url =
# Courriel de contact de l'organisation (type EMAIL, recommandé, ex. contact@libre-service.eu).
organization.contact.email =
# Lien de la page des mentions légales de l'organisation (type URL, recommandé, ex. https://www.libre-service.eu/mentions_legales.html).
organization.legal.url =
# Lien de la documentation technique de l'organisation (type URL, recommandé, ex. https://forge.devinsy.fr/libre-service.eu/documentation).
organization.guide.technical =
# Lien de la documentation utilisateur de l'organisation (type URL, recommandé, ex. https://www.libre-service.eu/services.xhtml).
organization.guide.user =
# Statut de l'organisation (un parmi {ACTIVE, IDLE, AWAY}, obligatoire).
organization.status.level =
# Description du statut de l'organisation (type STRING, optionnel, ex. en sommeil).
organization.status.description =
# Date de création de l'organisation (type DATE, recommandé, ex. 08/11/2018).
organization.startdate =
# Date de fermeture de l'organisation (type DATE, optionnel, ex. 22/02/2022).
organization.enddate =
# Date d'entrée dans le collectif (type DATE, obligatoire, ex. 08/11/2018).
organization.memberof.chatons.startdate =
# Date de sortie du collectif (type DATE, optionnel, ex. 08/11/2019).
organization.memberof.chatons.enddate =
# Statut en tant que membre de l'organisation (un parmi {ACTIVE, IDLE, AWAY}, obligatoire).
organization.memberof.chatons.status.level =
# Description du statut en tant que membre de l'organisation (type STRING, optionnel, ex. en sommeil).
organization.memberof.chatons.status.description =
# Pays de l'organisation (type STRING, recommandé, ex. France).
organization.country.name =
# Code pays de l'organisation (type COUNTRY_CODE sur 2 caractères, obligatoire, ex.ex. FR ou BE ou CH ou DE ou GB).
# Table ISO 3166-1 alpha-2 : https://fr.wikipedia.org/wiki/ISO_3166-1#Table_de_codage
organization.country.code =
# Géolocalisation de l'organisation
# Coordonnées GPS
# Latitude (type DECIMAL_DEGREE, recommandé, format DD, ex. 15,23456).
organization.geolocation.latitude =
# Longitude (type DECIMAL_DEGREE, recommandé, format DD, ex. -30,67890).
organization.geolocation.longitude =
# Adresse (type STRING, recommandé, ex. 1 rue croquette, 92370 Chaville).
organization.geolocation.address =
# Liens vers les services de paiement permettant de récolter des donations pour votre chaton (type URL, optionnel).
# Dans le cas d'une solution maison (exemple : pont vers votre banque), choisir "organization.funding.custom".
# Liste non exhaustive à laquelle vous pouvez ajouter d'autres services.
organization.funding.liberapay =
organization.funding.tipee =
organization.funding.helloasso =
organization.funding.paypal =
organization.funding.custom =
# [Subs]
# Un lien vers un fichier properties complémentaire (type URL, optionnel)
# Une clé (nomination libre) pour chacun de vos services,
# par exemple pour un service etherpad : subs.etherpad = https://www.monchaton.ext/.well-known/etherpad.properties
subs.foo1 =
# [Metrics]

View file

@ -1,163 +0,0 @@
# service.properties
#WARNING : cette fiche service ne concerne que les services logiciels
# [File]
# Classe du fichier (valeur parmi {Federation, Organization, Service, Device}, obligatoire).
file.class = service
# Version de l'ontologie utilisée utilisé (type STRING, recommandé).
file.protocol = ChatonsInfos-0.5
# Date et horaire de génération du fichier (type DATETIME, recommandé, ex. 2020-07-06T14:23:20).
file.datetime =
# Nom du générateur du fichier (type STRING, recommandé).
file.generator =
# [Service]
# Nom du service (type STRING, obligatoire).
service.name =
# Description du service (type STRING, recommandé).
service.description =
# Lien du site web du service (type URL, obligatoire - si pas possible, merci de créer une page de présentation du service).
service.website =
# Lien du logo du service (type URL, recommandé, ex. https://www.libre-service.eu/.well-known/statoolinfos/logo.svg).
service.logo =
# Lien de la page web des mentions légales du service (type URL, recommandé).
service.legal.url =
# Lien de la documentation web du service (type URL, recommandé).
service.guide.technical =
# Lien des aides web pour le service (type URL, recommandé).
service.guide.user =
# Lien de la page de support du service (type URL, recommandé).
service.contact.url =
# Courriel du support du service (type EMAIL, recommandé).
service.contact.email =
# Date d'ouverture du service (type DATE, obligatoire, ex. 22/02/2022).
service.startdate =
# Date de fermeture du service (type DATE, optionnel, ex. 22/02/2022).
service.enddate =
# Statut du service (un parmi {OK, WARNING, ALERT, ERROR, OVER, VOID}, obligatoire).
# OK : tout va bien (service en fonctionnement nominal).
# WARNING : attention (service potentiellement incomplet, maintenance prévue, etc.).
# ALERT : alerte (le service connait des dysfonctionnements, le service va bientôt fermer, etc.).
# ERROR : problème majeur (service en panne).
# OVER : terminé (le service n'existe plus).
# VOID : indéterminé (service non ouvert officiellement, configuration ChatonsInfos en cours, etc.).
service.status.level =
# Description du statut du service (type STRING, optionnel, ex. mise à jour en cours)
service.status.description =
# Inscriptions requises pour utiliser le service (un ou plusieurs parmi {None, Free, Member, Client}, obligatoire, ex. Free,Member).
# None : le service s'utilise sans inscription.
# Free : inscription ouverte à tout le monde et gratuite.
# Member : inscription restreinte aux membres (la notion de membre pouvant être très relative, par exemple, une famille, un cercle damis, adhérents d'association…).
# Client : inscription liée à une relation commerciale (facture…).
service.registration =
# Capacité à accueillir de nouveaux utilisateurs (un parmi {OPEN, FULL}, obligatoire).
# OPEN : le service accueille de nouveaux comptes.
# FULL : le service n'accueille plus de nouveau compte pour l'instant.
service.registration.load =
# Type d'installation du service, une valeur parmi {DISTRIBUTION, PROVIDER, PACKAGE, TOOLING, CLONEREPO, ARCHIVE, SOURCES, CONTAINER}, obligatoire.
# DISTRIBUTION : installation via le gestionnaire d'une distribution (apt, yum, etc.).
# PROVIDER : installation via le gestionnaire d'une distribution configuré avec une source externe (ex. /etc/apt/source.list.d/foo.list).
# PACKAGE : installation manuelle d'un paquet compatible distribution (ex. dpkg -i foo.deb).
# TOOLING : installation via un gestionnaire de paquets spécifique, différent de celui de la distribution (ex. pip…).
# CLONEREPO : clone manuel d'un dépôt (git clone…).
# ARCHIVE : application récupérée dans un tgz ou un zip ou un bzip2…
# SOURCES : compilation manuelle à partir des sources de l'application.
# CONTAINER : installation par containeur (Docker, Snap, Flatpak, etc.).
# L'installation d'un service via un paquet Snap avec apt sous Ubuntu doit être renseigné CONTAINER.
# L'installation d'une application ArchLinux doit être renseignée DISTRIBUTION.
# L'installation d'une application Yunohost doit être renseignée DISTRIBUTION.
service.install.type =
# [Software]
# Nom du logiciel (type STRING, obligatoire).
software.name =
# Lien du site web du logiciel (type URL, recommandé).
software.website =
# Lien web vers la licence du logiciel (type URL, obligatoire).
software.license.url =
# Nom de la licence du logiciel (type STRING, obligatoire).
software.license.name =
# Version du logiciel (type STRING, recommandé).
software.version =
# Lien web vers les sources du logiciel (type URL, recommandé).
software.source.url =
# Liste de modules optionnels installés (type VALUES, optionnel, ex. Nextcloud-Calendar,Nextcloud-Talk).
software.modules =
# [Host]
# Nom de l'hébergeur de la machine qui fait tourner le service, dans le cas d'un auto-hébergement c'est vous ! (type STRING, obligatoire).
host.name =
# Description de l'hébergeur (type STRING, optionnel).
host.description =
# Nom générique de la distribution installée sur le serveur (type STRING, obligatoire, ex. YunoHost).
host.server.distribution =
# Type de serveur (un parmi {NANO, PHYSICAL, VIRTUAL, SHARED, CLOUD}, obligatoire, ex. PHYSICAL).
# NANO : nano-ordinateur (Raspberry Pi, Olimex…)
# PHYSICAL : machine physique
# VIRTUAL : machine virtuelle
# SHARED : hébergement mutualisé
# CLOUD : infrastructure multi-serveurs
host.server.type =
# Type d'hébergement (un parmi {HOME, HOSTEDBAY, HOSTEDSERVER, OUTSOURCED}, obligatoire, ex. HOSTEDSERVER).
# HOME : hébergement à domicile
# HOSTEDBAY : serveur personnel hébergé dans une baie d'un fournisseur
# HOSTEDSERVER : serveur d'un fournisseur
# OUTSOURCED : infrastructure totalement sous-traitée
host.provider.type =
# Si vous avez du mal à remplir les champs précédents, ce tableau pourra vous aider :
# NANO PHYSICAL VIRTUAL SHARED CLOUD
# HOME pm pm vm shared cloud
# HOSTEDBAY pm pm vm shared cloud
# HOSTEDSERVER -- pm vm shared cloud
# OUTSOURCED -- -- vps shared cloud
# Légendes : pm : physical machine ; vm : virtual machine ; vps : virtual private server.
# Nom du logiciel hyperviseur (type STRING, optionnel, ex. KVM).
host.provider.hypervisor =
# Pays de l'hébergeur (type STRING, recommandé).
host.country.name =
# Code pays de l'hébergeur (type COUNTRY_CODE sur 2 caractères, obligatoire, ex. FR ou BE ou CH ou DE ou GB).
# Table ISO 3166-1 alpha-2 : https://fr.wikipedia.org/wiki/ISO_3166-1#Table_de_codage
host.country.code =
# [Subs]
# Un lien vers un fichier properties complémentaire (type URL, optionnel).
# Une clé (nomination libre) pour chacun de vos métriques spécifiques,
# par exemple pour un service etherpad : subs.metrics-etherpad = https://www.monchaton.ext/.well-known/metrics-etherpad.properties
subs.foo =

View file

@ -1,43 +0,0 @@
# File + organization + services + machines + metrics
# [File]
file.class=organization
file.generator=Cpm
file.datetime=2022-07-22T17:35:58
file.protocol=StatoolInfos-0.5
# [Organization]
organization.name=Services au public
organization.description=Les services publics de Libre-service.eu
organization.website=https://www.libre-service.eu/
organization.logo=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu-logo-public.png
organization.status.level=ACTIVE
organization.status.description=En activité
organization.owner.name=Libre-service.eu
organization.owner.website=https://www.libre-service.eu/
organization.owner.logo=https://www.libre-service.eu/.well-known/statoolinfos/libre-service.eu-logo-carre.png
organization.contact.url=https://www.libre-service.eu/contact.xhtml
organization.contact.email=contact@libre-service.eu
organization.socialnetworks.mastodon=
organization.legal.url=https://www.libre-service.eu/mentions-legales.xhtml
organization.guide.user=https://www.libre-service.eu/
organization.guide.technical=https://forge.devinsy.fr/libre-service.eu/documentation
organization.startdate=01/10/2021
organization.enddate=
organization.memberof.libreserviceeu.status.level=ACTIVE
organization.memberof.libreserviceeu.status.description=
organization.memberof.libreserviceeu.startdate=01/10/2021
organization.memberof.libreserviceeu.enddate=
organization.country.name=France
organization.country.code=FR
organization.type=INFORMAL
# [Subs]
subs.audio=https://audio.libre-service.eu/.well-known/statoolinfos/audio.libre-service.eu.properties
subs.minetest=https://minetest.libre-service.eu/.well-known/statoolinfos/minetest.libre-service.eu.properties
subs.pad=https://pad.libre-service.eu/.well-known/statoolinfos/pad.libre-service.eu.properties
subs.paste=https://paste.libre-service.eu/.well-known/statoolinfos/paste.libre-service.eu.properties
subs.qrcode=https://qrcode.libre-service.eu/.well-known/statoolinfos/qrcode.libre-service.eu.properties
subs.visio=https://visio.libre-service.eu/.well-known/statoolinfos/visio.libre-service.eu.properties
# [Metrics]

View file

@ -72,8 +72,8 @@ public final class StatoolInfosLauncher
else else
{ {
// //
Level logEnvLevel = Level.getLevel(StringUtils.defaultString(System.getenv().get("LOG4J_LEVEL"), " ")); Level logEnvLevel = Level.getLevel(StringUtils.defaultString(System.getenv().get("LOG4J_LEVEL"), ""));
Level logPropertyLevel = Level.getLevel(StringUtils.defaultString(System.getProperty("log4j2.level", " "))); Level logPropertyLevel = Level.getLevel(StringUtils.defaultString(System.getProperty("log4j2.level", "")));
Level level; Level level;
if (logPropertyLevel != null) if (logPropertyLevel != null)

View file

@ -308,20 +308,6 @@ public class HttpAccessLogAnalyzer
/** /**
* Checks if is page. * Checks if is page.
* *
* <pre>
* null => false
* "" => false
* " " => false
* "GET /foo" => true
* "GET /foo42" => true
* "GET /foo.cgi" => true
* "GET /foo.htm" => true
* "GET /foo.html" => true
* "GET /foo.php" => true
* "GET /foo.xhtml" => true
* "GET /foo.jpg" => false
* </pre>
*
* @param request * @param request
* the request * the request
* @return true, if is page * @return true, if is page

View file

@ -34,7 +34,7 @@ public class HttpAccessLogParser
// '"$request" $status $body_bytes_sent ' // '"$request" $status $body_bytes_sent '
// '"$http_referer" "$http_user_agent"'; // '"$http_referer" "$http_user_agent"';
public static final Pattern COMBINED_PATTERN = Pattern.compile( public static final Pattern COMBINED_PATTERN = Pattern.compile(
"^(?<remoteAddress>[a-fA-F0-9\\:\\.]+) - (?<remoteUser>[^\\[]+) \\[(?<time>[^\\]]+)\\] \"(?<request>(?>[^\"]|\\\")*)\" (?<status>\\d+) (?<bodyBytesSent>\\d+) \"(?<referer>(?>[^\"]|\\\")*)\" \"(?<userAgent>(?>[^\"]|\\\")*)\".*$"); "^(?<remoteAddress>[a-fA-F0-9\\:\\.]+) - (?<remoteUser>[^\\[]+) \\[(?<time>[^\\]]+)\\] \"(?<request>([^\"]|\\\")*)\" (?<status>\\d+) (?<bodyBytesSent>\\d+) \"(?<referer>([^\"]|\\\")*)\" \"(?<userAgent>([^\"]|\\\")*)\".*$");
public static final DateTimeFormatter DATETIME_FORMATTER = DateTimeFormatter.ofPattern("dd/MMM/yyyy:HH:mm:ss Z").withLocale(Locale.ENGLISH); public static final DateTimeFormatter DATETIME_FORMATTER = DateTimeFormatter.ofPattern("dd/MMM/yyyy:HH:mm:ss Z").withLocale(Locale.ENGLISH);

View file

@ -40,7 +40,7 @@ public class HttpAccessLogParserTest
* the exception * the exception
*/ */
@Test @Test
public void testParseLog01() throws Exception public void test01() throws Exception
{ {
// Nominal line. // Nominal line.
String line = "172.104.137.47 - - [30/Apr/2023:23:25:15 +0200] \"GET /default.jsp HTTP/1.1\" 302 534 \"-\" \"curl/7.54.0\""; String line = "172.104.137.47 - - [30/Apr/2023:23:25:15 +0200] \"GET /default.jsp HTTP/1.1\" 302 534 \"-\" \"curl/7.54.0\"";
@ -59,22 +59,6 @@ public class HttpAccessLogParserTest
line = "10.9.74.87 - - [03/Aug/2023:21:11:59 +0200] \"GET /plus/flink.php?dopost=save&c=cat%20/etc/passwd HTTP/1.1\" 302 5438 \"<?php \\\"system\\\"($c);die;/*ref\" \"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36\""; line = "10.9.74.87 - - [03/Aug/2023:21:11:59 +0200] \"GET /plus/flink.php?dopost=save&c=cat%20/etc/passwd HTTP/1.1\" 302 5438 \"<?php \\\"system\\\"($c);die;/*ref\" \"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36\"";
Assert.assertNotNull(HttpAccessLogParser.parseLog(line)); Assert.assertNotNull(HttpAccessLogParser.parseLog(line));
line = "41.80.118.70 - - [04/Jan/2024:00:38:58 +0100] \"GET /services/audio.xhtml?url=http://go.a.gle.email.2.%5C%5Cn1@sarahjohttonw.estbrooxxertrew.e.r@hu.fe.ng.k.ua.ngniu.bi..uk41@www.zaxxle@silxxa.wooxx.o.r.t.h@ba.tt.le9.578@jxd.1.4.7m.nb.v.3.6.9.cx.z.9"
+ "51.4@ex.p.lo.si.v.edhq.g@sixxia.wooxx.o.r.t.h@r.eces.si.v.e.x.g.z@leaxxa.lanxxon@vi.rt.u.ali.rd.j@h.att.ie.m.c.d.o.w.e.ll2.56.6.3@buxxon.rene@fullxxuestickyriddl.edyxxmi.c.t.r.a@johndf.gfjhfgjf.ghfdjfhjhjhjfdgh@sybbr%3Er.eces.si.v.e.x.g.z@lexxna.l"
+ "axxton@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5C%5C%5C%5C%5C%5C%5Cn1@saxxhjohxxonw.estbxxokbertrew.e.r@hu.fe.ng.k.ua.nxxiu.bi..uk41@www.zxxele@sixxia.woxxw.o.r.t.h@fullgluxxtickyriddl.edyxxmi.c.t.r.a@johndf.gfjhfgjf.ghfdjfhjhjhjfdgh@sybbr%3Er.eces"
+ ".si.v.e.x.g.z@lexxna.lxxgton@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5C%5C%5C%5C%5C%5C%5Cn1@sarahxxnsonw.estbrookbxxtrew.e.r@hu.fe.ng.k.ua.nxxiu.bi..uk41@www.zaxxle@silxxa.wooxx.o.r.t.h@p.a.r.a.ju.mp.e.r.sj.a.s.s.en20.14@maxxalena.tuxx@h.att.ie.m."
+ "c.d.o.w.e.ll2.56.6.3buxxon.rene@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5Cn1@saxxhjohnsonw.estbrooxxertrew.e.r@hu.fe.ng.k.ua.nxxiu.bi..uk41@www.zxxele@silvxx.wooxx.o.r.t.h@winkxxr-sanxxini.it/info/mwst01i.pdf/rk=0/rs=fzqfiq9omocv.7bggtuxxnthpge-%3F"
+ "a%5B%5D=%3Ca+href%3Dhttps%3A%2F%2Fwww.rssxxg.com%2Frequest.php%3Freq%3Dfurl%26i%3D3759037%26r%3D17559%26url%3Dhttp%253A%252F%252Fbuxxssayuk.us%3xxuy+essays%3C%2Fa%3E%3Cmeta+http-equiv%3Drefresh+content%3D0%3Burl%3Dhttps%3A%2F%2Fwww.spxxtsbook.ag%2"
+ "Fctr%2Facctmgt%2Fpl%2FopxxLink.ctr%3FctrPage%3Dhttps%3A%2F%2Fbuxxssayuk.us%2F+%2F%3E HTTP/1.0\" 200 4521 \"https://www.toxoyz.com/t/spip_cookie.php?url=http://go.a.gle.email.2.%5C%5Cn1@sarxxxohnsonw.estbrxxkbertrew.e.r@hu.fe.ng.k.ua.nxxiu.bi..uk"
+ "41@www.zaxxle@silvia.wooxx.o.r.t.h@ba.tt.le9.578@jxd.1.4.7m.nb.v.3.6.9.cx.z.951.4@ex.p.lo.si.v.edhq.g@sixxia.wooxx.o.r.t.h@r.eces.si.v.e.x.g.z@xxnna.lxxxton@vi.rt.u.ali.rd.j@h.att.ie.m.c.d.o.w.e.ll2.56.6.3@burton.rene@fullgxxestickyriddl.exxxami."
+ "c.t.r.a@johndf.gfjhfgjf.ghfdjfhjhjhjfdgh@sybbr%3Er.eces.si.v.e.x.g.z@leaxxa.laxxton@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5C%5C%5C%5C%5C%5C%5Cn1@sarxxjohnxxnw.estbroxxbertrew.e.r@hu.fe.ng.k.ua.ngxxu.bi..uk41@www.zxxele@silxxa.woxxw.o.r.t.h@fullgl"
+ "uesxxckyriddl.exxnami.c.t.r.a@johxxf.gfjhfgjf.ghfdjfhjhjhjfdgh@sybbr%3Er.eces.si.v.e.x.g.z@lexxna.laxxton@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5C%5C%5C%5C%5C%5C%5Cn1@sarxxjohnxxnw.estbxxokbertrew.e.r@hu.fe.ng.k.ua.ngniu.bi..ux41@www.zaxxle@sxxvi"
+ "a.wooxx.o.r.t.h@p.a.r.a.ju.mp.e.r.sj.a.s.s.en20.14@magxxlena.txxn@h.att.ie.m.c.d.o.w.e.ll2.56.6.3burxxn.rene@c.o.nne.c.t.tn.tu@go.a.gle.email.2.%5C%5Cn1@sarahjohxxonw.estbrookxxrtrew.e.r@hu.fe.ng.k.ua.nxxiu.bi..ux41@www.zxxele@sixxia.woxxw.o.r.t.h"
+ "@winxxer-sanxxini.it/info/mwst01i.pdf/rk=0/rs=fzqfixxomocv.7bggtuxxnthpge-%3Fa%5B%5D=%3Ca+href%3Dhttps%3A%2F%2Fwww.rsxxng.com%2Frequest.php%3Freq%3Dfurl%26i%3D3759037%26r%3D17559%26url%3Dhttp%253A%252F%252Fbuyexxayuk.us%3Ebuy+essays%3C%2Fa%3E%3Cme"
+ "ta+http-equiv%3Drefresh+content%3D0%3Burl%3Dhttps%3A%2F%2Fwww.sportxxook.ag%2Fctr%2Facctmgt%2Fpl%2Fopenxxnk.ctr%3FctrPage%3Dhttps%3A%2F%2Fbuxxssayuk.us%2F+%2F%3E\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Ch"
+ "rome/114.0.0.0 Safari/537.36\"";
Assert.assertNotNull(HttpAccessLogParser.parseLog(line));
} }
/** /**

View file

@ -38,27 +38,6 @@ import fr.devinsy.statoolinfos.util.FilesUtils;
*/ */
public class HttpAccessLogsAnalyzerTest public class HttpAccessLogsAnalyzerTest
{ {
/**
* Checks if is page test 01.
*/
@Test
public void isPageTest01()
{
Assert.assertFalse(HttpAccessLogAnalyzer.isPage(null));
Assert.assertFalse(HttpAccessLogAnalyzer.isPage(""));
Assert.assertFalse(HttpAccessLogAnalyzer.isPage(" "));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo42"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo.html"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo.HTML"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo.HtMl"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo.HtMl"));
Assert.assertTrue(HttpAccessLogAnalyzer.isPage("GET /foo/bar/foo.php"));
Assert.assertFalse(HttpAccessLogAnalyzer.isPage("GET /foo.jpg"));
}
/** /**
* @throws Exception * @throws Exception
*/ */