statool/logar
statool/logar
1
0
Fork 0
Code Issues Pull requests Releases Activity
logar/README.md

3.8 KiB
Raw Blame History

Logar

Logar is a simple log archive tool to manage HTTP log files.

LICENSE

Logar is released under the GNU AGPLv3+ license. Enjoy!

Authors:

Author: Christian Pierre MOMON christian@momon.org

Based on https://publicdomainvectors.org/en/free-clipart/Wood-log-clip-art/45623.html

License: Public Domain.

Other stuff

License of other stuff:

  • other: without any specific information, default license of images is CC-BY-SA 3+.

DOCUMENTATION

cpm@ocmstar (13:19:32) ~/Projets/Logar/Dev/logar/dist/logar-0.1.1-SNAPSHOT_20210425131921 40 > ./logar.sh -h
2021-04-25T13:19:34.176928 Logar call: -h
Logar CLI version 0.1.1-20210425131921
Usage:
    logar [ -h | -help | --help ]
    logar [ -v | -version | --version ]
    logar anonymize      fileordirectory [mapfile]    anonymize ip and user
    logar archive [-dry] source target                archive previous month from /var/log/nginx/ tree
    logar check          fileordirectory              check line format in log file
    logar checksort      fileordirectory              check sort in log file
    logar sort           fileordirectory              sort log files by datetime
    logar testconcate    fileordirectory              test line concate in log file

Done.

Command anonymize

This command is useful to anonymize log. It can anonymized accces log file and Nginx error log file from default Nginx combined format. The result is a new file whose name contains "-anon".

Optionally a map file can be set as parameter. Then the file is load and used to run. At the end, the file will be erased by a new one.

The anonymization does not change the line length:

  • ip address: replaced with a valid random generate ip with the same length
  • remote user: replaced with a random [a..z] string with the same length

Examples:

  • logar anonymize test.log
  • logar anonymize test.log.gz table

Command archive

This command archive previous month log lines from a /var/log/nginx/ directory to single file by domain.

The optionally -dry parameter is useful to test the archive process without produced any target file.

Example:

  • logar /var/log/nginx/ target/
  • logar -dry /var/log/nginx/ target/

Command check

This command check bad format line in log files (access or error). It is useful to detect malformed log file or test the logar parsing feature.

Example:

  • logar check /var/log/nginx/

Command checksort

This command check if log lines are sorted by datetime. It is useful to detect unsorted log file. Reminder: sorted file are mandatory to Webalizer.

Example:

  • logar checksort /var/log/nginx/www.devinsy.fr/
  • logar checksort /var/log/nginx/www.devinsy.fr/www.devinsy.fr-access-2021-01.log.gz

Command identify

This command identifies the files log format.

Example :

> logar identify .
Access/*            www.libre-service.eu-access.log
Access/*            www.libre-service.eu-access.log.1
Error/Apache        www.libre-service.eu-error.log
Error/Apache        www.libre-service.eu-error.log.1

Command sort

This command sorts log files. A backup file is produced. A check is made between original file and produced file.

Caution: as the log file is fully load in memory, huge file are potentially difficult.

Example:

  • logar checksort /var/log/nginx/date.chapril.org/
  • logar checksort /var/log/nginx/date.chapril.org/date.chapril.org-access-2021-01.log.gz

Command testconcate

This command tests a advanced feature of rebuild log line. Perhaps useful feature in future.

Requirements

  • Java 17
  • Eclipse 4.16 (202006).

INSTALL

TODO

Unit test environment

For unit tests, install the TestNG: