statool/statoolinfosweb
statool/statoolinfosweb
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Activity

Split configuration file, huge refactoring.

Browse source
This commit is contained in:
Christian P. MOMON 2020-09-17 18:54:29 +02:00
parent dbd0f2f975
commit 8f46a23342
8 changed files with 833 additions and 165 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

202
GENERAL.md Normal file
View file

@ -0,0 +1,202 @@
# StatoolInfos
StatoolInfos is…
Data are stored in property files. Each entry line uses key defined by convention.
## Primitive Types
| Type | Description | Pattern | Example |
| ------ | ------ | ------ | ------ |
| DATE | A date in ISO format | ISO_DATE | 2020-07-06 |
| DATETIME | A date and time in ISO format | ISO_DATETIME | 2020-07-06T14:23:20 |
| DAYS | List of 366 NUMERIC separated by SEPARATOR | STU | 1;2;3;4;5;…;365 |
| EMAIL | An electronic address. | ^.+@.+$ | christian@momon.org |
| MONTHS | List of 12 NUMERIC separated by SEPARATOR | STU | 1;2;3;4;5;6;7;9;10;11;12 |
| NUMERIC | A numeric value | STU | -1234.567 |
| NUMERICS | List of NUMERIC separated by SEPARATOR | STU | 12.34;56.78;9 |
| SEPARATOR | The default separator character | `;` | Foo;foo;;foo |
| STATUS | A status. | ON/MAINTENANCE/DOWN/OFF/DRAFT | ON |
| STRING | Foo string. With `<br/>` or `\n`. | ^.*$ | Foo is foo. |
| URL | A web link | ^https?://.+\..+/+$ | https://foo.foo/ |
| VALUE | String not containg SEPARATOR character | Foo |
| VALUES | List of VALUE separated by SEPARATOR | STU | foo1;foo2;foo3 |
| WEEKS | List of 52 NUMERIC separated by SEPARATOR | STU | 1;2;3;4;5;…;51;52 |
Note for list:
* empty value or null value are available;
* shorter list designs empty values.
## Sections
### Federation section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------| ------ | ------ |
| federation.name | STRING | Yes | Name of federation | CHATONS |
| federation.description | STRING | Yes | Desciption of the federation | Collectif CHATONS |
| federation.website | URL | Yes | Official website URL of the federation | https://www.chatons.org/ |
| federation.logo | URL | Wished | URL of the organization logo | https://chatons.org/logo_chatons_v2.png |
| federation.favicon | URL | Wished | URL of the organization favicon | https://chatons.org/sites/default/files/chatons_logo_tiny.png |
| federation.contact.url | URL | No | Contact webpage of the federation | https://www.chatons.org/contact |
| federation.contact.email | EMAIL | Yes | Contact email of the federation | contact@chatons.org |
| federation.legal.url | URL | No | Legal webpage of the federation | https://www.chatons.org/page/mentions-l%C3%A9gales |
| federation.documentation.url | URL | No | Documentation webpage of the federation | https://wiki.chatons.org/ |
| federation.documentation.technical.url | URL | No | Technical documentation webpage of the federation | |
| federation.documentation.tutorial.url | URL | No | Tutorial documentation webpage of the federation | |
| federation.birthdate | DATE | No | Date of birth of the federation | 09/02/2016 |
### File section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| file.class | Federation/Organization/Service/Device | Yes | Class of the content | Service |
| file.protocol | STRING | Yes | Version of the StatoolInfos schema | StatoolInfos-0.1 |
| file.datetime | DATETIME | Yes | Date and time of the file build. | 2020-07-06T14:23:20 |
| file.generator | STRING | Yes | Generator of the file. | Cpm hands |
| file.url | URL | Yes | Origin URL of the file. | Cpm hands |
### Host section (dratf)
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| host.name | STRING | Yes | Name of the host | foo |
| host.provider | STRING | Yes | Provider of the host | foo |
| host.description | STRING | Wished | Description of the host | The virtual machine called foo |
| host.type | VPS / CLOUD / LOCATEDSERVER / HOSTSERVER / HOMESERVER / RASPERRY | Wished | Type of the host | HOMESERVER |
| host.country.name | STRING | Wished | Country name of the host | France |
| host.country.code | STRING | Wished | ISO country code of the host | FR |
### Metrics section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| metrics.foo1.foo2.name | STRING | Wished | Name of metric | visitor Ip |
| metrics.foo1.foo2.description | STRING | Wished | Description of metric | Nombre d'ip ayant visitées |
| metrics.foo1.foo2.2020 | NUMERIC | No | Count of foo1.foo2 | 123 |
| metrics.foo1.foo2.2020.months | MONTHS | No | Month count of foo1.foo2 for year 2020 | 100;200;300;;;; |
| metrics.foo1.foo2.2020.weeks | WEEKS | No | Week count of foo1.foo2 for year 2020 | 100;200;300;;;; |
| metrics.foo1.foo2.2020.days | DAYS | No | Day count of foo1.foo2 for year 2020 | 100;200;300;;;; |
Examples:
```
metrics.visitors.ipv4.2020=123
metrics.visitors.ipv6.2020=123
metrics.visitors.total.2020=246
metrics.visitors.ipv4.2020.months=12;34;56;
metrics.visitors.ipv6.2020.months=12;34;56;
metrics.visitors.total.2020.months=24;68;112;
metrics.visitors.ipv4.2020.weeks=123;456;
metrics.visitors.ipv6.2020.weeks=123;456;
metrics.visitors.total.2020.weeks=246;912;
metrics.visitors.ipv4.2020.days=123;456;
metrics.visitors.ipv6.2020.days=123;456;
metrics.visitors.total.2020.days=246;912;
```
### Organization section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| organization.name | STRING | Yes | Name of the organization | Chapril |
| organization.description | STRING | Wished | Description of the organization | Chapril est le chaton de l'April |
| organization.website | URL | Y | Official website of the organization | httsp://www.chapril.org/ |
| organization.logo | URL | Wished | Logo URL of the organization | https://date.chapril.org/Chapril-banner/v1/chapril-logo-small.png |
| organization.favicon | URL | Wished | Favicon URL of the organization | https://date.chapril.org/favicon.png |
| organization.owner.name | STRING | No | Name of the organization owner | April |
| organization.owner.website | URL | No | Official website of the organization owner | https://www.april.org/ |
| organization.owner.logo | URL | No | Logo URL of the organization owner | https://www.april.org/sites/default/themes/zen_april/logo.png |
| organization.owner.favicon | URL | No | Favicon URL of the organization owner | https://www.april.org/sites/default/themes/zen_april/favicon.ico |
| organization.contact.url | URL | Wished | Contact webpage of the organization | https://www.chapril.org/contact.html |
| organization.contact.email | EMAIL | Contact email of the organization | contact@chapril.org |
| organization.legal.url | URL | Wished | Legal webpage of the organization | https://www.chapril.org/cgu.html |
| organization.documentation.technical.url | URL | Wished | Technical documentation webpage of the organization | https://admin.chapril.org/ |
| organization.birthdate | DATE | Wished | Birth date of the organization | 08/11/2018 |
### Service section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| service.name | STRING | Yes | Service name. | MumbleChaprilOrg |
| service.description | STRING | Yes | Service description. | Service libre de conférence audio |
| service.organization.name | STRING | Yes | Organization name. | Chapril |
| service.website.url | URL | Wished | Website of the service | https://mumble.chapril.org/ |
| service.legal.url | URL | Wished | URL | Legal webpage of the organization | https://www.chapril.org/cgu.html |
| service.documentation.technical.url | URL | No | Technical documentation webpage of the service | https://admin.chapril.org/doku.php?id=admin:services:mumble.chapril.org |
| service.documentation.tutorial.url | URL | No | Tutorial documentation webpage of the service | https://www.chapril.org/Mumble.html |
| service.contact.url | URL | Wished | Contact webpage of the service | contact@chapril.org |
| service.contact.email | EMAIL | Wished | Contact email of the service | mumble-support@chapril.org |
| service.birthdate | DATETIME | Wished | Birth date of the service | 20/03/2020 |
| service.deathdate | DATETIME | No | Death date of the service | |
| service.status | STATUS | Yes | Service status. | ON |
| service.registration | None;Free;Member;Client | Yes | Registration requirement | Free; Member|
### Subs section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| subs.foo | URL | No | URL of a sub node. | Firefox Send |
Example:
```
subs.datechaprilorg=https://date.chapril.org/.well-known/datechaprilorg.properties
subs.pastechaprilorg=https://paste.chapril.org/.well-known/pastechaprilorg.properties
subs.dropchaprilorg=https://drop.chapril.org/.well-known/dropchaprilorg.properties
```
### Software section
| Field | Type | Mandatory | Description | Example |
| ------ | ------ | ------ | ------ | ------ |
| software.name | STRING | Yes | Name of the software | Firefox Send |
| software.website | URL | Yes | Offical website of the software | https://send.firefox.com/ |
| software.license.url | URL | Yes | Webpage of the software license | https://forge.april.org/Chapril/drop.chapril.org-firefoxsend/src/branch/chapril-v3.0.21/LICENSE |
| software.license.name | STRING | Yes | Name of the license | Mozilla Public License Version 2.0 |
| software.version | STRING | Yes | Version of the software | Chapril-3.0.21 |
| software.source.url | URL | Yes | URL of the software source | https://forge.april.org/Chapril/drop.chapril.org-firefoxsend/ |
## Classes
### Federation class
Federation = file + federation + subs + metrics
### Organization class
Organization = file + organization + subs + metrics
### Service class
Service = file + service + host + software + metrics + extras
### Device class
Device = file + device + system
# Draft
| Field | Type | Mandatory | Description |
| ------ | ------ | ------ | ------ |
| services.* | URL | Y | URL of the Statool file of the service |
| Field | Type | Mandatory | Description |
| ------ | ------ | ------ | ------ |
| services.urls.* | URL | Y | URL of services |

372
src/fr/devinsy/statoolinfos/core/Configuration.java Normal file
View file

@ -0,0 +1,372 @@
/*
* Copyright (C) 2020 Christian Pierre MOMON <christian@momon.org>
*
* This file is part of StatoolInfos, simple service statistics tool.
*
* StatoolInfos is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* StatoolInfos is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with StatoolInfos. If not, see <http://www.gnu.org/licenses/>.
*/
package fr.devinsy.statoolinfos.core;
import java.io.File;
import org.apache.commons.lang3.StringUtils;
import fr.devinsy.statoolinfos.properties.PathProperties;
import fr.devinsy.statoolinfos.properties.PathPropertyList;
/**
* The Class PathProperty.
*/
public class Configuration extends PathPropertyList
{
private static final long serialVersionUID = 2085950843956966741L;
/**
* Instantiates a new federation.
*/
public Configuration()
{
super();
}
/**
* Instantiates a new federation.
*
* @param properties
* the properties
*/
public Configuration(final PathProperties properties)
{
super(properties);
}
/**
* Gets the builds the directory.
*
* @return the builds the directory
*/
public File getBuildDirectory()
{
File result;
String path = getBuildDirectoryPath();
if (StringUtils.isBlank(path))
{
result = null;
}
else
{
result = new File(path);
}
//
return result;
}
/**
* Gets the builds the directory path.
*
* @return the builds the directory path
*/
public String getBuildDirectoryPath()
{
String result;
result = get("conf.build.directory");
//
return result;
}
/**
* Gets the builds the directory valid.
*
* @return the builds the directory valid
*/
public File getBuildDirectoryValid()
{
File result;
result = getBuildDirectory();
if (result == null)
{
throw new IllegalArgumentException("Undefined build directory.");
}
else if (!result.exists())
{
throw new IllegalArgumentException("Build directory does not exist: " + result.getAbsolutePath());
}
//
return result;
}
/**
* Gets the builds the input.
*
* @return the builds the input
*/
public File getBuildInput()
{
File result;
String path = get("conf.build.input");
if (StringUtils.isBlank(path))
{
result = null;
}
else
{
result = new File(path);
}
//
return result;
}
/**
* Gets the class name.
*
* @return the class name
*/
public String getClassName()
{
String result;
result = get("conf.class");
//
return result;
}
/**
* Gets the cache.
*
* @return the cache
* @throws StatoolInfosException
*/
public CrawlCache getCrawlCache() throws StatoolInfosException
{
CrawlCache result;
String path = getCrawlCachePath();
result = new CrawlCache(new File(path));
//
return result;
}
/**
* Gets the crawl cache path.
*
* @return the crawl cache path
*/
public String getCrawlCachePath()
{
String result;
result = get("conf.crawl.cache");
//
return result;
}
/**
* Gets the crawl input.
*
* @return the crawl input
*/
public File getCrawlInput()
{
File result;
String path = getCrawlInputPath();
if (StringUtils.isBlank(path))
{
result = null;
}
else
{
result = new File(path);
}
//
return result;
}
/**
* Gets the crawl input path.
*
* @return the crawl input path
*/
public String getCrawlInputPath()
{
String result;
result = get("conf.crawl.input");
//
return result;
}
/**
* Gets the htmlize directory.
*
* @return the htmlize directory
*/
public File getHtmlizeDirectory()
{
File result;
String path = getHtmlizeDirectoryPath();
if (StringUtils.isBlank(path))
{
result = null;
}
else
{
result = new File(path);
}
//
return result;
}
/**
* Gets the htmlize directory path.
*
* @return the htmlize directory path
*/
public String getHtmlizeDirectoryPath()
{
String result;
result = get("conf.htmlize.directory");
//
return result;
}
/**
* Gets the htmlize input.
*
* @return the htmlize input
*/
public File getHtmlizeInput()
{
File result;
String path = getHtmlizeInputPath();
if (StringUtils.isBlank(path))
{
result = null;
}
else
{
result = new File(path);
}
//
return result;
}
/**
* Gets the htmlize input path.
*
* @return the htmlize input path
*/
public String getHtmlizeInputPath()
{
String result;
result = get("conf.htmlize.input");
//
return result;
}
/**
* Checks for valid cache.
*
* @return true, if successful
*/
public boolean hasValidCache()
{
boolean result;
String path = get("conf.crawl.cache");
if (StringUtils.isBlank(path))
{
result = false;
}
else if (!new File(path).exists())
{
result = false;
}
else
{
result = true;
}
//
return result;
}
/**
* Checks if is federation.
*
* @return true, if is federation
*/
public boolean isFederation()
{
boolean result;
result = StringUtils.equals(getClassName(), "federation");
//
return result;
}
/**
* Checks if is organization.
*
* @return true, if is organization
*/
public boolean isOrganization()
{
boolean result;
result = StringUtils.equals(getClassName(), "organization");
//
return result;
}
/**
* Checks if is service.
*
* @return true, if is service
*/
public boolean isService()
{
boolean result;
result = StringUtils.equals(getClassName(), "service");
//
return result;
}
}

2
src/fr/devinsy/statoolinfos/core/CrawlCache.java
View file

@ -57,7 +57,7 @@ public class CrawlCache
} }
else if (StringUtils.isBlank(directory.getName())) else if (StringUtils.isBlank(directory.getName()))
{ {
throw new IllegalArgumentException("Blank directory."); throw new IllegalArgumentException("Undefined directory.");
} }
else if (!directory.exists()) else if (!directory.exists())
{ {

83
src/fr/devinsy/statoolinfos/core/Factory.java
View file

@ -28,6 +28,7 @@ import org.slf4j.LoggerFactory;
import fr.devinsy.statoolinfos.properties.PathProperties; import fr.devinsy.statoolinfos.properties.PathProperties;
import fr.devinsy.statoolinfos.properties.PathProperty; import fr.devinsy.statoolinfos.properties.PathProperty;
import fr.devinsy.statoolinfos.properties.PathPropertyUtils;
/** /**
* The Class PathProperty. * The Class PathProperty.
@ -44,58 +45,56 @@ public class Factory
} }
/** /**
* Load federation. * Load configuration.
* *
* @return the federation * @param configurationFile
* the configuration file
* @return the configuration
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred.
*/ */
public static Federation loadFederation(final PathProperties properties) throws StatoolInfosException, IOException public static Configuration loadConfiguration(final File configurationFile) throws StatoolInfosException, IOException
{ {
Federation result; Configuration result;
// PathProperties properties = PathPropertyUtils.load(configurationFile);
String crawlCachePath = properties.get("conf.crawl.cache"); result = new Configuration(properties);
logger.info("Cache setting: {}", properties.get("conf.crawl.cache"));
CrawlCache cache = new CrawlCache(new File(crawlCachePath));
result = new Federation(properties);
PathProperties subs = result.getByPrefix("subs");
for (PathProperty property : subs)
{
if (StringUtils.startsWith(property.getValue(), "http"))
{
URL url = new URL(property.getValue());
result.setLocalFile(cache.restoreFile(url));
PathProperties subProperties = cache.restoreProperties(url);
Organization organization = loadOrganization(subProperties, cache);
result.getOrganizations().add(organization);
}
}
// //
return result; return result;
} }
/** /**
* Load organization. * Load federation.
* *
* @return the organization * @param federationFile
* the federation file
* @return the federation
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred.
*/ */
public static Organization loadOrganization(final PathProperties properties) throws StatoolInfosException, IOException public static Federation loadFederation(final File federationFile, final CrawlCache cache) throws StatoolInfosException, IOException
{ {
Organization result; Federation result;
// PathProperties properties = PathPropertyUtils.load(federationFile);
String crawlCachePath = properties.get("conf.crawl.cache"); result = new Federation(properties);
logger.info("Cache setting: {}", properties.get("conf.crawl.cache")); result.setLocalFile(federationFile);
CrawlCache cache = new CrawlCache(new File(crawlCachePath));
result = loadOrganization(properties, cache); PathProperties subs = result.getByPrefix("subs");
for (PathProperty property : subs)
{
if (StringUtils.startsWith(property.getValue(), "http"))
{
File subFile = cache.restoreFile(new URL(property.getValue()));
Organization organization = loadOrganization(subFile, cache);
result.getOrganizations().add(organization);
}
}
// //
return result; return result;
@ -111,20 +110,21 @@ public class Factory
* @return the organization * @return the organization
* @throws IOException * @throws IOException
*/ */
public static Organization loadOrganization(final PathProperties properties, final CrawlCache cache) throws IOException public static Organization loadOrganization(final File organizationFile, final CrawlCache cache) throws IOException
{ {
Organization result; Organization result;
PathProperties properties = PathPropertyUtils.load(organizationFile);
result = new Organization(properties); result = new Organization(properties);
result.setLocalFile(organizationFile);
PathProperties section = result.getByPrefix("subs"); PathProperties subs = result.getByPrefix("subs");
for (PathProperty property : section) for (PathProperty property : subs)
{ {
if (StringUtils.startsWith(property.getValue(), "http")) if (StringUtils.startsWith(property.getValue(), "http"))
{ {
PathProperties subProperties = cache.restoreProperties(new URL(property.getValue())); File subFile = cache.restoreFile(new URL(property.getValue()));
Service service = loadService(subFile);
Service service = loadService(subProperties);
result.getServices().add(service); result.getServices().add(service);
} }
} }
@ -137,12 +137,15 @@ public class Factory
* Load service. * Load service.
* *
* @return the service * @return the service
* @throws IOException
*/ */
public static Service loadService(final PathProperties properties) public static Service loadService(final File serviceFile) throws IOException
{ {
Service result; Service result;
PathProperties properties = PathPropertyUtils.load(serviceFile);
result = new Service(properties); result = new Service(properties);
result.setLocalFile(serviceFile);
// //
return result; return result;

13
src/fr/devinsy/statoolinfos/core/Organization.java
View file

@ -18,6 +18,8 @@
*/ */
package fr.devinsy.statoolinfos.core; package fr.devinsy.statoolinfos.core;
import java.io.File;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
import fr.devinsy.statoolinfos.properties.PathProperties; import fr.devinsy.statoolinfos.properties.PathProperties;
@ -30,6 +32,7 @@ public class Organization extends PathPropertyList
{ {
private static final long serialVersionUID = -2709210934548224213L; private static final long serialVersionUID = -2709210934548224213L;
private Services services; private Services services;
private File localFile;
/** /**
* Instantiates a new organization. * Instantiates a new organization.
@ -70,6 +73,11 @@ public class Organization extends PathPropertyList
return result; return result;
} }
public File getLocalFile()
{
return this.localFile;
}
/** /**
* Gets the name. * Gets the name.
* *
@ -119,4 +127,9 @@ public class Organization extends PathPropertyList
// //
return result; return result;
} }
public void setLocalFile(final File localFile)
{
this.localFile = localFile;
}
} }

13
src/fr/devinsy/statoolinfos/core/Service.java
View file

@ -18,6 +18,8 @@
*/ */
package fr.devinsy.statoolinfos.core; package fr.devinsy.statoolinfos.core;
import java.io.File;
import fr.devinsy.statoolinfos.properties.PathProperties; import fr.devinsy.statoolinfos.properties.PathProperties;
import fr.devinsy.statoolinfos.properties.PathPropertyList; import fr.devinsy.statoolinfos.properties.PathPropertyList;
@ -27,6 +29,7 @@ import fr.devinsy.statoolinfos.properties.PathPropertyList;
public class Service extends PathPropertyList public class Service extends PathPropertyList
{ {
private static final long serialVersionUID = 3629841771102288863L; private static final long serialVersionUID = 3629841771102288863L;
private File localFile;
/** /**
* Instantiates a new service. * Instantiates a new service.
@ -62,6 +65,11 @@ public class Service extends PathPropertyList
return result; return result;
} }
public File getLocalFile()
{
return this.localFile;
}
/** /**
* Gets the name. * Gets the name.
* *
@ -116,4 +124,9 @@ public class Service extends PathPropertyList
// //
return result; return result;
} }
public void setLocalFile(final File localFile)
{
this.localFile = localFile;
}
} }

242
src/fr/devinsy/statoolinfos/core/StatoolInfos.java
View file

@ -43,112 +43,151 @@ public class StatoolInfos
/** /**
* Builds the. * Builds the.
* *
* @param input * @param configurationFile
* the input * the input
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception * the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred. * Signals that an I/O exception has occurred.
*/ */
public static void build(final File input) throws StatoolInfosException, IOException public static void build(final File configurationFile) throws StatoolInfosException, IOException
{ {
logger.info("Build {}", input.getAbsolutePath()); logger.info("Build {}", configurationFile.getAbsolutePath());
PathProperties inputProperties = PathPropertyUtils.load(input);
String buildDirectoryName = inputProperties.get("conf.build.directory"); Configuration configuration = Factory.loadConfiguration(configurationFile);
if (StringUtils.isBlank(buildDirectoryName)) logger.info("Build input setting: {}", configuration.getBuildInput());
logger.info("Build directory setting: {}", configuration.getBuildDirectoryPath());
File inputFile = configuration.getBuildInput();
File buildDirectory = configuration.getBuildDirectory();
if (inputFile == null)
{ {
throw new StatoolInfosException("Build directory target is undefined."); throw new StatoolInfosException("Input is undefined.");
}
else if (!inputFile.exists())
{
throw new StatoolInfosException("Input does not exist.");
}
else if (!inputFile.isFile())
{
throw new StatoolInfosException("Input is not a file.");
}
else if (buildDirectory == null)
{
throw new StatoolInfosException("Build directory is undefined.");
}
else if (!buildDirectory.exists())
{
throw new StatoolInfosException("Build directory does not exist.");
}
else if (!buildDirectory.isDirectory())
{
throw new StatoolInfosException("Build directory is not a directory.");
} }
else else
{ {
File targetDirectory = new File(buildDirectoryName); // Build file section.
PathProperties target = new PathPropertyList();
target.put("file.class", configuration.get("conf.class"));
target.put("file.generator", "StatoolInfos");
target.put("file.datetime", LocalDateTime.now().toString());
target.put("file.protocol", configuration.get("conf.protocol"));
if (targetDirectory.exists()) // Load input properties.
{ PathProperties input = PathPropertyUtils.load(inputFile);
// Load configuration file.
PathProperties targetProperties = new PathPropertyList(); // Add input properties with file section ones.
target.add(input);
// Add generator paths. // Save the build properties.
PathProperties fileSection = new PathPropertyList(); File targetFile = new File(buildDirectory, configurationFile.getName());
fileSection.put("file.class", inputProperties.get("conf.class")); PathPropertyUtils.save(targetFile, target);
fileSection.put("file.generator", "StatoolInfos");
fileSection.put("file.datetime", LocalDateTime.now().toString());
fileSection.put("file.protocol", inputProperties.get("conf.protocol"));
targetProperties.add(fileSection);
//
targetProperties.add(inputProperties);
// Clear configuration paths.
targetProperties.removeSection("conf");
// Save target file.
File targetFile = new File(targetDirectory, input.getName());
PathPropertyUtils.save(targetFile, targetProperties);
}
else
{
throw new StatoolInfosException("Build directory target does not exist.");
}
} }
} }
/** /**
* Clear. * Clear.
* *
* @param input * @param configurationFile
* the input * the input
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception * the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred. * Signals that an I/O exception has occurred.
*/ */
public static void clear(final File input) throws StatoolInfosException, IOException public static void clear(final File configurationFile) throws StatoolInfosException, IOException
{ {
PathProperties inputProperties = PathPropertyUtils.load(input); Configuration configuration = Factory.loadConfiguration(configurationFile);
{ {
String crawlCacheName = inputProperties.get("conf.crawl.cache"); logger.info("Build directory setting: {}", configuration.getBuildDirectoryPath());
if (StringUtils.isBlank(crawlCacheName))
String path = configuration.getBuildDirectoryPath();
if (StringUtils.isBlank(path))
{ {
throw new StatoolInfosException("Crawl cache directory is undefined."); logger.warn("Undefined build directory.");
}
else if (!new File(path).exists())
{
logger.warn("Build directory does not exist: {}.", path);
} }
else else
{ {
File crawlCacheDirectory = new File(crawlCacheName); File buildDirectory = configuration.getBuildDirectory();
if (crawlCacheDirectory.exists()) for (File file : buildDirectory.listFiles())
{ {
CrawlCache cache = new CrawlCache(crawlCacheDirectory); if ((file.isFile()) && (StringUtils.endsWithAny(file.getName(), ".properties")))
{
logger.info("Deleting " + file.getName());
file.delete();
}
}
}
}
{
logger.info("Cache setting: {}", configuration.getCrawlCachePath());
String path = configuration.getCrawlCachePath();
if (StringUtils.isBlank(path))
{
logger.warn("Undefined crawl cache.");
}
else if (!new File(path).exists())
{
logger.warn("Crawl cache does not exist: {}.", path);
}
else
{
CrawlCache cache = configuration.getCrawlCache();
cache.clear(); cache.clear();
} }
else
{
throw new StatoolInfosException("Crawl cache directory does not exist.");
}
}
} }
{ {
String buildDirectoryName = inputProperties.get("conf.build.directory"); logger.info("Htmlize directory setting: {}", configuration.getHtmlizeDirectoryPath());
if (StringUtils.isBlank(buildDirectoryName))
{
throw new StatoolInfosException("Build directory is undefined.");
}
else
{
File buildDirectory = new File(buildDirectoryName);
if (buildDirectory.exists()) String htmlDirectoryPath = configuration.getHtmlizeDirectoryPath();
if (StringUtils.isBlank(htmlDirectoryPath))
{ {
CrawlCache cache = new CrawlCache(buildDirectory); logger.warn("Undefined htmlize directory.");
cache.clear(); }
else if (!new File(htmlDirectoryPath).exists())
{
logger.warn("Htmlize directory does not exist: {}.", htmlDirectoryPath);
} }
else else
{ {
throw new StatoolInfosException("Crawl cache directory does not exist."); File htmlizeDirectory = configuration.getHtmlizeDirectory();
for (File file : htmlizeDirectory.listFiles())
{
if ((file.isFile()) && (StringUtils.endsWithAny(file.getName(), ".properties", ".js", ".html", ".ico", ".css", ".jpg", ".xhtml")))
{
logger.info("Deleting " + file.getName());
file.delete();
}
} }
} }
} }
@ -157,29 +196,32 @@ public class StatoolInfos
/** /**
* Crawl. * Crawl.
* *
* @param input * @param configurationFile
* the input * the input
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception * the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred. * Signals that an I/O exception has occurred.
*/ */
public static void crawl(final File input) throws StatoolInfosException, IOException public static void crawl(final File configurationFile) throws StatoolInfosException, IOException
{ {
PathProperties configuration = PathPropertyUtils.load(input); Configuration configuration = Factory.loadConfiguration(configurationFile);
String crawlCachePath = configuration.get("conf.crawl.cache"); logger.info("Crawl input setting: {}", configuration.getCrawlInputPath());
logger.info("Cache setting: {}", configuration.get("conf.crawl.cache")); logger.info("Crawl cache setting: {}", configuration.getCrawlCachePath());
CrawlCache cache = new CrawlCache(new File(crawlCachePath));
cache.storeQuietly(configuration.getURL("federation.logo")); CrawlCache cache = configuration.getCrawlCache();
cache.storeQuietly(configuration.getURL("federation.logo.url"));
cache.storeQuietly(configuration.getURL("organization.logo"));
cache.storeQuietly(configuration.getURL("organization.logo.url"));
cache.storeQuietly(configuration.getURL("service.logo"));
cache.storeQuietly(configuration.getURL("service.logo.url"));
PathProperties subs = configuration.getByPrefix("subs"); PathProperties input = PathPropertyUtils.load(configuration.getCrawlInput());
cache.storeQuietly(input.getURL("federation.logo"));
cache.storeQuietly(input.getURL("federation.logo.url"));
cache.storeQuietly(input.getURL("organization.logo"));
cache.storeQuietly(input.getURL("organization.logo.url"));
cache.storeQuietly(input.getURL("service.logo"));
cache.storeQuietly(input.getURL("service.logo.url"));
PathProperties subs = input.getByPrefix("subs");
for (PathProperty property : subs) for (PathProperty property : subs)
{ {
URL url = new URL(property.getValue()); URL url = new URL(property.getValue());
@ -232,35 +274,61 @@ public class StatoolInfos
/** /**
* Htmlize. * Htmlize.
* *
* @param input * @param configurationFile
* the input * the input
* @throws StatoolInfosException * @throws StatoolInfosException
* the statool infos exception * the statool infos exception
* @throws IOException * @throws IOException
* Signals that an I/O exception has occurred. * Signals that an I/O exception has occurred.
*/ */
public static void htmlize(final File input) throws StatoolInfosException, IOException public static void htmlize(final File configurationFile) throws StatoolInfosException, IOException
{ {
PathProperties properties = PathPropertyUtils.load(input); Configuration configuration = Factory.loadConfiguration(configurationFile);
String crawlCachePath = properties.get("conf.crawl.cache"); logger.info("Cache setting: {}", configuration.getCrawlCachePath());
logger.info("Cache setting: {}", properties.get("conf.crawl.cache")); logger.info("Htmlize input setting: {}", configuration.getHtmlizeInputPath());
CrawlCache cache = new CrawlCache(new File(crawlCachePath)); logger.info("Htmlize directory setting: {}", configuration.getHtmlizeDirectoryPath());
String className = properties.get("conf.class"); File htmlizeInput = configuration.getHtmlizeInput();
if (StringUtils.equals(className, "federation")) File htmlizeDirectory = configuration.getHtmlizeDirectory();
if (htmlizeInput == null)
{ {
Federation federation = Factory.loadFederation(properties); throw new IllegalArgumentException("Htmlize input undefined.");
Htmlizer.htmlize(federation, cache);
} }
else if (StringUtils.equals(className, "organization")) else if (!htmlizeInput.exists())
{ {
Organization organization = Factory.loadOrganization(properties); throw new IllegalArgumentException("Htmlize input is missing.");
Htmlizer.htmlize(organization, cache); }
else if (htmlizeInput.isDirectory())
{
throw new IllegalArgumentException("Htmlize input is a directory.");
}
else if (htmlizeDirectory == null)
{
throw new IllegalArgumentException("Htmlize directory undefined.");
}
else if (!htmlizeDirectory.exists())
{
throw new IllegalArgumentException("Htmlize directory is missing.");
}
else if (!htmlizeDirectory.isDirectory())
{
throw new IllegalArgumentException("Htmlize directory is not a directory.");
} }
else else
{ {
// TODO if (configuration.isFederation())
{
Htmlizer.htmlizeFederation(configuration);
}
else if (configuration.isOrganization())
{
Htmlizer.htmlizeOrganisation(configuration);
}
else
{
logger.warn("No htmlize for this input: {}.", configuration.getClassName());
}
} }
} }
} }

37
src/fr/devinsy/statoolinfos/htmlize/Htmlizer.java
View file

@ -26,7 +26,9 @@ import org.apache.commons.io.FileUtils;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import fr.devinsy.statoolinfos.core.Configuration;
import fr.devinsy.statoolinfos.core.CrawlCache; import fr.devinsy.statoolinfos.core.CrawlCache;
import fr.devinsy.statoolinfos.core.Factory;
import fr.devinsy.statoolinfos.core.Federation; import fr.devinsy.statoolinfos.core.Federation;
import fr.devinsy.statoolinfos.core.Organization; import fr.devinsy.statoolinfos.core.Organization;
import fr.devinsy.statoolinfos.core.Service; import fr.devinsy.statoolinfos.core.Service;
@ -82,37 +84,30 @@ public class Htmlizer
* @throws IOException * @throws IOException
* @throws StatoolInfosException * @throws StatoolInfosException
*/ */
public static void htmlize(final Federation federation, final CrawlCache cache) throws IOException, StatoolInfosException public static void htmlizeFederation(final Configuration configuration) throws IOException, StatoolInfosException
{ {
File targetDirectory = new File(federation.get("conf.htmlize.directory")); CrawlCache cache = configuration.getCrawlCache();
logger.info("Htmlize target directory: {}", targetDirectory.getAbsoluteFile()); File htmlizeInput = configuration.getHtmlizeInput();
File htmlizeDirectory = configuration.getHtmlizeDirectory();
if (!targetDirectory.exists()) Federation federation = Factory.loadFederation(htmlizeInput, cache);
{
throw new IllegalArgumentException("Htmlize target directory is missing."); copyStuff(htmlizeDirectory);
} cache.restoreLogoTo(federation.getLogoURL(), new File(htmlizeDirectory, federation.getTechnicalName() + "-logo.jpg"));
else if (!targetDirectory.isDirectory())
{
throw new IllegalArgumentException("Htmlize target directory is not a directory.");
}
else
{
copyStuff(targetDirectory);
cache.restoreLogoTo(federation.getLogoURL(), new File(targetDirectory, federation.getTechnicalName() + "-logo.jpg"));
// //
String page = FederationPage.build(federation); String page = FederationPage.build(federation);
FileUtils.write(new File(targetDirectory, "index.xhtml"), page, StandardCharsets.UTF_8); FileUtils.write(new File(htmlizeDirectory, "index.xhtml"), page, StandardCharsets.UTF_8);
for (Organization organization : federation.getOrganizations()) for (Organization organization : federation.getOrganizations())
{ {
page = OrganizationPage.build(organization); page = OrganizationPage.build(organization);
FileUtils.write(new File(targetDirectory, organization.getTechnicalName() + ".xhtml"), page, StandardCharsets.UTF_8); FileUtils.write(new File(htmlizeDirectory, organization.getTechnicalName() + ".xhtml"), page, StandardCharsets.UTF_8);
for (Service service : organization.getServices()) for (Service service : organization.getServices())
{ {
page = ServicePage.build(service); page = ServicePage.build(service);
FileUtils.write(new File(targetDirectory, organization.getTechnicalName() + "-" + service.getTechnicalName() + ".xhtml"), page, StandardCharsets.UTF_8); FileUtils.write(new File(htmlizeDirectory, organization.getTechnicalName() + "-" + service.getTechnicalName() + ".xhtml"), page, StandardCharsets.UTF_8);
} }
} }
@ -126,7 +121,6 @@ public class Htmlizer
// Download service stuff (favicon, logo…). // Download service stuff (favicon, logo…).
// Build service page. // Build service page.
} }
}
/** /**
* Htmlize organization. * Htmlize organization.
@ -138,8 +132,11 @@ public class Htmlizer
* @throws IOException * @throws IOException
* @throws StatoolInfosException * @throws StatoolInfosException
*/ */
public static void htmlize(final Organization organization, final CrawlCache cache) throws IOException, StatoolInfosException public static void htmlizeOrganisation(final Configuration configuration) throws IOException, StatoolInfosException
{ {
CrawlCache cache = configuration.getCrawlCache();
Organization organization = Factory.loadOrganization(configuration.getBuildInput(), cache);
File targetDirectory = new File(organization.get("conf.htmlize.directory")); File targetDirectory = new File(organization.get("conf.htmlize.directory"));
logger.info("Htmlize target directory: {}", targetDirectory.getAbsoluteFile()); logger.info("Htmlize target directory: {}", targetDirectory.getAbsoluteFile());