SEPA - SPARQL Event Processing Architecture

Table of Contents

• Introduction
• Demo
• Quick start
• Configuration
• Usage
• Build with Maven
• Contributing
• History
• Credits

Introduction

SEPA (SPARQL Event Processing Architecture) is a publish-subscribe architecture designed to support information level interoperability. The architecture is built on top of generic SPARQL endpoints (conformant with SPARQL 1.1 protocol) where publishers and subscribers use standard SPARQL 1.1 Updates and Queries. Notifications about events (i.e., changes in the RDF knowledge base) are expressed in terms of added and removed SPARQL binding results since the previous notification. To know more about SEPA architecture and vision please refer to this paper. SEPA proposal has been formalized in the following unofficial drafts:

• SPARQL Event Processing Architecture (SEPA) contribute here
• SPARQL 1.1 Secure Event Protocol contribute here
• SPARQL 1.1 Subscribe Language contribute here
• JSON SPARQL Application Profile (JSAP) contribute here

Demo

Quick start

• Download the SEPA Engine and run it: java -jar engine-x.y.z.jar

• Download Blazegraph (or use any other SPARQL 1.1 Protocol compliant service) and run it as shown here

• Use the SEPA Playground to check basic functionalities of the engine.

• To build SEPA from source, see Build with Maven.

For Hackers

Configuration

The SEPA engine can be used with different SPARQL endpoints which must support SPARQL 1.1 protocol. The endpoint can be configured using a JSON file endpoint.jpar. Furthermore, the engine has various parameters that can be used to configure the standard behavior; they can be set using another JSON file called engine.jpar.
In the repository, you will find some versions of endpoint-{something}.jpar file. According to your underlying SPARQL endpoint, you have to rename the correct file to endpoint.jpar. The default version of endpoint.jpar configures the engine to use a local running instance of Blazegraph as SPARQL 1.1 Protocol Service.

{
"host":"localhost",
"sparql11protocol":{
  "protocol":"http",
  "port":9999,
  "query":{
    "path":"/blazegraph/namespace/kb/sparql",
    "method":"POST",
    "format":"JSON"},
  "update":{
    "path":"/blazegraph/namespace/kb/sparql",
    "method":"POST",
    "format":"JSON"}}}

The default version of engine.jpar configures the engine to listen for incoming SPARQL 1.1 SE Protocol requests at the following URLs:

1. Query: http://localhost:8000/query
2. Update: http://localhost:8000/update
3. Subscribe/Unsubscribe: ws://localhost:9000/subscribe
4. SECURE Query: https://localhost:8443/secure/query
5. SECURE Update: https://localhost:8443/secure/update
6. SECURE Subscribe/Unsubscribe: wss://localhost:9443/secure/subscribe
7. Registration: https://localhost:8443/oauth/register
8. Token request: https://localhost:8443/oauth/token

{"parameters":{
  "scheduler":{
   "queueSize":100,
   "timeout":5000},
  "processor":{
   "updateTimeout":5000,
   "queryTimeout":5000,
   "maxConcurrentRequests":5,
   "reliableUpdate":true},
  "spu":{"timeout":5000},
  "gates":{
   "security":{
    "tls":false,
    "enabled":false,
    "type":"local"},
   "paths":{
    "secure":"/secure",
    "update":"/update",
    "query":"/query",
    "subscribe":"/subscribe",
    "unsubscribe":"/unsubscribe",
    "register":"/oauth/register",
    "tokenRequest":"/oauth/token"},
   "ports":{
    "http":8000,
    "https":8443,
    "ws":9000,
    "wss":9443}}}}

Logging

SEPA uses log4j2 by Apache. A default configuration is stored in the file log4j2.xml provided with the distribution. If the file resides in the engine folder, but it is not used, add the following JVM directive to force using it:

java -Dlog4j.configurationFile=./log4j2.xml -jar engine-x.y.z.jar

Security

By default, the engine implements a simple in-memory OAuth 2.0 client-credential flow. It uses a JKS for storing the keys and certificates for SSL and JWT signing/verification. A default sepa.jks is provided including a single X.509 certificate (the password for both the store and the key is: sepa2017). If you face problems using the provided JKS, please delete the sepa.jks file and create a new one as follows: keytool -genkey -keyalg RSA -alias sepakey -keystore sepa.jks -storepass sepa2017 -validity 360 -keysize 2048 Run java -jar engine-x.y.z.jar -help for a list of options. The Java Keytool can be used to create, access and modify a JKS. SEPA also implements other two security mechanisms:

• LDAP: it extends the default one by storing clients's information into an LDAP server (tested with Apache Directory)
• KEYCLOAK: authentication based on OpenID Connect in managed by Keycloak

Security is configured within the engine.jpar as follows:

{"gates":{
  "security":{
    "tls": false,
    "enabled": true,
    "type": "local"
}}}

where

• type can assume one of the following values: local,ldap,keycloak
• tls is used when type=ldap to enable or not LDAP StartTLS

JMX monitoring

The SEPA engine is also distributed with a default JMX configuration jmx.properties (including the jmxremote.password and jmxremote.access files for password and user grants). Remember to change password file permissions using: chmod 600 jmxremote.password. To enable remote JMX, the engine must be run as follows: java -Dcom.sun.management.config.file=jmx.properties -jar engine-x.y.z.jar. Using jconsole is possible to monitor and control the most important engine parameters. By default, the port is 5555 and the root:root credentials grant full control (read/write).

Usage

The SEPA engine can be configured from the command line. Run java -jar engine-x.y.z.jar -help for the list of available settings.

java [JMX] [JVM] [LOG4J] -jar SEPAEngine_X.Y.Z.jar [-help] [-secure true] [-engine engine.jpar] [-endpoint endpoint.jpar] [JKS OPTIONS] [LDAP OPTIONS] [ISQL OPTIONS]

• secure : overwrite the current secure option of engine.jpar
• engine : can be used to specify the JSON configuration parameters for the engine (default: engine.jpar)
• endpoint : can be used to specify the JSON configuration parameters for the endpoint (default: endpoint.jpar)
• help : to print this help

[JMX]

• Dcom.sun.management.config.file=jmx.properties : to enable JMX remote managment

[JVM]

• XX:+UseG1GC

[LOG4J]

• Dlog4j.configurationFile=path/to/log4j2.xml

[JKS OPTIONS]

• sslstore : JKS for SSL CA (default: ssl.jks)
• sslpass : password of the JKS (default: sepastore)
• jwtstore : JKS for the JWT key (default: jwt.jks)
• jwtalias : alias for the JWT key (default: jwt)
• jwtstorepass : password for the JKS (default: sepakey)
• jwtaliaspass : password for the JWT key (default: sepakey)

[LDAP OPTIONS]

• ldaphost : host (default: localhost)
• ldapport : port (default: 10389)
• ldapdn : domain (default: dc=sepatest,dc=com)
• ldapusersdn : domain (default: null)
• ldapuser : username (default: null)
• ldappwd : password (default: null)

[ISQL OPTIONS]

• isqlpath : location of isql (default: /usr/local/virtuoso-opensource/bin/)
• isqlhost : host of Virtuoso (default: localhost)
• isqluser : user of Virtuoso (default: dba)
• isqlpass : password of Virtuoso (default: dba)

Build with Maven

SEPA is a Maven multi-module project composed of four sub-projects:

• Client API
• Engine
• Dashboard
• Chat

Java 25 and Maven are required to build the current project.

Build and install all modules locally with tests skipped:

mvn clean install -DskipTests -Dgpg.skip=true

-DskipTests is needed for the local reactor build because the client-api module includes integration tests that require a running SEPA engine. -Dgpg.skip=true skips artifact signing for local builds.

The build creates executable shaded JARs in the module target directories, including the SEPA engine JAR:

engine/target/sepa-engine-<version>.jar

Run the client-api integration tests with a Maven-started in-memory SEPA engine:

mvn -pl client-api verify -Pwith-sepa-engine -Dgpg.skip=true

The with-sepa-engine profile requires the engine JAR to have already been built by the previous reactor command. It starts engine/target/sepa-engine-<version>.jar during the Maven pre-integration-test phase using engine/src/main/resources/endpoints/jena-in-memory.jpar, waits for ports 8000 and 9000, runs the Failsafe integration tests, and stops the engine when Maven exits. Engine output is written to:

client-api/target/sepa-engine.log

The complete local verification flow is therefore:

mvn clean install -DskipTests -Dgpg.skip=true
mvn -pl client-api verify -Pwith-sepa-engine -Dgpg.skip=true

To know more about Maven, refer to the official documentation.

Contributing

You are very welcome to be part of SEPA community. If you find any bug feel free to open an issue here on GitHub, but also feel free to ask any question. For more details check Contributing guidelines. Besides, if you want to help the SEPA development follow this simple steps:

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Check some IDE specific instruction below
4. Do your stuff
5. Provide tests for your features if applicable
6. Commit your changes: git commit -am 'Add some feature'
7. Push to the branch: git push origin my-new-feature
8. Submit a pull request :D

Pull request with unit tests have an higher likelihood to be accepted, but we are not to restrictive. So do not be afraid to send your contribution!

History

SEPA has been inspired and influenced by Smart-M3. SEPA authors have been involved in the development of Smart-M3 since its origin.

The main differences between SEPA and Smart-M3 are the protocol (now compliant with the SPARQL 1.1 Protocol) and the introduction of a security layer (based on TLS and JSON Web Token for client authentication).

All the SEPA software components have been implemented from scratch.

Credits

SEPA stands for SPARQL Event Processing Architecture. SEPA is promoted and maintained by the Dynamic linked data and Web of Things Research Group @ ARCES, the Advanced Research Center on Electronic Systems "Ercole De Castro" of the University of Bologna.

License

SEPA Engine is released under the GNU GPL, SEPA APIs are released under the GNU LGPL