Skip to content

Data processing service

Viktor Batytskyi edited this page Apr 10, 2015 · 13 revisions

Data processing service

With the help of data processing service, which can be accessed via URL localhost:8080/api/data/process, it is possible to send requests for data processing to one of the predefined modules.

All the modules are located under proc/ directory and are implemented as usual node.js modules. Entry points to the modules are described in config/executables.js file, where logical name of a module maps to its entry point.

To send a request for data processing you should make HTTP POST request for localhost:8080/api/data/process URL, where request body is a JSON object that corresponds to the following schema:

{
    [{"name": "data", "type": "json"} or {"name": "data_id", "type": "string"}],
    {"name": "proc_name", "type": "string"},
    [{"name": "params", "type": "json"} or nothing],
    {"name": "response_type", "type": ["data", "data_id"]}
}

Next two examples represent valid request objects:

{
    "data": {
        "values": [1, 2, 4]
    },
    "params": {
        "multiplier": 2
    },
    "proc_name": "multiplier",
    "response_type": "data_id"
}

or

{
    "data_id": "55184973dbbadcf05d65df0e",
    "params": {
        "precision": 4
    },
    "proc_name": "avg",
    "response_type": "data"
}

The first example request sends JSON data object to be processed in its data field and expects data_id as a result. It makes request to the module called multiplier.

The second example request sends data_id which represents existent data_id returned by previous computations and expects JSON data object as a result. It makes request to the module called avg.

After making all necessary computations, as a response to client's request, server sends an object that corresponds to the following schema (HTTP: OK, code 200):

{
    {"name": "status_code", "type": "int"},
    [{"name": "data", "type": "json"} or nothing],
    {"name": "data_id", "type": "string"},
    {"name": "err", "type": "string"},
    {"name": "parent", "type": "string"}
}

Server may respond with HTTP: Bad request, code 400 when a request was malformed or data_id was nonexistent.

Server responds with HTTP: Internal server error, code 500 if some error happened while performing the computations.

Sample response of "response_type": "data":

{ 
    "data": { 
        "values": [ 2, 4, 8 ] 
    },
    "status_code": 0,
    "data_id": "55184973dbbadcf05d65df0e"
}

Sample response of "response_type": "data_id":

{ 
    "status_code": 0,
    "data_id": "5518498fdbbadcf05d65df0f",
    "parent": "55184973dbbadcf05d65df0e" 
}
All the paths are relative to src/main/webapp.

Clone this wiki locally