JSON API

The machine learning and signal processing classes are able to save and load their data in JSON format. This allows for data to be:

  • Moved between different instantiations and switch between different setups
  • Shared between C++, JavaScript, Wekinator, etc.
  • Uploaded to and Downloaded from RepoVizz2

The JSON API has the following methods:

getJSON() — returns a JSON string describing the model or data

writeJSON(<filepath>) — writes a file with the JSON description

putJSON() — updates the data or model with a JSON description

readJSON(<filepath>) — reads a JSON description from a file and updates the object

The JSON representations produced and consumed by this API follow the specification below.

 

RAPID-MIX JSON Specification

The RAPID-MIX API specifies a set of JSON formats to simplify the exchange of training data, training configurations, trained models, and signal processing graph descriptions between any RAPID-MIX based applications.

  • The simplest object used in RAPID-MIX is what we call an example, in other words a labelled recording of a time series of vectors of numbers, of length 1 or greater.

JavaScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
    docType: “rapid-mix:ml-example”,
    docVersion: “1.0.0”,
    payload: {
        label: “label1”,
        inputDimension: 3,
        outputDimension: 1,
        columnNames: { // optional
            input: [‘accelX’, ‘accelY’, ‘accelZ’],
            output: [‘filter‘], // optional
        },
        input: [
            [1, 2, 3],
            [2, 3, 4],
            ...
        ],
        output: [ // can be a single empty array if outputDimension is 0
            [ 0.5 ],
            [ 0.65 ],
            ...
        ]
    }
}

  • Then we have the training set, which is mostly constituted by a set of examples.

JavaScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
    docType: “rapid-mix:ml-training-set”,
    docVersion: “1.0.0”,
    payload: {
        inputDimension: 3,
        outputDimension: 0,
        columnNames: { // optional
            input: [‘accelX’, ‘accelY’, ‘accelZ’],
            output: [] // optional
        },
        data: [
            {
                input: [
                    [1, 2, 3], // frame 0
                    [1.1, 1.9, 2.5], // frame 1
                    ...
                ],
                output: [], // can be a single empty array if outputDimension is 0
                label: “label1”, // optional
            },
            ...
        ]
    }
}

  • In addition to the actual training data shown above, we need a couple of extra informations to be able to train a model. One of them is the training configuration.

JavaScript
1
2
3
4
5
6
7
8
9
{
    docType: “rapid-mix:ml-configuration”,
    docVersion: “1.0.0”,
    target: {
        name: “machine-learning-library”, // e.g: "xmm"
        version: “1.0.1”
    },
    payload: { /* library-specific configuration parameters */ }
}

  • Once a model has been trained from data and configuration, it also needs to fit the following specification in order to be exchanged and loaded properly.

JavaScript
1
2
3
4
5
6
7
8
9
{
    docType: “rapid-mix:ml-model”,
    docVersion: “1.0.0”,
    target: {
        name: “machine-learning-library”, // e.g: "xmm"
        version: “1.0.1”
    },
    payload: { /* library-specific model representation */ }
}

  • Finally, it is also possible to describe signal processing graphs in the following way.

JavaScript
1
2
3
4
5
6
7
8
9
{
    docType: “rapid-mix:signal-processing”,
    docVersion: “1.0.0”,
    target: {
        name: “signal-processing-formalism”, // e.g: "waves-lfo" or "rapid-stream"
        version: “1.0.1”
    },
    payload: { /* description and parameter values */ }
}

  • Built on top of these specifications, we can send properly formatted http requests and responses around. That’s what the Como web service does, using the following format.

JavaScript
1
2
3
4
5
6
7
8
9
10
11
{
    docType: “rapid-mix:ml-http-request”,
    docVersion: “1.0.0”,
    target: {
        name: “web-service-name”, // e.g: "como-web-service"
        version: “1.0.0”
    },
    // Como needs a configuration and a trainingSet fields in its request payload,
    // in the rapid-mix format
    payload: { /* web service specific data */ }
}

JavaScript
1
2
3
4
5
6
7
8
9
10
11
{
    docType: “rapid-mix:ml-http-response”,
    docVersion: “1.0.0”,
    target: {
        name: “web-service-name”, // e.g: "como-web-service"
        version: “1.0.0”
    },
    // Como puts the original configuration and a trained model fields in its response payload,
    // in the rapid-mix format
    payload: { /* web service specific data */ }
}