REST API Endpoints

In addition to the Process Endpoint designed for triggering the ingestion of data into the Transformer, there is a selection of built-in exposed endpoints for you to call.

API	HTTP Request	URL Template	Description
Process	GET	/process	Manually triggers SQL Transformer process.
Config	GET	/config	Displays configuration as JSON string
GET	/config?paths=<config-options>	Displays all Transformer configuration specified in the comma-separated list
Update Config	PUT	/updateConfig?configEntry=<config-entry>&configValue=<config-value>	Update configuration options on a running Transformer
Upload Config Backup	PUT	/uploadConfigBackup	Uploads the current configuration to the specified config backup location so that it can be restored at a later date.
Test DB Connection	GET	/testDBConnection?jdbc=<jdbc>&url=<url>&user=<user>&password=<base64-encoded-password>&jdbcParamKey=<jdbcParamKey>&jdbcParamValue=<jdbcParamValue>	Tests whether the database credentials you have provided successfully connect to your SQL DB. Please note that the password must be Base64 encoded.
Query Database	GET	/queryDatabase?jdbc=<jdbc>&url=<url>&user=<user>&password=<base64-encoded-password>&jdbcParamKey=<jdbcParamKey>&jdbcParamValue=<jdbcParamValue>&query=<query>&returnCSV=<returnCSV>	Runs a SQL query against the specified database. Returns a CSV if returnCSV is set to true, otherwise a json object is returned. Please note that the password must be Base64 encoded. In addition, you can provide JDBC parameters into your connection. This is done by passing any number of jdbcParamKey/jdbcParamValue pairs into your request.
License	GET	/license	Displays license information
RML	GET	/rml	Displays the current RML mapping file, this is displayed in Turtle RDF serialisation
PUT	/rml	Deploys a new mapping file into Transformer specified in the request body
Functions	GET	/updateCustomFunctions	Allows for the deployment of new custom functions to the Transformer. It will download the files set in the CUSTOM_FUNCTION_JAR_URL and CUSTOM_FUNCTION_TTL_URL config options.
Update Cron	PUT	/updateCron?cronValue=<cron-value>	Updates the cron expression used for scheduling. If called without a cron value, the Transformer with turn off the scheduler.
Restart Kafka	GET	/restartKafka	Turns Kafka on or off depending on its current state.

Config

GET /config

The config endpoint is a GET request that allows you to view the configuration settings of a running Transformer. By sending GET http://<Transformer-ip>:<Transformer-port>/config (for example http://127.0.0.1:8080/config), you will receive the entire configuration represented as a JSON object, as seen in this small snippet below. All confidential values (such as credentials) are are hidden. This endpoint is also useful as a means of Health Checking the Transformer.

{
    "graphBuildTransformerConfig": {
        "provenance": {
            "kafka.kafkaBrokers": "localhost:9092",
            "outputDirUrl": "s3://graphbuild-test/output/",
            "kafka.topicNameSuccess": "prov_success_queue",
            "recordProvo": true,
            "switchedOffActivities": "",
            "kafka.topicNameDLQ": "prov_dead_letter_queue"
        },
        "kafka": {
            "topicNameSource": "source_urls",
            ............. }
}

Update Config

PUT /updateConfig

The configuration on running Transformers can now be edited without having to restart. This is done through the update config endpoint. For example, by running the following /updateConfig?configEntry=friendlyName&configValue=GraphBuilder we have changed the friendly name of the Transformer to GraphBuilder. To see a list of the configuration entry names, consult the SQL Transformer Configurable Options.

License

GET /license

The license endpoint is a GET request that allows you to view information about your license key that is in use on a running Transformer. By sending GET http://<transformer-ip>:<transformer-port>/license (for example: http://127.0.0.1:8080/license), you will receive a JSON object response containing the following values.

{
    "license": "# GraphBuild License (id: xxxx)xxxxxxxxx",
    "licenseInformation": {
        "registeredTo": "<name> - <company> - <e-mail>",
        "expires": "Sat Feb 31 00:01:00 GMT 2015 - (xxx days to go)",
        "productName": "GraphBuild"
    },
    "validationStatus": "LICENSE_VALID"
}

Process

GET /process

As previously outlined, using the process endpoint is one way of triggering the Transformer to start ingestion your database data. When an execution of the Transformer fails after being triggered in this way, the response will be a status 400 Bad Request and contain a response message similar to that sent to the dead letter queue.

{
    "name": "SQL-Transformer",
	"time": "2022-04-21T11:36:10.002",
	"type": "SQL-Transformer",
	"error": "Record could not be processed due to: Invalid Database Credentials.",
	"version": "2.0.3",
	"provenance": "...[prov]..."
} 

RML

The RML endpoint is all about the model file that you would have already created. It consists of a GET and a PUT endpoint, allowing you to get the current master mapping file currently in use on the Transformer, as well as replacing the master model file with a new one.

GET /rml

By sending GET http://<Transformer-ip>:<Transformer-port>/rml you will receive a response containing the contents of the model file written in RDF/Turtle.

PUT /rml

By sending PUT http://<Transformer-ip>:<Transformer-port>/rml with a turtle mapping file in the body of the request, it will upload it to the file location specified in the MAPPINGS_DIR_URL and MASTER_MAPPING_FILE options in the configuration and replace the existing file. The mapping file should be in RDF/Turtle format and the declared HTTP Content-Type should be text/turtle. The successful upload is then indicated by an empty response with HTTP status OK (Ref. RFC-7231) and will be functional immediately.

Custom Functions

If when designing your model file for your Transformer, you require a function to be executed that cannot perform your required operation simply by using the built-in functions, it is possible to create and use your own. This can be done by setting your CUSTOM_FUNCTION_JAR_URL and CUSTOM_FUNCTION_TTL_URL config options to point at your jar and ttl files, and calling this endpoint to download and set these files. For further instructions on how to correctly carry out this process, please see our guide.