# REST API Endpoints In addition to the Process Endpoint designed for triggering the ingestion of data into the Transformer, there is a selection of built-in exposed endpoints for you to call. | **API** | **HTTP Request** | **URL Template** | **Description** | | -------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Process | GET | `/process?inputFileURL=&logicalSources=` | Tells the Transformer to ingest the file located at the specified URL location using the correlating logical source | | `/process?inputFileURL=&logicalSources=&mappingURL=` | When an input file needs to be transformed against a specific mapping file, an additional optional parameter for the mapping file or mapping directory URL can be given. | | | | `/process?inputFileURL=&logicalSources=&inputFileURL=&logicalSources=` | You also have the ability to process multiple input source files at once, this is for instances where there are joins in your mapping, creating links between your data. Each input file requires a correlating logical source pairing it with the source in the mapping. (This approach can also use the specific mapping parameter). | | | | Config | GET | `/config` | Displays configuration as JSON string | | GET | `/config?paths=` | Displays all Transformer configuration specified in the comma-separated list | | | Update Config | PUT | `/updateConfig?configEntry=&configValue=` | Update configuration options on a running Transformer | | Upload Config Backup | PUT | `/uploadConfigBackup` | Uploads the current configuration to the specified config backup location so that it can be restored at a later date. | | License | GET | `/license` | Displays license information | | RML | GET | `/rml?fileName=` | Displays the RML mapping file at the specified location, this is displayed in Turtle RDF serialisation | | PUT | `/rml?fileName=` | Deploys a new mapping file into Transformer specified in the request body | | | Functions | GET | `/updateCustomFunctions` | Allows for the deployment of [new custom functions](#custom-functions) to the Transformer. It will download the files set in the `CUSTOM_FUNCTION_JAR_URL` and `CUSTOM_FUNCTION_TTL_URL` config options. | | Restart Kafka | GET | `/restartKafka` | Turns the Transformer Kafka connection on or off depending on its current state. | ### Process #### GET `/process` As previously [outlined](/EGeX4aTAJLlpg9Hh8kfl/semi-structured-transformer/triggering-the-transformer-ingesting-data.md), using the process endpoint is one way of triggering the Transformer to ingest your source data. When an execution of the Transformer fails after being triggered in this way, the response will be a status `400 Bad Request` and contain a JSON response message similar to that sent to the dead letter queue. ``` { "name": "Semi-Structured-Transformer", "time": "2022-04-21T11:36:10.002", "type": "Semi-Structured-Transformer", "error": "Record 'file:///var/local/basic-input.xyz' could not be processed due to: Input file is of an invalid file type: (.xyz) - Expected XML, JSON, CSV, XLSX, or ODS.", "version": "2.0.3", "entity": "file:///var/local/basic-input.xyz", "provenance": "...[prov]..." } ``` ### Config #### GET `/config` The config endpoint is a GET request that allows you to view the configuration settings of a running Transformer. By sending `GET http://:/config` (for example `http://127.0.0.1:8080/config`), you will receive the entire configuration represented as a JSON, as seen in this small snippet below. All confidential values (such as credentials) are hidden. This endpoint is also useful as a means of Health Checking the Transformer. ``` { "graphBuildTransformerConfig": { "provenance": { "kafka.kafkaBrokers": "localhost:9092", "outputDirUrl": "s3://graphbuild/testing/output/", "kafka.topicNameSuccess": "prov_success_queue", "recordProvo": true, "kafka.topicNameDLQ": "prov_dead_letter_queue", "switchedOffActivities": "" }, "kafka": { "topicNameSource": "source_urls", "topicNameDLQ": "dead_letter_queue", ............ ``` ### Update Config #### PUT `/updateConfig` The configuration on a running Transformer can now be edited without having to restart. This is done through the update config endpoint. For example, by running the following `/updateConfig?configEntry=friendlyName&configValue=GraphBuilder` we have changed the friendly name of the Transformer to GraphBuilder. To see a list of the configuration entry names, consult the Semi Structured Transformer [Configurable Options](/EGeX4aTAJLlpg9Hh8kfl/semi-structured-transformer/all-configuration-options.md). ### License #### GET `/license` The license endpoint is a GET request that allows you to view information about your license key that is in use on a running Transformer. By sending `GET http://:/license` (for example: `http://127.0.0.1:8080/license`), you will receive a JSON response containing the following values. ``` { "license": "# graphbuild License (id: xxxx)xxxxxxxxx", "licenseInformation": { "registeredTo": " - - ", "expires": "Sat Feb 31 00:01:00 GMT 2023 - (xxx days to go)", "productName": "graphbuild" }, "validationStatus": "LICENSE_VALID" } ``` ### RML The RML endpoint is all about the mapping file that you would have created. It consists of a GET and a PUT endpoint, allowing you to get the current master mapping file currently in use on the Transformer, and well as replacing the master mapping file with a new one. #### GET `/rml` By sending `GET http://:/rml?fileName=` you will receive a response containing the contents of the mapping file at the location specified written in [RDF/Turtle](https://www.w3.org/TR/turtle/). #### PUT `/rml` By sending `PUT http://:/rml?fileName=` with a turtle mapping file in the body of the request, it will upload it to the file location specified. If a file already exists in that location, it will be replaced, otherwise a new file will be created. The mapping file should be in RDF/Turtle format and the declared HTTP `Content-Type` should be `text/turtle`. The successful upload is then indicated by an empty response with HTTP status `OK` (Ref. [RFC-7231](https://tools.ietf.org/html/rfc7231#section-6.3.1)) and will be functional immediately. ### Custom Functions If when designing your mapping file for your Transformer, you require a function to be executed that cannot perform your required operation simply by using the built-in functions, it is possible to create and use your own. This can be done by setting your `CUSTOM_FUNCTION_JAR_URL` and `CUSTOM_FUNCTION_TTL_URL` config options to point at your jar and ttl files, and calling this endpoint to download and set these files. For further instructions on how to correctly carry out this process, please see the [walkthrough](#custom-functions).
--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.graph.build/EGeX4aTAJLlpg9Hh8kfl/semi-structured-transformer/rest-api-endpoints.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.