Writing to Graph Databases

Graph Writer

To write to a graph database, you will need to use the Graph Writer. This applies to both Labelled Property Graphs and Semantic Graphs.

The Graph Writer supports Update and Insert modes.

Automatic processing with Kafka integration

If you have integrated Kafka into your stack, graph models can be automatically written, by the graph writer, to a configured target graph database. See Stack Run Configurations for examples.

The process is as follows:

1. A transformer successfully completes a transformation task, which results in

   1. New graph models successfully written to disk or S3

   2. New success message posted to Kafka "success" topic, containing the path to the new files

2. The writer, configured to listen to the Kafka 'success' topic, performs the following tasks

   1. Consume a 'success' message from the Kafka topic

   2. Read the new graph models into memory

   3. Persist to the configured target graph database

Manual processing using the REST endpoint

If however you do not have a Kafka service running, or you wish to push data manually to your graph, you can use the Graph Writer's RESTful API endpoint.

Firstly, get the URL of the file you wish to import, ensuring that the Writer has direct access to it. If generated by the Transformers, this will be the same file URL shown in the successful transformation report. Using this file URL, run a GET request on the following endpoint:

http://<graph-writer-endpoint>:<graph-writer-port>/process?inputRdfURL=<graph-data-file-URL>

For example:

http://127.0.0.1:9610/process?inputRdfURL=s3://graphbuild/output/SS-Transformer-1234.nq

Upon completion, a processing report will be provided in the response message.