# Configuring the Writer

## Configuring the Writer <a href="#configuring-the-writer" id="configuring-the-writer"></a>

As with the Transformers supplied by Graph.Build, the Graph Writer has a wide array of user configuration, all of which can be set and altered both before the startup of the Writer and during the running of a Writer. The former is done through the use of environment variables in your Docker container or ECS Task Definition, and latter is done through the use of an exposed endpoints, as seen below. For a breakdown of every configuration option in the Graph Writer, see the full list [here](/EGeX4aTAJLlpg9Hh8kfl/graph-writer/all-configuration-options.md).

### Configuration Access <a href="#configuration-manipulation" id="configuration-manipulation"></a>

#### Accessing the Config <a href="#accessing-the-config" id="accessing-the-config"></a>

Once the Writer has started and is operational, you can request to view the current configuration by calling the `/config` endpoint. This is expanded upon below, including the ability to specify specific config properties.

#### Editing config <a href="#editing-config" id="editing-config"></a>

As explained below, the configuration on a running Writer can be edited through the `/updateConfig` endpoint.

#### Backup and Restore Config <a href="#backup-and-restore-config" id="backup-and-restore-config"></a>

A useful feature in the Writer, is the ability to backup and restore your configuration. This is particularly beneficial when you’ve made multiple changes to the config on a running Writer, and want to be able to restore this without rerunning any update config commands. To backup your config, simply call the `/uploadConfigBackup` endpoint, and all changes you’ve made to the config will be uploaded to the storage location specified in your `CONFIG_BACKUP` env var.

To restore your configuration, this must be done on the startup of a Writer, therefore, by setting the `CONFIG_BACKUP` config option as an environment variable in your startup script / task definition. This must however be a remote directory such as S3, as anything local will be deleted if a task or container is stopped.

### Configuration Categories <a href="#configuration-categories" id="configuration-categories"></a>

#### Mandatory Configuration (Local Deployment) <a href="#mandatory-configuration-local-deployment" id="mandatory-configuration-local-deployment"></a>

* License - `TRANSFORMER_LICENSE`
  * This is the license key required to operate the Writer when being run on a local machine outside of AWS Marketplace, request your new unique license key [here](https://graph.build/).

#### Graph Database Configuration <a href="#graph-database-configuration" id="graph-database-configuration"></a>

* Graph Database Endpoint - `GRAPH_DATABASE_ENDPOINT`
  * This is the endpoint for your Graph Database you wish to upload your Graph Data to and therefore required for the Graph Writer to work.
* Graph Database Type - `GRAPH_DATABASE_TYPE`
  * This is your Graph Database type, some graphs will support the default `sparql` type (e.g. AllegroGraph), however certain graphs require specific type declaration, these include `graphdb`, `stardog`, `blazegraph`, `neptune-sparql`, and `rdfox`.
  * If you are using a Property Graph, you can set a specific Graph provider, including `neo4j`, `neptune-cypher`, `neptune-gremlin`, or the traversal language `cypher` or `gremlin`.
* Graph Database Username and Password - `GRAPH_DATABASE_USERNAME` and `GRAPH_DATABASE_PASSWORD`
  * This is the username and password of your Graph Database. You can leave these fields blank if your Graph does not require any authentication.

#### AWS Configuration <a href="#aws-configuration" id="aws-configuration"></a>

When running the Writer in ECS, these settings are not required as all credentials are taken directly from the EC2 instance running the Writer. If you wish to use AWS cloud services while running the Writer on-prem, you need to specify an AWS Access Key and Secret Key, and AWS Region. By providing your AWS credentials, this will give you permission for accessing, downloading, and uploading remote files to S3 Buckets. The S3 Region option specifies the region of where in AWS your files and services reside. To do this, the Writer utilises the [AWS Default Credential Provider Chain](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html), allowing for a number of methods to be used. The simplest is by setting the environment variables for `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_REGION`. Please note that all services must be in the same region, including if you choose to run the Writer in an EC2 instance.

#### Kafka Configuration <a href="#kafka-configuration" id="kafka-configuration"></a>

One of the many ways to interface with the Writer is through the use of Apache Kafka. With the Graph Writer, a Kafka Message Queue can be used for managing the input of RDF data into the Writer. To properly set up your Kafka Cluster, see the instructions [here](/EGeX4aTAJLlpg9Hh8kfl/stack-run-configurations/docker/full-stack-with-rdfox-connection.md). Once complete, use the following Kafka configuration variables to connect the cluster with your Writer. If you do not wish to use Kafka, please set the variable `TRANSFORMER_RUN_STANDALONE` to true.

The Kafka Broker is what tells the Writer where to look for your Kafka Cluster, so set this property as follows: `<kafka-ip>:<kafka-port>`. The recommended port is `9092`.

#### Provenance Configuration <a href="#provenance-configuration" id="provenance-configuration"></a>

Currently, the Graph Writer does not generate its own provenance meta-data and so the `RECORD_PROVO` configuration option is set to false. However, any provenance previously generated is separate to this option, and will still be ingested into your Knowledge Graph. If you are using Kafka, ensure that your Kafka source topic is correctly configured if your Writer provenance is pushed to a separate queue from your generated output data.

#### Logging Configuration <a href="#logging-configuration" id="logging-configuration"></a>

Logging in the Graph Writer works the same way as the Transformers, and like with most functionality is configurable through the use of environment variables. When running the Graph Writer locally from the command line, the Writer will automatically log to your terminal instance. In addition to this, the archives of logs will be saved within the docker container at `/var/log/graphbuild/text/archive/` and `/var/log/graphbuild/json/archive/` for text and JSON logs respectively, where the current logs can be found at `/var/log/graphbuild/text/current/` and `/var/log/graphbuild/json/current/`. By default, a maximum of 7 log files will be archived for each file type, however this can be overridden. If running a Writer on cloud in an AWS environment, then [connect to your instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html) via SSH or PuTTY, and the previously outlined logging locations apply.

By default, the Writer logs at `INFO` level, this can be changed by overriding the `LOG_LEVEL_TRANSFORMER` option, however can only be done on Writer startup and will require a reboot if not.

#### Optional Configuration <a href="#optional-configuration" id="optional-configuration"></a>

There is also a further selection of optional configurations for given situations, see [here](/EGeX4aTAJLlpg9Hh8kfl/graph-writer/all-configuration-options.md) for the full list.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.graph.build/EGeX4aTAJLlpg9Hh8kfl/graph-writer/configuring-the-writer.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.