Skip to content

feat: NiFi 2.x upgrade, ABACEnforcement processor, and passthrough mode for ConvertToZTDF #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JBCongdon wants to merge 10 commits into
base: main
Choose a base branch
from
Open

feat: NiFi 2.x upgrade, ABACEnforcement processor, and passthrough mode for ConvertToZTDF #63

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d33d31f
feat: NiFi 2.x upgrade, ABAC enforcement processor, and passthrough mode
JBCongdon Mar 26, 2026
bd679ad
fix(test): add nifi-security-utils-api to resolve TlsConfiguration Cl…
JBCongdon Mar 27, 2026
106ce8f
fix: validate non-empty resource FQNs and evaluate ENVIRONMENT expres…
JBCongdon Mar 27, 2026
70a7405
docs: fix copy-paste Javadoc on ENABLE_ENCRYPTION property
JBCongdon Mar 27, 2026
9dbea2f
feat: add ParseJREAPC processor for JREAP-C binary protocol parsing
JBCongdon Mar 27, 2026
9bb917a
fix(test): declare IOException on assertContentEquals test method
JBCongdon Mar 27, 2026
a704774
docs: update README for NiFi 2.x, ParseJREAPC, and ABACEnforcement
JBCongdon Mar 27, 2026
d4db67c
refactor: move ParseJREAPC to dedicated nifi-jreapc-processors module
JBCongdon Apr 7, 2026
3b7c9f5
test: add ABACEnforcement unit tests and JREAP-C pipeline integration…
JBCongdon Apr 7, 2026
b1f3ec2
docs: fix ParseJREAPC module path and add Testing section to README
JBCongdon Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 76 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,92 @@
# OpenTDF NiFi
Integration of the [OpenTDF Platform](https://github.com/opentdf/platform) into [NiFi](https://nifi.apache.org/)

Components:
* "Zero Trust Data Format" (ZTDF) Processors:
* [ConvertToZTDF](./nifi-tdf-processors/src/main/java/io/opentdf/nifi/ConvertToZTDF.java): A NiFi processor that converts FlowFile content to ZTDF format.
* [ConvertFromZTDF](./nifi-tdf-processors/src/main/java/io/opentdf/nifi/ConvertFromZTDF.java): A NiFi processor that converts ZTDF formatted FlowFile content to its plaintext representation
Integration of the [OpenTDF Platform](https://github.com/opentdf/platform) into [Apache NiFi](https://nifi.apache.org/). Provides processors for TDF encryption/decryption and attribute-based access control (ABAC) enforcement on any flow file — including binary protocol streams that cannot be TDF-wrapped.

* Controller Services:
* [OpenTDFControllerService](./nifi-tdf-controller-services-api/src/main/java/io/opentdf/nifi/OpenTDFControllerService.java): A NiFi controller service providing OpenTDF Platform Configuration
## Processors

### TDF Encryption

| Processor | Description |
|-----------|-------------|
| [ConvertToZTDF](./nifi-tdf-processors/src/main/java/io/opentdf/nifi/ConvertToZTDF.java) | Encrypts flow file content as a Zero Trust Data Format (ZTDF) object, binding OpenTDF policy attributes to the ciphertext. Set **Enable Encryption = false** to tag a flow file with policy attributes without encrypting the payload (ABAC-only / tag-only mode). |
| [ConvertFromZTDF](./nifi-tdf-processors/src/main/java/io/opentdf/nifi/ConvertFromZTDF.java) | Decrypts a ZTDF-formatted flow file back to plaintext using the configured KAS endpoint. |

### Binary Protocol Support

| Processor | Description |
|-----------|-------------|
| [ParseJREAPC](./nifi-jreapc-processors/src/main/java/io/opentdf/nifi/ParseJREAPC.java) | Parses JREAP-C (Joint Range Extension Applications Protocol Category C) binary message headers and extracts policy-relevant fields — classification, J-series word type, track number, source/destination addressing, and timestamp — as flow file attributes. Optionally populates `tdf_attribute` automatically from the classification level when a **Classification Attribute Namespace** is configured, making it directly consumable by `ABACEnforcement` downstream. Payload bytes are passed through unmodified. |
| [ABACEnforcement](./nifi-tdf-processors/src/main/java/io/opentdf/nifi/ABACEnforcement.java) | Calls the OpenTDF Authorization Service `GetDecisions` endpoint to make an ABAC permit/deny decision for the flow file. Uses the `tdf_attribute` flow file attribute as the resource context. Routes to **permit**, **deny**, or **failure** relationships. Supports a **Fail Open** property to control behavior when the authorization service is unreachable. Designed to enforce policy on binary protocol streams (JREAP-C, sensor feeds, telemetry) that cannot be encrypted as TDF but still require access control. |

### Controller Services

| Service | Description |
|---------|-------------|
| [OpenTDFControllerService](./nifi-tdf-controller-services-api/src/main/java/io/opentdf/nifi/OpenTDFControllerService.java) | Shared controller service providing OpenTDF Platform configuration (endpoint, OIDC credentials, KAS URL) to all TDF processors. |

## What This Enables

**Standard TDF flow** — encrypt data in NiFi and enforce policy downstream:

```
[Source] → [ConvertToZTDF] → [storage / transit] → [ConvertFromZTDF] → [Consumer]
```

**ABAC-only flow for binary protocols** — enforce policy without modifying payload bytes:

```
[JREAP-C source] → [ParseJREAPC] → [ABACEnforcement] → permit → [forward]
→ deny → [drop / audit]
→ failure → [error handling]
```

This pattern is the NiFi equivalent of the `GATEWAY_ABAC_ENCRYPT_EMAIL=0` mode in gateway deployments: attribute tagging and policy enforcement happen without wrapping the content as TDF. It applies wherever the payload format is fixed (Link 16/JREAP-C, sensor feeds, telemetry streams) and the flow requires access control without encryption.

**Tag-only mode with `ConvertToZTDF`** — enforce policy and optionally encrypt in a single processor:

```
[Source] → [UpdateAttribute tdf_attribute=...] → [ConvertToZTDF Enable Encryption=false] → [ABACEnforcement]
```

## Using a Custom TrustStore

## Using a custom TrustStore
Communicating over TLS with self-signed or other untrusted certs can be configured using NiFi's standard [SSL Context Service](https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.25.0/org.apache.nifi.ssl.StandardSSLContextService/index.html)
and then wired into the processors by setting their respective SSL Context Service properties to use a configured
SSL Context Service.
and wired into processors via their **SSL Context Service** property.

## Example
## Configuration

To use these processors in NiFi:
* Configure the OpenTDFControllerService properties
* set then OpenTDF compliant endpoint
* set OIDC Client credentials (client id and client secret)
* set the data policy (UpdateAttribute Processor)
* set the KAS URL: ConvertToZTDF processor
1. Configure the **OpenTDFControllerService**:
- OpenTDF platform endpoint
- OIDC client credentials (client ID and client secret)
2. Wire the controller service into each processor's **OpenTDF Config Service** property
3. Set `tdf_attribute` on flow files (via `UpdateAttribute`, `ParseJREAPC`, or other means) with one or more OpenTDF attribute value FQNs in the format `https://namespace/attr/name/value/val`

#### FlowChart: Generic ZTDF Nifi Flows
#### FlowChart: Generic ZTDF NiFi Flows

![diagram](./docs/diagrams/generic_ztdf_nifi_flows.svg)


## Testing

Unit and integration tests are included for all processors. Run the full suite with:

```shell
export GITHUB_ACTOR=your-gh-username
export GITHUB_TOKEN=your-gh-token
mvn --batch-mode clean verify -s settings.xml
```

Test coverage includes:

| Test class | What it covers |
|------------|----------------|
| `ParseJREAPCTest` | 32-byte header parsing: all classification codes, word types, exercise/simulation flags, `tdf_attribute` slug generation, content pass-through |
| `ABACEnforcementTest` | Permit, deny, fail-open/closed, missing/blank/empty `tdf_attribute`, default resource attributes, empty decision list, multi-decision deny-wins |
| `JREAPCPipelineTest` | Full pipeline chain: `ParseJREAPC` → `ABACEnforcement` → `ConvertToZTDF` with synthetic JREAP-C binary data, asserting byte-for-byte preservation through all stages |
| `ConvertToZTDFTest` | TDF wrapping, attribute binding, assertion signing |
| `ConvertFromZTDFTest` | TDF decryption routing |

# Quick Start - Docker Compose

1. Build the NiFi Archives (NARs) and place in the docker compose mounted volumes. The opentd
Expand Down
110 changes: 110 additions & 0 deletions nifi-jreapc-nar/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.opentdf.nifi</groupId>
<artifactId>nifi-pom</artifactId>
<version>0.10.0</version><!-- {x-version-update:nifi:current} -->
</parent>
<artifactId>nifi-jreapc-nar</artifactId>
<name>nifi-jreapc-nar</name>
<description>NiFi JREAP-C Processor NAR Archive</description>
<properties>
<source.skip>true</source.skip>
</properties>
<packaging>nar</packaging>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>nifi-jreapc-processors</artifactId>
<version>0.10.0</version><!-- {x-version-update:nifi:current} -->
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.3.1</version>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.8.0</version>
<executions>
<execution>
<id>attach-javadocs</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.nicoulaj.maven.plugins</groupId>
<artifactId>checksum-maven-plugin</artifactId>
<version>1.11</version>
<executions>
<execution>
<id>create-checksums</id>
<phase>package</phase>
<goals>
<goal>files</goal>
</goals>
<configuration>
<algorithms>
<algorithm>MD5</algorithm>
<algorithm>SHA-1</algorithm>
<algorithm>SHA-256</algorithm>
<algorithm>SHA-512</algorithm>
</algorithms>
<failOnError>true</failOnError>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>*.nar</include>
</includes>
</fileSet>
</fileSets>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-prepare-agent</id>
<phase>none</phase>
</execution>
<execution>
<id>report-aggregate</id>
<phase>none</phase>
</execution>
<execution>
<id>default-report-aggregate</id>
<phase>none</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
126 changes: 126 additions & 0 deletions nifi-jreapc-processors/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.opentdf.nifi</groupId>
<artifactId>nifi-pom</artifactId>
<version>0.10.0</version><!-- {x-version-update:nifi:current} -->
</parent>
<artifactId>nifi-jreapc-processors</artifactId>
<name>nifi-jreapc-processors</name>
<description>NiFi processor for parsing JREAP-C binary protocol messages</description>
<packaging>jar</packaging>

<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-utils</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mock</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.17.2</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.3.1</version>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<encoding>UTF-8</encoding>
<source>${maven.compiler.source}</source>
</configuration>
<executions>
<execution>
<id>attach-javadocs</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.nicoulaj.maven.plugins</groupId>
<artifactId>checksum-maven-plugin</artifactId>
<version>1.11</version>
<executions>
<execution>
<id>create-checksums</id>
<phase>package</phase>
<goals>
<goal>files</goal>
</goals>
<configuration>
<algorithms>
<algorithm>MD5</algorithm>
<algorithm>SHA-1</algorithm>
<algorithm>SHA-256</algorithm>
<algorithm>SHA-512</algorithm>
</algorithms>
<failOnError>true</failOnError>
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<includes>
<include>*.jar</include>
</includes>
</fileSet>
</fileSets>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Loading
Loading