Security Dataset Generation Platform Overview

[sehkone]

Security Dataset Generation Platform

Vision and Architecture Overview

1. Background and Problem Definition

The modern security landscape is undergoing rapid structural change.

• Most network traffic is encrypted with TLS, and HTTP/3 (QUIC) adoption is growing.
• Some services use End-to-End Encryption (E2E), making mid-path decryption structurally impossible.
• Organizations have different architectures:
  • Office environments with few internal servers
  • Service environments with TLS termination points
  • Government/public-sector environments that perform policy-based HTTPS decryption
  • Environments where E2E encryption limits visibility

At the same time, the nature of threat actors is also evolving.

• Attackers may be human.
• They may be automated scripts.
• They may be attackers who use AI as an assistant.
• They may be autonomous AI Actors that explore and adapt on their own.

In addition, AI Agents in internal environments are increasingly interacting with systems — performing task automation, code generation, data queries, and external LLM integration. These AI Agents can:

• Access the file system
• Call internal APIs
• Query databases
• Maintain persistent communication with external LLM services

Without proper control structures, this creates risks of privilege abuse or runaway behavior.

Existing public datasets do not adequately reflect these encryption structures, actor types, or AI-driven automation environments. Therefore, a raw-data-based dataset generation system that accounts for encryption architecture, environment structure, actor types, and AI Agent activity is needed.

This platform was designed for that purpose.

2. Vision

The Security Dataset Generation Platform aims to:

• Automatically generate raw data (PCAP, Sysmon, Falco, DB/Cloud logs)
• Systematically model environment structures and encryption architectures
• Realistically blend normal and threat behaviors
• Simulate behaviors from human, automated, and AI Actor sources
• Generate internal AI Agent activity traffic and provide verification inputs for AI Agent Proxy
• Validate through the same QFlow-based production pipeline
• Maintain a structure that scales from local to large-scale

This platform is not a simple testing tool — it is an engineering platform for structurally verifying the security landscape of the AI era.

3. Scope

In Scope

• Raw data generation
  • PCAP
  • Sysmon (JSONL)
  • Falco (JSONL)
  • DB access logs
  • Cloud access logs
• Normal and threat behavior blending
• AI Actor-based attack scenarios
• Internal AI Agent activity traffic generation
• Input traffic generation for AI Agent Proxy (normal activity, runaway, privilege violations, etc.)
• Dataset Bundle standardization
• Ground Truth recording
• Automated validation (Validator)
• Scalable design for local and cloud deployment

Out of Scope

• Detection model implementation
• Real-time response system implementation
• Complete replication of specific customer environments
• AI Agent Proxy implementation itself (exists as a separate solution)

Currently, only the input traffic that Proxy will process is generated; actual Proxy-based verification is performed at the QFlow Replay stage. Future expansion may include Proxy within the emulation environment for end-to-end generation and verification.

4. Design Principles

4.1 Raw-First

The platform generates raw originals, not processed data:

• PCAP
• Sysmon JSONL
• Falco JSONL
• DB/Cloud logs

Processed data tied to specific detection models is not generated.

4.2 Pipeline Consistency

Generated data must pass through the same path as real customer data: QFlow collection/parsing → internal storage → AI analysis.

No test-only bypass paths are allowed.

5. N-Dimensional Environment Modeling

"N-dimensional" refers to the independent design axes that define an environment. Each axis is independent, and combinations define diverse environments.

5.1 Base Environment Axes

1) Scale

• Small
• Medium
• Large

2) Encryption Architecture

• TLS-based encryption
• TLS termination point present
• HTTPS proxy decryption
• End-to-End Encryption (E2E)

3) Workload Profile

• Endpoint-centric
• Server-centric
• Cloud services

Multiple selections can express hybrid configurations.

4) Threat Category

• Insider activity
• External intrusion
• Data exfiltration
• Automated attacks
• etc.

6. AI-Related Dimensions

AI environments cannot be represented by a single axis, because the attacker identity and the presence of internal AI Agents are separate concepts.

6.1 Attacker Model

• Human attacker
• Scripted automation
• AI-assisted attacker
• Autonomous AI Actor

This axis defines who is attacking.

6.2 AI Agent Deployment Model

• Internal AI Agent (for task automation)
• External AI Service integration
• If no AI Agent exists, this axis is omitted.

Multiple selections can express hybrid configurations.

This axis defines how AI operates within the organization.

6.3 AI Agent Control Model

• Monitoring (observe only, no blocking)
• Enforcement (block on policy violation)
• If no AI Agent exists, this axis is omitted.

Used together with the AI Agent Deployment Model.

This axis defines which control mode Proxy is tested under, and is used to generate corresponding traffic patterns. The Proxy itself is not currently included in the emulation environment; generated datasets are injected into the Proxy at the QFlow Replay stage for verification.

Each AI-related dimension is independent of the others. For example:

• The attacker may be an AI Actor, but there may be no internal AI Agent.
• An internal AI Agent may exist, but the attacker may be human.

These combinations enable structural modeling of:

• AI-based attack environments
• AI-based task automation environments
• AI Agent runaway monitoring environments

Specific behavior definitions and traffic generation methods for each axis are covered in separate documents (AI Actor Behavior Model, AI Agent Traffic Generation — TBD).

7. Combinatorial Explosion and Phased Expansion

With multiple dimensions and multiple options per dimension, the number of possible combinations grows exponentially. For example, with 6 axes and 4 options each:

4 × 4 × 4 × 4 × 4 × 4 = 4,096

This is known as combinatorial explosion.

Generating all combinations simultaneously is impractical. Therefore, this platform follows this strategy:

• Select representative anchor cases
• Expand based on priority
• Gradually include AI Actor and AI Agent environments

Anchor case selection criteria and specific lists are covered in a separate document (Anchor Case Selection).

8. High-Level Architecture

Scenario Definition (declares environment structure, behavior composition, axis combinations)
    ↓
Generator → see Generator Design
    ↓
Raw Artifacts (PCAP / JSONL)
    ↓
Dataset Bundle (+ Ground Truth)
    ↓
Validator → see Validator Design
    ↓
QFlow Replay → see QFlow Replay Integration
    ↓
Storage
    ↓
AI Analysis

AI Actor and AI Agent environments also pass through the same pipeline.

The scenario definition format, field definitions, and axis combination expressions are covered in a separate document (Scenario Definition Schema).

9. Dataset Bundle Structure

dataset_bundle/
  meta.json
  net/
    edge.pcap
    inner.pcap
  host/
    win-001/sysmon.jsonl
    linux-001/falco.jsonl
  db/
  cloud/
  ground_truth/
    manifest.jsonl

• Backend-independent
• Can be generated in parallel and merged
• Supports automated validation
• Can include AI Actor and Agent scenario metadata

Ground Truth is the answer data recording whether each event in the generated data is normal or malicious, the attack stage, technique, etc. The granularity, format, and generation method are covered in a separate document (Ground Truth Definition).

10. Phased Roadmap

Phase 1 – Base Environment Setup

• 3 encryption architecture variants
• Human-based attack scenarios
• Related: Generator Design, Scenario Definition Schema, Anchor Case Selection

Phase 2 – AI Actor Inclusion

• AI-based attack scenarios
• Internal AI Agent activity traffic generation
• Related: AI Actor Behavior Model, AI Agent Traffic Generation (TBD)

Phase 3 – AI Agent Control Environment

• Input traffic generation per Monitoring/Enforcement mode
• Runaway and privilege violation scenarios
• Related: AI Agent Traffic Generation (TBD)

Phase 4 – Large-Scale Parallel Generation

• Large-volume AI Actor activity
• Monthly large-scale dataset generation

11. Success Criteria

• Both human and AI Actor behaviors are observable
• Diverse input traffic for AI Agent Proxy can be verified
• Same QFlow-based pipeline is maintained
• Large-scale environment simulation is possible
• Encryption architecture diversity is ensured

12. Related Documents

Document	Content
Generator Design	Implementation approach, per-scenario generation strategy, extension structure
Scenario Definition Schema	Input format, field definitions, axis combination expressions, examples
Anchor Case Selection	Selection criteria, Phase 1 target combination list, priority rationale
Ground Truth Definition	Granularity, format, answer recording method at generation time
Validator Design	Validation item list, pass/fail criteria
AI Actor Behavior Model	Behavior definition approach, attack scenario catalog, phased sophistication
AI Agent Traffic Generation (TBD)	Agent activity types, runaway/privilege violation traffic, per-control-mode generation methods
QFlow Replay Integration (TBD)	Bundle injection method, compatibility requirements