CogniCode

CogniCode turns a declarative intent (input and desired output, e.g., an English PDF into an English MP3) into an executable composition of cloud Backend-as-a-Service (BaaS) services. An ontology of the available services determines which services to use and in what order. An LLM then generates the integration code, executes it, and refines it on runtime errors.

Folder structure

Folder	Contents
application/	Source code and packaged JAR of CogniCode. README.md explains setup and usage, DEVELOPERS.md the modules, ONTOLOGY.md the ontology.
generated_code/	Code generated by CogniCode on AWS for the three use cases.
Results/	Raw measurements (iterations, time) on AWS and GCP.
extension_example/	Adding a service to the ontology without code changes.

Paper artifacts (IEEE Internet Computing)

This section maps the paper COGNICODE: Toward an LLM-powered system for serverless service composition automation (IEEE Internet Computing) to this repository. The use cases UC1 to UC3 of the paper appear as Workflow 1 to Workflow 3 in the results and as WF1 to WF3 in generated_code/.

Paper element	Location
Intent	application/input_aws.json (UC1)
Service ontology (JSON-LD)	application/services_aws.jsonld, structure in ONTOLOGY.md
SPARQL query and composition planning	OntologyPathFinder.java
Orchestration prompt	example-prompt.md, built in LLMRequestExecutor.java
Iterative refinement and iteration cap	LLMRequestExecutor.java (MAX_ITERATIONS = 10)
Evaluation inputs	application/input.pdf (UC1), input.mp3 (UC2), input.mp4 (UC3)
Evaluation results	Results/aws_experiments.csv, rows with model Claude Sonnet 3.5v2
Code given to the reviewing engineers	generated_code/, claude.py per workflow
Service-extension example	extension_example/

Intent

The paper writes the intent of UC1 as input=text.pdf#dt:PDF&dt:language=en and output=audio.mp3#dt:MP3&dt:language=en. In the configuration file, the same intent is given by the fields inputType, inputLanguage, outputType, and outputLanguage, each holding the ontology term, together with inputFilePath for the input file.

Composition planning

The Composition Synthesizer derives the plan in two steps:

1. A SPARQL CONSTRUCT query (buildEdgeGraph()) extracts from the ontology which services accept which data types.
2. A breadth-first search over these links (findChain()) starts from the input features of the intent. At each step, the output features of a service must match the input features required by the next one. Features a service preserves are carried forward. The first sequence that reaches the declared output features, which is the one with the fewest services, becomes the composition plan.

For the three evaluated workflows, the composition plans are:

• UC1: PDF -> AWSTextract -> Text -> AWSPolly -> MP3
• UC2: MP3 -> AWSTranscribe -> Text -> AWSTranslate -> Text -> AWSPolly -> MP3
• UC3: MP4 -> AWSElementalMediaConvert -> MP3 -> AWSTranscribe -> Text

Data types and services alternate. In UC1, Textract turns the PDF into text, and Polly turns the text into MP3.

Evaluation setup

• Model: Claude 3.5 Sonnet v2 on Amazon Bedrock (us.anthropic.claude-3-5-sonnet-20241022-v2:0). This model has since been deprecated, so runs with a current model are not directly comparable to the reported results.
• Temperature: set in ClaudeLLMService.java (default 0.7). The evaluation ran each use case ten times at each of 0.0, 0.4, 0.7, and 1.0, giving 120 runs.
• Documentation-based prompt tuning: for AWS, the prompt includes the official MediaConvert job settings when the plan contains MediaConvert (UC3).

Output validation

The output of every run that ended without an error was checked manually. We first checked the output file for plausibility (type, size, length), then listened to or read its content and compared it with the input. The check targets the integration. The output must carry the content of the input in the requested form and language, for example the transcript rather than the transcription job metadata. Recognition errors of an individual service, such as a misheard word, are outside its scope.

Results format

aws_experiments.csv columns: model;wf;temp;iter;time_s. iter is the number of iterations until the code ran without an error. The one run with iter = 10 reached the iteration cap without success. time_s is the orchestration time in seconds.

The rows of the other three models in this file, generalization_gcp.csv, and the files mistral.py and openai.py in generated_code/ belong to the companion study: S. Ristov et al., "Evaluating large language models for automated serverless workflow generation," in Proc. ACM Int. Conf. Distributed and Event-Based Systems (DEBS), 2026.

Adding a service

Add one entry to services_aws.jsonld with its accepted, produced, and preserved features. Each entry also needs a category field, which must be one of TextServices, SpeechServices, VisionServices, DocumentServices, or MediaServices. Plans are derived per intent at runtime, so no other change is needed. extension_example/ demonstrates this with AWS Comprehend.

Note: The category only groups the services. It does not affect planning or the prompt, so any of the five values works.