Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,11 @@ with **synchronization semantics** for files and directories.
Designed to facilitate **transparent overlap between computation and I/O operations**, CAPIO-CL allows multiple
producer–consumer application modules to coordinate efficiently using a **JSON-based syntax**.

For detailed documentation and examples, please visit:
👉 [https://capio.hpc4ai.it/docs/coord-language/](https://capio.hpc4ai.it/docs/coord-language/)
For detailed documentation and examples, please visit: [https://capio.hpc4ai.it/docs/coord-language/](https://capio.hpc4ai.it/docs/coord-language/)

---

## 📘 Overview
## Overview

The **CAPIO Coordination Language (CAPIO-CL)** allows applications to declare:
- **Data objects**, **I/O dependencies**, and **access modes**
Expand All @@ -42,7 +41,7 @@ At runtime, CAPIO-CL’s parser and engine components analyze, track, and manage

---

## ⚙️ Building
## Building

### Requirements & dependencies
- C++17 or greater
Expand Down Expand Up @@ -70,7 +69,7 @@ By default, this will:

---

## 📦 Integration as a Subproject
## Integration as a Subproject

**CAPIO-CL** can be included directly into another CMake project using:

Expand Down Expand Up @@ -106,13 +105,13 @@ When included this way, tests and python bindings are **not built**, keeping int

---

## 🐍 Python Bindings
## Python Bindings

CAPIO-CL now provides native **Python bindings** built using [pybind11](https://github.com/pybind/pybind11).
These bindings expose the core C++ APIs—such as `Engine`, `Parser`, directly
to Python, allowing the CAPIO-CL logic to be used within python projects.

### 🔧 Building the Bindings
### Building the Bindings
You can build and install the Python bindings directly from the CAPIO-CL source tree using:

```bash
Expand All @@ -125,7 +124,7 @@ Now you will be able to directly import the package **py_capio_cl** in your proj

---

## 🧩 API Snapshot
## API Snapshot

A simplified example of CAPIO-CL usage in C++:

Expand Down
50 changes: 29 additions & 21 deletions capiocl.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#define HOST_NAME_MAX 1024
#endif

/// @brief Namespace containing all the CAPIO-CL related code
namespace capiocl {
class Serializer;

Expand All @@ -24,13 +25,13 @@ constexpr char CLI_LEVEL_WARNING[] = "[\033[1;33mCAPIO-CL\033[0m";
constexpr char CLI_LEVEL_ERROR[] = "[\033[1;31mCAPIO-CL\033[0m";
constexpr char CLI_LEVEL_JSON[] = "[\033[1;34mCAPIO-CL\033[0m";

// CAPIO streaming semantics

/// @brief Namespace for CAPIO-CL Firing Rules
namespace fire_rules {
constexpr char NO_UPDATE[] = "no_update";
constexpr char UPDATE[] = "update";
} // namespace fire_rules

/// @brief Namespace for CAPIO-CL Commit Rules
namespace commit_rules {
constexpr char ON_CLOSE[] = "on_close";
constexpr char ON_FILE[] = "on_file";
Expand Down Expand Up @@ -60,12 +61,10 @@ inline void print_message(const std::string &message_type = "",

/**
* @brief Engine for managing CAPIO-CL configuration entries.
*
* The CapioCLEngine class stores and manages configuration rules for files
* and directories as defined in the CAPIO-CL configuration file.
* It maintains producers, consumers, commit rules, fire rules, and other
* metadata associated with files or directories.
*
* Each entry in the configuration associates a path with:
* - Producers and consumers
* - Commit and fire rules
Expand All @@ -80,6 +79,8 @@ class Engine {
std::string node_name;
bool store_all_in_memory = false;

/// @brief Internal CAPIO-CL Engine storage entity. Each CapioCLEntry is an entry for a given
/// file handled by CAPIO-CL
struct CapioCLEntry {
std::vector<std::string> producers;
std::vector<std::string> consumers;
Expand Down Expand Up @@ -125,9 +126,7 @@ class Engine {
const auto *getLocations() const { return &_locations; }

public:
/**
* Class constructor
*/
/// @brief Class constructor
explicit Engine() {
node_name = std::string(1024, '\0');
gethostname(node_name.data(), node_name.size());
Expand All @@ -142,7 +141,6 @@ class Engine {

/**
* @brief Check whether a file is contained in the configuration.
*
* The lookup is performed by exact match or by regex globbing.
*
* @param file Path of the file to check.
Expand Down Expand Up @@ -188,9 +186,10 @@ class Engine {

/**
* @brief Add a new file dependency, when rule is commit_on_file
*As a side effect, the file identified by path, has the commit rule set to Commit on Files
*
* @param path
* @param file_dependency
* @param path targeted file path
* @param file_dependency the new file for this the path is subject to commit rule
*/
void addFileDependency(const std::filesystem::path &path,
std::filesystem::path &file_dependency);
Expand Down Expand Up @@ -278,8 +277,7 @@ class Engine {

/**
* @brief Set the dependencies of a file.
*
* Used for commit-on-file rules.
* This method as a side effect sets the commit rule to Commit on Files.
*
* @param path File path.
* @param dependencies List of dependent files.
Expand Down Expand Up @@ -392,26 +390,28 @@ class Engine {
/**
* @brief Check if a file is stored in memory.
*
* @param path File path.
* @param path File path to query
* @return true if stored in memory, false otherwise.
*/
bool isStoredInMemory(const std::filesystem::path &path);

/**
* @brief Check if file should remain on file system after workflow terminates
*
* @param path
* @return
* @param path File path to query
* @return True if file should persist on storage after workflow termination.
*/
bool isPermanent(const std::filesystem::path &path);

/**
* @brief Check for equality between two instances of Engine
* @param other reference to another Engine class instance
* @return true if both this instance and other are equivalent. false otherwise.
*/
bool operator==(const capiocl::Engine &other) const;
};

/**
* @brief Contains the code to parse a JSON based CAPIO-CL configuration file
*
*/
/// @brief Contains the code to parse a JSON based CAPIO-CL configuration file
class Parser {
public:
/**
Expand All @@ -429,7 +429,7 @@ class Parser {
};

/**
* Custom exception for errors occurring within the Parser component
* @brief Custom exception thrown when parsing a CAPIO-CL configuration file by Parser
*/
class ParserException : public std::exception {
std::string message;
Expand All @@ -438,13 +438,21 @@ class ParserException : public std::exception {
explicit ParserException(const std::string &msg) : message(msg) {
print_message(CLI_LEVEL_ERROR, msg);
}

/**
* Get the description of the error causing the exception
* @return
*/
const char *what() const noexcept override { return message.c_str(); }
};

/// @brief Dump the current loaded CAPIO-CL configuration from class Engine to a CAPIO-CL
/// configuration file.
class Serializer {
public:
/**
* Dump the current configuration loaded into the Engine to a CAPIO-CL configuration file.
* @brief Dump the current configuration loaded into the Engine to a CAPIO-CL configuration
* file.
*
* @param engine instance of @class capiocl::Engine to dump
* @param workflow_name Name of the current workflow
Expand Down
4 changes: 4 additions & 0 deletions doxygen/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
generated/**
.Doxyfile
.README.md
*.pdf
Loading
Loading