check-cucumber pull fails on empty filename and loses features with duplicate source paths

[PierrickTassery]

Summary

check-cucumber pull can fail and/or export an incomplete set of .feature files when the Testomat BDD project contains:

1. a suite with no stored source file path; and/or
2. multiple BDD suites sharing the same source .feature path.

This was found while validating the new BDD pull workflow from github:testomatio/check-cucumber#0.7.0.

Environment

• macOS
• Node/NPM via npx
• npx check-cucumber@latest from npm resolves to 0.6.0 and does not expose pull
• npx github:testomatio/check-cucumber#0.7.0 --help exposes pull and push, although the CLI still prints v0.6.0

Command used

TESTOMATIO=<project-api-key> npx github:testomatio/check-cucumber#0.7.0 pull -d features

Also reproduced when running from inside the target folder:

cd features
TESTOMATIO=<project-api-key> npx github:testomatio/check-cucumber#0.7.0 pull

Actual result

The pull starts creating files, then fails with:

❌ Failed to pull files: EISDIR: illegal operation on a directory, open '.'
Pull failed: EISDIR: illegal operation on a directory, open '.'

A dry-run shows one blank file path:

Would-create lines: 46
Blank would-create lines: 1
Valid would-create lines: 45

API investigation

check-cucumber pull calls:

GET /api/test_data?api_key=<project-api-key>&with_files=true

The response contains tests, suites, and files.

Case 1: empty filename

One entry in data.files had an empty string as the key:

{
  "files": {
    "": "Feature: Some feature created without a source path\n..."
  }
}

In data.suites, the same suite appears with a key starting with #, which looks like an empty source path followed by the feature title:

#Some feature created without a source path => @Sxxxxxxxx

The CLI then tries to write path.join(targetDir, ""), which resolves to the target directory itself / ., causing the EISDIR failure.

Case 2: duplicate source paths cause lost feature files

data.files is an object keyed by filename/path:

{
  "features/example.feature": "feature content"
}

But data.suites can contain multiple suites with the same source path:

features/example.feature#Feature A => @Saaaaaaaa
features/example.feature#Feature B => @Sbbbbbbbb

Because data.files is keyed by path, only one content can be represented for features/example.feature. The other feature is not present as a separate file in data.files.

The Testomat ZIP export handles this kind of collision by generating unique suffixed filenames, for example:

features/example_aaaaaaaa.feature
features/example_bbbbbbbb_2.feature

But /api/test_data?with_files=true does not return all these feature files uniquely in data.files.

Impact

For the project where this was observed:

• ZIP export contained more .feature files than the API response.
• API response contained one invalid empty filename.
• Some source paths had suite collisions.
• check-cucumber pull failed with EISDIR.
• After manually skipping the empty filename, local export was still incomplete compared with ZIP export.

So check-cucumber pull is not currently fidelity-safe for this BDD project.

Expected behavior

Ideally, check-cucumber pull should be able to export all BDD features that the Testomat ZIP export can export.

Potential fixes:

1. API: do not return files[""]; generate a valid filename when a suite has no source path.
2. API: avoid returning files as a path-keyed object when duplicate source paths are possible, or include a unique filename per suite.
3. CLI: validate filenames before writing and skip/fallback on invalid names instead of crashing.
4. CLI/API: align pull output with ZIP export behavior, including unique suffixes for duplicate source paths.
5. Publish the pull-enabled version to npm, since check-cucumber@latest does not currently expose pull.

Additional note

This may be partly an API/backend issue, but it currently surfaces through the check-cucumber pull user workflow, so I am reporting it here first.

I can provide the exact project-specific examples privately if needed.