[BUG]: agentskills.io tools/ directory is ignored by quality, lint, and create-eval-dataset

[mimran-khan]

What happened?

Schema already treats tools/ as the agentskills.io name for executable helpers (schema.py comments, DEFAULT_ALLOWED_SKILL_DIRS includes tools). Quality, script lint, and dataset generation only look at scripts/.

• quality_score.py detect_skill_type uses skill_path / "scripts"
• script_lint.py returns success with No scripts/ directory found
• generate_dataset.py sets scripts = [] unless scripts/ exists

Expected: a spec-compliant skill with tools/run.py is treated as script-based, gets linted, and shows up in create-eval-dataset as a real script.

Actual: quality types it as guide-only, lint-scripts reports no scripts and PASSES, and create-eval-dataset --full --no-llm prints Scripts: ['none']. Put the same file under scripts/ and all three treat it as a script skill.

Reproduction steps

tmp=$(mktemp -d)
mkdir -p "$tmp/tools-skill/tools"
cat > "$tmp/tools-skill/SKILL.md" <<'EOF'
---
name: tools-skill
description: Spec-compliant skill that keeps executables in tools/.
---
# Tools skill
EOF
echo 'print("hello")' > "$tmp/tools-skill/tools/run.py"

skillevaluator lint-scripts "$tmp/tools-skill" -r cli
skillevaluator quality-check "$tmp/tools-skill" -r cli
skillevaluator create-eval-dataset "$tmp/tools-skill" --full --no-llm --force

Observed:

• lint: PASS, No scripts/ directory found
• quality: skill type guide-only (not script-based)
• dataset: Scripts: ['none']

Move run.py to scripts/run.py and rerun. Lint and dataset both see the script. Quality types it as script-based.

SkillEvaluator version or commit

009aa300be7925c7ba75760592baeb941cc29ba8 (0.2.1)

Environment

• macOS 15, arm64
• Python 3.12.2
• uv sync --python 3.12 --all-extras
• No API key (--no-llm)

Before submitting

• I agree to follow this project's Code of Conduct
• I have searched the open issues and found no duplicate of this report

Happy to send a PR that treats tools/ as an executable dir next to scripts/ in quality, lint, and dataset parsing, if that matches the schema contract.