Feature/runner setup

.github/workflows/ci.yml

@@ -21,3 +21,35 @@ jobs:
           pip install flask pytest
       - name: Run tests
         run: pytest -q
+
+  smoke-termux:
+    # This job is intended to run on a self-hosted Termux runner (Android device)
+    runs-on: [self-hosted, termux-android]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Make smoke script executable
+        run: chmod +x tests/integration/smoke_termux.sh
+      - name: Run Termux smoke test (self-hosted)
+        run: |
+          ./tests/integration/smoke_termux.sh
+
+  smoke-ubuntu:
+    # Best-effort smoke test on ubuntu-latest. This tries to install udocker and run the same smoke script.
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+      - name: Install udocker (best-effort)
+        run: |
+          python -m pip install --upgrade pip
+          pip install udocker || true
+          udocker --version || true
+      - name: Make smoke script executable
+        run: chmod +x tests/integration/smoke_termux.sh
+      - name: Run smoke test (best-effort)
+        run: |
+          ./tests/integration/smoke_termux.sh

QUICKSTART.md

@@ -120,6 +120,17 @@ sqlite3 udocker_state.db
 
 ---
 
+## CI smoke test (optional)
+
+You can run an integration smoke test on an Android Termux device by registering it as a self-hosted runner (label it `termux-android`) and running the `smoke-termux` job in CI, or run it locally:
+
+```bash
+chmod +x tests/integration/smoke_termux.sh
+./tests/integration/smoke_termux.sh
+```
+
+---
+
 ## ✨ Features
 
 ✅ **40+ Docker API endpoints**  

README.md

@@ -124,6 +124,17 @@ curl -X POST http://localhost:2375/v1.52/containers/create \
 curl http://localhost:2375/v1.52/containers/CONTAINER_ID/logs
 ```
 
+Options: `tail`, `since` (unix seconds), `timestamps=1`, `follow=1` (stream), `multiplex=1|0` (force multiplex), `heartbeat` (seconds, keepalive), `idle_timeout` (seconds to close follow when idle).
+
+Examples:
+```bash
+# Stream logs with timestamps and follow
+curl -N "http://localhost:2375/v1.52/containers/CONTAINER_ID/logs?follow=1&timestamps=1"
+
+# Get logs since timestamp
+curl "http://localhost:2375/v1.52/containers/CONTAINER_ID/logs?since=1710000000"
+```
+
 ### Stop and Delete a Container
 ```bash
 curl -X POST http://localhost:2375/v1.52/containers/CONTAINER_ID/stop
@@ -268,6 +279,24 @@ For issues:
 3. Check database: `sqlite3 udocker_state.db`
 4. Test endpoint: `curl http://localhost:2375/_ping`
 
+## CI Self-hosted Termux Runner (smoke tests)
+
+If you want CI to run a smoke-test on an Android device running Termux, register that device as a **self-hosted runner** in your repository and add the label `termux-android` to it. The workflow includes a `smoke-termux` job which will execute `tests/integration/smoke_termux.sh` on that runner.
+
+Steps:
+
+1. On GitHub, go to your repository > Settings > Actions > Runners > Add runner and follow the registration steps for your Android device (Termux supports the runner binary via `chmod +x` and running the provided script).
+2. When registering the runner, add the label `termux-android` (so the job matches `runs-on: [self-hosted, termux-android]`).
+3. Ensure `udocker` is installed on the device and available in PATH.
+4. Run the `smoke-termux` job from a PR or workflow run; it will start the dashboard (if not running), exercise container create/start/logs/stop/delete, and report success or failure.
+
+You can also run the test locally on the device:
+
+```bash
+chmod +x tests/integration/smoke_termux.sh
+./tests/integration/smoke_termux.sh
+```
+
 ---
 
 **Created**: 2025-12-21

RUNNER_STATUS.md

@@ -0,0 +1,21 @@
+# Runner Registration Status
+
+Repository: xeniosrahi/Termux-Udocker-API
+Branch: v1.52
+
+Runner Registered: false
+Runner Label: termux-android
+Runner Name: termux-android-1 (suggested)
+
+Notes:
+- Use this file to mark whether a Termux self-hosted runner has been registered for CI smoke tests.
+- To register the runner, follow the instructions in `README.md` > "CI Self-hosted Termux Runner (smoke tests)" or the `docs/SDK-vs-shim.md` runner section.
+- After successful registration and a green smoke run, update `Runner Registered: true` and optionally record the runner name & timestamp.
+
+Example record after registration:
+
+Runner Registered: true
+Registered At: 2025-12-21T12:34:56Z
+Runner URL: https://github.com/xeniosrahi/Termux-Udocker-API/actions/runners
+Runner Notes: udocker installed; tested smoke-termux job
+

core/container_manager.py

@@ -5,6 +5,8 @@
 import threading
 from datetime import datetime
 from db import ContainerDB
+import json
+import models
 
 db = ContainerDB()
 
@@ -125,7 +127,7 @@ def inspect_container(cid):
             "Created": datetime.fromtimestamp(db_info['created_at']).isoformat() + "Z",
             "PortBindings": ports,
             "RestartCount": db_info['restart_count'],
-            "Env": (lambda e: [f"{k}={v}" for k, v in e.items()] if isinstance(e, dict) else e)(json.loads(db_info['env_vars'] or '[]'))
+            "Env": models.normalize_env(json.loads(db_info['env_vars'] or '[]'))
         }
 
     @staticmethod

core/dashboard.py

@@ -6,6 +6,7 @@
 import threading
 from flask import Flask, jsonify, request, Response
 from container_manager import ContainerManager, db
+import models
 from datetime import datetime
 import sqlite3
 
@@ -54,10 +55,7 @@ def container_to_json(container, inspect=False):
     if inspect:
         # Normalize Env to a list of strings "KEY=VALUE"
         env_raw = json.loads(container['env_vars'] or '[]')
-        if isinstance(env_raw, dict):
-            env_list = [f"{k}={v}" for k, v in env_raw.items()]
-        else:
-            env_list = env_raw
+        env_list = models.normalize_env(env_raw)
 
         base.update({
             "Created": datetime.fromtimestamp(container['created_at']).isoformat() + "Z",
@@ -259,25 +257,42 @@ def mux_header(stream_type, size):
     def generate_stream():
         sent = 0
         last_ts = since_val or 0
-        timeout = 5  # seconds to poll for new logs before closing
-        start = time.time()
+
+        # Heartbeat and multiplex control
+        heartbeat = int(request.args.get('heartbeat', '15'))  # seconds
+        multiplex_param = request.args.get('multiplex')  # '1'|'0'|None
+        auto_multiplex = stdout and stderr
+        def should_multiplex():
+            if multiplex_param is None:
+                return auto_multiplex
+            return multiplex_param == '1'
 
         # Send current entries first
         entries = db.get_log_entries(container_id, tail, since=since_val)
         for ts, out in entries:
-            line = out
-            if timestamps:
-                line = f"{datetime.fromtimestamp(ts).isoformat()} {out}"
-            if stdout and not stderr:
-                # plain text
-                yield (line + "\n").encode('utf-8')
-            else:
-                # multiplexed: determine stream type
-                stream_type = 1 if stdout else 2
-                payload = line.encode('utf-8') + b"\n"
-                yield mux_header(stream_type, len(payload)) + payload
-            last_ts = max(last_ts, ts)
-            sent += 1
+            try:
+                line = out
+                if timestamps:
+                    line = f"{datetime.fromtimestamp(ts).isoformat()} {out}"
+                if not should_multiplex():
+                    yield (line + "\n").encode('utf-8')
+                else:
+                    stream_type = 1 if stdout else 2
+                    payload = line.encode('utf-8') + b"\n"
+                    try:
+                        yield mux_header(stream_type, len(payload)) + payload
+                    except Exception:
+                        # fall back to plain payload on send error
+                        yield payload
+                last_ts = max(last_ts, ts)
+                sent += 1
+            except GeneratorExit:
+                return
+            except BrokenPipeError:
+                return
+            except Exception:
+                # ignore and continue
+                continue
 
         # Follow: keep streaming until idle_timeout expires or generator is closed by client
         idle_timeout = int(request.args.get('idle_timeout', '300'))  # seconds
@@ -290,18 +305,48 @@ def generate_stream():
                 for ts, out in new_entries:
                     if ts <= last_ts:
                         continue
-                    line = out
-                    if timestamps:
-                        line = f"{datetime.fromtimestamp(ts).isoformat()} {out}"
-                    if stdout and not stderr:
-                        yield (line + "\n").encode('utf-8')
-                    else:
-                        stream_type = 1 if stdout else 2
-                        payload = line.encode('utf-8') + b"\n"
-                        yield mux_header(stream_type, len(payload)) + payload
-                    last_ts = max(last_ts, ts)
-                    last_activity = time.time()
-                    pushed = True
+                    try:
+                        line = out
+                        if timestamps:
+                            line = f"{datetime.fromtimestamp(ts).isoformat()} {out}"
+                        if not should_multiplex():
+                            yield (line + "\n").encode('utf-8')
+                        else:
+                            stream_type = 1 if stdout else 2
+                            payload = line.encode('utf-8') + b"\n"
+                            try:
+                                yield mux_header(stream_type, len(payload)) + payload
+                            except Exception:
+                                yield payload
+                        last_ts = max(last_ts, ts)
+                        last_activity = time.time()
+                        pushed = True
+                    except GeneratorExit:
+                        return
+                    except BrokenPipeError:
+                        return
+                    except Exception:
+                        continue
+
+                # send heartbeat if requested and no data pushed
+                if not pushed and heartbeat and (time.time() - last_activity) >= heartbeat:
+                    try:
+                        hb = b"\n"
+                        if should_multiplex():
+                            try:
+                                yield mux_header(1, len(hb)) + hb
+                            except Exception:
+                                yield hb
+                        else:
+                            yield hb
+                        last_activity = time.time()
+                    except GeneratorExit:
+                        return
+                    except BrokenPipeError:
+                        return
+                    except Exception:
+                        # ignore heartbeat send errors
+                        pass
 
                 # if no new data for idle_timeout, exit
                 if not pushed and (time.time() - last_activity) > idle_timeout:
@@ -313,7 +358,8 @@ def generate_stream():
             # On any other exceptions, stop streaming
             return
 
-    return Response(generate_stream(), mimetype='application/octet-stream')
+    headers = {'Transfer-Encoding': 'chunked'}
+    return Response(generate_stream(), mimetype='application/octet-stream', headers=headers)
 
 @app.route('/containers/<container_id>/stats', methods=['GET'])
 @app.route('/v1.52/containers/<container_id>/stats', methods=['GET'])
@@ -543,16 +589,16 @@ def create_container():
     cid = f"udocker_{name}_{int(time.time())}"
 
     # Normalize port bindings to: { proto: [(host_port, container_port), ...] }
-    ports = {}
-    if data.get('HostConfig', {}).get('PortBindings'):
-        for container_port, bindings in data['HostConfig']['PortBindings'].items():
-            proto = container_port.split('/')[1] if '/' in container_port else 'tcp'
-            container_port_num = int(container_port.split('/')[0])
-            if bindings:
-                host_port = int(bindings[0].get('HostPort', 0))
-                ports.setdefault(proto, []).append((host_port, container_port_num))
+    ports = models.normalize_port_bindings(data.get('HostConfig', {}).get('PortBindings'))
 
     try:
+        # Basic payload validation
+        try:
+            models.validate_container_create_payload(data)
+        except Exception:
+            # allow create to proceed with best-effort defaults if payload is missing Image
+            pass
+
         db.create_container(cid, name, image, ports=ports, env_vars=data.get('Env', {}))
         return jsonify({
             "Id": cid,

core/db.py

@@ -2,6 +2,7 @@
 import json
 from datetime import datetime
 import os
+from models import normalize_env
 
 DB_PATH = 'udocker_state.db'
 
@@ -98,7 +99,7 @@ def create_container(self, cid, name, image, script=None, ports=None, env_vars=N
                 int(datetime.now().timestamp()),
                 'created',
                 json.dumps([]),
-                json.dumps(env_vars or [])
+                json.dumps(normalize_env(env_vars))
             ))
 
             # Add port bindings if provided. Accept several formats: