Skip to content
View shivpratapsinghpanwar's full-sized avatar

Block or report shivpratapsinghpanwar

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse

Shivpratap Singh Panwar

I teach humanoid robots to move and machines to see — from scratch, on hardware that actually ships.

Computer Vision Research Engineer · Humanoid RL · Robot Fleet Perception · Edge Inference · Medical AI

Email LinkedIn Kaggle


🤖 23-DoF humanoid ⚡ INT4 on Jetson 🎥 Multi-cam robot fleets 📄 3 peer-reviewed 🔬 500K+ imgs pipelined
locomotion from scratch full quantization stack perception in production IEEE · IBM · Springer 100+ hrs video

🤖 Humanoid locomotion — from scratch, and still going

An ongoing program training whole-body locomotion for the Unitree G1 (23-DoF) with no pretrained policy, no imitation data — reward design up. Task registry, gait mechanics, perturbation curricula, and the training infrastructure are all custom:

graph LR
    A["Task & reward design<br/>(custom registry)"] --> B["Massively parallel sim<br/>MuJoCo Warp · mjlab"]
    B --> C["PPO · RSL-RL<br/>multi-GPU T4"]
    C --> D["Checkpoint / resume<br/>across session limits"]
    D --> E["Failure diagnosis<br/>dedicated analysis runs"]
    E -->|curriculum update| A
Loading
  • Running that is actually running — a speed-adaptive gait clock shortens the cycle with commanded velocity and pushes stance fraction below 0.5, opening a true flight phase. Stock fixed clocks make "running" an unlearnable fast walk; mine makes it a gait.
  • Take hits from any direction — perturbation curriculum mixing instantaneous velocity impulses with sustained horizontal drags from uniformly random headings. The policy doesn't memorize one shove; it learns balance recovery as a skill.
  • Reverse locomotion as a first-class command (lin_vel_x ∈ [−1.2, 2.0]) — walking backwards is trained, not hoped for.
  • Anti-cheat reward shaping — standing-still and feet-air terms that stop the classic failure mode of a robot marching in place to farm gait rewards.
  • Debugging like an engineer, not a gambler — when a policy failed backward pushes, it got its own isolation run, diagnosis, and targeted curriculum fix before retraining. Every experiment checkpointed and resumable across free-tier GPU session limits.
  • Foundation-model track: custom design work on τ0-VLA — world-model-guided test-time computation for robot control. Base stack: my fork of Unitree's RL suite on mjlab/MuJoCo Warp.

🎥 Perception for robot fleets

At Kody Technolab (full-stack robotics company) I build the vision layer for multi-camera robots operating as fleets — security, advertisement, and data-gathering platforms plus the Mahindra Assistant:

graph LR
    A["Multi-camera ingest<br/>sync & calibration"] --> B["Multi-task perception<br/>detect · segment · pose · depth"]
    B --> C["Quantize<br/>FP16 → INT4"]
    C --> D["Edge deploy<br/>Jetson · Android robots"]
    D --> E["Fleet in production"]
    E -->|"data flywheel: field footage → retraining"| A
Loading
  • Multi-task heads sharing one latency budget: detection, instance segmentation, pose and depth estimation running together on embedded compute.
  • The whole quantization ladder — FP16 / INT8 / UINT8 / UINT4 via TensorRT, ONNX Runtime, TFLite, OpenVINO — chosen per platform, per model, per latency budget.
  • Pipelines that have processed 500K+ images and 100+ hours of robot video; preprocessing time cut 40%, production throughput up ~30%.
  • Model strategy per constraint: YOLO family, SAM-1/2/3, D-FINE, RF-DETR, MobileOne×ArcFace, custom architectures — the right tool for the hardware in the room, with classical projective geometry as the fallback where deep learning runs out of data.

🏥 Vision where none exists — medical AI

Building perception for medical robotics targeting rare anomalies — conditions with no existing CV solution and almost no training data when development starts. That's the point: low-data strategies, geometry-first fallbacks where deep learning runs out of examples, and rigorous evaluation on curated clinical splits.

When real medical data runs out, I manufacture it — and measure whether it actually helps, not whether it looks pretty:

  • Pluggable generative backends: SD 1.5 + per-class LoRA, and a from-scratch DDPM with zero natural-image prior for sensitive domains.
  • Every synthetic image is provenance-tracked and screened for memorization of real patient images before it may train anything — privacy treated as a hard gate, not a footnote.
  • Honest paired multi-seed A/B on HAM10000: rare-class augmentation moved vascular-lesion F1 +0.050 ± 0.013 and melanoma recall +0.116 ± 0.048 (full measured results).
  • The entire train→evaluate loop executes remotely on free Kaggle GPUs through a git-pinned execution runner built for autonomous agent iteration — every run reproducible to the commit.

⚡ The toolbox

Quantization FP16 · INT8 · UINT8 · UINT4 — TensorRT, ONNX Runtime, TFLite, OpenVINO
Edge targets NVIDIA Jetson (Nano / Xavier / Orin) · Android robots · custom embedded boards · GPU servers
Perception YOLO family · SAM-1/2/3 · D-FINE · RF-DETR · Faster/Mask R-CNN · pose · depth · anomaly detection
Robot learning MuJoCo Warp · mjlab · RSL-RL (PPO) · reward & curriculum design · sim-to-real thinking
Core PyTorch · TensorFlow · OpenCV · MediaPipe · C++ · Python · classical CV & projective geometry

Public lab notebookKaggle: 50+ notebooks, 19 datasets — D-FINE fine-tuning with ONNX/OpenVINO export, SlowFast action recognition, MoveNet+LSTM pose pipelines, face embedding, and the G1 locomotion runs above, all reproducible.

📄 Publications

  • KrishiDisha: Revolutionizing Agriculture with Intelligent Recommendations using Computer VisionIEEE ICoEIT, Jul 2025. Multi-task CV platform (F1 0.99 / precision 0.96 / R² 0.98), field-validated by 150+ farmers.
  • Web-BCD: A Machine and Deep Learning Approach for Breast Cancer DetectionIBM Technical Report, Dec 2024. 89%→94% accuracy (ROC-AUC 0.96), deployed live for clinician use.
  • Understanding the Patterns of Student Dropout: A ReviewSpringer, Smart Technology, Jun 2024 (chapter).

💼 The short version

ML Engineer · Kody Technolab (Nov 2025 – present; intern Apr–Oct 2025) · DL Research Mentee · IBM India (2024) B.Tech CSE (AI/ML) · Medi-Caps University · CGPA 8.67 · Head of Research & Astronomy, Science Club


I do the boring analysis of research papers, design for compute-constrained reality, merge approaches, and ship the result.

Popular repositories Loading

  1. KrishiDisha.ai KrishiDisha.ai Public

    Forked from AbhishekChourasia29/KrishiDisha.ai

    HTML 2

  2. mediapipe mediapipe Public

    Forked from google-ai-edge/mediapipe

    Cross-platform, customizable ML solutions for live and streaming media.

    C++ 1

  3. Synthetic_Data_Factory Synthetic_Data_Factory Public

    There is one thing that is still unsolved in Deep Learning era that is generating data real enough to be used for training in industry to produce products be it in medical, security or any anomaly …

    Python 1

  4. unitree_rl_mjlab_custom_robot unitree_rl_mjlab_custom_robot Public

    Forked from unitreerobotics/unitree_rl_mjlab

    This is a repository for reinforcement learning implementation for Unitree robots, based on Mujoco.

    C++ 1

  5. tau-0-vla_Custom_Design tau-0-vla_Custom_Design Public

    Forked from sii-research/tau-0-vla

    This repo is the official implementation of "τ0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation".

    Python 1

  6. shivpratapsinghpanwar shivpratapsinghpanwar Public

    Config files for my GitHub profile.