Is your feature request related to a problem? Please describe.
no
Describe the solution you'd like
provide a dockerfile like this:
# Hexagon-MLIR 构建镜像(Ubuntu 22.04)
# 用法:docker build -t hexagon-mlir-build .
# 说明:本镜像只准备构建环境 + 下载依赖,真正的 LLVM/Triton 构建由
# scripts/build.sh 在 docker run 时执行(便于挂载缓存、多次迭代)。
# LLVM 全量构建约 1.5~3 小时(8 核)、峰值磁盘 40~60GB,建议挂载缓存卷。
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8
# ---- 基础构建工具 ----
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
cmake \
ninja-build \
git \
wget \
curl \
unzip \
pkg-config \
ccache \
libstdc++-12-dev \
libtinfo5 \
libtinfo6 \
python3-dev \
python3-pip \
python3-venv \
openssh-client \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# ---- 依赖下载目录(所有组件均在 Docker build 阶段预先下载好,后续离线可重复用)----
RUN mkdir -p /opt/deps && cd /opt/deps && \
echo "[1/3] clang+llvm-13.0.1 host toolchain" && \
wget -q https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.1/clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04.tar.xz && \
tar -xf clang+llvm-13.0.1-x86_64-linux-gnu-ubuntu-18.04.tar.xz --strip-components=1 -C /opt/deps && mkdir -p /opt/host_toolchain && \
mv bin lib include share /opt/host_toolchain/ 2>/dev/null; \
echo "[2/3] Hexagon_SDK 6.4.0.2" && \
wget -q https://softwarecenter.qualcomm.com/api/download/software/sdks/Hexagon_SDK/Linux/Debian/6.4.0.2/Hexagon_SDK_lnx.zip -O /opt/deps/Hexagon_SDK_lnx.zip && \
unzip -q /opt/deps/Hexagon_SDK_lnx.zip -d /opt/ && \
echo "[3/3] Hexagon Tools 19.0.02 + HexKL 1.0.0" && \
wget -q https://softwarecenter.qualcomm.com/api/download/software/tools/Hexagon_open_access/Linux/Debian/19.0.02/Hexagon_open_access.Core.19.0.02.Linux-Any.tar.gz -O /opt/deps/tools.tar.gz && \
tar -xzf /opt/deps/tools.tar.gz -C /opt/ && \
wget -q https://softwarecenter.qualcomm.com/api/download/software/tools/Hexagon_KL/Linux/1.0.0/Hexagon_KL.Core.1.0.0.Linux-Any.zip -O /opt/deps/hexkl.zip && \
unzip -q -j /opt/deps/hexkl.zip "hexkl-1.0.0-beta1-6.4.0.0.zip" -d /opt/deps/hexkl_out && \
unzip -q /opt/deps/hexkl_out/"hexkl-1.0.0-beta1-6.4.0.0.zip" -d /opt/deps/hexkl_dir && \
HEXKL_ADDON=$(find /opt/deps/hexkl_dir -type d -name hexkl_addon | head -1) && \
mkdir -p /opt/hexkl && cp -r "$HEXKL_ADDON" /opt/hexkl/hexkl_addon && \
ls -d /opt/Hexagon_SDK/* /opt/Tools/ /opt/hexkl/hexkl_addon /opt/host_toolchain/bin/clang > /dev/null && \
rm -rf /opt/deps
# ---- Python 3.10 环境(与官方 CI 一致;torch-mlir 的 wheel 只构建到 cp310)----
RUN python3 -m venv /opt/mlir-env && \
/opt/mlir-env/bin/pip install --upgrade pip setuptools wheel
ENV PATH="/opt/host_toolchain/bin:/opt/mlir-env/bin:${PATH}" \
CC=/opt/host_toolchain/bin/clang \
CXX=/opt/host_toolchain/bin/clang++ \
HEXAGON_SDK_ROOT=/opt/Hexagon_SDK/6.4.0.2 \
HEXAGON_TOOLS=/opt/Tools \
HEXKL_ROOT=/opt/hexkl/hexkl_addon \
HEXAGON_ARCH_VERSION=75 \
LLVM_PROJECT_BUILD_DIR=/work/llvm-project/build
WORKDIR /work
COPY scripts/build.sh /opt/build.sh
RUN chmod +x /opt/build.sh
and an end2end build script for docker:
#!/usr/bin/env bash
# Hexagon-MLIR 容器内一键构建脚本(宿主机挂载项目模式)
#
# 用法:
# docker build -f docker/Dockerfile -t hexagon-mlir-build .
# docker run --rm -it \
# -v ~/projects/hexagon-mlir:/work/hexagon-mlir \
# -v hexagon-mlir-work:/work/llvm-project \
# hexagon-mlir-build bash /opt/build.sh
#
# 约定:
# - 主仓库已在宿主机 clone 并挂载到 /work/hexagon-mlir —— 本脚本【不 clone 主仓库】
# - triton / triton_shared(含 Qualcomm 补丁)为官方 pin 的 commit,宿主机已通过
# ci/setup_submodules.sh 拉取;容器内仅在缺失/版本不符时 fallback
# - LLVM 是 Triton 的锁定依赖(triton/cmake/llvm-hash.txt),clone 到 /work/llvm-project
# (建议挂命名卷,构建产物 40-60GB 不落到镜像层)
set -euo pipefail
export HEXAGON_MLIR_ROOT=/work/hexagon-mlir
export HEXAGON_SDK_ROOT=/opt/Hexagon_SDK/6.4.0.2
export HEXAGON_TOOLS=/opt/Tools
export HEXKL_ROOT=/opt/hexkl/hexkl_addon
# 先保存用户传入的架构版本(set_local_env.sh 会把它覆盖为 75)
export USER_HEXAGON_ARCH_VERSION=${HEXAGON_ARCH_VERSION:-75}
export HEXAGON_ARCH_VERSION=${HEXAGON_ARCH_VERSION:-75}
export CC=/opt/host_toolchain/bin/clang
export CXX=/opt/host_toolchain/bin/clang++
export PATH="/opt/host_toolchain/bin:/opt/mlir-env/bin:${PATH}"
git config --global --add safe.directory /work/hexagon-mlir
git config --global --add safe.directory /work/hexagon-mlir/triton
git config --global --add safe.directory /work/hexagon-mlir/triton_shared
source /opt/mlir-env/bin/activate
echo "=============================================="
echo " [1/5] 校验挂载的 hexagon-mlir 项目(不 clone)"
echo "=============================================="
if [ ! -d "${HEXAGON_MLIR_ROOT}/.git" ]; then
echo "❌ /work/hexagon-mlir 不是 git 仓库!请确认挂载:"
echo " docker run -v ~/projects/hexagon-mlir:/work/hexagon-mlir ..."
exit 1
fi
cd "${HEXAGON_MLIR_ROOT}"
# 子模块校验:triton 需收敛到官方 pin 的 commit(setup_submodules.sh 内硬编码)
TRITON_PIN_SHORT="df38505e45"
if [ ! -d triton ] || [ "$(git -C triton rev-parse --short HEAD 2>/dev/null)" != "${TRITON_PIN_SHORT}" ]; then
echo "⚠️ triton 子模块缺失或版本不符,运行 ci/setup_submodules.sh(clone + 打补丁)..."
bash ./ci/setup_submodules.sh
else
echo "✅ 子模块已就绪: triton @ $(git -C triton rev-parse --short HEAD), triton_shared @ $(git -C triton_shared rev-parse --short HEAD 2>/dev/null || echo MISSING)"
fi
echo "=============================================="
echo " [2/5] Python 依赖"
echo "=============================================="
pip install --upgrade pip setuptools wheel
pip install -r ci/requirements.txt
echo "=============================================="
echo " [3/5] 构建 LLVM"
echo " 锁定版本: $(cat triton/cmake/llvm-hash.txt)(llvm-project main 快照,2026-01-29 前后)"
echo " 预计 1.5~3 小时 / 40-60GB 磁盘,ccache 已开启,中断可续"
echo "=============================================="
LLVM_HASH_FILE="${HEXAGON_MLIR_ROOT}/triton/cmake/llvm-hash.txt"
[ -f "${LLVM_HASH_FILE}" ] || { echo "❌ 找不到 ${LLVM_HASH_FILE},子模块未就绪"; exit 1; }
LLVM_SHA=$(cat "${LLVM_HASH_FILE}" | tr -d '[:space:]')
export LLVM_PROJECT_BUILD_DIR=/work/llvm-project/build
if [ ! -f "${LLVM_PROJECT_BUILD_DIR}/bin/mlir-opt" ]; then
mkdir -p /work/llvm-project
cd /work/llvm-project
if [ ! -d llvm-project/.git ]; then
# --filter=blob:none: 按需取 blob,避免全量 clone 拉下 ~1GB+ 历史
git clone --filter=blob:none https://github.com/llvm/llvm-project.git
fi
cd llvm-project
git fetch origin
git checkout ${LLVM_SHA}
echo "✅ LLVM 已锁定: $(git log -1 --format='%h %ad %s' --date=short)"
mkdir -p "${LLVM_PROJECT_BUILD_DIR}"
cd "${LLVM_PROJECT_BUILD_DIR}"
# ⚠️ 源码在 /work/llvm-project/llvm-project/llvm/,从 build/ 出发是 ../llvm-project/llvm
cmake -G Ninja ../llvm-project/llvm \
-DLLVM_ENABLE_PROJECTS="llvm;mlir;lld" \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_TARGETS_TO_BUILD="AMDGPU;NVPTX;X86;Hexagon" \
-DCMAKE_BUILD_TYPE="RelWithDebInfo" \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_CCACHE_BUILD:BOOL=ON \
-DLLVM_ENABLE_EH=ON \
-DLLVM_BUILD_EXAMPLES:BOOL=OFF \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-unknown-linux-gnu" \
-DCMAKE_INSTALL_PREFIX="${LLVM_PROJECT_BUILD_DIR}/install"
cmake --build . -j$(nproc)
cmake --install .
fi
echo "=============================================="
echo " [4/5] 构建 Triton + qcom_hexagon_backend(TRITON_PLUGIN_DIRS)"
echo "=============================================="
cd "${HEXAGON_MLIR_ROOT}"
# ⚠️ set_local_env.sh 的 BASE_DIR="$(cd .. && pwd)" 按「项目上一级」推导依赖路径;
# 容器内依赖在 /opt,必须重新指正,否则 HEXAGON_SDK_ROOT 等会指向 /work/... 导致失败
source scripts/set_local_env.sh
# ⚠️ set_local_env.sh 末尾的 "set +euxo pipefail" 会关闭错误处理,必须恢复;
# 否则 Triton 构建失败时脚本仍会继续执行并误报"构建完成"
set -euo pipefail
# ⚠️ set_local_env.sh 会把 HEXAGON_ARCH_VERSION 硬编码为 75,覆盖用户传入值,重新导出
export HEXAGON_ARCH_VERSION=${USER_HEXAGON_ARCH_VERSION:-75}
export HEXAGON_SDK_ROOT=/opt/Hexagon_SDK/6.4.0.2
export HEXAGON_TOOLS=/opt/Tools
export HEXKL_ROOT=/opt/hexkl/hexkl_addon
cd triton
TRITON_BUILD_WITH_CLANG_LLD=1 \
TRITON_BUILD_WITH_CCACHE=true \
LLVM_INCLUDE_DIRS="${LLVM_PROJECT_BUILD_DIR}/install/include" \
LLVM_LIBRARY_DIR="${LLVM_PROJECT_BUILD_DIR}/install/lib" \
LLVM_SYSPATH="${LLVM_PROJECT_BUILD_DIR}/install" \
pip install -e . --no-build-isolation --verbose
echo "=============================================="
echo " [5/5] LIT 测试(无需 NPU 硬件,验证编译侧正确性)"
echo "=============================================="
cd "${HEXAGON_MLIR_ROOT}"
BUILD_DIR=triton/build/cmake.linux-x86_64-cpython-$(python3 -c "import sys;print(f'{sys.version_info.major}.{sys.version_info.minor}')")
if command -v lit >/dev/null 2>&1; then
lit ${BUILD_DIR}/third_party/qcom_hexagon_backend/test/ || echo "(LIT 有失败用例,详见上方输出)"
else
echo "lit 未安装,跳过(pip install lit 后可单独运行)"
fi
echo "=============================================="
echo " ✅ 构建完成"
echo " - 编译器二进制: ${HEXAGON_MLIR_ROOT}/${BUILD_DIR}/third_party/qcom_hexagon_backend/bin/"
echo " - 无 NPU 设备可用: 编译 Triton kernel → 查看 lowering / LLVM-IR / Hexagon 汇编"
echo " - 真实 NPU 执行需 v73/v75/v79/v81 设备 + 访问权限(见 docs/user-guide.md)"
echo " - 注: triton/build 生成在挂载的项目目录内(回写宿主机约 5-10GB)"
echo "=============================================="
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe.
no
Describe the solution you'd like
provide a dockerfile like this:
and an end2end build script for docker:
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.