🏛️ MassToRender: Architectural Multi-View AI Consistency Pipeline

MassToRender is a production-grade, hybrid generative AI framework designed to solve the critical challenge of spatial and material consistency across multi-view architectural visualisations. By bridging generative text-to-image models with precise 3D Structural Latents (LRM), MassToRender converts single-view concepts or volumetric mass models into a complete, synchronized 6-view architectural presentation (Front, Back, Left, Right, Perspective, Aerial) with total geometry and texture fidelity.

---

🎯 The Core Problem & Our Solution

The Challenge

Standard diffusion models excel at synthesizing high-fidelity single-perspective architectural renders. However, they lack inherent 3D spatial awareness. When tasked with generating sequential views (e.g., moving from a front facade to a lateral or rear view), the network introduces catastrophic drift: window alignments break, structural grids shift, floor heights change, and material textures lose coherent scale and distribution.

Our Solution

MassToRender eliminates random probabilistic generation by anchoring the diffusion process to explicit geometric and semantic ground truths:

1. 3D Structural Priors: Utilizes Microsoft TRELLIS (Large Reconstruction Model) to generate sharp, water-tight structural mesh layouts (.glb/.obj) from conceptual inputs, eliminating standard soft Poisson surface artifacts.
2. Deterministic Control Stacks: Extracts multi-view multi-pass geometry from Blender to force spatial conditioning via a coordinated 4-Layer ControlNet Pipeline.
3. Semantic Material Invariance: Implements a dynamic, localized segmentation map parser (color_dict.json) that maps RGB regions directly to targeted structural prompts, guaranteeing that a material chosen for an anchor view propagates flawlessly across all coordinates.

---

⚙️ Pipeline Architecture

The workflow is completely modularized into three execution layers:

[ Conceptual Image / Massing ] 
             │
             ▼
 ┌───────────────────────┐
 │ Stage 1: TRELLIS LRM  │ ──► Generates Crisp Structural 3D Mesh (.glb)
 └───────────────────────┘
             │
             ▼
 ┌───────────────────────┐
 │  Stage 2: Blender     │ ──► Auto-extracts 6 Orthogonal/Perspective Views
 └───────────────────────┘     (Beauty, Depth, Normal, Segmap Passes)
             │
             ▼
 ┌───────────────────────┐
 │ Stage 3: ControlNet   │ ──► Executes Coordinated Diffusion Model Inference
 │    (MassToRender)     │     Using Anchor Seed Style Propagation
 └───────────────────────┘
             │
             ▼
[ Coherent 6-View High-Fi Render Pafta ]

1. 3D Volume Reconstruction (TRELLIS)

• Automated background removal cleans boundary topology.
• TRELLIS transforms raw pixels into structured lattice/slat volumes.
• Outputs clean, sharp-edged solid geometry perfectly optimized for crisp architectural lintels, slabs, and curtain walls.

2. Multi-Pass Extraction (Blender)

The generated mesh is loaded into a local environment where a camera array extracts synchronized orthographic and perspective passes for 6 views (front, back, left, right, perspective, aerial):

• Beauty Pass: Pure clay/white model setup providing lighting gradients and acting as the initialization latent for img2img.
• Depth Pass: Ground-truth distance metrics preventing any volumetric distortion.
• Normal Pass: Detailed surface normals mapping vertex directionality to control precise light orientation and micro-reflections.
• Segmap Pass: Pixel-isolated color regions separating distinct structural materials.

3. Synchronized Diffusion Inference (masstorender_v2.py)

The Python engine runs a multi-conditioned stable diffusion configuration:

• MLSD ControlNet: Automatically extracts structural wireframe linework from the beauty pass to retain clean architectural grids.
• Depth & Normal ControlNets: Enforces 1:1 geometry retention.
• Segmentation ControlNet: Matches real-time color counts to the localized prompt generator.

---

🎨 Dynamic Material Configuration (color_dict.json)

To achieve seamless material synchronization, the pipeline decouples prompt orchestration from the core source code. Material descriptions are mapped using sharp hexadecimal/RGB fields inside a localized dictionary configuration:

{
  "FF0000": "modern brutalist building with exposed structural concrete walls, architectural concrete texture, sharp lintels",
  "808080": "smooth concrete pavement, minimalist architectural courtyard ground, gray stone tiles",
  "00FF00": "manicured green grass lawn, low shrubbery, architectural landscaping foliage"
}

Benefits of this Approach:

• Zero-Touch Code Execution: Swapping an entire building from raw timber cladding to polished limestone requires editing only three lines in a JSON configuration file.
• Absolute Cross-View Invariance: The model evaluates pixel layouts across every view, ensuring the same semantic prompt tags the exact spatial location regardless of the camera matrix.

---

🚀 Getting Started & Local Environment

Prerequisites

• Python 3.10+
• CUDA-capable GPU (Minimum 6GB VRAM supported via advanced execution strategies)
• PyTorch (optimized with WHL cu121)

Local Directory Structure

For clean portability, the pipeline automatically resolves relative execution coordinates. Maintain the following tree directory layout:

E:\MassToRender\
│
├── current_run/               <-- Copy your active Blender pass exports here
│   ├── front_beauty.png
│   ├── front_depth.png
│   ├── front_normal.png
│   ├── front_segmap.png
│   ├── back_beauty.png
│   ├── back_depth.png
│   ├── back_normal.png
│   ├── back_segmap.png
│   ├── left_beauty.png
│   ├── left_depth.png
│   ├── left_normal.png
│   ├── left_segmap.png
│   ├── right_beauty.png
│   ├── right_depth.png
│   ├── right_normal.png
│   ├── right_segmap.png
│   ├── perspective_beauty.png
│   ├── perspective_depth.png
│   ├── perspective_normal.png
│   ├── perspective_segmap.png
│   ├── aerial_beauty.png
│   ├── aerial_depth.png
│   ├── aerial_normal.png
│   └── aerial_segmap.png
│
├── log/                       <-- Automated execution traces
├── outputs/                   <-- High-fidelity multi-view render sheets
├── color_dict.json            <-- Material-to-color configuration map
└── masstorender_v2.py         <-- Core engine file

Installation & Run Routine

1. Setup your virtual environment or Conda container:

   conda create -n MassToRender python=3.10 -y
   conda activate MassToRender
   pip install torch torchvision torchaudio --index-url [https://download.pytorch.org/whl/cu121](https://download.pytorch.org/whl/cu121)
   pip install diffusers transformers controlnet_aux accelerate xformers numpy pillow

2. Export your multi-pass images from Blender into the current_run/ folder.

3. Run the automated inference execution:

   python masstorender_v2.py

Hardware Optimization Strategy

The engine is explicitly engineered for production on local setups utilizing consumer GPUs (e.g., RTX 2060 6GB VRAM):

• enable_model_cpu_offload: Dynamically drops sequential ControlNet components out of active VRAM when not in execution cycle.
• xformers Memory Attention: Implements highly optimized memory scaling.
• VAE Slicing & Tiling: Processes heavy high-res latent maps sequentially to prevent out-of-memory errors.

---

📊 Auto-Anchor Style Propagation

To maintain impeccable lighting and contextual style consistency, the pipeline runs an automated scoring module based on structural complexity:

$$ \text{Score} = (\text{Edge Density} \times 0.6) + (\text{Depth Variance} \times 0.4) $$

The view returning the maximum score is automatically declared the Anchor View. The engine renders the Anchor view using multiple seeds, locks the ideal stylistic latent variance, and propagates that exact environmental style signature to the remaining 5 viewpoints, ensuring perfect visual synchronization across the final presentation layout.

---

✉️ Contact & Freelance Consultation

This pipeline represents a highly advanced, engineering-first approach to commercial architectural visualization automations. For specialized pipeline deployments, Custom LoRA trainings, or workflow scaling optimizations on platforms like Upwork, please reach out via GitHub or professional profiles.