Skip to content

How many pipelines compile at once follows the cores, not the memory each compile takes #91

Open

Description

@5hif7y

Related to #7, #15, #83

What happens

Measured on RX 580 (no ray queries, so the software tracer), amdvlk 31.0.21924.61, Windows 10, i5-7400, 16 GB:

  • Each stage of the software tracer takes 1.5 to 5 GB of private memory to compile. After The coverage of a program in a traversal records no texture feedback #84, reflect.comp takes 5.4 GB and 144 s.
  • More threads is slower. create_pipelines with 4 threads instead of 2 took 637 s instead of 620 s: private memory rose to 10 GB, free memory fell to 0.17 GB and Windows paged.
  • Too few threads is slow too. Pipeline_Thread has clamp(cores/2-1,1,16) workers, which is 1 on 4 cores. A cold mirai capture of demo waits for its stages about 17 min, beyond the 600 s limit of an RPC.

Both counts come from the cores, and neither from what a compile costs in memory or what the machine has free.

Expected

The number of compiles running at once is bounded by memory as well as cores, so a cold start neither pages nor leaves cores idle. A fixed ratio from one machine is not the answer (see the review of #15). The bound should come from what the compiles of this run actually took, or from what the system reports free.

🤖 Generated with Claude Code

Activity

  1. self-assigned this
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions