throughput-engine-v2 Tasks

1. OpenSpec and failing tests

  • 1.1 Add the throughput-engine-v2 proposal, design, tasks, and spec delta
  • 1.2 Add failing tests for allocator mode controls and capability reporting
  • 1.3 Add failing tests for scheduler graph execution state
  • 1.4 Add failing tests for true batch execution metadata and operator invocation semantics

2. Runtime throughput features

  • 2.1 Add async-ready device allocator mode controls to MemoryManager
  • 2.2 Add graph capture/replay state and controls to DAGScheduler
  • 2.3 Rework Pipeline::executeBatch() into a real batch execution path
  • 2.4 Update built-in operators for fixed-shape batch runtime support

3. Observability and validation

  • 3.1 Turn profiling seam markers into concrete scheduler instrumentation
  • 3.2 Add a lightweight benchmark target or benchmark mode
  • 3.3 Update CI and docs for GPU-capable validation paths
  • 3.4 Run build/tests and close the change