NVIDIA has released TensorRT Model Connect (TRTMC) in public preview as an Apache‑2.0‑licensed open‑source project that converts a supported Hugging Face or local checkpoint directly into a versioned .bundle artifact for native C++ inference. The workflow requires only two commands—trtmc build and trtmc run—and eliminates the intermediate ONNX export step traditionally needed to move from PyTorch to TensorRT. The build phase, performed in Python, resolves the checkpoint, constructs the TensorRT engine, and packages it into the bundle; the runtime phase executes inference through C++ task APIs such as generate(), transcribe(), generate_image(), embed(), and solve(), allowing deployment in services, embedded systems, or robotics stacks without a PyTorch runtime.
The project ships as a family‑owned set of reference implementations rather than a single generic converter. Hybrid profiles that invoke a helper Python executable declare this dependency explicitly in their manifests. The trtmc inspect command exposes bundle metadata (kind, model family, precision, runtime identity, and engines) for auditability. Supported deployment environments are limited to Linux aarch64 wheels requiring Python 3.10 or 3.12, glibc 2.39 or newer, and TensorRT 11.1.0.106; x86_64 users must rely on a Docker source‑build path.
Why this matters
The removal of the ONNX conversion stage reduces a known source of numerical divergence and integration effort, which is evidenced by the source’s claim that TRTMC avoids “export gaps, repeated per‑model integration, and validation spread across several conversion artifacts.” By delivering a self‑contained .bundle that can be loaded directly via C++ APIs, the approach simplifies deployment pipelines for latency‑sensitive or resource‑constrained environments, aligning with the project’s focus on native inference without PyTorch dependencies. This architectural shift may lower operational overhead for teams targeting edge or embedded hardware, though actual performance gains would depend on benchmark validation not detailed in the announcement.