TornadoVM vs CUDA C++
GPU computing in Java has traditionally meant one of two paths: accept CPU-only performance, or switch to CUDA C++ and maintain JNI bindings between the two codebases. The NVIDIA CUDA toolkit is free to download, but the real cost is the second language, the separate build toolchain, and the maintenance burden when JDK versions change and JNI signatures break.
TornadoVM keeps the entire codebase in Java. The same @Parallel or KernelContext method that runs on the CPU in development compiles to CUDA PTX at runtime and executes on the GPU in production. Teams that have written CUDA-style kernels before can adopt the KernelContext API directly; teams new to GPU programming can start with @Parallel and let TornadoVM infer the launch configuration.
On NVIDIA hardware, TornadoVM also calls into cuBLAS, cuFFT, and cuDNN directly from the TaskGraph, so there is no longer a performance reason to drop to CUDA C++ for library-heavy workloads. The SGEMM benchmark on an RTX 4090 shows 160 TFLOP/s with cuBLAS FP16 through the Hybrid API, against 5.8 TFLOP/s for TornadoVM's own JIT kernel.
TornadoVM vs TensorRT
TensorRT is NVIDIA's proprietary ML inference optimizer. It delivers high throughput on NVIDIA GPUs by applying layer fusion, precision calibration, and engine serialization, but it requires C++ or Python bindings, an NVIDIA GPU, and a model export step (ONNX or TensorRT engine). It has no Java API and no support for AMD, Intel, or Apple Silicon.
TornadoVM's jitLLM runs Llama 3, Mistral, Qwen 3, Phi-3, and DeepSeek GGUF models inside the JVM via the CUDA backend, reaching 117 tokens per second on an RTX 5090. For teams that need GPU-accelerated inference inside a Java application, with no Python layer and no ONNX export step, jitLLM is an open source path that runs on NVIDIA hardware today.
TensorRT remains stronger for maximum throughput on NVIDIA GPUs for models that benefit from TensorRT-specific graph optimization. TornadoVM is the better fit when the application is Java-first, the GPU mix includes non-NVIDIA hardware, or the team wants to write and debug the same kernel on CPU before running it on GPU.