GPU backends
How Maxwell loads its native CUDA FDTD library, which GPUs the wheels target, CUDA graphs, multi-GPU options, and the environment variables involved.
Maxwell's FDTD solver is a native CUDA library driven from PyTorch. This page explains how that library is found and loaded, what hardware it supports, and the options that change how it runs on one or several GPUs.
One native library per platform#
The release wheels for Windows x86_64 and Linux x86_64 each contain one prebuilt FDTD library (witwin_maxwell_fdtd_cuda). It is built against the LibTorch Stable ABI that PyTorch 2.10 introduced, so the same binary works with every PyTorch from 2.10 on, without a rebuild per PyTorch version. It does not depend on the Python version either.
The library is a fat binary built with CUDA 12.8. It contains native code for compute capabilities 7.0, 7.5, 8.0, 8.6, 8.7, 8.9, 9.0, 10.0, 10.1 and 12.0, plus PTX for 12.0, which covers Volta, Turing, Ampere, Ada, Hopper and Blackwell GPUs.
How the library is loaded#
On the first FDTD run in a process, Maxwell:
- Looks for the packaged prebuilt library and its ABI marker file. If the marker matches the expected ABI version, it loads that library and stops.
- Otherwise, builds the CUDA sources that ship in the package with PyTorch's JIT extension builder, into
<temp>/witwin_maxwell_fdtd_cuda/<abi version>, and loads the result. This needsnvccand, on Windows, the Visual Studio C++ build tools.
There is no CPU backend and no automatic fallback. If CUDA is not available, preparation fails with an error.
Environment variables#
| Variable | Effect |
|---|---|
WITWIN_MAXWELL_FDTD_CUDA_SKIP_PREBUILT=1 | Ignore the packaged library and use the JIT build |
WITWIN_MAXWELL_FDTD_CUDA_BUILD_DIR | Folder for the JIT build (default: the temp folder above) |
WITWIN_MAXWELL_FDTD_CUDA_PREBUILT=1 | Load an existing JIT build from the build folder without invoking the compiler, for example under a profiler |
WITWIN_CUDA_GENCODE_ARCHES | Semicolon-separated architectures for a JIT build, such as 8.6;12.0+PTX |
WITWIN_MAXWELL_FDTD_BACKEND | cuda (default) or auto; both select the CUDA backend |
Precision and memory#
FDTD fields and material coefficients are stored in 32-bit floating point on the GPU, and complex results (phasors, Bloch fields) in complex64. Memory grows with the number of cells, the field and material arrays the scene needs (dispersive poles, anisotropy and nonlinearity add arrays), PML regions, and monitors. result.stats() reports the CPML memory mode and allocation, and full_field_dft=True adds one complex array per component and frequency.
CUDA graphs#
By default (Simulation.fdtd(cuda_graph=True)), Maxwell captures the field-update core of one time step into a CUDA graph and replays it, which removes most kernel-launch overhead on small and medium grids. If a scene cannot be captured, it falls back to ordinary kernel launches. result.stats()["cuda_graph_active"] tells you which path ran.
More than one GPU#
Maxwell has three ways to use several GPUs, for three different problems:
| Problem | Feature | Entry point |
|---|---|---|
| One simulation too large or slow for one GPU | Domain decomposition along one axis, with halo exchange between devices | Simulation.fdtd(..., parallel=FDTDParallelConfig(devices=("cuda:0", "cuda:1"), decomposition_axis="x")) |
| Many independent simulations | Ensemble scheduling, one task per GPU | run_many(simulations, execution=MultiGPUExecution.ensemble(devices=...)) |
| Many variants of one scene (sweeps, design populations) | Batched lanes that share compiled data, on one or several GPUs (experimental) | Simulation.batch(simulation, execution=BatchExecution(devices=...), parameters=...) |
FDTDParallelConfig.transport chooses how devices exchange halos: "cuda_p2p" (peer-to-peer in one process), "nccl" (one process per GPU), or "auto". Multi-GPU FDTD runs without CUDA graphs and has its own distributed adjoint for supported trainable scenes. See Batch and multi-GPU runs and the Execution API.