rf-detr

RF-DETR's default amp_dtype="auto" selects bf16-mixed because torch.cuda.is_bf16_supported() returns True on T4/V100 GPUs even when bfloat16 is only emulated. T4 (compute capability 7.5) and V100 (7.0) lack native bf16 tensor cores, so emulated bf16 is slower than fp16.

Immediate workaround: force fp16 mixed precision in TrainConfig:

config = TrainConfig(
    dataset_dir=dataset_dir,
    amp_dtype="fp16",
)

Verify that the trainer reports 16-mixed, not bf16-mixed. On a T4, this can reduce training-step time from roughly 443 ms to 226 ms and lower memory usage.

The proper code fix is to check for native bf16 support instead of emulated support:

native_bf16 = torch.cuda.is_bf16_supported(including_emulation=False)
precision = "bf16-mixed" if native_bf16 else "16-mixed"

For older PyTorch versions without the including_emulation argument, fall back to a compute-capability check and use bf16 only for Ampere or newer GPUs, i.e. compute capability >= (8, 0). Explicitly setting amp_dtype="bf16" on T4/V100 should remain allowed but emit a warning recommending "fp16".

Python's -O/-OO mode removes assert statements, so the validation in BestMetricSingle.__init__() is skipped. Replace the assertion in src/rfdetr/training/model_ema.py with an explicit check that runs in optimized mode:

if better not in {"large", "small"}:
    raise ValueError(
        f"'better' must be either 'large' or 'small', got {better!r}"
    )
self.better = better

As a temporary workaround, run without -O, or ensure better is always either "large" or "small". Without the fix, an invalid value may only fail later when isbetter() raises ValueError: Unexpected value for 'better'.

This affects keypoint models because they use manual optimization. In 16-mixed/fp16, PyTorch Lightning keeps gradients multiplied by the GradScaler factor until optimizer.step(), but RF-DETR was clipping them before unscaling. With a scale of 65536, an intended clip value of 0.1 becomes about 1.5e-6, causing AdamW updates to nearly vanish.

Fix: upgrade RF-DETR to a release containing PR #1549 / commit 8f8d2d77. To install that commit directly:

pip install "rfdetr @ git+https://github.com/roboflow/rf-detr.git@8f8d2d77"

The code-level fix is to unscale the optimizer before manual gradient clipping in _step_optimizer, equivalent to calling Lightning's optimizer unscaling method before clip_grad_norm_/clip_grad_value_.

Temporary workarounds:
- Use bf16-mixed on hardware with native bf16 support, such as Ampere or newer GPUs.
- Train keypoints in fp32.
- Avoid explicit precision="16-mixed" / amp_dtype="fp16" on MPS, CUDA fp16, T4, or V100 until upgraded.

Detection and segmentation are unaffected because they use automatic optimization, where Lightning clips after unscaling.

This affects RF-DETR 1.8.0–1.11.0. strip_checkpoint() removes model_config from checkpoint_best_total.pth, so RFDETR.from_checkpoint() rebuilds the model with class defaults such as resolution=384, num_select=300, or dec_layers=2. The trained weights load, but the architecture does not match them.

Permanent fix: upgrade to an RF-DETR release containing the fix that preserves model_config in checkpoint_best_total.pth, or install the patched branch:
pip install -U git+https://github.com/roboflow/rf-detr.git@develop

Immediate workarounds:
1. Load the unstripped source checkpoint instead, usually EMA:
RFDETR.from_checkpoint("output/checkpoint_best_ema.pth")
The source name is stored in the stripped checkpoint as best_total_source.
2. Or reconstruct the model with every non-default architecture argument before loading the stripped weights:
RFDETRNano(pretrain_weights="checkpoint_best_total.pth", resolution=224, num_select=50, dec_layers=3, device="cpu")
You must pass all changed architecture values—not only resolution. Values such as num_queries, mask_downsample_ratio, and dec_layers must also match training or loading can fail or predictions/exports will be incorrect.

This affects RF-DETR 1.9.0–1.10.1 because predict() hardcodes antialias=False, even for models trained with the default torchvision backend, RF-DETR ≤1.4, or Roboflow platform Resize preprocessing, all of which use antialiased resizing.

Fix: upgrade to an RF-DETR release containing PR #1525 (or install the latest develop branch) and explicitly opt in:

pip install -U rfdetr
# or, if the released version does not include the fix yet:
pip install "rfdetr @ git+https://github.com/roboflow/rf-detr.git@develop"

result = model.predict(image, antialias=True)

The default remains antialias=False for compatibility, so passing antialias=True is required for antialiased checkpoints. In tests on such checkpoints, this reproduced RF-DETR 1.8.3 outputs with zero detection differences.

Workaround for 1.9.0–1.10.1: pre-resize the image yourself before calling predict(). Resize to model.model.resolution using PIL bilinear resampling or torchvision.transforms.functional.resize(..., antialias=True), then scale the returned boxes from the resized resolution back to the original image dimensions. This relies on an internal attribute and should be replaced with antialias=True after upgrading.