Build error for nvidia GPG key

Version/Environment (if relevant): This issue affects Ubuntu-based environments (Ubuntu 16.04, 18.04, 20.04, 22.04) that use NVIDIA CUDA repositories in Dockerfiles for custom compute environments. This is relevant to all Domino versions where users build custom environments with NVIDIA/CUDA dependencies. Issue: Build failures occur during apt-get update commands with GPG signature verification errors. Common symptoms include:

Reading package lists... Done
W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Root Cause: NVIDIA’s GPG signing key for their CUDA repositories expired, causing package managers to reject repository updates for security reasons. The old GPG key (7fa2af80) is no longer valid and must be replaced with the updated keyring package. Resolution: Install the new NVIDIA CUDA keyring package in your Dockerfile before any apt-get update operations. Add the following commands early in your Dockerfile:

# Fix NVIDIA GPG key - place before any apt-get update commands
# Remove old repository files if they exist (optional cleanup)
RUN sudo rm /etc/apt/sources.list.d/cuda.list || echo "cuda.list not found, skipping"
RUN sudo rm /etc/apt/sources.list.d/nvidia-ml.list || echo "nvidia-ml.list not found, skipping"

# Install new NVIDIA keyring (adjust Ubuntu version as needed)
# For Ubuntu 18.04:
RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-keyring_1.0-1_all.deb && \
    sudo apt-key del 7fa2af80 && \
    sudo dpkg -i cuda-keyring_1.0-1_all.deb

# For Ubuntu 20.04, use:
# RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb && \
#     sudo apt-key del 7fa2af80 && \
#     sudo dpkg -i cuda-keyring_1.0-1_all.deb

# For Ubuntu 22.04, use:
# RUN wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb && \
#     sudo apt-key del 7fa2af80 && \
#     sudo dpkg -i cuda-keyring_1.0-1_all.deb

Notes/Information:

Ubuntu Version Identification: Check your Ubuntu version in a workspace terminal using: lsb_release -a
URL Pattern: Update the URL to match your Ubuntu version: ubuntu1804 (18.04), ubuntu2004 (20.04), ubuntu2204 (22.04)
Execution Order: These commands must run before any other apt-get update or package installation operations
GPG/PGP/SBT Fixes: If you have other GPG-related fixes in your Dockerfile, place the NVIDIA fix before them
Error Handling: The || echo statements prevent build failures if repository list files don’t exist
Alternative Approach: For environments that don’t require specific CUDA versions, consider using Domino’s pre-built GPU environments from quay.io/repository/domino/

References/Internal Records: For the most current NVIDIA guidance, refer to: https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/