SageMaker Training Jobs Now Support SOCI Lazy-Loaded Images: Pulling a 35 GB Deep Learning Image Drops From Over 7 Minutes to Under 10 Seconds, at the Cost of 3–16 Extra Seconds in the First Epoch

Every job pays for downloading the image

Before a SageMaker AI training or processing job can run your code, it pulls the entire container image from ECR and decompresses every layer to disk. Deep learning images routinely exceed 10 GB, and those minutes are billed as job time without producing any training progress. In hyperparameter sweeps that launch hundreds of short jobs, image download can even take up most of the run time.

AWS announced on October 27 that SageMaker AI training and processing jobs now support Seekable OCI (SOCI) indexed images. SOCI is an open-source technology for lazy-loading containers: an index maps every file in the image, and at startup the job fetches only the files needed to launch the container, streaming the rest in the background as the code runs. The index is stored in ECR as a separate artifact, so the original image digest and signatures stay valid, and jobs fall back to a standard download when an image has no index, so it never breaks a job.

AWS's measurements

• Download time: standard pulls grew from 48 seconds for a 1.8 GB image to 7 minutes 17 seconds at 35.4 GB; lazy loading stayed at 3–11 seconds regardless of size, 8x to 46x faster.
• Across instances: a 13.2 GB image took 3 min 16 s to 4 min 20 s with a standard pull on 13 GPU and CPU instance configurations, versus 6–8 seconds lazy-loaded, about 34x faster.
• Training speed: on ResNet-50 image classification and full-parameter fine-tuning of Qwen2.5-0.5B, the first epoch took 3–16 seconds longer while files were read for the first time; from the second epoch on, epoch times matched the standard approach, for a net saving of 1–6 minutes per job. Even when the entry point ran torch.compile first, or a 6 GB model was baked into the image, jobs still finished more than 3 minutes and 2–2.5 minutes sooner respectively.

How to use it, and when it helps less

The simplest route is switching to an AWS Deep Learning Container tag with the -soci suffix; pre-indexed versions exist for PyTorch Training, TensorFlow Training, vLLM and SGLang. For your own images, you generate an index on a Linux host with containerd and the SOCI snapshotter, or deploy an AWS-provided template that builds an index on every image push. Job definitions, API calls and data channels stay the same.

The limits are spelled out too: only Amazon ECR private repositories are supported; images smaller than about 4 GB see little benefit; if your entry point scans, checksums or preloads most of the image before training, the first epoch waits for nearly the same bytes as a full pull and the net gain is small; and SOCI images run on a containerd-based runtime, so scripts that call the Docker CLI or read /var/run/docker.sock need checking. Images derived from a DLC base also need their own index, since the base image's index doesn't carry forward. The feature is available in all AWS Regions.