Skip to content

ECR Pull Through Cache

A pull through cache rule mirrors container images from an upstream registry (like GitHub Container Registry) into the account's private ECR registry. Workloads pull the image from ECR instead of reaching out to the upstream registry directly, which keeps pulls inside AWS and works from subnets that will eventually have no internet access.

The cache is lazy: nothing is copied when the rule is created. The first pull through the ECR URI creates the cached repository and imports the image. Re-pulling a tag (e.g. latest) is what refreshes it — ECR revalidates against the upstream at most once every 24 hours per tag. There is no push-driven sync from the upstream registry.

The image URI follows this pattern, where the prefix comes from the cache rule:

<account>.dkr.ecr.<region>.amazonaws.com/<prefix>/<upstream image path>:<tag>

For example, with a rule prefix of ghcr:

docker pull 271851283454.dkr.ecr.us-west-2.amazonaws.com/ghcr/sapphire-health/ansible-epic/aws:latest

Step 1: Create the Credential Secret

ghcr.io (along with Docker Hub, Azure and GitLab) requires authentication even for public images — AWS rejects rule creation without a credential. Only ECR Public, registry.k8s.io and Quay support anonymous rules.

Create a GitHub PAT with only the read:packages scope (ideally from a service account so it doesn't depend on one person's credentials), then create the secret manually. Like all other secrets in this repo, it is created outside Terraform and only referenced by ARN — see Secrets Manager Overview.

NOTE

  • The secret name must start with ecr-pullthroughcache/ or ECR will refuse to use it.
  • The secret must be in the same account and region as the cache rule.
  • Copy the full ARN from the output — Secrets Manager appends a random 6-character suffix, and ECR requires the exact ARN.
1
2
3
4
5
aws secretsmanager create-secret `
  --name "ecr-pullthroughcache/ghcr" `
  --description "GHCR credentials for ECR pull-through cache" `
  --secret-string '{"username":"<github-username>","accessToken":"<PAT>"}' `
  --region us-west-2

To rotate the PAT later, use aws secretsmanager put-secret-value — the ARN stays the same, so no Terraform change is needed.

Step 2: Define the Cache Rule

The map key becomes the repository prefix in the pull URI (set ecr_repository_prefix to override it). The optional creation_template controls the settings ECR applies to repositories it auto-creates under that prefix — without it, cached repos get AWS defaults (no lifecycle policy at all).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
ecr_pull_through_cache_rules = {
    ghcr = {
        upstream_registry_url = "ghcr.io"
        credential_arn = "arn:aws:secretsmanager:us-west-2:271851283454:secret:ecr-pullthroughcache/ghcr-5uQ9Uu"
        creation_template = {
            description = "Applied to repos auto-created by the ghcr pull through cache"
            //image_tag_mutability must stay MUTABLE so re-pulls of a tag (e.g. latest) can refresh the cached image
            lifecycle_policy = {
                rules = [
                    {
                        rulePriority = 1
                        description = "Expire untagged manifests after 14 days"
                        selection = {
                            tagStatus = "untagged"
                            countType = "sinceImagePushed"
                            countUnit = "days"
                            countNumber = 14
                        }
                        //action defaults to { type = "expire" }, the only valid action type
                    }
                ]
            }
        }
    }
}

The lifecycle_policy is written as regular HCL and converted to the JSON ECR expects by the module. Expiring untagged manifests keeps superseded image versions (each refresh of a tag untags the previous manifest) from accumulating storage costs.

NOTE

  • Do not set image_tag_mutability = "IMMUTABLE" on a cache repo — it prevents ECR from refreshing existing tags, which defeats the purpose of the cache.
  • If the upstream image is multi-architecture, the architecture-specific child images are untagged, so an expire-untagged lifecycle rule will break them. Drop or lengthen the rule for multi-arch images.

Step 3: Grant Pull Permissions

Principals pulling through the cache need two permissions on top of the standard ECR pull permissions (ecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer): ecr:BatchImportUpstreamImage to trigger the import/refresh, and ecr:CreateRepository because the first pull creates the repository. Scope them to the rule's prefix.

The example below adds these to the ECS execution role that pulls the Coder dev container image:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
policy_documents = {
    ecr_pull_through_cache_policy = {
        statement = [{
            effect = "Allow"
            actions = [
                "ecr:BatchImportUpstreamImage",
                "ecr:CreateRepository"
            ]
            resources = ["arn:aws:ecr:us-west-2:271851283454:repository/ghcr/*"]
        }]
    }
}

iampolicies = {
    ECRPullThroughCachePolicy = {
        policy = "ecr_pull_through_cache_policy"
    }
}

iamroles = {
    CoderExecution = {
        policies = [
            "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy",
            "CoderExecutionPolicy",
            "ECRPullThroughCachePolicy"
        ]
        assume_role_policy = "ecs_tasks_assume_role"
    }
}

If a principal without ecr:BatchImportUpstreamImage pulls a cached image, it still gets the cached copy — it just can't trigger a refresh, so the image slowly goes stale.

Step 4: Verify

After applying, confirm ECR can authenticate to the upstream with the stored credential:

1
aws ecr validate-pull-through-cache-rule --ecr-repository-prefix ghcr --region us-west-2

Then trigger the first pull, which creates the cached repository:

1
2
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 271851283454.dkr.ecr.us-west-2.amazonaws.com
docker pull 271851283454.dkr.ecr.us-west-2.amazonaws.com/ghcr/sapphire-health/ansible-epic/aws:latest

Confirm the repository was created with the creation template settings applied:

1
2
aws ecr describe-repositories --repository-names ghcr/sapphire-health/ansible-epic/aws --region us-west-2
aws ecr get-lifecycle-policy --repository-name ghcr/sapphire-health/ansible-epic/aws --region us-west-2

Behavior and Operational Notes

  • Freshness: ECR checks the upstream for a newer version of a tag at most once every 24 hours, so a newly published upstream image can take up to a day to appear even with frequent pulls. To force it immediately, pull by digest or delete the cached image.
  • Cached repositories are not managed by Terraform. The ECR service creates them on first pull; terraform destroy removes the rule and template but leaves cached repositories (and their storage billing) behind — delete them manually with aws ecr delete-repository.
  • Private subnets: pulling from ECR without internet access requires ecr.api and ecr.dkr interface endpoints plus an S3 gateway endpoint (image layers are stored in S3) — see VPC Endpoints. Note that AWS documents that the first pull of an image may still need an internet route; subsequent pulls work fully privately.
  • Cost: the rule and creation template are free; the secret is $0.40/month and cached image storage bills at the normal ECR rate ($0.10/GB-month). Same-region pulls to EC2/ECS/Fargate are free.