Docker Layer Cache doesn't seem to be working in AWS CodeBuild

I have been trying to speed up Docker image building using multistage builds and Docker Layer Cache. However, all the steps get executed at every build and the layer cache doesn't seem to work at all. Here is my build spec, can anyone help identify anything I might be missing here. I am using the build/6.0 image and have the privileged mode turned on as recommended in the docs.

version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging into Amazon ECR
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ECR_IMAGE_URL
      - export DOCKER_BUILDKIT=1
      - docker pull $AWS_ECR_IMAGE_URL:compile-stage || true
      - docker pull $AWS_ECR_IMAGE_URL:$IMAGE_TAG || true
  build:
    commands:
      - echo Building the Docker image ...
      - export PYPI_USER=aws
      - export PYPI_TOKEN=`aws codeartifact get-authorization-token --domain $PYPI_DOMAIN --domain-owner $ACCOUNT_ID --query authorizationToken --output text`
      - cd production_code
      - |
        docker build --target compile-image \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --build-arg PYPI_USER \
        --build-arg PYPI_TOKEN \
        --cache-from=$AWS_ECR_IMAGE_URL:compile-stage \
        --tag $AWS_ECR_IMAGE_URL:compile-stage .
      - |
        docker build --target runtime-image \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --cache-from=$AWS_ECR_IMAGE_URL:compile-stage \
        --cache-from=$AWS_ECR_IMAGE_URL:$IMAGE_TAG \
        --tag $AWS_ECR_IMAGE_URL:$IMAGE_TAG .
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the compile image
      - docker push $AWS_ECR_IMAGE_URL:compile-stage
      - echo Pushing the runtime image
      - docker push $AWS_ECR_IMAGE_URL:$IMAGE_TAG