Skip to content

Scaling Python 3.8/3.9 → 3.12 Lambda Runtime Migration Using Poetry + GitHub Copilot Agent Skills — Validating Approach Before Full Rollout

0

We are a Platform Engineering team managing a large-scale AWS Lambda estate. AWS has reached End-of-Life for Python 3.8/3.9 runtimes, requiring us to migrate to Python 3.12. The scope is significant:

170+ internal Python libraries packaged and published to AWS CodeArtifact 800+ Lambda functions consuming those libraries Poetry is our standard tool for dependency management, packaging, and publishing

AWS Transform was evaluated but could not be approved internally due to security policy constraints.

Our Proposed Approach:

We are using GitHub Copilot (our approved AI coding assistant) with a custom Agent Skills Library — a curated set of prompt templates and reusable instructions — to standardize and semi-automate the migration in two sequential phases:

Phase 1 — Library Migration Each library's pyproject.toml is updated to target Python 3.12, dependencies are resolved and compatibility-tested, and a new version is published to CodeArtifact.

Phase 2 — Lambda Function Migration Once updated libraries are available in CodeArtifact, each Lambda function is updated to reference the new runtime and library versions, followed by smoke testing and deployment. The Agent Skills Library enforces a consistent migration pattern across all 170+ libraries and 800+ functions, reducing manual effort and human error.

Questions:

  1. Is this a sound and scalable approach for a migration at this scope, or are there known failure patterns we should account for upfront?
  2. Are there AWS-native guardrails or tools (e.g., CodeGuru, Lambda Powertools compatibility checks, or CodeArtifact upstream proxies) that should be incorporated alongside the AI-assisted workflow?
  3. What is the recommended sequencing — should all libraries be migrated and stabilized in CodeArtifact before touching any Lambda functions, or is a progressive, library-by-library rollout alongside function updates preferable?
  4. Are there any Poetry-specific considerations when publishing Python 3.12 packages to CodeArtifact that differ from 3.9 (e.g., wheel compatibility, metadata format changes)?

What We're Looking For: Guidance on whether this approach is architecturally sound at scale, and any AWS best-practice recommendations to complement the AI-assisted workflow before we commit to a full rollout.

1 Answer
1

Your proposed approach is architecturally sound and aligns with AWS best practices for large-scale Lambda migrations. The phased strategy—libraries first, then functions—is the recommended sequencing pattern and will minimize risk and dependency conflicts.

Regarding Your Specific Questions:

  1. Scalability and Failure Patterns: Your approach is sound. The key risk to account for upfront is the time-intensive nature of manual migrations at scale. For 800+ Lambda functions, manual updates would typically require 15-30 minutes per function, translating to 200-500 person-hours of effort. Your AI-assisted automation should reduce this by 97-99%. Critical failure patterns to watch for include: runtime-specific code that may break between Python versions, dependency version conflicts when libraries are updated incrementally, and functions that may have hardcoded references to older library versions.

  2. AWS-Native Guardrails: Yes, several should be incorporated:

  • Implement thorough testing in non-production environments before production rollout
  • Back up all Lambda function configurations before migration
  • Implement the upgrade in phases rather than all at once
  • Verify function behavior after each upgrade
  • Consider implementing logging functionality to track migration progress and issues
  • Add error handling and rollback capabilities to your automation
  • Use a dry-run mode for testing before actual updates
  1. Recommended Sequencing: Your instinct is correct—migrate and stabilize ALL libraries in CodeArtifact before touching Lambda functions. This "libraries-first" approach prevents dependency resolution failures and ensures that when you begin function updates, all required dependencies are available and tested. A progressive library-by-library approach alongside function updates would create a complex dependency matrix that's difficult to manage at your scale and increases the risk of runtime failures.

  2. Poetry-Specific Considerations: While the sources don't detail Poetry-specific differences between Python 3.9 and 3.12 for CodeArtifact, be aware that you'll need to ensure your pyproject.toml files correctly specify Python 3.12 compatibility and that all dependencies are resolved against the new runtime. Test the full publish-consume cycle in a non-production CodeArtifact repository first.

Additional Best Practice Recommendations:

  • Timeline Awareness: After December 15, 2025, Python 3.9 will no longer receive security patches. After February 15, 2026, you won't be able to update existing Python 3.9 functions at all, making this migration time-critical.

  • Phased Rollout Strategy: Consider grouping your 800 functions by criticality and business impact. Migrate lower-risk functions first to validate your process before touching mission-critical workloads.

  • Automated Validation: Build automated smoke tests that can be run against each migrated function to verify basic functionality before considering the migration complete.

  • Documentation and Tracking: Maintain detailed records of which libraries and functions have been migrated, including version numbers and any issues encountered. This will be invaluable for troubleshooting and rollback scenarios.

Your AI-assisted approach with standardized patterns is exactly the right strategy for managing this scale of migration efficiently while maintaining consistency and reducing human error.
Sources
Automated Migration Script for AWS Lambda Python 3.9 End-of-Support | AWS re:Post

answered 18 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.