Sharing files/environments across multiple actions in a codecatalyst workflow

Question

I'm putting together a test workflow working with python files. My workflow has two stages - 
1. Build environment - this installs the required python tools and libraries via pip. I'm doing a `pip install --user` so these files end up under `~/.local`
2. Run tests - the aim of this is to run the actual tests against the code installed from the library.

Questions:
1. When I get to step 2, the files installed in step 1 aren't visible so the tests fail. Am I right in understanding that different steps run in different containers, and if so ...
2. Is it possible to either a) use the same containiner as in step 1 or b) store the files installed in step 1, and restore them to step 2. For b, I've tried caching where I create a cache in step 1 and then try to use it in step 2.

Below is my workflow yaml:

```
---
Name: Initial_workflow
SchemaVersion: "1.0"
Triggers:
  - Type: Push
Actions:
  Build_Phase:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
    Configuration:
      Steps:
        - Run: |
            echo "Install required environment libraries"
            pip install --user -r env_requirements.txt
        - Run: |
            echo "Install required code libraries"
            cd src
            pip install --user -r requirements.txt
        - Run: |
            echo "Debug for python libraries"
            find ~/.local -type d          
    Caching:
      FileCaching:
        PythonLibs:
          Path: ~/.local
    Compute:
      Type: EC2
  Test_42:
    Identifier: aws/managed-test@v1
    Inputs:
      Sources:
        - WorkflowSource
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
    Configuration:
      Steps:
        - Run: echo "Running unittests !"
        - Run: |
            echo "Debug for python libraries"
            find ~/.local -type d             
        - Run: |
            cd src
            python -m pytest
    Caching:
      FileCaching:
        PythonLibs:
          Path: ~/.local
    Compute:
      Type: EC2
    DependsOn:
      - Build_Phase
```

Accepted Answer

Hi there -

For sharing the output of one action with another action in the same workflow, you can define an [artifact](https://docs.aws.amazon.com/codecatalyst/latest/userguide/workflows-working-artifacts.html) and use it as an input in additional actions.

[File caching](https://docs.aws.amazon.com/codecatalyst/latest/userguide/workflows-caching.html) is used to cache data between multiple workflow runs, so I don't think this is the right capability for what you're trying to accomplish based on your description.

I hope that this helps!

Answer

Thanks @JonJ-AWS, artifacts were the answer and all working now :)

For anyone interested, sharing the workflow below:

```
---
Name: Initial_workflow
SchemaVersion: "1.0"
Triggers:
  - Type: Push
Actions:
  Build_Phase:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
      Artifacts:
        - Name: pythonLibs
          Files:
            - /root/.local/**/*
    Configuration:
      Steps:
        - Run: |-
            echo "Install required environment libraries on ${HOSTNAME}"
            pip install --user -r env_requirements.txt
        - Run: |-
            echo "Install required code libraries"
            cd src
            pip install --user -r requirements.txt
    Compute:
      Type: EC2
  Test_Phase:
    Identifier: aws/managed-test@v1
    Inputs:
      Sources:
        - WorkflowSource
      Artifacts:
        - pythonLibs
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
    Configuration:
      Steps:
        - Run: |
            echo "Copying artifacts from build on ${HOSTNAME}"
            mv ${CATALYST_SOURCE_DIR_pythonLibs}/root/.local ~
            export PATH=$PATH:$HOME/.local/bin
        - Run: |
            echo "Running Bandit security scan"
            cd src
            mkdir junit
            bandit -r . -o junit/bandit.xml -f xml
        - Run: |
            echo "Running unit tests"
            python -m pytest
    Compute:
      Type: EC2
    DependsOn:
      - Build_Phase
```

Answer

RE:  "Am I right in understanding that different steps run in different containers?"  
I will assume that you are referring to different actions (e.g. "Build_Phase" and "Test_Phase").
If that's correct then yes, they run in different containers.

RE: "Is it possible to use the same container [to run both actions]?"  
This isn't supported today. If this experience is preferable, we'd know to know that and your reasoning :)

Sharing files/environments across multiple actions in a codecatalyst workflow

相关内容