To download zipped file from S3 bucket to EC2 location

0

Hi AWS, I am writing the code for CI/CD pipeline to download the zipped folder from S3 bucket to EC2 instance location using GitHub Actions. I have tweaked my code in order to keep the artifacts instead of overriding the file. The format is like s3://<bucket-name>/<folder-name>/file.zip where the folder name is 4 digits git-commit sha. Now while I am trying to download the zipped file from s3 bucket to ec2 instance, the command is executed successfully with no error but the file is not getting downloaded. I have tried aws cp --recursive and aws sync command but nothing worked out.

This is the pipeline code:

name: "Deploying a CI/CD for .NET sample app in ASP.NET 4.7 using GitHub Actions and store artifacts in Amazon S3"

on:
  workflow_dispatch:
  push:
    branches: [ "master" ]
  pull_request:
    branches: [ "master" ]

env:
  S3_BUCKET_NAME: ${{ secrets.S3_BUCKET_NAME }}
  ZIP_DOWNLOAD_LOCATION: ${{ secrets.ZIP_DOWNLOAD_LOCATION }}
  
jobs:
  build:
    runs-on: windows-latest
    defaults:
      run:
        shell: cmd

    steps:
    - name: Checkout code repository
      uses: actions/checkout@v3
  
    - name: Setup MSBuild
      uses: microsoft/setup-msbuild@v
    - name: Setup NuGet
      uses: NuGet/setup-nuget@v1.0.
    - name: Restore Packages
      run: nuget restore HelloWorldSampleApp.sl
    - name: Build solution
      run: msbuild HelloWorldSampleApp.sln /p:Configuration=Release /p:DeployOnBuild=true

    - name: Set AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ${{ secrets.AWS_REGION }}
        
    - name: Get Git commit SHA and upload artifacts to Amazon S3
      shell: powershell
      env:
        BUILD: ${{ secrets.BUILD }}
      run: |
        $output = git rev-parse --short=4 HEAD
        aws s3 cp ${{ env.BUILD }} s3://${{ env.S3_BUCKET_NAME }}/$output/

  deploy:
    needs: build
    runs-on: [ self-hosted, Windows, X64 ]
    defaults:
      run:
        shell: cmd
    
    steps:
      - name: Set AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}
       
      - name: Download the zip file from S3 to EC2 folder
        shell: powershell
        run: |
          $latest_folder = aws s3 ls ${{ env.S3_BUCKET_NAME }} --recursive | Sort-Object LastWriteTime | Select-Object -Last 1
          C:\"Program Files"\Amazon\AWSCLIV2\aws s3 sync s3://${{ env.S3_BUCKET_NAME }}/$latest_folder ${{ env.ZIP_DOWNLOAD_LOCATION }}

Here is the policy attached to the IAM user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AccessToGetBucketLocation",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "AccessToWebsiteBuckets",
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketWebsite",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket-name>",
                "arn:aws:s3:::<bucket-name>/*"
            ]
        }
    ]
}

Please help.

profile picture
Arjun
質問済み 8ヶ月前413ビュー
1回答
0

Where is your GitHub actions running? On GitHub cloud runners?

profile picture
エキスパート
回答済み 8ヶ月前
  • Hi Gary, the first one is GitHub Hosted Runner where the build job is running but the step where I am facing issue is running on GitHub self-hosted runner which is configured on EC2 Windows instance.

  • Hi @Gary Mclean, I am trying to run git commands on windows self-hosted runner which is configured on Amazon EC2 Windows instance. I have installed the git bash on the same machine where my runner is configured and set it's path in the Environment Variables as well.

    Now when I am running this piece of code:

    deploy:
      needs: test # This test job is deployed on GitHub Hosted Runner
      runs-on: [ self-hosted, Windows, X64 ]
      defaults:
        run:
          shell: cmd
    
      steps:
        - name: Checkout code repository
          uses: actions/checkout@v2
        
        - name: Git Commit SHA command
          shell: powershell
          run: |
            $output = C:\"Program Files"\Git\cmd\git rev-parse --short=4 HEAD
            echo $output
    

    I am getting this error: Run C:"Program Files"\Git\cmd\git rev-parse HEAD fatal: not a git repository (or any of the parent directories): .git Error: Process completed with exit code 1.

    I got the solution for this that I need to clone the repository inside the EC2 instance where the runner is configured and post that I need to run git init command to initialize it as a git repo but I have a reason not to clone it as being a client I don't want the third party will able to access my code. Is there any alternative which helps me to get rid of this error?

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ