How do I troubleshoot issues associated with connecting Git repositories to SageMaker notebook instances?

Resolution

To add a new Git repository to your SageMaker notebook instance, see Add a Git repository to your Amazon SageMaker account.

If you try to add a Git repository that's not hosted on AWS CodeCommit, then you might get the following errors.

Errors associated with connectivity

If there are connectivity issues, then you might get one of the following errors:

• "fatal: unable to access <Git repository URL>: the requested URL returned error: 403"
• "fatal: unable to access <Git repository URL>: failed to connect to ...: port 443: connection timed out"
• "fatal: unable to access <Git repository URL> could not resolve the host:"

To resolve these errors, do the following:

• Verify the notebook instance connectivity to the Git repository.
• Verify the accuracy of the Git repository URL.

Verify the notebook instance connectivity

Be sure that your notebook instance has successfully connected to the Git repository. This repository might be located either on an on-premises Git repository server or a public cloud server, such as GitHub, Bitbucket, or GitLab.

Check the network configuration that your notebook instance uses. The notebook instance might have one of the following network configurations:

• The instance isn't attached to an Amazon Virtual Private Cloud (VPC).
• Your Amazon VPC has direct internet access.
• Your Amazon VPC doesn't have direct internet access.

Then, verify that the subnets, security groups, network access control lists (NACLs), NAT gateway, internet gateway, and route tables are configured correctly. For more information, see Understanding Amazon SageMaker notebook instance networking configurations and advanced routing options.

Verify the accuracy of the Git repository URL

Be sure that the Git repository URL that you add is in the following format:

• GitHub URL: https://github.com/workspace/repository.git
• Bitbucket URL: https://bitbucket.org/workspace/repository.git
• GitLab URL: https://gitlab.com/workspace/repository.git

Replace workspace with your Git workspace name, and repository.git with the code repository name that has the *.git file extension.

Errors associated with authentication

If there are authentication issues, then you might get one of the following errors:

• "fatal: could not read Username for <Git repository URL>: terminal prompts disabled"
• "fatal: could not read Password for <Git repository URL>: terminal prompts disabled"
• "fatal: authentication failed for <Git repository URL>"

To resolve these errors, do the following:

• Verify the authentication details.
• Verify that the AWS Identity and Access Management (IAM) user or roles has the required permissions to access AWS Secrets Manager.

Verify that the authentication details are accurate

Be sure that the username and password are accurate in Secrets Manager.

If you use GitHub or GitLab and have two-factor authentication turned on, then you must use a personal access token instead of a password. After the token is generated, you must update the password in Secrets Manager with the new token. For more information, see Managing your personal access tokens on the GitHub website, or Personal access tokens on the GitLab website.

Bitbucket doesn't support account passwords for Git authentication. It's a best practice to create App passwords in your personal settings. After you create the App password, you must update the password in Secrets Manager.

Verify that the notebook instance's IAM role has the required permissions

Verify that the IAM role has the secretsmanager:GetSecretValue permission attached in its IAM policy. If the IAM role doesn't have the required permissions, then you can add the following policy:

{  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": [
        "example-secret-ARN"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "secretsmanager:ListSecrets",
      "Resource": "*"
    }
  ]
}

Note: example-secret-ARN is the ARN in Secrets Manager that has the username and password for the Git repository authentication.

Errors associated with storage

If there are storage related issues, then you might get the following error: "No space left on device while cloning Git repo. Please try using a larger volume".

To resolve this error, verify that the notebook instance has enough storage. The size of the free space in the notebook instance's volume must be larger than that of the Git repository that you use to clone. If you run out of disk space, specify a larger value for VolumeSizeInGB. For more information, see Customize your notebook volume size, up to 16 TB, with Amazon SageMaker.

Related information

Associate Git repositories with SageMaker notebook instances

Using AWS CodeCommit with interface VPC endpoints