Gentlefolks, why is the the token exchange service (TES) failing to fetch credentials? I have greengrass installed on Ubuntu 20.x.x running on a virtual machine. At the end of the numbered items is an error log (truncated) obtained from /greengrass/v2/logs/greengrass.log
. Thank you
What I've done or what I think you should know:
- There exist
GreenGrassServiceRole
which contains AWSGreengrassResourceAccessRolePolicy
, and every other policy containing the word "greengrass" in its name. It also has a trust relationship as below. This was created when I installed Greengrass, but I added additional policies.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "greengrass.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "<MY_ACCOUNT_NUMBER>"
},
"ArnLike": {
"aws:SourceArn": "<ARN_THAT_SHOWS_MY_REGION_AND_ACC_NUMBER>:*"
}
}
},
{
"Effect": "Allow",
"Principal": {
"Service": "credentials.iot.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
aws.greengrass.Nucleus
component is deployed with the following configuration update. The alias also exists.
{
"reset": [],
"merge": {
"ComponentConfiguration": {
"DefaultConfiguration": {
"iotRoleAlias": "GreengrassV2TokenExchangeRoleAlias",
"awsRegion": "us-east-1",
"iotCredEndpoint": "https://sts.us-east-1.amazonaws.com",
"iotDataEndpoint": "<ENDPOINT_OBTAINED_FROM_IOT_CORE_SETTINGS>"
}
}
}
}
-
aws.greengrass.TokenExchangeService
is deployed.
-
There's a custom component that uses the Greengrass SDK to publish to IoT Core. It has the following configuration update.
{
"reset": [],
"merge": {
"ComponentDependencies": {
"aws.greengrass.TokenExchangeService": {
"VersionRequirement": "^2.0.0",
"DependencyType": "HARD"
}
}
}
}
- There is an IoT policy from a previous exercise which attached to the core device (Ubuntu on virtual machine) certificate. It allows all
actions. There's also another
GreengrassTESCertificatePolicyGreengrassV2TokenExchangeRoleAlias
which is associated with the thing's certificate. The policy allows iot:AssumeRoleWithCertificate
ERROR LOG BELOW
2022-05-21T21:07:39.592Z [ERROR] (pool-2-thread-28) com.aws.greengrass.tes.CredentialRequestHandler: Error in retrieving AwsCredentials from TES. {iotCredentialsPath=/role-aliases/GreengrassV2TokenExchangeRoleAlias/credentials, credentialData=Failed to get connection}
2022-05-21T21:08:38.071Z [WARN] (pool-2-thread-28) com.aws.greengrass.tes.CredentialRequestHandler: Encountered error while fetching credentials. {iotCredentialsPath=/role-aliases/GreengrassV2TokenExchangeRoleAlias/credentials}
com.aws.greengrass.deployment.exceptions.AWSIotException: Unable to get response
at com.aws.greengrass.iot.IotCloudHelper.getHttpResponse(IotCloudHelper.java:95)
at com.aws.greengrass.iot.IotCloudHelper.lambda$sendHttpRequest$1(IotCloudHelper.java:80)
at com.aws.greengrass.util.BaseRetryableAccessor.retry(BaseRetryableAccessor.java:32)
at com.aws.greengrass.iot.IotCloudHelper.sendHttpRequest(IotCloudHelper.java:81)
at com.aws.greengrass.tes.CredentialRequestHandler.getCredentialsBypassCache(CredentialRequestHandler.java:207)
at com.aws.greengrass.tes.CredentialRequestHandler.getCredentials(CredentialRequestHandler.java:328)
at com.aws.greengrass.tes.CredentialRequestHandler.getAwsCredentials(CredentialRequestHandler.java:337)
at com.aws.greengrass.tes.LazyCredentialProvider.resolveCredentials(LazyCredentialProvider.java:24)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.resolveCredentials(AwsExecutionContextBuilder.java:165)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:102)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsSyncClientHandler.java:69)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:78)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56)
at software.amazon.awssdk.services.s3.DefaultS3Client.getBucketLocation(DefaultS3Client.java:3382)
at com.aws.greengrass.componentmanager.builtins.S3Downloader.lambda$getRegionClientForBucket$2(S3Downloader.java:134)
at com.aws.greengrass.util.RetryUtils.runWithRetry(RetryUtils.java:50)
at com.aws.greengrass.componentmanager.builtins.S3Downloader.getRegionClientForBucket(S3Downloader.java:133)
at com.aws.greengrass.componentmanager.builtins.S3Downloader.getDownloadSize(S3Downloader.java:115)
at com.aws.greengrass.componentmanager.ComponentManager.prepareArtifacts(ComponentManager.java:420)
at com.aws.greengrass.componentmanager.ComponentManager.preparePackage(ComponentManager.java:377)
at com.aws.greengrass.componentmanager.ComponentManager.lambda$preparePackages$1(ComponentManager.java:338)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.UnknownHostException: https
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at jdk.internal.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:80)
at com.sun.proxy.$Proxy15.connect(Unknown Source)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:253)
at software.amazon.awssdk.http.apache.ApacheHttpClient.access$500(ApacheHttpClient.java:106)
at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:232)
at com.aws.greengrass.iot.IotCloudHelper.getHttpResponse(IotCloudHelper.java:88)
... 27 more
2022-05-21T21:08:38.073Z [ERROR] (pool-2-thread-28) com.aws.greengrass.tes.CredentialRequestHandler: Error in retrieving AwsCredentials from TES. {iotCredentialsPath=/role-aliases/GreengrassV2TokenExchangeRoleAlias/credentials, credentialData=Failed to get connection}
Unfortunately, it wouldn't budge. Thank you for the relatively quick response and provided options. Following the links you provided, I can confirm that I did install with automatic provisioning.
What I did or what I think you should know after your last reply
GreengrassV2TokenExchangeRoleAlias
has 'credentials.iot.amazonaws.com'.I don't know what else to try.
This is the python script that should should oublish to IoT Core
OK, I see that you are not using MERGE correctly. MERGE will change the configuration options using a map. You should not have ComponentConfiguration or DefaultConfiguration as part of MERGE.
So it should look like:
Read more here: https://docs.aws.amazon.com/greengrass/v2/developerguide/update-component-configurations.html#merge-configuration-update