By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Glue job error - Error while List shards

0

Getting error while connecting streaming data from kinesis to redshift with few transformations using visual ETL. (using amazon kinesis - glue data catalog table as source ). Schema is already defined prior, hence crawler is not used. Error Category: UNCLASSIFIED_ERROR; StreamingQueryException: Query [id = 30dc3a82xxx, runId = ba421exxxx] terminated with exception: Error while List shards

  • Vanishree
    4 months ago

    Thank you. Adding kinesis endpoint worked. If I'm connecting through external source to redshift (eg. psycopg2) getting same kind of error to establish connection to redshift and same to connect tableau. Is it a similar problem?

Topics
AnalyticsDatabaseInternet of Things (IoT)
Tags
AWS GlueExtract Transform & Load DataAmazon RedshiftAmazon Kinesis
Language
English
Vanishree
asked 5 months ago334 views
1 Answer
1
Accepted Answer

Check the driver error logs, that could be a permission issue, that the stream is in a different region or similar

profile pictureAWS
EXPERT
Gonzalo Herreros
answered 5 months ago
profile picture
EXPERT
Oleksii Bebych
reviewed 5 months ago
  • Vanishree
    5 months ago

    Checked region and permissions which seems correct. Tried another method to directly stream into redshift using external schema and materialized view , even then getting error - error: Unable to describe stream code: 42001 ,Timeout was reached Stream name: name query: -1[child_sequence:1] location: kinesis_client.

  • Gonzalo Herreros EXPERT
    5 months ago

    Timeout normally means you are running in a VPC without internet or a kinesis endpoint (could also be the security group but is less likely)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content