En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post

Google BigQuery Connector for AWS Glue / Request to update to 0.28

0

The Glue Marketplace Connector "Google BigQuery Connector for AWS Glue" is currently at 0.24.2. Can you please update this connector to 0.28? The newest versions provide direct write using the GRPC streaming write, as well as more forgiving schema handling when using this method. Additionally the current 0.24.2 seems to have issues with passing credentials properly when using indirect write (via cloud storage), which is the only option.

Thanks!

https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases Current connector is: https://709825985650.dkr.ecr.us-east-1.amazonaws.com/amazon-web-services/glue/bigquery:0.24.2-glue3.0

Sujets
Analytique
Balises
AWS Glue
Langue
English
Chris
demandé il y a un an462 vues
1 réponse
0

I managed to get this working with a custom connector using spark-bigquery-with-dependencies_2.12-0.28.0.jar from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases

Looks like that's all the AWS package is anyways. It's even pulling in my AWS secret ok. Having the AWS connector updated would still be nice.

Chris
répondu il y a un an
  • Khawaja Abdul Ahad
    il y a un an

    Can you elaborate on how did you add this custom connector using the updated version? Did you use the same way as the previous version? I am new to AWS Glue

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents