Usando AWS re:Post, accetti AWS re:Post Termini di utilizzo
AWS re:Postre:Post

Google BigQuery Connector for AWS Glue / Request to update to 0.28

0

The Glue Marketplace Connector "Google BigQuery Connector for AWS Glue" is currently at 0.24.2. Can you please update this connector to 0.28? The newest versions provide direct write using the GRPC streaming write, as well as more forgiving schema handling when using this method. Additionally the current 0.24.2 seems to have issues with passing credentials properly when using indirect write (via cloud storage), which is the only option.

Thanks!

https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases Current connector is: https://709825985650.dkr.ecr.us-east-1.amazonaws.com/amazon-web-services/glue/bigquery:0.24.2-glue3.0

Argomenti
Analisi
Tag
AWS Glue
Lingua
English
Chris
posta un anno fa462 visualizzazioni
1 Risposta
0

I managed to get this working with a custom connector using spark-bigquery-with-dependencies_2.12-0.28.0.jar from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases

Looks like that's all the AWS package is anyways. It's even pulling in my AWS secret ok. Having the AWS connector updated would still be nice.

Chris
con risposta un anno fa
  • Khawaja Abdul Ahad
    un anno fa

    Can you elaborate on how did you add this custom connector using the updated version? Did you use the same way as the previous version? I am new to AWS Glue

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande

Contenuto pertinente