Al usar AWS re:Post, aceptas las AWS re:Post Términos de uso
AWS re:Postre:Post

Google BigQuery Connector for AWS Glue / Request to update to 0.28

0

The Glue Marketplace Connector "Google BigQuery Connector for AWS Glue" is currently at 0.24.2. Can you please update this connector to 0.28? The newest versions provide direct write using the GRPC streaming write, as well as more forgiving schema handling when using this method. Additionally the current 0.24.2 seems to have issues with passing credentials properly when using indirect write (via cloud storage), which is the only option.

Thanks!

https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases Current connector is: https://709825985650.dkr.ecr.us-east-1.amazonaws.com/amazon-web-services/glue/bigquery:0.24.2-glue3.0

Temas
Análisis
Etiquetas
AWS Glue
Idioma
English
Chris
preguntada hace un año462 visualizaciones
1 Respuesta
0

I managed to get this working with a custom connector using spark-bigquery-with-dependencies_2.12-0.28.0.jar from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases

Looks like that's all the AWS package is anyways. It's even pulling in my AWS secret ok. Having the AWS connector updated would still be nice.

Chris
respondido hace un año
  • Khawaja Abdul Ahad
    hace un año

    Can you elaborate on how did you add this custom connector using the updated version? Did you use the same way as the previous version? I am new to AWS Glue

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas

Contenido relevante