En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post

Talend to Glue migration

0

Customer using Talend for ETL and Postgres as DWH. They are looking to migrate ETL workload on to Glue ETL and DWH on Redshift. They want to know how to implement the following use cases with AWS Glue:

  1. Extracting data from MS Excel in Glue jobs for further processing.
  2. Extracting data from SharePoint. Currently they use Apigee between SharePoint and Talend.
  3. Data Lineage
  4. Sending email with count of processed/rejected records to Admin when jobs complete or fail.

Please provide your inputs on how to implement above use cases with Glue. Any reference information, learning related to Talend to Glue migration will really help. Thanks.

Sujets
Analytique
Balises
AWS Glue
Langue
English
AWS-User-4429230
demandé il y a 4 ans1,2 k vues
1 réponse
0
Réponse acceptée

This seems like relatively light load use case, in that case, this all can be implemented within pure-Python libs with Python Glue Shell.

  1. Use python libs for excel to extract/process data from excel files.
  2. Use python REST API for Sharepoint to extract Sharepoint data.
  3. Append columns to track source for data lineage.
  4. Get processing counts and sent data over SNS after processing
AWS
Vivek_S
répondu il y a 4 ans
profile picture
EXPERT
Giovanni Lauria
vérifié il y a 8 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents