I have a glue job that is reading a couple of CSV files form S3. 1. I manually choose examples files to infer the schema, that works fine. 2. Then I add a join action but I don't want to infer the schema form the data preview session, so I manually Edit and Apply the schema, but my changes on the Schema are not saved. 3. This is causing the join action to not have any output 4. if I want to use that join action as input to another join I'm not able apply conditions base on the first join. 5. I'm not using any Catalog or database All of this was working fine a couple of weeks ago, I'm not sure if an update on the Glue Studio can be causing the problem. Any help is really apreciated, thank you!

Glue Job Studio - Output Schema - Edit and Apply does not save changes

I have a glue job that is reading a couple of CSV files form S3.

I manually choose examples files to infer the schema, that works fine.
Then I add a join action but I don't want to infer the schema form the data preview session, so I manually Edit and Apply the schema, but my changes on the Schema are not saved.
This is causing the join action to not have any output
if I want to use that join action as input to another join I'm not able apply conditions base on the first join.
I'm not using any Catalog or database

All of this was working fine a couple of weeks ago, I'm not sure if an update on the Glue Studio can be causing the problem.

Any help is really apreciated, thank you!

Sary Al-Assad
5 个月前
Any luck with this? I'm facing the same issue with empty schema tabs after joins. I've been running the same jobs for months now without an issue and without a need to run the data preview to infer the schema. Suddenly last week this changed and many of my jobs are showing empty schema tabs.

主题

分析

标签

AWS Glue

语言

English

Joel 0

已提问 5 个月前199 查看次数

1 回答

最新
投票最多
评论最多

这些答案有用吗？为正确答案投票，以帮助社区从您的知识中受益。

I have reproduce that for the join field, will report it to the team.
However, bear in mind that the schema is meant to reflect the data, not the other way around, if your schema doesn't match the preview then you need to do mapping/casting as required.

专家

Gonzalo Herreros

已回答 5 个月前

Joel 0
5 个月前
Thanks Gonzalo for your support. I beleive there are other problems with the Join (inner join) https://repost.aws/questions/QUd70zPw7JTkm_YGH9MsFyLw/glue-job-inner-join-unexpected-data-results
Sary Al-Assad
5 个月前
Any luck with this? I'm facing the same issue with empty schema tabs after joins. I've been running the same jobs for months now without an issue and without a need to run the data preview to infer the schema. Suddenly last week this changed and many of my jobs are showing empty schema tabs.
Joel 0
5 个月前
Hi Gonzalo, any unpdate on this? Thank you!

Glue Job Studio - Output Schema - Edit and Apply does not save changes

相关内容