1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
3
Hello,
Seems like you are getting UnsupportedOperationException when reading the parquet data. There might be two cases as far as I aware. Either the underlying parquet file/files might be corrupted or the schema/datatype reference interpreted incorrectly. If you have partitioned data in s3 data source, try reading different data and see if you are getting the same issue when specifically reading particular partitioned data. If the files are not corrupted on the other hand, check if any column is in different type for an example, it may also throw this kind of exception. Refer this Jira - https://issues.apache.org/jira/browse/SPARK-24828
관련 콘텐츠
AWS 공식업데이트됨 2년 전- AWS 공식업데이트됨 2년 전
Thank you!!. I got the issue when querying particular partition, not sure though but I recreated that partition and the issue is resolved.