Is there a hello world on running pydeequ in glue?

0

Hi there I'm trying to use pydeequ in glue (3.0 ETL) using this tutorial.

However I get this error:

2023-07-01 13:57:35,492 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(73)): Error from Python:Traceback (most recent call last):
  File "/tmp/glue_job_integration.py", line 144, in <module>
    .onData(sdf)
  File "/home/spark/.local/lib/python3.7/site-packages/pydeequ/analyzers.py", line 52, in onData
    return AnalysisRunBuilder(self._spark_session, df)
  File "/home/spark/.local/lib/python3.7/site-packages/pydeequ/analyzers.py", line 124, in __init__
    self._AnalysisRunBuilder = self._jvm.com.amazon.deequ.analyzers.runners.AnalysisRunBuilder(df._jdf)
TypeError: 'JavaPackage' object is not callable
2023-07-01 13:57:35,492 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(73)): Error from Python:Traceback (most recent call last): File "/tmp/glue_job_integration.py", line 144, in <module> .onData(sdf) File "/home/spark/.local/lib/python3.7/site-packages/pydeequ/analyzers.py", line 52, in onData return AnalysisRunBuilder(self._spark_session, df) File "/home/spark/.local/lib/python3.7/site-packages/pydeequ/analyzers.py", line 124, in __init__ self._AnalysisRunBuilder = self._jvm.com.amazon.deequ.analyzers.runners.AnalysisRunBuilder(df._jdf) TypeError: 'JavaPackage' object is not callable

I realize the example was made for Sagemaker. Anybody have a suggestion? This is my first time using deequ. Thank for reading!

질문됨 일 년 전368회 조회
1개 답변

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인