- 최신
- 최다 투표
- 가장 많은 댓글
Given the error message contains CatalogSink, It seems like the connection to MySQL is failing. There may be some portion that points to the exact error codes. To identify the exact error messages, searching through the ERROR logs with GlueExceptionAnalysisListener
typically would lead me to the exact line that errors and the Error message. In this example, you should be able to find a specific error such as:
- Driver not found
- user name or password incorrect
- database not found
- error in query (table not found, column not found, query malformed, etc)
If you need to troubleshoot, please refer to this link - https://aws.amazon.com/premiumsupport/knowledge-center/glue-test-connection-failed/. To give you some pointers:
- Ensure you are using correct connection parameters:
jdbc:mysql://***aws_connection_url***amazonaws.com:3306/database_name
- Glue connection refers to the correct VPC and subnet (if you are using a VPC)
Also, it would be useful if your Glue version is compatible with the MySQL version you are connecting to. If you are using Glue V3.0, it covers until MySQL V8.0.23. If your version is not compatible, you would have to write your own code.
- Download the JDBC driver for MySQL version you are using and Upload to S3
- Edit the Glue Job properties and add S3 URI in Dependent jars path - e.g. s3://yourS3bucket/path/mysql-connector-java-8.0.21.jar
- Modify the Glue Job e.g. Python script
datasink4 = glueContext.write_dynamic_frame.from_jdbc_conf
(frame = dropnullfields3, catalog_connection = “GLUE-CONNECTION-NAME”,
connection_options = {
"customJdbcDriverS3Path": "s3://yourS3Bucket/path/mysql-connector-java-8.0.21.jar",
"customJdbcDriverClassName": "com.mysql.cj.jdbc.Driver",
"user": “#DBuserName#”, "password": “DbUsersPassword”,
"url":"jdbc:mysql://dbname.url.region.rds.amazonaws.com/schemaName,
"connectionType": "mysql", "dbtable": “tablenameInDb”,
"database": “schemaName”}, transformation_ctx = "datasink4")
관련 콘텐츠
- AWS 공식업데이트됨 2년 전
- AWS 공식업데이트됨 3년 전