Athena-DocDb connector - Getting "Unable to make field long java.nio.Buffer.address accessible" error

0

I am new to Athena, and referred this page to create Document DB connector.
Background: I had to create the lambda manually by forking the code as the assumed role doesn't have permissions to access the Serverless Application Repository; defined only one environment variable 'default_docdb' that contains the db url. The lambda uses Java 17.

When I access any collection from this database, I see this error from the lambda logs. I found a stackoverlfow question discussing the same issue. Did I miss any configuration steps while deploying the lambda manually?

java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with `--add-opens=java.base/java.nio=ALL-UNNAMED`? (See https://arrow.apache.org/docs/java/install.html)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:143)|
at org.apache.arrow.memory.ArrowBuf.setZero(ArrowBuf.java:1175) |
at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:445)|
at org.apache.arrow.vector.BaseFixedWidthVector.handleSafe(BaseFixedWidthVector.java:850) |
at org.apache.arrow.vector.IntVector.setSafe(IntVector.java:203)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.setValue(BlockUtils.java:309)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:151)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:130)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doGetTableLayout(MetadataHandler.java:406)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doHandleRequest(MetadataHandler.java:283) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:154) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:122) |
at com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:907)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:239)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:191)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:181)|
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @35bbe5e8|
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)|
at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:88) |
... 15 more |```
asked a month ago36 views
1 Answer
1
Accepted Answer

Through the document - https://arrow.apache.org/docs/java/install.html. You can see for Jave 9 or later, some JDK internals must be exposed by adding --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED to the Java.

Explain:

“add-opens” is a command-line option in Java used to resolve module system access restrictions. Since the introduction of the module system in Java 9, certain internal APIs (such as java.nio) are hidden and restricted by default. If a library or program (such as Apache Arrow) needs access to these internal APIs, it will encounter access restrictions, resulting in runtime errors like module java.base does not "opens java.nio" to unnamed module. The command -add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED allows a Java program at runtime to grant the org.apache.arrow.memory.core module access to the java.nio package within the java.base module, thereby lifting the access restriction.

AWS
Eric_L
answered a month ago
profile picture
EXPERT
reviewed a month ago
profile picture
EXPERT
reviewed a month ago
  • Thank you - after adding this option, the error disappeared, but I cannot query any collections. I tried a select * (getting "SELECT * not allowed from relation that has no columns") as well as a select column (getting "Column 'createdat' cannot be resolved or requester is not authorized to access requested resources".

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions