Help us improve the AWS re:Post Knowledge Center by sharing your feedback in a brief survey. Your input can influence how we create and update our content to better support your AWS journey.
All Content tagged with Data Lakes
Content language: English
Filter content
Select tags to filter
Sort by
Sort by most recent
101 results
I am attempting to use Hudi 1.1.1 in an AWS 5.0 Glue job. To avoid conflicts with the pre-installed Glue Hudi libraries, I have set the job parameter `--datalake-formats` to an empty string (""). Howe...
We keep getting following error sometimes when we are wrting data to S3 table. Can anyone let me know what is the issue and work around? Its not happening for all tables but only specific tables only....
I have existing Iceberg tables in gov cloud. I have created them using AWS Glue ETL spark py jobs.
My dbt debug command says "All checks have passed" But my dbt run command gives me the below error:
...
Hi,
We are planning to use S3 table for storing our clients' data but we want to have RBAC-based feature so that we can make sure that data is access based on the permission. We are planning to crea...
Using Airbyte "AWS Datalake" destination to push data into AWS Glue/S3. I have to use the S3 accesspoint ARN, since the location where Airbyte is running does not have open TCP:443 egress. ....so I...
Hi all,
I'm trying to create a Iceberg table through Athena. The detailed S3 bucket path looks like this 's3://my_bucket/company_id={company_id}/date={YYYY-MM-DD}/data.parquet
However, the 'date' ...
I am creating a data mesh architecture on datazone domain with associating multiple accounts from the Central Account but when I am adding IAM role as a user into root domain of Datazone it does not a...
AWS rePost LiveEXPERT
published a year ago0 votes237 views
Don't miss us live on [Twitch.tv](https://bit.ly/4anH9WR) on Monday, November 25th to learn how you can Build Next-Gen Data Platforms using Apache Iceberg.
Darren RobackEXPERT
published a year ago1 votes1.6K views
The purpose of this article is to provide a guide on leveraging AWS Lambda and the Python-Magic library to accurately detect and categorize file types. This approach enables businesses to build more r...
In AWS DMS, I have a Serverless replication, but I want to modify it now justs to add an extra table. No matter what I change, I get this error: Task Settings CloudWatchLogGroup or CloudWatchLogStream...
This is the third time I've run into this error.
I don't know is it a real issue or I just forget to set something up.
So basically when I try to add data permission to all table in Lakeformation ba...
Config of Redshift Cluster:
- Enhanced VPC routing has enabled
- Redshift subnet in the same subnet as S3 vpc endpoint
Config of S3
- VPC endpoints created for S3
- Routing has configured to rout...