For the most recent update on ongoing service disruptions affecting the AWS Middle East (UAE) Region (ME-CENTRAL-1), refer to the AWS Health Dashboard. For information on AWS Service migration, see How do I migrate my services to another region?
Questions tagged with Data Lakes
Content language: English
Filter questions
Select tags to filter
Sort by
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
96 results
Hi Team,
I couldn't find list/details of the tools to which Amazon S3 integrates with by using a S3 connector. Which tools integrate with S3 to provide in-place querying of S3 data (i.e. data shouldn'...
I need to load data from multiple tables in a SQL server to S3 for some batch processing. Can AWS Glue read data from different SQL Server table, generate csv files and zipping it to S3?
And can AWS G...
asked 4 years ago
Hey I am trying to learn what others and what the best practices are with glue for development automation and testing/validation.
I have a large dataset (table) with >1e9 records (rows) in Glue. The tables are partitioned by column A, which is a n-letters subtring of column B. For example:
| A (partition key) | B | ... |
| --- ...
Hi, I'm relatively new to AWS glue and was having trouble in the following transformation codes:
```
DataSource4 = glueContext.create_dynamic_frame.from_catalog(database = "beta", table_name = "[tabl...
We are exploring usecases where we want to achieve in-place transformation and querying of S3 data lake data. We don't want to provision database and create tables (so we are not keen to consider Reds...
I have a table with columns A, B, C, D, ..., where A is a partition key. In a Glue job I want to group records of this table by column A. Is there a way to make the glue workers aware of the partition...
What is the best way to scale cross-account AWS KMS–encrypted Amazon S3 bucket access using ABAC?
Tag Name – scaling-cross-account-kms-encrypted-s3-access-using-ABAC
asked 4 years ago
My blueprint needs require an S3 PutObject event to start a Glue ETL job. I only see On-Demand and schedule based triggers as options when creating a blueprint. Does anyone know of a method to creat...
I have an AWS Data Lake that is ready to be used at the moment.
My use case for the Data Lake is to be able to, ingest data from different API connectors (coming from other data vendors and service ...
I ran an Lake Formation BluePrint and result was as follows:
| method to run | result |
| --- | --- |
| by hand | COMPLETED |
| scheduled | IMPORT FAILED(NoCredentialsError: Unable to locate credent...
Hi all,
I'm using a glue job that reads CSV files from S3 and injects data to MySQL RDS.
at the end of my pyspark glue script, I call a stored procedure. the issue is the glue job status end by succ...