Questions tagged with Data Lakes
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I have a database with around 40 tables. However, some end users don't need to see all tables in the database. I'm using Lake Formation Tagging and know that if a tag is added to the database...
1
answers
0
votes
625
views
asked 2 years agolg...
Hi everyone,
I have 270GB of data in my NAS. So what we are doing right now is that we have set up bidirectional sync from dropbox. Through windows explorer, I have given access to NAS to all users....
2
answers
0
votes
510
views
asked 2 years agolg...
I have implemented LakeFormation on my data bucket.
I have a step function in which one step consists of running a GlueJob that reads and writes to the data catalog.
I have upgraded my DataLake...
1
answers
0
votes
6902
views
asked 2 years agolg...
## Problem
I want to know, understand and correct my knowledge, approach on, Setting up an Data Ingestion pipeline, which collects "events" or "data" from any possible external application sources...
1
answers
0
votes
452
views
asked 2 years agolg...
Hi Team,
I couldn't find list/details of the tools to which Amazon S3 integrates with by using a S3 connector. Which tools integrate with S3 to provide in-place querying of S3 data (i.e. data...
1
answers
0
votes
253
views
asked 2 years agolg...
Can AWS Glue read data from different SQL Server table, generate csv files and zipping it to S3?lg...
I need to load data from multiple tables in a SQL server to S3 for some batch processing. Can AWS Glue read data from different SQL Server table, generate csv files and zipping it to S3?
And can AWS...
1
answers
0
votes
514
views
asked 2 years agolg...
Hey I am trying to learn what others and what the best practices are with glue for development automation and testing/validation.
1
answers
0
votes
268
views
asked 2 years agolg...
I have a large dataset (table) with >1e9 records (rows) in Glue. The tables are partitioned by column A, which is a n-letters subtring of column B. For example:
| A (partition key) | B | ... |
| ---...
1
answers
0
votes
249
views
asked 2 years agolg...
Hi, I'm relatively new to AWS glue and was having trouble in the following transformation codes:
```
DataSource4 = glueContext.create_dynamic_frame.from_catalog(database = "beta", table_name =...
1
answers
0
votes
4264
views
asked 2 years agolg...
We are exploring usecases where we want to achieve in-place transformation and querying of S3 data lake data. We don't want to provision database and create tables (so we are not keen to consider...
5
answers
0
votes
974
views
asked 2 years agolg...
I have a table with columns A, B, C, D, ..., where A is a partition key. In a Glue job I want to group records of this table by column A. Is there a way to make the glue workers aware of the...
1
answers
0
votes
686
views
asked 2 years agolg...
What is the best way to scale cross-account AWS KMS–encrypted Amazon S3 bucket access using ABAC?
Tag Name – scaling-cross-account-kms-encrypted-s3-access-using-ABAC
1
answers
0
votes
407
views
asked 2 years agolg...