Ways to access S3 data

-1

I wanted to know, what are the different ways through which we an access s3 data for different usecases . I have mentioned below about the ways i know :

  • Accessing s3 data directly over internet
  • Copying the S3 data to DynamoDB at backend and then accessing data from DDB in real time ( for better latency)
  • Accessing S3 data using AWS Athena

Apart from these, what are the other ways ?

3 Answers
1
Accepted Answer

Hello, there are different ways to retrieve your data from S3.

  1. Direct access through the S3 console or API/SDK calls over the internet.
  2. As you said, data can be copied to DynamoDB for lower latency applications, enabling access from there.
  3. Moreover, AWS Athena facilitates running SQL queries directly on S3-stored data.
  4. Temporary credentials via access grants allow access to S3 objects.
  5. S3 File Gateway enables direct access to objects in AWS
  6. Within a VPC, private access is established using interface VPC endpoints or NAT/internet gateways, controlled by bucket and access point policies.
  7. Other services like Lambda, EMR, and Glue are capable of accessing S3 data for various data processing tasks.

I know this can gather confusion, but on the long run, disserting what your use case is, you may find the perfect way to access your data in the most efficient why.

Thanks!

profile picture
EXPERT
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago
profile pictureAWS
EXPERT
reviewed 2 months ago
1

The answer from Giovanni Lauria above is good but to really answer the question we need to ask you: What are you trying to do?

There are going to be a large number of ways that you can abstract access to S3. But without knowing what you're trying to achieve it would be easy to get tied up in details which aren't helpful.

profile pictureAWS
EXPERT
answered 2 months ago
profile picture
EXPERT
reviewed 2 months ago
profile picture
EXPERT
reviewed 2 months ago
  • my data is less than 50 MB in the form of parquet files which is stored in s3 bucket and I want to access data in real time with a low latency of about 1 or 2 seconds through my web application?

  • That's a good start, but now it becomes about what "access data" means. If you intend to query the data using some sort of SQL query then Glue is definitely the way to go. If you're just going to read it into your application and process it there then a direct S3 connection is probably good enough. You haven't mentioned where the application is hosted nor whether it is the client or server side which will access the data.

  • I have two use cases of data access : 1 - Simply read the data into my application 2 - Run a sql query to derive an attribute and then accessing it into my application This data will be accessed at client side .

  • If you have many clients then it makes sense to optimise by only reading the data at the server-side (which is closer to S3 in terms of latency) and doing the query there; then returning the result to the client. This allows to (perhaps) to cache the data (which makes queries faster and perhaps less expensive); and maybe to cache the query responses (again, faster). You could use Glue to do this; but for the size of data that you're talking about it might be simpler to hold it in memory on the server side.

0

Can you suggest which of these ways are favorable, if my data is less than 50 MB and I want to access data in real time with a low latency of about 1 or 2 seconds ?

answered 2 months ago
  • Overall, considering your specific needs of real-time access with low latency for data less than 50 MB, direct access over the internet via S3 console or API/SDK calls is likely the most favorable option. All the rest is more costly than beneficial, on a multitude of factors.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions