EC2 instance comparison analysis
We are trying to compare EC2 instances performance for reading files from S3 bucket.
We are using python script to read parquet file from S3 bucket.
Python script uses pandas.read_parquet(S3 object key) function to read parquet file from S3 bucket.
The EC2 instances considered for comparison are c6in.xlarge and c6in.2xlarge.
Instance name | vCPU | Memory | Storage | Network performance | EBS Volume |
---|
c6in.xlarge | 4 | 8 GiB | EBS Only | Up to 30000 Megabit | 8GiB-gp3 |
c6in.2xlarge | 8 | 16 GiB | EBS Only | Up to 40000 Megabit | 8GiB-gp3 |
Steps followed for comparison
- Create EC2 instances (c6in.xlarge and c6in.2xlarge).
- Run python script.
- Note the time taken for reading all files from S3 bucket.
Issue Description
When we are trying to compare read time in c6in.xlarge and c6in.2xlarge instances, we have observed that read time is higher in c6in.2xlarge instance. Since c6in.2xlarge uses double the number of vCPU and Network performance also higher compared to c6in.xlarge, we are expecting read time should be less in c6in.2xlarge compared to c6in.xlarge instance.
Metrics observed read time in c6in.xlarge and c6in.2xlarge instances
c6in.xlarge -> 247.705972 sec
c6in.2xlarge -> 279.7610955 sec
Comparison between gp3, io1 and io2 EBS volume types
We tried by changing EBS volume types by keeping EBS volume size(8GiB) as constant.
We changed EBS volume types to gp3, io1 and io2 respectively and used default value suggested.
Observations for changing EBS volume types
In this we have observed that there is no much difference between gp3, io1 and io2 EBS volume types in c6in.xlarge instance and little difference in c6in.2xlarge instance.
But instance point of view still it is higher read time in c6in.2xlarge compared to c6in.xlarge instance.
EBS Volume Type | Default IOPS | c6in.xlarge Read Time (sec) | c6in.2xlarge Read Time (sec) |
---|
gp3 | 3000 | 203.922494409999 | 219.475556271999 |
io1 | 400 | 203.525826416999 | 228.025478206 |
io2 | 4000 | 203.134390571999 | 221.881286345 |
Please help me to identify the root cause and how to solve it.
Thanks in advance.