Questions tagged with Database

Hello, we're having some difficulty with our primary AWS Aurora Clusters and the various webservers that connect to them. Whenever a scaling operation occurs - both scaling up and scaling down - a large number of connections to the database are rejected ( 2,000+ ). We have noted the following errors from the RDS event log and our webservers: > The DB cluster failed to scale from 64 capacity units to 32 capacity units for this reason: A scaling point wasn’t found. > OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") We have attempted to debug this problem with the following steps, but nothing has helped us to resolve it so far: * Reviewed MySQL error logs in Cloudwatch ( these only provided the server-level errors, nothing from RDS specifically ) * Turned off general MySQL logging * Ensured Apache / MySQL connection timeouts did not exceed the RDS Autoscaling Timeout * Verified all VPC, IGW, Route Table, and subnet settings associated with the instances connecting to the database. Below is some information regarding the cluster. > **Cluster Version**: > * Aurora MySQL (compatible with MySQL 5.6.1.22.3) > > **Capacity Settings:** > * Minimum ACUs 16 - 32 GiB RAM > * Maximum ACUs 64 - 122 GiB RAM > > **Additional Scaling Configuration:** > * Autoscaling timeout and action: 00:05:00 > * Do this: Roll back the capacity change **Any advice or insight into possible avenues of troubleshooting or debugging would be highly appreciated at this point.**lg...

DMS replication task, which has source as RDS - MySQL and destination as S3, it is ongoing cdc replication task and is showing status as runnning 100%, for 4 hours it was not replicating any data, when we checked RDS, it was having new data of those 4 hours, so why did DMS didn't and also didn't put error status. Later we just stopped and resumed, 3-4 times and then after 3rd attempt data started coming from 4 hours before checkpoint. This issue has happended for our omnifin database (comparitively big Db, issue happened for the first time) and for knab database (very small Db, on every 27th day of the month this issue happens) So why does DMS without showing any error, stops performing its replication task, and in status it is green with runnning 100% display. Please look into this issue. Thanks.lg...

Hi friends, My RDS databases are encrypted using Default AWS-managed keys, and everything works as expected. However, I'm confronted with the idea of using Customer managed keys which looks like additional work. I do not think it is needed at this moment, are there any security issues for remaining using default AWS encryption keys ? what strategy will be best for you? our environments, for the moment, do not have a workload that deserves this configuration. I'm not very sure about rebuilding my entire environment to implement a customer-managed key. Any ideas ? Best Regards,lg...

I have a serverless database set up in Redshift, created a table, and am now trying to load that table from a .csv file I have uploaded to an S3 bucket. When I created the table I set the primary key as an identity key as follows: customerid integer NOT NULL identity(0,1) When I load the table using the COPY query, the key increments by 128 starting with 64 rather than by 1 starting at 1. For example, my customerID field has values of 64, 192, 320, 448, etc. I've read numerous articles that this is due to compression and parallelism. I've tried including the "COMPUPDATE off" command as part of my COPY query but that did not change the results. I've truncated my table each time before I've tried to reload it to reset the seed. How can I load a table and have the identity key start with 1 and increment by 1?lg...

How can I return to the RDS disconnected from the ec2 instance I was using?lg...

 I'm guessing I don't have access to the datalake I'm querying, BUT I've already onboarded my AWS accounts to access the database. Can someone tell me where I can check these permissions?lg...

Hi, I'm trying to use Glue to get a table from SQL Server (I have the schema). This table has a "Location" with an space (I cannot change the "Location" in AWS). This generate an error: py4j.protocol.Py4JJavaError: An error occurred while calling o96.getDynamicFrame. : com.microsoft.sqlserver.jdbc.SQLServerException: Sintaxis incorrecta cerca de la palabra clave 'of'. After investigating, I detected that it is because the space. I tried to add [] but I couldn't. Also I cannot change the database. Any idea? Thanks for the helplg...

**Gist:** Cannot gain terminal access to then run Oracle or SQL Plus commands to import a database dump into Oracle RDS. Struggling to import an Oracle database dump as there is no way to do this via AWS's RDS web console, and I have not been able to gain terminal access. Indeed, it does not seem that AWS RDS really wants the owner to use SSH access, as the IP address of the server instance is not readily displayed, but instead the expectation is that all work is done via the DB instance, and not the host server (it is a managed service, after all!). **Situation** * An Oracle RDS instance was created a few months ago. At that time an 'impdp' import was done. * Now wish to redo this step, yet do not have means to gain terminal access. Alas, the knowledge of how the first import was done is no longer in the company... and we cannot gain terminal access using the Master username + password (known good credentials as these are used to log into the Oracle DB using DBeaver or Oracle SQL Developer tools). * Endpoint is of format oracle.clpf12ab34.us-east-1.rds.amazonaws.com * Master username: known (displayed on RDS console's configuration tab) * Password: known **Failed approaches to gain terminal access:** 1. When I attempt to ssh into the server, having stripped the DB Name "oracle" from the front, I observe error "ssh: Could not resolve hostname ... nodename nor servname provided, or not known" 2. Managed to ping the server, so attempted to ssh using masterusernam@ipaddress - yet this returns error "ssh: connect to host <<IPADDRESS>> port 22: Operation timed out" Note: on all my ECS instances access is done via PEM key files, and not using username/password pair. However, no PEM key-pair has been defined on AWS RDS. Therefore my only credential is the Oracle DB username/password. **What next?** I am aware that AWS services cannot reissue new credentials. Yet I would have anticipated the ability to use the information that I have to be able to do some administration of the Oracle DB server... and specifically to be able to run a database dump import (using Oracle's 'impdp' command). The next approach is to manually conduct import... table by table, and with care for foreign key or constraint dependencies. This shall be laborious. Another approach is to dump this existing RDS instance and to create another. Ideally I would need to make use of Oracle DB version 12.1, as this is what I am needing to use for a project -- and importantly I need to have a patch level that would allow me to conduct the import (as the source database was Oracle 12.1 with patch from July 2021). As always, your guidance and pointers are most welcome. Wish you great end-of-week & weekend!lg...

Hi, We have just created a brand new account in Amazon EC2. We have also created a default MS-SQL Server database along with a new EC2 machine. However, when we try to connect to this database via SQL Server Management Studio installed on my local laptop, we are running into errors. These errors are not unique. These are typical errors which come while trying to configure connectivity between Management studio and remote database. We have followed all articles and other information available both at Microsoft Technet as well as AWS, however the error have remained unchanged. Details of the error are given below. I would greatly appreciate it if someone have a look and answer them ASAP. Many many Thanks, Manoj Kumar Error Message 1 - Cannot connect to vibexchange.c3uqqhmm412p.ap-south-1.rds.amazonaws.com. Details - Cannot connect to vibexchange.c3uqqhmm412p.ap-south-1.rds.amazonaws.com. Error Message 2 - A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) (Framework Microsoft SqlClient Data Provider) ------------------------------ For help, click: https://docs.microsoft.com/sql/relational-databases/errors-events/mssqlserver-53-database-engine-error ------------------------------ Error Number: 53 Severity: 20 State: 0 ------------------------------ Program Location: at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at Microsoft.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, Boolean withFailover, Boolean isFirstTransparentAttempt, SqlAuthenticationMethod authType, String certificate, ServerCertificateValidationCallback serverCallback, ClientCertificateRetrievalCallback clientCallback, Boolean useOriginalAddressInfo, Boolean disableTnir) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover, Boolean isFirstTransparentAttempt, Boolean disableTnir) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, ServerCertificateValidationCallback serverCallback, ClientCertificateRetrievalCallback clientCallback, DbConnectionPool pool, String accessToken, SqlClientOriginalNetworkAddressInfo originalNetworkAddressInfo, Boolean applyTransientFaultHandling) at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at Microsoft.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open() at Microsoft.SqlServer.Management.SqlStudio.Explorer.ObjectExplorerService.ValidateConnection(UIConnectionInfo ci, IServerType server) at Microsoft.SqlServer.Management.UI.ConnectionDlg.Connector.ConnectionThreadUser() Details - A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) (Framework Microsoft SqlClient Data Provider) For help, click: https://docs.microsoft.com/sql/relational-databases/errors-events/mssqlserver-53-database-engine-error Error Number: 53 Severity: 20 State: 0 at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at Microsoft.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, Boolean withFailover, Boolean isFirstTransparentAttempt, SqlAuthenticationMethod authType, String certificate, ServerCertificateValidationCallback serverCallback, ClientCertificateRetrievalCallback clientCallback, Boolean useOriginalAddressInfo, Boolean disableTnir) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover, Boolean isFirstTransparentAttempt, Boolean disableTnir) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, ServerCertificateValidationCallback serverCallback, ClientCertificateRetrievalCallback clientCallback, DbConnectionPool pool, String accessToken, SqlClientOriginalNetworkAddressInfo originalNetworkAddressInfo, Boolean applyTransientFaultHandling) at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at Microsoft.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at Microsoft.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open() at Microsoft.SqlServer.Management.SqlStudio.Explorer.ObjectExplorerService.ValidateConnection(UIConnectionInfo ci, IServerType server) at Microsoft.SqlServer.Management.UI.ConnectionDlg.Connector.ConnectionThreadUser() Error 3 - The network path was not found Details - The network path was not foundlg...

I have this problem: Failed to stabilize Instance with id. My CF looks like: Resources: DocumentDBSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupName: 'DocumentDB SG' GroupDescription: !Sub 'Security Group for the DocumentDb' VpcId: !Ref VPC DocumentDBSubnetGroup: Type: AWS::DocDB::DBSubnetGroup Properties: DBSubnetGroupDescription: "Subnet group for Document DB cluster" DBSubnetGroupName: "document-db-subnet-group" SubnetIds: !Ref PrivateSubnetIds DocumentDBParameterGroup: Type: AWS::DocDB::DBClusterParameterGroup Properties: Description: "Parameter group for Document DB cluster" Family: docdb4.0 Name: "document-db-paramater-group" Parameters: audit_logs: "disabled" DocumentDBCluster: Type: AWS::DocDB::DBCluster Properties: BackupRetentionPeriod: 7 DBClusterIdentifier: "docdb" DBSubnetGroupName: !Ref DocumentDBSubnetGroup DBClusterParameterGroupName: !Ref DocumentDBParameterGroup Port: 27017 PreferredBackupWindow: "07:00-09:30" PreferredMaintenanceWindow: "tue:07:00-tue:11:00" VpcSecurityGroupIds: - !Ref DocumentDBSecurityGroup StorageEncrypted: true DocumentDBInstance: Type: AWS::DocDB::DBInstance DependsOn: - DocumentDBCluster Properties: DBClusterIdentifier: !Ref DocumentDBCluster DBInstanceClass: db.t3.medium DBInstanceIdentifier: "docdb" PreferredMaintenanceWindow: "tue:07:00-tue:11:00" If i search this problem i find information about RDS (snapshot), but i don't use snapshot in this deployment..lg...

Hi there, I am having recurring issues migrating Aurora MySQL 5.6.10 on db.r5.large to Aurora MySQL 5.7.12 on db.r6g.large. I started by trying to replicate all schemas I had created, but this failed with an unknown error. I then broke this down into one schema per replication group, this also failed with an unknown error. I then turned on CloudWatch logging for all tasks. This worked other than one table repeatedly fails to replicate. If I use the mysql cli to drop or repair the table, mysql drops the connection! When I look at the table in phpmyadmin, it says 'unknown storage engine' and/or table in use. When I try to drop the schema using phpmyadmin, it logs me out straight away! I've waited a few minutes and now can log back in, and can see the schema has been dropped successfully. This looks like a bug in DMS creating the table, or in Aurora somehow locking the table and putting it into an inconsistent state. I've now resolved the issue and moved on, but the service team might want to be aware of this. The table schema is very simple: CREATE TABLE IF NOT EXISTS `lkcities` ( `state` varchar(2) DEFAULT NULL, `city` varchar(16) DEFAULT NULL, `country_id` varchar(2) NOT NULL, UNIQUE KEY `country_id` (`country_id`,`state`,`city`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; -- -- Dumping data for table `lkcities` -- INSERT INTO `lkcities` (`state`, `city`, `country_id`) VALUES ('AK', 'Akhiok', 'US'), ('AK', 'Akiachak', 'US'), ('AK', 'Akiak', 'US'), ('AK', 'Akutan', 'US'), ('AK', 'Alakanuk', 'US'), ('AK', 'Aleknagik', 'US'), ('AK', 'Allakaket', 'US'), ('AK', 'Ambler', 'US'), ('AK', 'Anaktuvuk Pass', 'US'), ('AK', 'Anchorage', 'US'), etc ~25,705 rows.lg...

I am using RDS Aurora t3.medium DB instance type in ohio region. we observed there is high CPU usages on mine RDS so we want to inscreased the instance type according to CPU like 4vCPU and 8 GB RAM but found out that RDS doesn't provide compute instance type like it is available in EC2 due to that i am forced to go for high cost db instance type r6g.xlarge. Why AWS is not providing the db instance type in compute section like other provider provide.lg...