AWS Lambda using NET8 image hangs on database access

1

We moved to NET8, so therefor we started using a NET8 image for a Lambda. The Lambda accesses two MySql databases in RDS. On the Lambda we see in Configuration - RDS Databases that both databases should be accessible with the provided VPC's and security groups. But the problem is that we can only access one of them. The moment we reference the other database the Lambda hangs until the timeout is triggered. There is no error/exception whatsoever. We are using Entity Framework Core 8.0 with Pomelo 8-beta2 Both databases are 5.7, For both connectionstrings => ServerVersion.AutoDetect(connectionString) returns "5.7.12-mysql" In the Pomelo code I can see that it actually does a connection.Open() for the AutoDetect, so that must mean it can access the database there right?

in C# database1Context.Database.CanConnect() works database2Context.Database.CanConnect() hangs

var query = database1Context.EntitySet1.Where( x => x.Id ==1) works var query = database2Context.EntitySet1.Where( x => x.Id ==1) hangs

And these last ones only build the query, they don't even actually execute it yet.

The Lambda was working fine when using NET6 (without an image). And we are using the same VPC and securitygroups as with the NET6 version.

We tried adding Pipelining=False to the connectionstring, no difference.

I really need some help here on how to attack the problem. How can I find out what is wrong? Why can it access one database and not the other one? Highly frustrating the lack of errors. Been trying all kind of things for two full days. Any hints appreciated!

asked 2 months ago214 views
5 Answers
0

Hangs are always frustrating without actionable errors. Are you able to change some experimental variables to eliminate or narrow down the issue? For example:

AWS
Beau
answered 2 months ago
profile picture
EXPERT
reviewed 10 days ago
  • Does it also happen with a .NET 6 container image Lambda? => We don't know, not simple to test but I may try that. Does it happen with .NET 8 outside of Lambda? (can you run this from an EC2 instance or development machine?) => We have multiple other parts of the application, api's, containers etc running where it works fine and can access both databases Does it still work fine with .NET 6 Lambda? => yes it does Are you using native AOT? => I guess not, I was not even aware of it that it existed :) Would it make sense to also open an issue here: https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/issues ? => maybe

    I was hoping to actually include the Pomelo source code into my project, hoping I could add some existing debug information. But I could not get the github package compiled. Thanks Beau, for the suggestions.

0

Are you using public.ecr.aws/lambda/dotnet:8 or public.ecr.aws/lambda/dotnet:8-preview ? Or something else for the container image base?

AWS
Beau
answered 2 months ago
0

Some other things to consider:

  • Is it always the second call that hangs or always database2Context? What if you switch the order you connect to 2 then 1?
  • What are the differences between DB 1 and DB 2?
  • Have you done anything custom or non-default with certificates?
  • If you increase the Lambda timeout to the max 15 minutes, maybe it will give the .NET code enough time to hit it's own timeout so that at least you can get a stack trace.

For what it's worth, I was able to connect to 2 different RDS MySQL databases on a VPC using the .NET 8 image Lambda using MySqlConnector.MySqlConnection (I don't have any experience with Pomelo unfortunately). The MySQL version was v5.7.37, which is the oldest allowed in the dropdown.

AWS
Beau
answered 2 months ago
0

I upgraded the EntityFramework packages to 8.0.2 (were 8.0.1) and Pomelo to 8.0.0 (was beta2) and now it is no longer hanging. Don't know which packages were the cause. I may try to find out.

answered 2 months ago
0

Going back to the beta version (just for testing) did not reintroduce the problem. So we have no clue what was the actual cause of the problem

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions