.NET Lambda Performance Degradation Under Concurrent Load - In-Memory Search Workload

Based on your description of the performance degradation in Lambda compared to EKS for your .NET in-memory search workload, I can provide some insights into what might be happening.

1. Lambda's CPU allocation model doesn't explicitly throttle sustained CPU usage under high concurrency, but there are scaling considerations. Lambda can scale at a rate of 1,000 execution environments every 10 seconds (or 10,000 requests per second) per function in each AWS Region. With 70 concurrent invocations, you're likely not hitting this scaling limit, but the way Lambda allocates compute resources differs from dedicated instances.

2. Regarding .NET's JIT and GC behavior in Lambda, while there's no specific information provided about this interaction, the Lambda execution environment might handle memory management differently than a dedicated instance, potentially affecting JIT compilation and garbage collection performance.

3. At 10GB memory allocation, your Lambda function should have access to 6 vCPUs. However, Lambda's CPU allocation is proportional to memory, with 1.8GB providing one full vCPU and 10,240MB providing 6 vCPUs. While this is theoretically equivalent to your m6.2xlarge instance's CPU capacity, the virtualization and resource sharing model in Lambda differs from dedicated EC2 instances.

4. Lambda doesn't explicitly document a "burst" vs "sustained" CPU model, but the performance characteristics you're observing suggest there might be differences in how CPU resources are allocated over time compared to dedicated instances.

5. For CPU-bound in-memory search workloads that require consistent, predictable performance, Lambda might not be the optimal choice. Your workload appears to benefit from dedicated compute resources. Since you're seeing significantly better performance on EKS with dedicated instances, migrating to ECS/Fargate or continuing with EKS would likely provide more consistent performance for your specific use case.

For compute-intensive operations like your in-memory search, the computational capacity in Lambda might not keep pace with the work as effectively as dedicated instances, even with equivalent vCPU allocations on paper. This could explain the 3x performance difference you're experiencing.
Sources
Troubleshoot configuration issues in Lambda - AWS Lambda
Lambda scaling behavior - AWS Lambda

I don't believe there's any intentional throttling for CPU or memory access, and it sounds like network bandwidth between Lambda and S3 may be mainly needed during the cold start and not while running the in-memory searches for data already loaded from S3.

There isn't much to go on with regard to locating the bottleneck, but one thing to note is that vCPUs is not a relevant metric for processor capacity. An m6 family instance, for example, uses Simultaneous Multithreading (SMT) to share a single actual processor core between two virtual threads. This means the number of vCPUs is twice that of the raw processing power. You should look at the "cores" metric instead of vCPUs that you see in instance specifications to determine how many actual physical CPU cores of the given type are available to do work.

m6 is also rather an old instance family and uses Intel's processors. You'll get somewhat more general CPU power from AMD's processors, such as the 7th generation AMD instance families m7a/r7a/c7a. They are newer than the 6th generation of instances either from AMD or Intel, and instances with AMD's processors are generally 10% cheaper than the corresponding models with Intel's processors, despite AMD often being more powerful.

If you don't have hard dependencies on the x86_64 processor architecture, you could get useful comparison figures for pinpointing the bottleneck by switching to AWS's own Graviton series of processors, which use the ARM64 processor architecture. On Lambda, somewhat dated Graviton2 processors are mostly used when you choose the "arm64" architecture for your Lambda function (https://docs.aws.amazon.com/lambda/latest/dg/foundation-arch.html). For your Fargate environment, you could choose the most recent Graviton4-based instances, such as those from the m8g family (https://aws.amazon.com/ec2/instance-types/m8g/), or a bit older Graviton3-based instances, such as the m7g family. The Graviton processor family is AWS's own, so it's both optimised for server and cloud workloads and is offered at a cheaper price than processors they're having to buy from Intel or AMD.

The relevant thing to look for in the comparisons would be if performance changes drastically when the processor architecture or instance family changes. It would suggest that the optimisations in newer processors may be key to your function's performance, or that newer instance families access memory more efficiently in a way that helps your application. If, on the other hand, the differences are just circa 15-30%, that would suggest the opposite.