segmentation fault in Rust Lambda using rusoto_dynamodb

0

I have some code that starts out like this:

use flate2::read::GzDecoder;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use rusoto_core::Region;
use rusoto_dynamodb::{AttributeValue, DynamoDb, DynamoDbClient, QueryInput};
use serde_json::{json, Value};
use std::fs::File;

#[tokio::main]
async fn main() -> Result<(), Error> {
    let func = service_fn(lambda_handler);
    run(func).await
}

async fn lambda_handler(event: LambdaEvent<Value>) -> Result<Value, Error> {

    let id = event.payload.get("id").and_then(|id| id.as_str()).unwrap();
    println!("Querying for id: {}", id);

    let region = Region::default();
    let dynamo_client = DynamoDbClient::new(region); // also tried Default::default() here, and Region::UsWest2
    println!("Dynamo client created");

    ... the rest you don't need to see, because the above println is never reached ...

This is the Cargo.toml:

[package]
name = "query_by_id"
version = "0.1.0"
edition = "2021"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[profile.release]
debug = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
aws_lambda_events = "0.15.0"
lambda_runtime = "0.11.1"
rusoto_core = "0.48.0"
rusoto_dynamodb = "0.48.0"
rusoto_config = "0.48.0"
serde_json = "1.0.116"
flate2 = "1.0.30"
tokio = { version = "1.37.0", features = ["full"] }
tokio-util = { version = "0.7", features = ["compat"] }
futures = "0.3.17"
rusoto_credential = "0.48.0"

and I build it using this Dockerfile:

FROM public.ecr.aws/docker/library/rust:1.78-alpine
WORKDIR /build

# Setup Rust environment for MUSL target, pre-cache known dependencies
RUN apk add musl-dev openssl-dev zip make
ENV PATH="/root/.cargo/bin:${PATH}"
ENV CARGO_TARGET_DIR=/root/.cargo/target
RUN rustup target add x86_64-unknown-linux-musl

RUN apk add pkgconfig
ENV OPENSSL_DIR=/usr
ENV OPENSSL_LIB_DIR=/usr/lib
ENV OPENSSL_INCLUDE_DIR=/usr/include/openssl

RUN cargo install cargo-prefetch
RUN cargo prefetch $(sed -n '/[dependencies]/,/^$/p' Cargo.toml | awk '{print $1"@="$3}' | tr '\n' ' ')

# Build the Application
COPY Cargo.toml /build/
RUN mkdir src && echo "fn main() {}" > src/main.rs && cargo build --release && rm -rf src
COPY src/ /build/src/
RUN cargo build --release

# Place executable into path expected by CDK Construct
RUN mkdir -p bin && cp $CARGO_TARGET_DIR/release/bootstrap bin/bootstrap
RUN strip -s bin/bootstrap

and I deploy it using this CDK Construct:

        // Create a Lambda function for querying
        const queryFunction = new lambda.Function(this, 'QueryFunction', {
            runtime: lambda.Runtime.PROVIDED_AL2023,
            handler: 'bootstrap',
            code: lambda.Code.fromDockerBuild(path.join(__dirname, '..', 'query_by_id'), {
                file: 'Dockerfile',
                imagePath: '/build/bin/',
                outputPath: '/var/task',
            }),
            vpc,
            securityGroups: [lambdaSecurityGroup],
            vpcSubnets: {
                subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
            },
            filesystem: accessPoint,
            timeout: cdk.Duration.seconds(300),
            memorySize: 1024,
            environment: {
                DYNAMODB_ENDPOINT_URL: `http://dynamodb.${this.region}.amazonaws.com`,
                EFS_PATH: '/mnt/efs',
            },
            role: lambdaRole,
        });

and here's what I get when I try to execute the Lambda:

START RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6 Version: $LATEST
Querying for id: 123456
RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6 Error: Runtime exited with error: signal: segmentation fault
Runtime.ExitError
END RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6
REPORT RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6	Duration: 28.66 ms	Billed Duration: 56 ms	Memory Size: 1024 MB	Max Memory Used: 16 MB	Init Duration: 26.47 ms	

What could possibly cause this? I thought segfaults were "impossible" in Rust. Maybe be something wrong with the Lambda environment?

AlexR
asked 4 months ago572 views
1 Answer
0
Accepted Answer

I'm new to Rust and didn't realize until now that rusoto_dynamodb is essentially unmaintained. Cross-compiling and static linking to musl-libc is also apparently no longer necessary, but still shows up in many blogs and tutorials. My problem was resolved after making these changes:

  • replace rusoto_dynamodb with aws-sdk-dynamodb
  • compile on Amazon Linux 2023 with normal gcc
  • remove the fn main() {} pre-fetch step (this caused another issue which was masked by the initial problem)

Updated Dockerfile:

FROM amazonlinux:2023
RUN yum install -q -y openssl-devel gcc python3 python3-pip zip make tar gzip wget
# install Rust
VOLUME /root/.cargo
ENV CARGO_HOME=/root/.cargo
ENV CARGO_TARGET_DIR=/root/.cargo/target
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Build the Application
WORKDIR /build
COPY Cargo.toml /build/
COPY src/ /build/src/
RUN cargo build --release
# Place executable into path expected by CDK Construct
RUN mkdir -p bin && cp $CARGO_TARGET_DIR/*/bootstrap bin/bootstrap
RUN strip -s bin/bootstrap

Updated Cargo.toml:

[package]
name = "s3_cache"
version = "0.1.0"
edition = "2021"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[profile.release]
debug = true

[dependencies]
aws_lambda_events = "0.15.0"
lambda_runtime = "0.11.1"
aws-sdk-dynamodb = "1.32.0"
aws-sdk-s3 = "1.32.0"
aws-config = "1.5.1"
serde_json = "1.0.116"
serde ={ version = "1.0.116", features = ["derive"] }
flate2 = "1.0.30"
tokio = { version = "1.37.0", features = ["full"] }
tokio-util = { version = "0.7", features = ["compat"] }
futures = "0.3.17"
aws-types = "1.3.1"
csv = "1.1.6"
AlexR
answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions