How to introduce distributed mode in maps using aws CDK

0

I have been trying to get a mitigation for the error that pops up in the state machine's execution

The execution reached the maximum number of history events (25000).

I am trying to add a DISTRIBUTED mode for my maps that are calling the s3 api (listObjects). I want to add this functionality at the CDK code in typescript which can be used in deployment of CFN stack and we have a form of tracking and giving relevant permissions for the state machine. example code :

        const listLimitDirectories = new CallAwsService(this, 'listLimitDirectories', {
            service: 's3',
            action: 'listObjectsV2',
            parameters: {
                Bucket: bucketName,
                Prefix: 'limits/',
                Delimiter: '/',
            },
            iamResources:[bucketArn, bucketArn + '/*'],
            iamAction: 's3:getObject'
        });

        const iterateLimitDirectories = new Map(this, "iterateLimitDirectories", {
            maxConcurrency: 24,
            itemsPath: "$.CommonPrefixes",
            resultPath: "$.Contents"
        })

It would be great to get some hands on resources that give some ideas on how to introduce this DISTRIBUTED mode in the code level

preguntada hace 10 meses527 visualizaciones
1 Respuesta
0

Hi, look at this Github ticket: https://github.com/aws/aws-cdk/issues/23216

You have some code samples that you are looking for toward the bottom of the ticket

Also this one may be of interest for you: https://github.com/alessandromr/aws-sfn-map-demo with joint article at https://alessandromarinoac.com/unleashing-the-power-of-large-scale-parallel-data-processing-on-aws

Best,

Didier

profile pictureAWS
EXPERTO
respondido hace 10 meses
profile picture
EXPERTO
revisado hace 10 meses

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas