How to introduce distributed mode in maps using aws CDK

0

I have been trying to get a mitigation for the error that pops up in the state machine's execution

The execution reached the maximum number of history events (25000).

I am trying to add a DISTRIBUTED mode for my maps that are calling the s3 api (listObjects). I want to add this functionality at the CDK code in typescript which can be used in deployment of CFN stack and we have a form of tracking and giving relevant permissions for the state machine. example code :

        const listLimitDirectories = new CallAwsService(this, 'listLimitDirectories', {
            service: 's3',
            action: 'listObjectsV2',
            parameters: {
                Bucket: bucketName,
                Prefix: 'limits/',
                Delimiter: '/',
            },
            iamResources:[bucketArn, bucketArn + '/*'],
            iamAction: 's3:getObject'
        });

        const iterateLimitDirectories = new Map(this, "iterateLimitDirectories", {
            maxConcurrency: 24,
            itemsPath: "$.CommonPrefixes",
            resultPath: "$.Contents"
        })

It would be great to get some hands on resources that give some ideas on how to introduce this DISTRIBUTED mode in the code level

feita há 10 meses527 visualizações
1 Resposta
0

Hi, look at this Github ticket: https://github.com/aws/aws-cdk/issues/23216

You have some code samples that you are looking for toward the bottom of the ticket

Also this one may be of interest for you: https://github.com/alessandromr/aws-sfn-map-demo with joint article at https://alessandromarinoac.com/unleashing-the-power-of-large-scale-parallel-data-processing-on-aws

Best,

Didier

profile pictureAWS
ESPECIALISTA
respondido há 10 meses
profile picture
ESPECIALISTA
avaliado há 10 meses

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas