How do I use CloudFormation to set special parameters in an AWS Glue job?

Resolution

To set special parameters for your AWS Glue job, you must supply a key-value pair for the DefaultArguments property of the AWS::Glue::Job resource in CloudFormation. If you supply a key only in your job definition, then CloudFormation returns a validation error.

To resolve this issue, complete the following steps:

1. In your CloudFormation template, set the value of your special parameter to an empty string for the DefaultArguments property of your job definition.
   Example JSON:

   "MyJob": {
     "Type": "AWS::Glue::Job",
     "Properties": {
       "Command": {
         "Name": "glueetl",
         "ScriptLocation": "s3://my-test//test-job1"
       },
       "DefaultArguments": {
         "--job-bookmark-option": "job-bookmark-enable",
         "--enable-metrics": ""
       },
       "ExecutionProperty": {
         "MaxConcurrentRuns": 2
       },
       "MaxRetries": 0,
       "Name": "cf-job3",
       "Role": {
         "Ref": "MyJobRole"
       }
     }
   }

   Example YAML:

   MyJob:
     Type: 'AWS::Glue::Job'
     Properties:
       Command:
         Name: glueetl
         ScriptLocation: 's3://my-test//test-job1'
       DefaultArguments:
         '--job-bookmark-option': job-bookmark-enable
         '--enable-metrics': ''
       ExecutionProperty:
         MaxConcurrentRuns: 2 
       MaxRetries: 0
       Name: cf-job3
       Role: !Ref MyJobRole

   Note: In the preceding examples of JSON and YAML templates, --enable-metrics has an empty string value. The empty string validates the template and launches the resource that's configured with the special parameter.
2. To activate your special parameter, run your job.