- Newest
- Most votes
- Most comments
yes, this is possible. I have created a sampe below for you to reference
- The start trigger is a scheduled trigger. This triggers 2 jobs (GlueJob and JDBCETLGlueJob in the picture)
- There is conditional trigger that you added after both jobs complete. the option is to start after "ALL" requirements are met - i.e. both jobs complete successfully.
- The conditional trigger starts another Glue Job
If this helps, please accept answer and upvote so that it helps others. Here's an example of a Python program that uses the Boto3 library to create a Glue trigger that triggers "glue-job-1" and "glue-job-2", and another conditional trigger that triggers "glue-job-3" when both "glue-job-1" and "glue-job-2" complete successfully:
import boto3
# Create a Glue client
glue_client = boto3.client('glue')
# Define the names of the jobs to trigger
job_1_name = "glue-job-1"
job_2_name = "glue-job-2"
job_3_name = "glue-job-3"
# Define the trigger that triggers job_1 and job_2
job_1_2_trigger = {
"Name": "job_1_2_trigger",
"Type": "ON_DEMAND",
"Actions": [
{
"JobName": job_1_name
},
{
"JobName": job_2_name
}
]
}
# Create the trigger
response = glue_client.create_trigger(Trigger=job_1_2_trigger)
print(f"Successfully created trigger {response['Name']}")
# Define the condition for the second trigger
condition = f"SUCCEEDED({job_1_name}) AND SUCCEEDED({job_2_name})"
# Define the trigger that triggers job_3
job_3_trigger = {
"Name": "job_3_trigger",
"Type": "CONDITIONAL",
"Actions": [
{
"JobName": job_3_name
}
],
"Predicate": {
"Conditions": [
{
"LogicalOperator": "AND",
"JobName": condition
}
]
}
}
# Create the trigger
response = glue_client.create_trigger(Trigger=job_3_trigger)
print(f"Successfully created trigger {response['Name']}")
This program uses the glue_client.create_trigger() method to create the triggers and the glue_client.create_trigger() method to create the triggers. The job_1_2_trigger is of type ON_DEMAND that triggers job_1 and job_2 . The job_3_trigger is of type CONDITIONAL that triggers job_3 when both job_1 and job_2 complete successfully.
You would need to add appropriate error handling, input validation and other requirements specific to your use case.
Relevant content
- asked 3 months ago
- asked 2 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 17 days ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
This is exactly what I need. Thank you Ananth . How do I code using boto3 ?. It is allowing me to define action and type only once