Terraform JSON parsing issue

0

Hi AWS, I have to generate a JSON file from .tfvars file, i.e. terraform.tfvars will be changed to terraform.tfvars.json. However I found a plugin tfvars-json https://github.com/alisdair/tfvars-json but while I am running the command I got an error: tfvars-json: command not found.

Second way I tried to solve the issue was using data external source reading python script which is responsible for passing the value dynamically in terraform.tfvars.json file. Here are the code snippets:

data.tf:

data "external" "addons" {
  program = ["python", "${path.module}/test.py"]
}

variables.tf:

variable "addons" {
  type = list(object({
    name    = string
    version = string
  }))
}

test.py:

import json

# Load existing terraform.tfvars.json content
with open('terraform.tfvars.json', 'r') as f:
    existing_data = json.load(f)

# Define new values for the addons variable
addons_data = [
    {"name": "vpc-cni", "version": "v1.18.0-eksbuild.1"},
    {"name": "kube-proxy", "version": "v1.28.6-eksbuild.2"}
]

# Update the addons variable in the existing data
existing_data["addons"] = addons_data

# Print the updated data in JSON format
print(json.dumps(existing_data))

terraform.tfvars.json:

{
  "aws_region": "us-east-1",
  "aws_access_key": "AKIAXXXXXXXXXXXXXXXXXXXX",
  "aws_secret_access_key": "ny0oaSdwXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "default_tags": {
    "primary_owner": "Arjun Goel",
    "secondary_owner": "Harsh Goel",
    "project_name": "EKS-POC-PROJECT"
  },
  "cidr_block": "192.168.0.0/16",
  "public_subnet_count": 2,
  "private_subnet_count": 2,
  "cluster_name": "eks-cluster-poc",
  "cloudwatch_log_group_name": "eks_cluster_cw_log_group",
  "cloudwatch_log_stream_name": "eks_cluster_cw_log_stream",
  "eks_service_role": "eks_iam_service_role",
  "eks_workernode_role": "eks_iam_worker_node_role",
  "launch_template_name": "eks_worker_node_lt",
  "addons": []
}

When I ran the python script standalone it is adding up the values dynamically for key "addons" in terraform.tfvars.json while when I am running the command terraform plan though it is populating the values, but it is throwing this error too:

Error: Unexpected External Program Results │ │ with data.external.addons, │ on data.tf line 17, in data "external" "addons": │ 17: program = ["python", "${path.module}/test.py"] │ │ The data source received unexpected results after executing the program. │ │ Program output must be a JSON encoded map of string keys and string values. │ │ If the error is unclear, the output can be viewed by enabling Terraform's logging at TRACE level. Terraform documentation on logging: │ https://www.terraform.io/internals/debugging │ │ Program: C:\Users\amang\AppData\Local\Programs\Python\Python311\python.exe │ Result Error: json: cannot unmarshal object into Go value of type string

This is the original JSON file with content:

{
    "class": "AS3",         //Satatic
    "action": "deploy",     //Satatic
    "persist": true,        //Satatic
    "declaration": {          //Satatic
       "class": "ADC",        //Satatic
       "schemaVersion": "3.37.0",          //Satatic
       "id": "example-declaration-tenant5",          //Satatic
       "label":"Sample Tenant5",          //Satatic
       "remark": "Tenant5 test for AS3",          //Satatic
       
       "Tenant5": {           //Satatic
          "class": "Tenant",          //Satatic
          "Shared": {    //Static Block
             "class": "Application",          //Satatic
             "template": "shared",        //Satatic
 
             "as3-tenant5-test": {   //Dynamic Block  Application #1
                 "remark": "as3-tenant5-test",  //Dynamic Block
                 "class": "Service_HTTP", //Dynamic Block
                 "iRules": [//Dynamic Block
                     "AS3-TEST3-IRULE"   //Dynamic Block. listo of rules defined here will have a subsequent config definition like line #77
                 ],
                 "profileTCP": {  //Dynamic Block
                     "bigip": "/Tenant5/tcp"
                 },
                 "profileHTTP": {  //Dynamic Block.  
                     "use": "http_hsts_xf1"  //Dynamic Block.  http_hsts_xf1 will be further defined as line #98
                 },
                  
                 "virtualAddresses": [  //Dynamic Block
                     "${VIP_ADDRESS}"
                 ],
                 "virtualPort": 80,  //Dynamic Block
                 "pool": "pool30",                //Dynamic Block
                 "snat": "none"  //Dynamic Block
             },
             
             "as3-tenant5-test2": {
                 "remark": "as3-tenant5-test2",                 
                 "class": "Service_HTTP",
                 "iRules": [
                     "AS3-TEST3-IRULE"
                 ],
                 "profileTCP": {
                     "bigip": "/Tenant5/tcp"
                 },
                 "virtualAddresses": [
                     "${VIP_ADDRESS_2}"
                 ],
                 "virtualPort": 80,
                 "pool": "pool30",
                 "snat": "none"
             },
 
             "as3-tenant5-test3": {   //Dynamic Block
                 "remark": "as3-tenant5-test3", //Dynamic Block
                 "class": "Service_HTTPS",  //Dynamic Block.
                  "iRules": [
                       "AS3-TEST3-IRULE"
                 ],
                 "profileTCP": {
                     "bigip": "/Tenant5/tcp"
                 },
                 "virtualAddresses": [
                     "3.3.3.3"
                 ],
                 "virtualPort": 443,
                 "snat": "auto",
                 "pool": "pool30",
                 },     
            
                 "serverTLS": "webtls",  //Dynamic  Block.  THis will further be defined in webtls block like line #154
                 "redirect80": false     //Dynamic  Block
             },
 
             "AS3-TEST3-IRULE": {  //Dynamic Block 
                 "class": "iRule",
                 "iRule": "when HTTP_REQUEST { HTTP::respond 200 content {<html><head><title>Hi team</title</head><body><h1>Hello team!</h1></body></html> }}"
              },
 
             "pool30": {  //Dynamic Block
                 "class": "Pool",
                 "monitors": [
                     "http"
                 ],
                 "members": [
                 {
                     "servicePort": 8080,
                      "serverAddresses": [
                         "3.3.3.200",
                         "3.3.3.201"
                       ]
                 }
                 ]
             },
             
             "http_hsts_xf1": {  Dynamic Block
                     "class": "HTTP_Profile",
                     "knownMethods": [
                         "CONNECT",
                         "DELETE",
                         "GET",
                         "HEAD",
                         "LOCK",
                         "OPTIONS",
                         "POST",
                         "PROPFIND",
                         "PUT",
                         "TRACE",
                         "UNLOCK"
                     ],
                     "maxHeaderCount": 64,
                     "maxHeaderSize": 32768,
                     "maxRequests": 0,
                     "pipelineAction": "allow",
                     "truncatedRedirects": false,
                     "unknownMethodAction": "allow",
                     "hstsIncludeSubdomains": false,
                     "hstsPeriod": 31536000,
                     "hstsInsert": true,
                     "xForwardedFor": true,
                     "multiplexTransformations": true,
                     "proxyType": "reverse",
                     "rewriteRedirects": "none",
                     "requestChunking": "sustain",
                     "responseChunking": "sustain",
                     "serverHeaderValue": "BigIP",
                     "viaRequest": "preserve",
                     "viaResponse": "preserve"
                 },
             
             
              "aes_128-256_cipher-rules": {  //Dynamic Block
                     "class": "Cipher_Rule",
                     "cipherSuites": [
                         "ECDHE-ECDSA-AES256-GCM-SHA384",
                         "ECDHE-RSA-AES256-GCM-SHA384",
                         "ECDHE-RSA-AES256-SHA384",
                         "ECDHE-RSA-AES256-CBC-SHA",
                         "ECDHE-ECDSA-AES128-GCM-SHA256",
                         "ECDHE-RSA-AES128-GCM-SHA256",
                         "ECDHE-RSA-AES128-SHA256",
                         "ECDHE-RSA-AES128-CBC-SHA",
                         "AES256-GCM-SHA384",
                         "AES256-SHA256",
                         "AES256-SHA",
                         "AES128-GCM-SHA256",
                         "AES128-SHA256",
                         "AES128-SHA"
                     ]
                 },
             
             "webtls": {  //Dynamci Rule
                 "class": "TLS_Server",  
                 "certificates": [
                   {
                     "certificate": "webcert"
                   }
                 ],
                "ciphers": "ECDHE-A384:ECDHE-RSA-AES256-SHA384-SHA",
                     "renegotiationEnabled": false,
                     "class": "TLS_Server",
                     "tls1_0Enabled": false,
                     "tls1_1Enabled": false,
                     "tls1_2Enabled": true,
                     "tls1_3Enabled": false,
                     "singleUseDhEnabled": false,
                     "insertEmptyFragmentsEnabled": false
             },
             
             "webcert": {  //Dynamic Block.  Will need your help on passing the certs in a secure mannaer
                 "class": "Certificate",
                 "remark": "in practice we recommend using a passphrase",
                 "certificate": "-----BEGIN CERt-----\nJc\n-----END CERTIFICATE-----",
                 "privateKey": " RSA PRIVATE KEY-----",
                 "passphrase": {
                   "ciphertext": "==",
                   "protected": ""
                 }
             }
         }
 
    }    

where there are a couple of nested blocks having STATIC or DYNAMIC values, how to generate a proper formatted JSON file out of it. Please help!

profile picture
preguntada hace un mes101 visualizaciones
No hay respuestas

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas