Terraform JSON parsing issue

0

Hi AWS, I have to generate a JSON file from .tfvars file, i.e. terraform.tfvars will be changed to terraform.tfvars.json. However I found a plugin tfvars-json https://github.com/alisdair/tfvars-json but while I am running the command I got an error: tfvars-json: command not found.

Second way I tried to solve the issue was using data external source reading python script which is responsible for passing the value dynamically in terraform.tfvars.json file. Here are the code snippets:

data.tf:

data "external" "addons" {
  program = ["python", "${path.module}/test.py"]
}

variables.tf:

variable "addons" {
  type = list(object({
    name    = string
    version = string
  }))
}

test.py:

import json

# Load existing terraform.tfvars.json content
with open('terraform.tfvars.json', 'r') as f:
    existing_data = json.load(f)

# Define new values for the addons variable
addons_data = [
    {"name": "vpc-cni", "version": "v1.18.0-eksbuild.1"},
    {"name": "kube-proxy", "version": "v1.28.6-eksbuild.2"}
]

# Update the addons variable in the existing data
existing_data["addons"] = addons_data

# Print the updated data in JSON format
print(json.dumps(existing_data))

terraform.tfvars.json:

{
  "aws_region": "us-east-1",
  "aws_access_key": "AKIAXXXXXXXXXXXXXXXXXXXX",
  "aws_secret_access_key": "ny0oaSdwXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "default_tags": {
    "primary_owner": "Arjun Goel",
    "secondary_owner": "Harsh Goel",
    "project_name": "EKS-POC-PROJECT"
  },
  "cidr_block": "192.168.0.0/16",
  "public_subnet_count": 2,
  "private_subnet_count": 2,
  "cluster_name": "eks-cluster-poc",
  "cloudwatch_log_group_name": "eks_cluster_cw_log_group",
  "cloudwatch_log_stream_name": "eks_cluster_cw_log_stream",
  "eks_service_role": "eks_iam_service_role",
  "eks_workernode_role": "eks_iam_worker_node_role",
  "launch_template_name": "eks_worker_node_lt",
  "addons": []
}

When I ran the python script standalone it is adding up the values dynamically for key "addons" in terraform.tfvars.json while when I am running the command terraform plan though it is populating the values, but it is throwing this error too:

Error: Unexpected External Program Results │ │ with data.external.addons, │ on data.tf line 17, in data "external" "addons": │ 17: program = ["python", "${path.module}/test.py"] │ │ The data source received unexpected results after executing the program. │ │ Program output must be a JSON encoded map of string keys and string values. │ │ If the error is unclear, the output can be viewed by enabling Terraform's logging at TRACE level. Terraform documentation on logging: │ https://www.terraform.io/internals/debugging │ │ Program: C:\Users\amang\AppData\Local\Programs\Python\Python311\python.exe │ Result Error: json: cannot unmarshal object into Go value of type string

This is the original JSON file with content:

{
    "class": "AS3",         //Satatic
    "action": "deploy",     //Satatic
    "persist": true,        //Satatic
    "declaration": {          //Satatic
       "class": "ADC",        //Satatic
       "schemaVersion": "3.37.0",          //Satatic
       "id": "example-declaration-tenant5",          //Satatic
       "label":"Sample Tenant5",          //Satatic
       "remark": "Tenant5 test for AS3",          //Satatic
       
       "Tenant5": {           //Satatic
          "class": "Tenant",          //Satatic
          "Shared": {    //Static Block
             "class": "Application",          //Satatic
             "template": "shared",        //Satatic
 
             "as3-tenant5-test": {   //Dynamic Block  Application #1
                 "remark": "as3-tenant5-test",  //Dynamic Block
                 "class": "Service_HTTP", //Dynamic Block
                 "iRules": [//Dynamic Block
                     "AS3-TEST3-IRULE"   //Dynamic Block. listo of rules defined here will have a subsequent config definition like line #77
                 ],
                 "profileTCP": {  //Dynamic Block
                     "bigip": "/Tenant5/tcp"
                 },
                 "profileHTTP": {  //Dynamic Block.  
                     "use": "http_hsts_xf1"  //Dynamic Block.  http_hsts_xf1 will be further defined as line #98
                 },
                  
                 "virtualAddresses": [  //Dynamic Block
                     "${VIP_ADDRESS}"
                 ],
                 "virtualPort": 80,  //Dynamic Block
                 "pool": "pool30",                //Dynamic Block
                 "snat": "none"  //Dynamic Block
             },
             
             "as3-tenant5-test2": {
                 "remark": "as3-tenant5-test2",                 
                 "class": "Service_HTTP",
                 "iRules": [
                     "AS3-TEST3-IRULE"
                 ],
                 "profileTCP": {
                     "bigip": "/Tenant5/tcp"
                 },
                 "virtualAddresses": [
                     "${VIP_ADDRESS_2}"
                 ],
                 "virtualPort": 80,
                 "pool": "pool30",
                 "snat": "none"
             },
 
             "as3-tenant5-test3": {   //Dynamic Block
                 "remark": "as3-tenant5-test3", //Dynamic Block
                 "class": "Service_HTTPS",  //Dynamic Block.
                  "iRules": [
                       "AS3-TEST3-IRULE"
                 ],
                 "profileTCP": {
                     "bigip": "/Tenant5/tcp"
                 },
                 "virtualAddresses": [
                     "3.3.3.3"
                 ],
                 "virtualPort": 443,
                 "snat": "auto",
                 "pool": "pool30",
                 },     
            
                 "serverTLS": "webtls",  //Dynamic  Block.  THis will further be defined in webtls block like line #154
                 "redirect80": false     //Dynamic  Block
             },
 
             "AS3-TEST3-IRULE": {  //Dynamic Block 
                 "class": "iRule",
                 "iRule": "when HTTP_REQUEST { HTTP::respond 200 content {<html><head><title>Hi team</title</head><body><h1>Hello team!</h1></body></html> }}"
              },
 
             "pool30": {  //Dynamic Block
                 "class": "Pool",
                 "monitors": [
                     "http"
                 ],
                 "members": [
                 {
                     "servicePort": 8080,
                      "serverAddresses": [
                         "3.3.3.200",
                         "3.3.3.201"
                       ]
                 }
                 ]
             },
             
             "http_hsts_xf1": {  Dynamic Block
                     "class": "HTTP_Profile",
                     "knownMethods": [
                         "CONNECT",
                         "DELETE",
                         "GET",
                         "HEAD",
                         "LOCK",
                         "OPTIONS",
                         "POST",
                         "PROPFIND",
                         "PUT",
                         "TRACE",
                         "UNLOCK"
                     ],
                     "maxHeaderCount": 64,
                     "maxHeaderSize": 32768,
                     "maxRequests": 0,
                     "pipelineAction": "allow",
                     "truncatedRedirects": false,
                     "unknownMethodAction": "allow",
                     "hstsIncludeSubdomains": false,
                     "hstsPeriod": 31536000,
                     "hstsInsert": true,
                     "xForwardedFor": true,
                     "multiplexTransformations": true,
                     "proxyType": "reverse",
                     "rewriteRedirects": "none",
                     "requestChunking": "sustain",
                     "responseChunking": "sustain",
                     "serverHeaderValue": "BigIP",
                     "viaRequest": "preserve",
                     "viaResponse": "preserve"
                 },
             
             
              "aes_128-256_cipher-rules": {  //Dynamic Block
                     "class": "Cipher_Rule",
                     "cipherSuites": [
                         "ECDHE-ECDSA-AES256-GCM-SHA384",
                         "ECDHE-RSA-AES256-GCM-SHA384",
                         "ECDHE-RSA-AES256-SHA384",
                         "ECDHE-RSA-AES256-CBC-SHA",
                         "ECDHE-ECDSA-AES128-GCM-SHA256",
                         "ECDHE-RSA-AES128-GCM-SHA256",
                         "ECDHE-RSA-AES128-SHA256",
                         "ECDHE-RSA-AES128-CBC-SHA",
                         "AES256-GCM-SHA384",
                         "AES256-SHA256",
                         "AES256-SHA",
                         "AES128-GCM-SHA256",
                         "AES128-SHA256",
                         "AES128-SHA"
                     ]
                 },
             
             "webtls": {  //Dynamci Rule
                 "class": "TLS_Server",  
                 "certificates": [
                   {
                     "certificate": "webcert"
                   }
                 ],
                "ciphers": "ECDHE-A384:ECDHE-RSA-AES256-SHA384-SHA",
                     "renegotiationEnabled": false,
                     "class": "TLS_Server",
                     "tls1_0Enabled": false,
                     "tls1_1Enabled": false,
                     "tls1_2Enabled": true,
                     "tls1_3Enabled": false,
                     "singleUseDhEnabled": false,
                     "insertEmptyFragmentsEnabled": false
             },
             
             "webcert": {  //Dynamic Block.  Will need your help on passing the certs in a secure mannaer
                 "class": "Certificate",
                 "remark": "in practice we recommend using a passphrase",
                 "certificate": "-----BEGIN CERt-----\nJc\n-----END CERTIFICATE-----",
                 "privateKey": " RSA PRIVATE KEY-----",
                 "passphrase": {
                   "ciphertext": "==",
                   "protected": ""
                 }
             }
         }
 
    }    

where there are a couple of nested blocks having STATIC or DYNAMIC values, how to generate a proper formatted JSON file out of it. Please help!

profile picture
asked 21 days ago90 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions