Glue Tables - Tag Parameters in PartitionKeys

Question

Hello,

I'm creating a table following the documentation [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue/client/create_table.html#create-table), but when I run the create_table command, I get the error `An error occurred (InvalidInputException) when calling the CreateTable operation: Parameters not supported for partition columns`.

In the [AWS documentation ](https://docs.aws.amazon.com/glue/latest/webapi/API_CreateTable.html), the parameters tag inside the partitionKeys is supported but I tried to create the tag in the aws console and got the same error.

I used a code similar to this using the boto3 API

```
import boto3

glue = boto3.client('glue', region_name='us-east-1')

table_input = {
    'Name': 'dummyTable',
    'Description': 'Test parameters on partitionKeys',
    'TableType': 'EXTERNAL_TABLE',
    'StorageDescriptor': {
        'Columns': [
                      {
                        "Name": "id",
                        "Type": "int",
                        "Parameters": {
                          "type": "int",
                          "index": "true"
                        }
                      },
                      {
                        "Name": "name",
                        "Type": "string"
                      },
                      {
                        "Name": "age",
                        "Type": "bigint"
                      },
                      {
                        "Name": "city",
                        "Type": "string"
                      }
                    ],
        'Location': "s3://my-bucket/dummytable",
        "Parameters": {
            "classification": "parquet",
            "typeOfData": "file"
        },
        'InputFormat': 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat',
        'OutputFormat': 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat',
        'SerdeInfo': {'SerializationLibrary': 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
        }
    },
    'PartitionKeys': [
            {
                'Name': 'active',
                'Type': 'int',
                'Comment': 'Test Partition',
                'Parameters': {
                    'index': 'True'
                }
            },
        ]
}

createTable = glue.create_table(
        DatabaseName='sample',
        TableInput=table_input)
```

Answer

Ninar, do you think we can use this parameter in future updates?

Answer

Hello,

I have tested this and can confirm that adding `Parameters` under `PartitionKeys` is throwing below error.

**Error:** `botocore.errorfactory.InvalidInputException: An error occurred (InvalidInputException) when calling the CreateTable operation: Parameters not supported for partition columns`

I am going to further test this and will update soon.

Thanks
Nihar

Answer

Hi,

This seems to be a documentation issue. The PartitionKeys object is an array of Column objects, which contain the Parameters property. This property can be set in other Column objects, but not when that Column represents a partition column in a table, according to our code. It looks like our documentation was made with the knowledge that columns can have parameters - but not knowing this limitation. We support parameters on normal table columns for the Lake Formation Michigan use cases - so we are going to open discussion as to whether this capability should be opened everywhere. In the meantime, we have put in a request to change the documentation to reflect what is currently possible.

Thanks 
Nihar

---

*Note : Please upvote/accept my answer if it helped you.*

Glue Tables - Tag Parameters in PartitionKeys

Relevant content