Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

[Thank Goodness It’s Search]: Building top-down Faceted Navigation for Retail Catalogs: Part 2 - Nested queries in OpenSearch

9 minute read
Content level: Intermediate
1

This post offers a detailed, example-driven walkthrough on building faceted navigation that lets users explore a catalog both top-down and bottom-up. It's an essential skill for anyone working with semi-structured data—whether it's product catalogs, user reviews, or behavioral event logs.

Welcome to Thank Goodness It’s Search series—your Friday fix of OpenSearch learnings, feature drops, and real-world solutions. I will keep it short, sharp, and search-focused—so you can end your week a little more knowledge on Search than you started.

This article builds on Building Faceted Navigation for Retail Catalogs: Part1 - Nested field types in OpenSearch, where we explored the difference between using arrays and defining fields as nested in OpenSearch. Now, we’ll apply those concepts in practice by creating a faceted navigation experience using a synthetic cosmetics catalog.

Step 1: Create the cosmetics index. Keep in mind that OpenSearch automatically infers the schema. If you don’t explicitly define a field as "nested", it will be treated as a simple array—leading to incorrect query behavior when dealing with objects.

PUT /cosmetic-makeup-index
{ 
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  },
  "mappings": {
    "properties": {
    "category": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
    "subcategory": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "axiology": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "skintype": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "Smudge-Free": {
        "type": "long"
      },
      "skuinfo": {
        "type": "nested",
        "properties": {
          "availableInStore": {
            "type": "long"
          },
          "color": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "flavor": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

Note, in the above cosmetic index, the category, subcategory, axiology, skintype are product-level attribtues. The field skuinfo is also at the parent product-level, its defined with field type nested and represents the SKU variants. The field skuinfo contains multiple attributes such as skuid, availableInStore, color, flavor. A product can have one or more sku variants.

Step 2: Bulk ingest the synthetic cosmetics data

Let's index our synthetic cosmetics data with various product attributes and SKU variants

POST _bulk
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick",  "axiology": "Vegan", "skintype": "Normal", "Smudge-Free": 0 , "skuinfo" : [{ "skuid":"00001", "flavor" : "Rose", "availableInStore":1, "color" : "Sky Red"},{ "skuid":"00002","flavor" : "Strawberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"00003", "flavor" : "Raspberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"00004","flavor" : "Citrus", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"00005", "flavor" : "Rose", "availableInStore":1, "color" : "Hot Pink"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipgloss",  "axiology": "Vegan", "skintype": "Dry", "Smudge-Free": 0, "skuinfo" : [{ "skuid":"10001", "flavor" : "Rose", "availableInStore":1, "color" : "Sky Red"},{ "skuid":"10002","flavor" : "Strawberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"10003", "flavor" : "Raspberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"10004","flavor" : "Strawberry", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"10005", "flavor" : "Citrus", "availableInStore":1, "color" : "Hot Pink"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick", "axiology": "Vegan", "skintype": "Oily", "Smudge-Free": 1, "skuinfo" : [{ "skuid":"20001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Sky Red"},{ "skuid":"20002","flavor" : "Citrus", "availableInStore":1, "color" : "French Rose"},{ "skuid":"20003", "flavor" : "Raspberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"20004","flavor" : "Orange", "availableInStore":1, "color" : "French Rose"},{ "skuid":"20005", "flavor" : "Citrus", "availableInStore":1, "color" : "Hot Pink"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick",  "axiology": "Cruelty-Free", "skintype": "Normal", "Smudge-Free": 1, "skuinfo" : [{ "skuid":"30001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"30002","flavor" : "Strawberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"30003", "flavor" : "Strawberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"30004","flavor" : "Orange", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"30005", "flavor" : "Citrus", "availableInStore":1, "color" : "Hot Pink"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick",  "axiology": "Vegan", "skintype": "Dry", "Smudge-Free": 0, "skuinfo" : [{ "skuid":"40001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"40002","flavor" : "Strawberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"40003", "flavor" : "Strawberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"40004","flavor" : "Orange", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"40005", "flavor" : "Citrus", "availableInStore":1, "color" : "French Rose"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipgloss", "axiology": "Vegan", "skintype": "Dry", "Smudge-Free": 0, "skuinfo" : [{ "skuid":"50001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"50002","flavor" : "Strawberry", "availableInStore":1, "color" : "Sky Red"},{ "skuid":"50003", "flavor" : "Rose", "availableInStore":1, "color" : "Magenta"},{ "skuid":"50004","flavor" : "Orange", "availableInStore":1, "color" : "French Rose"},{ "skuid":"50005", "flavor" : "Citrus", "availableInStore":1, "color" : "French Rose"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick",  "axiology": "Cruelty-Free", "skintype": "Normal", "Smudge-Free": 0, "skuinfo" : [{ "skuid":"60001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"60002","flavor" : "Strawberry", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"60003", "flavor" : "Strawberry", "availableInStore":1, "color" : "Magenta"},{ "skuid":"60004","flavor" : "Apple", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"60005", "flavor" : "Orange", "availableInStore":1, "color" : "French Rose"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipgloss",  "axiology": "Cruelty-Free", "skintype": "Combination", "Smudge-Free": 1, "skuinfo" : [{ "skuid":"70001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"70002","flavor" : "Strawberry", "availableInStore":1, "color" : "Sky Red"},{ "skuid":"70003", "flavor" : "Rose", "availableInStore":1, "color" : "Magenta"},{ "skuid":"70004","flavor" : "Raspberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"70005", "flavor" : "Citrus", "availableInStore":1, "color" : "Hot Pink"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipstick",  "axiology": "Vegan", "skintype": "Oily", "Smudge-Free": 1, "skuinfo" : [{ "skuid":"80001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"80002","flavor" : "Strawberry", "availableInStore":1, "color" : "French Rose"},{ "skuid":"80003", "flavor" : "Citrus", "availableInStore":1, "color" : "Magenta"},{ "skuid":"80004","flavor" : "Strawberry", "availableInStore":0, "color" : "Flamingo"},{ "skuid":"80005", "flavor" : "Citrus", "availableInStore":1, "color" : "Sky Red"}]}
{ "index" : { "_index": "cosmetic-makeup-index"}}
{"category": "Makeup", "subcategory": "Lipgloss", "axiology": "Cruelty-Free", "skintype": "Combination", "Smudge-Free": 1, "skuinfo" : [{ "skuid":"90001", "flavor" : "Raspberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"90002","flavor" : "Strawberry", "availableInStore":1, "color" : "Hot Pink"},{ "skuid":"90003", "flavor" : "Rose", "availableInStore":1, "color" : "Magenta"},{ "skuid":"90004","flavor" : "Rose", "availableInStore":1, "color" : "French Rose"},{ "skuid":"90005", "flavor" : "Apple", "availableInStore":1, "color" : "Sky Red"}]}

Step3, Navigate the category 'Makeup' and view the available subcategories

Note, we have set the size to 0 in the query, which means we are not interested in the actual product results for this post. This is a common practice in faceted navigation where we want to retrieve only the facets (subcategories in this case) without the actual product details.

GET cosmetic-makeup-index/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "category.keyword": {
              "value": "Makeup"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "subcat": {
      "terms": {
        "field": "subcategory.keyword",
        "size": 10
      }
    }
  }
}

Step4, Select the subcategory 'Lipstick' and view the available attributes and nested sku variants

To view the attributes and nested SKU variants for the selected subcategory 'Lipstick', we can use a nested aggregation to explore the skuinfo field.

GET /cosmetic-makeup-index/_search
{
  "size": 0,
  "query": {
    "match": {
      "subcategory": "Lipstick"
    }
  },
  "aggs": {
    "attributes": {
      "nested": {
        "path": "skuinfo"
      },
      "aggs": {
        "flavors": {
          "terms": {
            "field": "skuinfo.flavor.keyword",
            "size": 10
          }
        },
        "colors": {
          "terms": {
            "field": "skuinfo.color.keyword",
            "size": 10
          }
        },
        "availableInStore": {
          "terms": {
            "field": "skuinfo.availableInStore",
            "size": 10
          }
        }
      }
    }
  }
}

Step5, Select the color variant "Hot Pink" and view all the refresh all the facets, applying the filter

In this example, we will filter the results for the color "Hot Pink". The query will return all products in the 'Lipstick' subcategory that have the specified color variant. Note, we are using a nested query to filter the skuinfo field for the color "Hot Pink" and the selected filter "Hot Pink" is applied to all aggregation that needs to display the correct set of filters.

GET /cosmetic-makeup-index/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "subcategory.keyword": {
              "value": "Lipstick"
            }
          }
        },
        {
          "nested": {
            "path": "skuinfo",
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "skuinfo.color": "Hot Pink"
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "attributes": {
      "nested": {
        "path": "skuinfo"
      },
      "aggs": {
        "colr": {
          "filter": {
            "term": {
              "skuinfo.color.keyword": "Hot Pink"
            }
          },  
          "aggs": {
            "c1": {
              "terms": {
                "field": "skuinfo.color.keyword",
                "size": 10
              }
            }
          }
        }, 
        "flavor": {
          "filter": {
            "term": {
              "skuinfo.color.keyword": "Hot Pink"
            }
          },  
          "aggs": {
            "c1": {
              "terms": {
                "field": "skuinfo.flavor.keyword",
                "size": 10
              }
            }
          }
        }, 
        "instore": {
          "filter": {
            "term": {
              "skuinfo.color.keyword": "Hot Pink"
            }
          },  
          "aggs": {
            "c1": {
              "terms": {
                "field": "skuinfo.availableInStore",
                "size": 10
              }
            }
          }
        }
      }
    }
  }
}

Awesome! You now have a top-down faceted navigation experience using the provided synthetic cosmetics catalog in OpenSearch. This setup allows users to filter products based on categories, subcategories, and specific attributes like color and flavor, all while leveraging the power of nested fields for accurate querying.

Conclusion In this post, we have successfully built a top-down faceted navigation experience for a synthetic cosmetics catalog using OpenSearch. We started by creating an index with the necessary fields, indexed synthetic data, and then navigated through categories and subcategories. We also explored how to filter products based on attributes and nested SKU variants, allowing users to refine their search results effectively.

In the next part of this series, you will learn to implement "reverse nested" queries to build a bottom-up faceted navigation experience. Stay tuned for more insights on building robust faceted navigation systems using OpenSearch. If you have any questions or need further assistance, feel free to reach out.

Next Steps Try these queries in your OpenSearch instance and see how they work with your data. Leave your comments or questions below, and I will be happy to help. Thank you for reading, and happy coding!

References

Try this out and let me know how it goes.

See you next Friday with another search solution in the Thank Goodness It's Search series! Until then, happy searching! 🔍

No comments

Relevant content