Building a Unified Namespace foundation: Implementing scalable manufacturing operations with AWS

Introduction

Manufacturing organizations stand at the threshold of a digital transformation that promises to revolutionize how they operate, compete, and innovate. At the heart of this transformation is a powerful concept: the UNS. This architectural approach is rapidly becoming the foundation for next-generation smart manufacturing so that organizations can unlock the full potential of their operational data.

At AWS, UNS implementations can transform manufacturing operations across the globe. When manufacturers create a standardized, real-time data framework, they can:

• Break down traditional barriers between systems.

• Create seamless integration from the factory floor to the cloud.

• Establish the foundation for advanced analytics and artificial intelligence applications.

Whether you're just starting to explore UNS, or looking to scale existing implementations, this article provides comprehensive insights into how to create a foundation for manufacturing excellence through UNS. This article also explores key architectural patterns that have proven successful across diverse manufacturing environments. Lastly, this article shares best practices on how to use AWS services to implement UNS, and includes practical steps that you can take to begin your UNS journey. Our goal is to equip you with the knowledge and tools that you need to transform your manufacturing operations through unified data architecture.

The manufacturing data challenge

Modern manufacturing facilities are powerhouses of digital innovation, where UNS has proven to be vital to scale operations and unlock new capabilities. Consider the sophisticated network of interconnected systems that make up a typical automotive manufacturing facility:

• Programmable Logic Controllers (PLCs) that control high-speed assembly lines

• Advanced quality inspection stations

• Automated material handling systems

• Enterprise applications and systems, such as Manufacturing Execution System (MES) and Enterprise Resource Planning (ERP)

Each of these systems generates valuable operational data that, when collected through UNS, creates opportunities for operational excellence. For example, consider a typical manufacturing plant with a facility that operates over 100 robots, 50 PLCs, and multiple enterprise systems. Data integration for this plant requires complex point-to-point connections and can be difficult to implement new use cases or scale existing solutions for. This plant needs a more flexible, scalable approach to data management than what currently exists.

Through our work with global manufacturers, we've seen how some of the largest manufacturing programs on AWS leverage UNS as their foundation for digital transformation. Leading automotive and industrial manufacturers have successfully scaled their UNS implementations from single-line proof-of-concepts to enterprise-wide deployments that span multiple continents. These global UNS implementations process millions of data points daily, creating real-time process optimization, predictive maintenance, and automated quality control across entire manufacturing networks. This proven scalability is what makes UNS not just another data integration solution, but a fundamental pillar of modern manufacturing operations.

Building the solution

To address these manufacturing data challenges, we developed a hybrid architecture that uses both AWS services and open-source components. At its core, an enterprise-wide MQTT message broker handles communication. Industrial messaging standards enhance the communication for structured data modeling and API specifications for standardized documentation. The edge computing layer provides local processing and data filtering, with options for container orchestration to enhance reliability and scalability. Organizations can choose between two options that both offer seamless integration from the factory floor to cloud analytics:

• A cloud-native architecture through purpose-built IoT services

• A more flexible, cloud-agnostic containerized approach

Core infrastructure

At the center of the UNS architecture is AWS IoT Core, a service that acts as the enterprise-wide MQTT broker. This service isn't merely a message router -- it's the central nervous system that processes millions of manufacturing messages daily across global facilities. The architecture transforms raw operational data into actionable insights, and maintains the reliability and security requirements of industrial operations.

When you consider the implementation of a UNS, it's a best practice to evaluate the standards and protocols that the UNS uses. These standards and protocols can significantly affect consistency and interoperability within the manufacturing environment. Among the various technologies available, Sparkplug and AsyncAPI are good options that can be foundational elements in building a robust UNS infrastructure.

While MQTT serves as our primary communication protocol, this protocol alone doesn't provide the structured data modeling that's needed for complex manufacturing environments. This is where Sparkplug becomes essential. As an open-source software specification, Sparkplug adds crucial structure to MQTT communications that are vital in industrial settings.

Sparkplug defines a rigid topic namespace structure that all clients must follow, as seen in the following example:

namespace/group_id/message_type/edge_node_id/[device_id]

The topic hierarchy mirrors the physical and logical organization of manufacturing operations, and follows a carefully designed structure that scales from individual sensors to enterprise-wide systems. For example, consider a global automotive manufacturer with plants across multiple continents. For this plant, their topic structure flows from enterprise level down to individual process measurements, and creates both granular control and broad analytical insights:

enterprise/north-america/detroit/assembly/line-3/robot-7/telemetry

This hierarchical approach allows for precise message routing while maintaining logical organization. When a critical quality measurement occurs on the production line, the data flows through this structure and gathers contextual information at each level.

The following is a payload example based on the MQTT topic hierarchy:

# Base topic structure following ISA-95 model
{enterprise}/{site}/{area}/{line}/{device}/{metric}

# Example implementations:

# Real-time equipment telemetry
manufacturing/detroit/assembly/line1/robot-01/telemetry
    └── payload: {
        "temperature": 45.2,
        "cycleTime": 23.4,
        "status": "RUNNING",
        "timestamp": "2024-01-15T14:22:31Z"
    }

# Quality inspection data
manufacturing/detroit/quality/station3/vision-system/measurements
    └── payload: {
        "partId": "A123-456",
        "inspectionPoints": [
            {"x": 10.2, "y": 15.3, "tolerance": 0.01},
            {"x": 12.4, "y": 18.7, "tolerance": 0.01}
        ],
        "decision": "PASS",
        "confidence": 0.998
    }

# Production metrics
manufacturing/detroit/assembly/line1/metrics/oee
    └── payload: {
        "availability": 0.95,
        "performance": 0.88,
        "quality": 0.997,
        "oee": 0.83,
        "interval": "1H",
        "timestamp": "2024-01-15T14:00:00Z"
    }

To maintain consistency across manufacturing operations, this solution uses AsyncAPI to document all message flows and data models. This approach proves to be vital, whether you use Sparkplug or implement custom data models. AsyncAPI provides a standard way to describe event-driven architectures, and makes it easier for teams to understand and implement message flows.

We can use AsyncAPI specifications to document the following information:

• Topic structures and hierarchies

• Message formats and payloads

• Security requirements

• Server configurations

• Client-server interactions

This documentation becomes particularly valuable in large manufacturing environments where multiple teams must understand and interact with the message infrastructure. The documentation serves as both a technical specification and contract between different systems and teams.

Edge computing layer

The edge computing layer in our UNS architecture serves as the critical interface between the physical world of manufacturing and digital realm of data analytics. For example, consider a welding station on the assembly line. Rather than sending every detail of the welding process to the cloud, edge processing can analyze the data in real-time and look for deviations from optimal parameters. Edge processing can transmit only significant events or summarized data upstream, and dramatically reduces network load while maintaining comprehensive oversight of production.

In this situation, AWS IoT Greengrass can provide a robust platform for local processing that addresses the unique challenges of industrial environments. Through custom components at the edge, we can implement intelligent data preprocessing and filtering directly at the source. This way, instead of overwhelming our central systems with raw data, we can distill the data into meaningful insights where it's generated.

Many manufacturers initially consider lightweight edge solutions to minimize upfront infrastructure costs and reduce operational complexity. However, our experience with large-scale implementations has shown the value of a more robust, enterprise-grade approach at the edge. With Amazon Elastic Kubernetes System (Amazon EKS) Anywhere or Amazon EKS Hybrid Nodes, manufacturers can deploy containerized MQTT brokers, such as EMQX or Mosquitto, with the full power of Kubernetes orchestration. The manufacturers then have the reliability and scalability needed for critical manufacturing operations. This architecture transforms edge computing from a potential point of failure into a resilient component of manufacturing infrastructure.

For a typical automotive production line, thousands of sensors generate critical data every second. Because of this, traditional edge solutions might struggle with intermittent connectivity or high message volumes. With a Kubernetes-orchestrated MQTT broker deployment, the system can maintain operations during partial failures by automatically redistributing loads and maintaining message persistence.

For manufacturers with legacy equipment, this architecture provides scalable protocol translation services. Rather than requiring expensive equipment upgrades, you can deploy and scale containerized protocol converters as needed, bringing older machinery into modern data fabric without compromising reliability. The result is an edge computing layer that does more than just collect and forward data. This layer provides a robust foundation for the entire UNS implementation that can scale from individual production lines to global manufacturing networks. Most importantly, this layer does all these tasks while maintaining the reliability and performance that modern manufacturing operations demand.

Developing your architecture

When establishing a UNS architecture for industrial data integration, organizations can follow two distinct strategic paths. The first approach uses AWS native architecture and purpose-built AWS IoT services that are designed specifically for industrial connectivity and data management.

The second approach implements a Kubernetes base architecture and employs containerized components that offer deployment flexibility and potential cloud provider independence.

Each option presents its own set of capabilities and considerations. This way, organizations can select the implementation strategy that best aligns with their technical requirements and long-term cloud strategy.

In the world of industrial IoT, flexibility is as valuable as functionality. This is where the Kubernetes-based architecture for UNS excels, offering organizations a cloud-agnostic approach to connect their operational technology with advanced analytics capabilities. At the edge, where industrial assets generate continuous streams of operational data, a containerized Site UNS Message Broker orchestrates the local data environment. This Kubernetes-powered system deploys the following specialized containers that work in harmony:

• An MQTT broker that manages the publish and subscribe messaging patterns from industrial equipment.

• Edge inferencing containers that process data locally to allow rapid decision-making.

• Data ingestion modules that prepare information for its journey to the enterprise level.

When this standardized data reaches the cloud, the data flows into AWS IoT Core or Amazon Managed Streaming for Apache Kafka (Amazon MSK). These services act as the enterprise-wide message broker and create a consistent UNS that spans from the factory floor to the cloud. Then, the architecture connects seamlessly with data analytics platforms and AI and machine learning (AI/ML) services that transform raw industrial data into actionable insights. This architecture is particularly powerful because of its containerized nature. With Kubernetes, organizations can deploy a consistent messaging infrastructure across diverse environments, making the solution portable across different cloud providers or in hybrid scenarios. This cloud-agnostic approach gives businesses the freedom to evolve their cloud strategy without disrupting their UNS implementation. The architecture maintains bidirectional communication throughout  the process, and allows cloud-generated insights to flow back to influence operations at the edge. Meanwhile, the consistent topic structure and hierarchy keep data properly organized and accessible, regardless of where they reside in the system.

For organizations that want a flexible yet robust approach to implement a UNS, this architecture offers an option that blends the benefits of UNS with the deployment flexibility of containerization.

Implementing your strategy

Our implementation follows a structured approach:

• We define the AsyncAPI specification for all message flows and create clear documentation of our data models and interaction patterns.

• For the on-premises infrastructure, we implement a hybrid architecture with local MQTT brokers at each manufacturing site. This architecture feeds into an enterprise-level broker in the cloud.

• The message broker implementation follows the Sparkplug specification or custom data model and creates a consistent handling of device states, data transmission, and command patterns.

• We build applications against this well-defined message infrastructure, with a clear understanding of the available data points and interaction patterns.

This standardized approach to message handling and documentation has proven crucial in maintaining consistency across large manufacturing operations, significantly reduces integration time, and improves system reliability. Through these standards, we create a robust foundation for scaling our manufacturing data infrastructure while maintaining interoperability across all components of the system.

Conclusion

UNS challenges reveal a profound truth about industrial data integration: the vision of a single, comprehensive source of truth remains constantly just beyond reach. This vision isn't reachable because of technical limitations, but because of the complex nature of industrial environments. The most successful implementations are an orchestrated diversity, rather than a forced uniformity. These implementations build architectures that respect specialized systems and create a coherent data flow. They recognize that high-volume historical data can benefit from direct replication paths, while real-time states require event-driven messaging. The truth of this complex issue isn't that unity is impossible or must mean absolute homogeneity. True unification comes from intelligently navigating differences rather than removing them, and using thoughtful orchestration within a coherent framework. When manufacturers understand this, they can stop pursuing unrealistic expectations and start building practical architectures that deliver real business value in the complex reality of industrial operations. To test out architecture patterns, see Build a Unified Namespace and connect factories at scale.

About the authors

Vanshaj Kochar

Vanshaj Kochar is a Senior Solutions Architect in Automotive and Manufacturing IBU who helps customers build secure, scalable, and robust cloud solutions in the AWS cloud. With a background in Electronics and Embedded Engineering, he's an active contributor to the automotive and IoT technical field communities. Outside of work, he likes hiking, playing cricket, and spending quality time with his family.

Robert Oikarinen

Robert Oikarinen is a Principal Cloud Architect with AWS Professional Services who focuses on supporting automotive and manufacturing customers. Based out of Detroit, Michigan, he has been with Amazon over 10 years and helps customers integrate IoT data with ML models.

Chirayu Parikh

Chirayu Parikh is a Senior Technical Account Manager within the AWS Enterprise Support's Automotive Strategic Industries organization. In his work, he supports large automotive companies with their cloud journey. Before AWS, he spent over a decade helping customers with enterprise networking solutions.