Under the Hood

What is
Data Product.

A true data product is more than just a table. It is an independent, deployable architectural quantum consisting of code, data, and the underlying infrastructure.

View Components Explore Interfaces

The Three Pillars

Every autonomous Data Product encapsulates three distinct elements that are deployed and managed together as a single unit.

Data & Metadata

The actual historical and current datasets, alongside rich semantics, schemas, and cataloging information that make the data understandable and usable.

Code & Logic

The pipelines, transformations, API endpoints, access control policies, and testing scripts required to ingest, process, and serve the data safely.

Infrastructure

The underlying physical or cloud resources (storage buckets, compute clusters, orchestrators) provisioned via Infrastructure-as-Code to run the product.

Standardized Interfaces (Ports)

To maintain autonomy while allowing interoperability across the organization, Data Products communicate with the outside world through well-defined "ports".

Output Ports

The primary interfaces for consumers. These include highly reliable REST APIs, GraphQL, standard SQL views, or event streams designed for easy integration.

Input Ports

The mechanisms used to securely ingest operational data or data from other data products into the current product's storage.

Control & Discovery Ports

Interfaces allowing central governance platforms to monitor SLA metrics, audit logs, schema registries, and enforce global security policies automatically.

// Sample Output Port config

{
  "product_id": "customer_360",
  "version": "v1.2.0",
  "output_ports": [
    {
      "type": "SQL",
      "endpoint": "snowflake://db/schema/vw_customers",
      "sla": "99.9%",
      "refresh_rate": "real-time"
    },
    {
      "type": "REST_API",
      "endpoint": "https://api.data.inc/v1/customers",
      "auth": "OAuth2"
    }
  ]
}

What is Data Product.