What is Substation?

Substation is an open source data pipeline and transformation toolkit written in Go that has these features:

  • Deploys modular, cloud native data pipelines that support 100s of unique designs
  • Offers event-driven ingest, transform, load (ITL) applications that evaluate, process, and deliver data in real-time
  • Provides Go packages for creating custom data processing applications


Modular, Cloud Native Data Pipelines and Microservices

  • Design unique data pipelines managed using infrastructure as code
    • Modular architecture supports 100s of unique pipeline designs
    • Use microservices to integrate with external systems and users
  • Affordable data processing at massive scale
    • Scales beyond 100,000 events per second (EPS)
    • Costs as low as $10/TB for simple pipelines and up to $40/TB for complex pipelines
  • Near zero maintenance by running on serverless cloud services

Evaluate, Process, & Deliver Data

Build Custom Data Processing Apps

Differentiating Benefits

Substation differentiates itself from other data pipeline solutions in several ways:

  • Fully serverless — you’ll never manage a server or think about sizing a cluster
  • Designed for scale — automatically scales from 1 to 100,000+ EPS with no intervention required by an engineer
  • Infrastructure and configurations as code — uses Terraform, Jsonnet, and AWS AppConfig, which means you can deploy unique, reusable data pipelines in minutes
  • Cost efficient — we went the extra mile to make things affordable, including creating a compliant, minimal version of the Kinesis Producer and Client Libraries in Go
  • Extensible — we expose the Substation core as Go packages, so you can build your own custom data processing applications