Hi! I'm an infrastructure engineering leader at HubSpot.
HubSpot is a marketing, sales, and services platform for growing businesses, and we're looking for senior engineer(s) for our lowest-level infrastructure provisioning / orchestration / tooling teams. One of our big projects revolves around building an end-to-end declarative representation of our infrastructure based entirely around Kubernetes and composable custom resource definitions (CRDs).
For our standard applications, a deploy is done with the push of a button and app developers do not need to think about Kubernetes manifests or even the existence of Kubernetes at all. So that portion of the problem is already completed at scale. What remains is handling all of the lower-level pieces of infrastructure and stateful services which are not easy to generalize for. We do this by extending Kubernetes with additional building blocks, rather than attempting to make an all-encompassing framework for doing so since such a framework would approach the complexity of Kubernetes itself. It can almost be thought of as building a lower-level PaaS for infrastructure teams.
With this model, each infra team builds CustomResourceDefinitions and operators to represent their services. For example:
- The Kafka team allows for creating a Kafka cluster via a KafkaCluster CRD
- The Kafka operator creates the ZooKeeper cluster (an underlying dependency) for that with a ZooKeeperCluster CRD, which is handled by the ZooKeeper team's operators
- Those operators get all of the secrets they need via our SecretTemplate CRD, handled by Infra Orch's kube secret operator (which injects Vault secrets into appropriate Kubernetes secrets)
The vision is to use CRDs for every un-automated corner of infrastructure, all the way down to the EC2 VPCs themselves. We're striving to reach a point where creating a new data center is done by applying the CRDs that define the top-level clusters, and everything else automatically is configured end-to-end with no manual human effort. For any case where there is a common problem that all teams running on Kubernetes must solve, our job is to find an abstraction for that problem.
Thanks for reading!
** This is just the role that I'm personally hiring for. HubSpot is hiring like crazy across the board -- check out https://www.hubspot.com/careers/jobs?q=engineer for more details!