Distinguished Architect, Data Platform
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Distinguished Architect leading the design of a next-generation data platform at CloudZero. Driving technical decisions and architectural innovations for scalable cloud cost management solutions.
Responsibilities:
- Define the Data Platform Architecture
- Lead end-to-end technical design for CloudZero's next-generation data platform, from event ingestion and stream processing through hot/cold storage and the query layer to the API surface
- Document architectural decisions, tradeoffs, and migration strategies with the rigor of an RFC-driven process
- Shape and drive every layer of the new architecture: event ingestion, stream processing and enrichment, real-time serving, analytical storage, query layer, and API
- Design and deliver CloudZero's real-time data pipeline from ingestion through enrichment to serving
- Establish SLOs for throughput, latency, and correctness, and build the operational playbooks that make this system trustworthy enough to replace the batch pipelines our entire product currently depends on
- Tackle real-time streaming at scale across thousands of customers simultaneously, with fault tolerance, backpressure awareness, and correctness as non-negotiables
- Redesign CloudZero's dimensional cost model to support high-cardinality, multi-dimensional cost attribution without runaway materialization costs
- Drive incremental, delta-based materialization strategies using modern open table formats, dramatically reducing expensive full-rebuild jobs and unlocking millions in annual infrastructure savings
- Assess CloudZero's current query infrastructure, drive in-flight migrations to completion, and lead the evolution of the query engine layer going forward
- Own performance optimization across partition pruning, predicate pushdown, and query planning, and set the vision for how the query layer grows as data volumes scale 10x
- Evolve CloudZero's proprietary cost attribution engine from a batch-oriented model to one that assigns complex cost dimensions by team, feature, and customer within seconds of resource usage
- Rethink enrichment, data lineage, and correctness guarantees in a streaming context
- Partner with product, infrastructure, and analytics engineering to define a multi-year data platform roadmap
- Build consensus across engineering leadership on foundational investments including table formats, streaming frameworks, query engines, and schema management
- Participate in architecture reviews, contribute to design patterns and best practices, and mentor senior and staff engineers through code review, pairing, and structured feedback
- Make everyone around you better, not by directing, but by raising the collective craft
Requirements:
- 10+ years in data engineering with a clear trajectory toward principal or staff-level architecture
- Built and operated large-scale data platforms serving tens of millions of events per day in production
- Deep experience with streaming systems such as Kafka, Kinesis, Flink, or Spark Streaming at real production throughput
- Strong hands-on fluency with modern open table formats including Apache Iceberg, Delta Lake, and Hudi, including compaction, partitioning strategy, and time-travel queries
- Designed hot/cold storage architectures with explicit latency SLOs per tier
- Proven ability to drive a data platform end to end, not just a single layer
Benefits:
- Offers Equity
- Offers Bonus
Similar Jobs
Data Architect
Data Architect evaluating and evolving data architecture for partners at EVT. Focus on ingestion, processing, and structuring of data with a collaborative team environment.
Data Engineer
Data Engineer responsible for designing, building, and maintaining data pipelines at Loopback Health. Collaborating with cross-functional teams to drive system and process improvements.
Data Engineer
Data Engineer optimizing and maintaining data pipelines in Blackline Safety's IoT-enabled safety ecosystem. Collaborating with product, engineering, and analytics teams on impactful data-driven initiatives.
Senior Ingeniero de Datos
Senior Data Engineer developing data solutions for transforming lives at Sofka. Collaborating in a close-knit team focused on continuous improvement and technical excellence.
Senior Data Engineer
Senior Data Engineer at Arsipa developing scalable data platforms and ETL/ELT pipelines. Collaboration with teams to ensure data quality, performance, and innovation in data architecture.
Data Engineer
Data Engineer shaping cloud-based data solutions for top U.S. companies. Engage with diverse projects remotely from Latin America.
Senior Data Engineer
Senior Technical Consultant delivering end-to-end data solutions and managing client relationships. Collaborating with technical team to drive project success with Microsoft technologies.
Senior Data Engineer – Data for Products
Lead the design and implementation of scalable batch and streaming data pipelines on Azure. Collaborate with product teams to ensure data quality, performance, and governance across the data lifecycle.
Data Engineer
Data Engineer maintaining scalable data pipelines for an international company in Hamburg. Requires experience with Snowflake, ETL processes and Azure Data Factory.
IS Data Warehouse Architect – III
Data Warehouse Architect designing and developing data models and ETL specifications. Collaborating on data architecture and vendor relations for a nonprofit health care organization.
Senior Director – Data Platform, Analytics
Leading the development of an integrated data platform at Harbor Compliance. Unifying fragmented data and establishing governance to empower decision-making.
Senior Data Engineer
Senior Data Engineer at D.A. Davidson optimizing data integrations and migrating legacy structures. Involves cloud architecture and coordination with product teams.
Data Architect
Professional Services Delivery Architect at Ciena guiding global delivery teams and ensuring high-quality outcomes for customer engagements. Leading complex transformations and influencing solution strategy.
Senior Data Engineer
Data Engineer designing and maintaining data pipelines for Arrow Electronics. Collaborating with teams in engineering and architecture discussions across global projects.
Data Engineer – AI, Spark, Databricks, Healthcare
Data Engineer managing advanced Spark scripts for data validation and integration at Cotiviti. Collaborating with engineering teams to ensure data integrity and compliance in healthcare data processes.
Senior Software Engineer, Data Platform Team
Senior Software Engineer designing and developing features for data platforms at Sysdig. Leading technical strategy and mentoring teams in cloud security innovations.
Data Engineering Architect – Manager
Data Engineering Architect managing scalable AWS Data Lakehouse architectures at Naveera Technology. Overseeing data engineering projects with a focus on analytics and reporting solutions.
AI Data Engineer
AI Data Engineer bridging data warehousing and AI-driven analytics capabilities. Collaborating on automated solutions and operational workflows in eCommerce with global brands.
AI Data Engineer
AI Data Engineer at Netwrix designing and building enterprise-grade data and AI platforms. Focused on stability and repeatability while adhering to security and governance standards.
Mid-level Data Cloud Developer – Data Engineer
Data Cloud Developer building cloud data platforms to support business decisions. Collaborating with tech teams to implement data solutions with a focus on quality and performance.