Databricks Solution Architect
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Databricks Solution Architect leading Clearstead’s financial advisory data and AI platform. Designing Azure Databricks architecture, production pipelines, and AI infrastructure.
Responsibilities:
- Lead the architecture and development of the Azure Databricks platform, including Delta Lake, Unity Catalog, and medallion architecture
- Establish technical standards, security controls, and scalable data integration frameworks
- Oversee platform performance, monitoring, operational readiness, and cost optimization
- Manage the transition of platform ownership from external implementation partners
- Design, build, and maintain production data pipelines using PySpark, SQL, and Databricks Workflows
- Integrate information from Salesforce, Orion, Addepar, PARis, and other enterprise systems
- Develop data transformation, reconciliation, and quality assurance processes
- Implement automated testing, deployment, and infrastructure management practices
- Develop data infrastructure supporting AI applications, intelligent agents, and advanced analytics
- Support document ingestion, vector search, model serving, and retrieval-augmented generation
- Collaborate with the AI Engineer and Data Management Lead to deliver secure, scalable solutions
- Serve as the technical design authority for the enterprise data platform
- Evaluate vendor solutions and oversee implementation partner deliverables
- Partner with business and technology leaders to translate requirements into effective solutions
- Mentor developing engineers and promote engineering best practices
- Design, implement, and maintain a secure, scalable, and high-performing Databricks platform
- Develop and optimize production data pipelines, ensuring data accuracy, system reliability, cost management, and timely resolution of technical issues
- Establish architecture standards, guide implementation partners, and collaborate with cross-functional teams to deliver solutions aligned with business objectives and regulatory requirements
Requirements:
- Bachelor's degree in Data, Information Systems, Computer Science, Business Analytics, or a related field, or equivalent combination of education and relevant professional experience
- 10+ years of experience in data engineering, data architecture, or related disciplines
- 4+ years designing and managing production Databricks environments
- Expert knowledge of Azure Databricks, Unity Catalog, Delta Lake, and data architecture principles
- Strong hands-on experience with PySpark, SQL, data pipelines, and data integration
- Experience supporting AI or machine learning workloads within Databricks
- Knowledge of Azure security, identity management, Terraform, and CI/CD practices
- Demonstrated experience managing enterprise data platforms and external technology partners
- Strong communication skills with the ability to translate technical concepts for business stakeholders
- Advanced degree in Information Systems, Data Analytics, or a related discipline preferred
- Certified Data Management Professional (CDMP), Databricks Data Engineer Associate or Professional, or comparable industry certifications preferred
- Databricks Data Engineer Professional certification and additional platform administration or architecture credentials preferred
- Experience with financial services, wealth management, or regulated data environments preferred
- Familiarity with Salesforce Financial Services Cloud, Orion, or Addepar preferred
- Experience with Power BI, Databricks AI/BI Genie, and Delta Sharing preferred
- Ability to perform sitting and keyboarding duties
- Attention, analysis, prioritization, interaction, and collaboration abilities
- Occasional flexibility in work hours may be required
Benefits:
- Fully remote work environment
- Limited to no travel
- Home-based work environment
- Potential flexibility in work hours for system implementations, production incidents, or critical business operations
Similar Jobs
Solutions Architect
Solutions Architect advancing Axiad Mesh, an identity-risk decision platform. Leading demos, proof-of-value engagements, and technical deal strategy for cybersecurity customers.
Senior Solutions Consultant – Speciality
Senior Solutions Consultant leading enterprise healthcare technology discovery, solution design, demonstrations, and deal strategy for Cencora. Mentoring consultants and influencing product direction.
Senior Solutions Consultant – Infusion
Senior Solutions Consultant leading healthcare technology pre-sales, solution design, and enterprise deal strategy for Cencora customers. Advising healthcare organizations while influencing products and mentoring consulting teams.
Senior Solutions Architect, AI Factory Infrastructure
Senior Solutions Architect mapping customer workloads to NVIDIA AI factory architectures. Designing scalable data-center solutions and driving technical adoption with customers and partners.
Senior Solutions Engineer – Tax
Senior Solutions Engineer demonstrating insightsoftware’s Longview Tax solution to clients and partners. Translating direct-tax requirements into tailored demos, POCs, proposals, and RFP responses.
Senior Solutions Engineer
Senior Solutions Engineer enabling Twilio’s communications products through technical pre-sales in the UK. Leading discovery, architectures, demonstrations, and API-based customer solutions.
Solutions Architect
Solutions Architect designing secure enterprise cloud architectures for Weekday’s client. Leading presales, integrations, deployment strategies, and technology governance.
Data Integration Engineer, Databricks, Python, SQL
Data Integration Engineer building Databricks, Python, and SQL integrations for a Big4 consultancy’s AI-driven KYC platform. Ensuring reliable data models, contracts, APIs, and quality controls.
IT Solution Architect – Azure, Databricks, AI Platforms
IT Solution Architect designing production Azure and Databricks architectures for AI-driven KYC platforms. Supporting financial institutions with integrations, resilience, operations, and service transition.
Senior SAP Commerce Cloud – Solutions Architect
Arquiteto de soluções SAP Commerce Cloud modernizando plataformas de e-commerce da Keyrus. Liderança técnica de arquiteturas escaláveis, integrações corporativas e ambientes cloud.
Data Integration Engineer
Data Integration Engineer building Python and SQL healthcare transaction pipelines for an episode-based benefits startup. Integrating claims, enrollment, provider, and partner data into core systems.
Principal Solution Architect – ServiceNow
Principal Solution Architect designing ServiceNow enterprise architectures for Canadian customers. Advising technology executives on AI, cloud platforms, integrations, and digital transformation.
Solutions Engineer
Solutions Engineer delivering German and English demos for Qargo’s cloud-based transport management platform. Translating logistics customer needs into technical solutions and product feedback.
Senior Solutions Engineer – Systems Integration, BESS
Senior Solutions Engineer integrating BESS subsystems for AES, a global power company. Leading system architecture, compliance, deployment support, and field troubleshooting.
JD Edwards Homebuilder Solution Architect
JD Edwards Homebuilder Solution Architect designing scalable ERP solutions for ERP Suites’ homebuilding clients. Leading discovery, presales, implementation guidance, and delivery architecture.
Solution Architect – JDE
Solution Architect designing JD Edwards and integrated enterprise solutions for consulting clients. Leading discovery, presales architecture, implementation guidance, and delivery oversight.
Marketing Automation Solutions Architect – Temp Contractor
Marketing automation architect maintaining Credible’s email platforms, data flows, and compliance-critical programs. Supporting a U.S. consumer finance marketplace during a 3–6 month team-leave coverage period.
Senior Solutions Consultant – Life Sciences
Senior Solutions Consultant implementing Talkdesk’s AI-powered CXA and CCaaS solutions for healthcare and life sciences customers. Designing Epic, Salesforce, and Veeva integrations.
Solutions Engineering Lead
Solutions Engineering Lead scaling pre-sales and customer infrastructure solutions at bunny.net. Building the Solutions Engineering function for a faster, safer internet.
Solutions Engineer
Solutions Engineer designing and pricing multimodal logistics solutions for Transportation Insight. Supporting consultative sales, supply chain assessments, client presentations, and implementation transitions.