Data Integration Engineer, Databricks, Python, SQL
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Data Integration Engineer building Databricks, Python, and SQL integrations for a Big4 consultancy’s AI-driven KYC platform. Ensuring reliable data models, contracts, APIs, and quality controls.
Responsibilities:
- Inventory and map all internal and external data sources.
- Define data contracts and canonical data structures for client, screening, evidence, audit, and reporting data.
- Build connectors to external screening providers, public registers, and local data sources.
- Connect Databricks with SQL Server, legacy databases, and core enterprise systems.
- Support API, database-link, batch, file, and SFTP-based integrations.
- Design and maintain data models supporting evidence, audit trails, and reporting.
- Implement data quality checks, contract tests, and retention rules.
- Validate incoming and outgoing data flows across production services.
- Support integration troubleshooting and production-readiness activities.
- Contribute to a business-critical platform where data accuracy and reliability are essential.
Requirements:
- Strong hands-on experience with Databricks, including Delta Lake and Unity Catalog.
- Strong experience with Python / PySpark.
- Advanced SQL skills.
- Experience integrating with SQL Server and legacy databases.
- Strong experience with REST APIs, DB links, batch processing, files, and SFTP integrations.
- Experience with data modelling and data-contract design.
- Experience implementing data-quality validation and integration testing.
- Fluent English and ability to work remotely within the EU.
Similar Jobs
Senior Solutions Engineer – Tax
Senior Solutions Engineer demonstrating insightsoftware’s Longview Tax solution to clients and partners. Translating direct-tax requirements into tailored demos, POCs, proposals, and RFP responses.
Senior Solutions Engineer
Senior Solutions Engineer enabling Twilio’s communications products through technical pre-sales in the UK. Leading discovery, architectures, demonstrations, and API-based customer solutions.
Solutions Architect
Solutions Architect designing secure enterprise cloud architectures for Weekday’s client. Leading presales, integrations, deployment strategies, and technology governance.
IT Solution Architect – Azure, Databricks, AI Platforms
IT Solution Architect designing production Azure and Databricks architectures for AI-driven KYC platforms. Supporting financial institutions with integrations, resilience, operations, and service transition.
Senior SAP Commerce Cloud – Solutions Architect
Arquiteto de soluções SAP Commerce Cloud modernizando plataformas de e-commerce da Keyrus. Liderança técnica de arquiteturas escaláveis, integrações corporativas e ambientes cloud.
Data Integration Engineer
Data Integration Engineer building Python and SQL healthcare transaction pipelines for an episode-based benefits startup. Integrating claims, enrollment, provider, and partner data into core systems.
Principal Solution Architect – ServiceNow
Principal Solution Architect designing ServiceNow enterprise architectures for Canadian customers. Advising technology executives on AI, cloud platforms, integrations, and digital transformation.
Solutions Engineer
Solutions Engineer delivering German and English demos for Qargo’s cloud-based transport management platform. Translating logistics customer needs into technical solutions and product feedback.
Senior Solutions Engineer – Systems Integration, BESS
Senior Solutions Engineer integrating BESS subsystems for AES, a global power company. Leading system architecture, compliance, deployment support, and field troubleshooting.
JD Edwards Homebuilder Solution Architect
JD Edwards Homebuilder Solution Architect designing scalable ERP solutions for ERP Suites’ homebuilding clients. Leading discovery, presales, implementation guidance, and delivery architecture.
Solution Architect – JDE
Solution Architect designing JD Edwards and integrated enterprise solutions for consulting clients. Leading discovery, presales architecture, implementation guidance, and delivery oversight.
Marketing Automation Solutions Architect – Temp Contractor
Marketing automation architect maintaining Credible’s email platforms, data flows, and compliance-critical programs. Supporting a U.S. consumer finance marketplace during a 3–6 month team-leave coverage period.
Senior Solutions Consultant – Life Sciences
Senior Solutions Consultant implementing Talkdesk’s AI-powered CXA and CCaaS solutions for healthcare and life sciences customers. Designing Epic, Salesforce, and Veeva integrations.
Solutions Engineering Lead
Solutions Engineering Lead scaling pre-sales and customer infrastructure solutions at bunny.net. Building the Solutions Engineering function for a faster, safer internet.
Solutions Engineer
Solutions Engineer designing and pricing multimodal logistics solutions for Transportation Insight. Supporting consultative sales, supply chain assessments, client presentations, and implementation transitions.
AI/ML Solutions Architect
AI/ML Solutions Architect shaping Databricks-based AI/ML architecture for Redhorse’s U.S. defense missions. Advancing aircraft maintenance, fleet availability, and military readiness across USAF and Army programs.
Lead AI Solutions Architect, SME
Lead AI Solutions Architect assessing AI use cases for a UK Cyber Security Operations Centre. Defining secure architecture, integrations, governance, and delivery roadmap.
Technical Solutions Architect – Contract
Technical Solutions Architect building enterprise AI, ML and cloud solutions for Satalia, a WPP company. Designing architectures, guiding engineering teams and engaging technical and business stakeholders.
Senior Treasury Solutions Consultant – Cash
Senior consultant designing cash-management and physical-currency solutions for Fifth Third Bank clients. Advising sales teams and guiding complex implementations.
Senior Solutions Architect – Field Service
Salesforce Field Service architect designing scalable solutions for service organizations at a global technology consultancy. Guiding discovery, integrations, deployment, adoption, and post-launch improvement.