Associate Principal Engineer, Azure Data Factory Architect
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Associate Principal Engineer leading Azure Data Factory architecture and development. Building scalable data integration solutions and developing ADF pipelines for near real-time processing.
Responsibilities:
- Architect and build scalable data integration solutions using Azure Data Factory
- Design, develop, and maintain ADF pipelines, triggers, datasets, and linked services
- Implement ETL / ELT workflows for batch and near real-time data processing
- Develop and optimize ADF Mapping Data Flows
- Integrate ADF with Azure services like Azure SQL, Synapse, Data Lake, and Blob Storage
- Handle performance tuning, monitoring, and cost optimization of pipelines
- Implement and manage CI/CD pipelines for ADF using Azure DevOps
- Troubleshoot production issues and perform root cause analysis
- Define architecture standards, best practices, and reusable frameworks
- Mentor and guide data engineers through hands-on support and code reviews
- Ensure data reliability, scalability, and high availability
- Collaborate with stakeholders to understand data integration requirements
Requirements:
- Total experience 9 years+
- Strong experience in Azure Data Engineering / Data Integration
- Deep hands-on expertise in Azure Data Factory (Must Have)
- Strong understanding of ETL / ELT concepts (Must Have)
- Hands-on experience with ADF pipelines, triggers, datasets, and linked services
- Experience with Mapping Data Flows and Integration Runtimes (Self-hosted & Azure)
- Strong experience with Azure Data Lake Storage (ADLS Gen2)
- Experience integrating on-premise and cloud data sources
- Strong SQL skills (Azure SQL / Synapse)
- Experience with Azure Synapse Analytics
- Familiarity with CI/CD pipelines using Azure DevOps
- Understanding of data orchestration, scheduling, and dependency management
- Experience with monitoring, logging, and error handling in ADF
- Good understanding of performance tuning and cost optimization
- Strong problem-solving and communication skills
- Good to Have: Java (Strong)
Similar Jobs
Software Engineer, Showroom
Software Engineer developing full-stack features for a home design startup. Collaborating with Product and Design partners to drive features end-to-end across the stack.
Senior Fullstack Developer, Patient Portal, DICOM
Senior Fullstack Developer at Dedalus developing patient portal solutions for healthcare technology. Focused on backend services, software design, and support for complex customer problems.
Senior Software Engineer – Integrations, Automations
Senior Software Engineer focusing on integrations and automations for hardware teams at AllSpice. Collaborating with internal teams and customers to enhance circuit design workflows in a flexible work environment.
Software Engineer, Intermediate
Intermediate Software Engineer at Fellow enhancing productivity through AI-powered tools. Building features and collaborating with teams to improve meeting efficiency.
Software Engineer
AI Engineer responsible for developing systems that orchestrate agents for a B2B SaaS startup. Focusing on AI-driven vendor risk assessment and managing intelligence infrastructure.
Senior Software Engineer, Intelligent Systems
Senior Software Engineer designing and building backend services for HostPapa's cloud commerce platform. Involving complex billing logic, API-driven services, and embedding AI solutions within a remote team.
Principal Software Engineer
C++ engineer modernizing payment processing systems at Ticketmaster. Engaging in design, development, and integration of payment solutions for live events.
Principal Software Engineer
C++ engineer modernizing Ticketmaster's high-throughput payment processing systems handling real financial transactions. Driving platform migrations and developing robust payment integrations with third-party gateways.
Senior Software Engineer – APIs
Senior Software Engineer developing customer-facing and internal APIs for Unacast's location intelligence analytics. Translating business problems into technology solutions with a focus on fast querying of large datasets.
Principal Software Engineer
Principal Software Engineer responsible for software engineering mentorship and integrating industry trends. Join Ad Hoc, delivering scalable digital services that transform government experience.
Senior Software Engineer III
Full Stack Software Engineer developing scalable digital services for Veterans Affairs using agile methods. Collaborating within a remote team to enhance government technology delivery.
Staff Software Engineer – Lead Developer
Staff Software Engineer leading full-stack projects to design impactful digital services for Veterans. Collaborating on user experiences while managing project scope and stakeholder communication.
Software Engineer III – Expert
Software Engineer III developing digital services at Ad Hoc. Engaging in leadership roles and influencing engineering processes in support of Veterans Affairs goals.
Software Engineer I
Software Engineer I developing high-quality software components for OneStream solutions. Collaborating with teams to deliver reliable, user-friendly software for enterprise performance management.
Senior Staff Software Engineer – Duck Creek
Senior Staff Software Engineer in Personal Lines IT organization leading Scrum teams to develop technology solutions. Overseeing design and development for Personal Lines' innovative systems.
Senior Staff Engineer, GenAI, Playwright
Senior Staff Engineer specializing in Quality Engineering and Test Automation at a Digital Product Engineering company. Focus on Playwright and AI-driven testing approaches.
Senior Staff Engineer, Gen AI, Azure
Senior Staff Engineer developing AI agents and orchestrating workflows at Nagarro. Requires strong experience in Java, Python, and Azure AI services.
Senior Software Engineer I
Application Developer responsible for developing e-commerce web solutions and managing large-scale projects at CDW. Collaborating with teams to ensure application efficiency and quality standards.
Software Architect
Software Architect leading technical teams in defining cloud architectures for data management platform. Collaborating on scalability, performance, and security with a focus on innovation.
Software Engineer – Reporting & Insights
Software Engineer developing reporting and analytics capabilities for Muck Rack's media platform. Designing and implementing solutions for data analysis in a remote-first work environment.