Information Technology & Software
23 Sept
Migration And Modernisation Data Engineer
Migration And Modernisation Data Engineer
Experience: 10-15 Years
Location: Bengaluru (Hybrid)
Job Role & Responsibilities
Build reusable migration accelerators, templates, utilities, libraries, and reference implementations for EDP onboarding.
Refactor legacy data pipelines, data models, and data products into scalable Lakehouse and Medallion architecture.
Develop production-ready data pipelines using Databricks, Spark, Python/Scala, SQL, and DBT.
Define engineering patterns, standards, and best practices for consistent migration and modernization delivery.
Support data validation, reconciliation, testing, performance optimization, and operational readiness.
Consulting and Advisory
Act as a technical consultant to business migration teams throughout assessment, planning, and execution.
Review current-state pipelines and data models, then recommend practical migration and refactoring approaches.
Facilitate technical workshops, design reviews, and migration planning discussions with business, architects, and engineers.
Help teams adopt EDP platform capabilities, governance, security, and engineering standards.
Specific Expertise Required
Technical Skills
Databricks: notebooks, workflows, jobs, Delta Lake, Unity Catalog, and Lakehouse implementation.
Apache Spark, Python/Scala, and advanced SQL for scalable data engineering and optimization.
DBT for modeling, transformations, testing, documentation, and deployment practices.
Modern data architecture: ETL/ELT, batch/streaming, Medallion design, data products, and enterprise data platforms.
Data formats and modeling: Parquet, JSON, CSV, XML, dimensional modeling, and analytics-ready structures.
AI and Automation Skills
Ability to use GenAI and agentic AI to accelerate migration, refactoring, code conversion, and documentation.
Experience or strong understanding of building AI agents, copilots, or automation assistants for engineering teams.
Prompt engineering, AI-assisted development, and workflow automation to improve delivery productivity.
Ability to identify migration tasks that can be automated using AI, scripts, reusable tools, or engineering frameworks.