Migration And Modernisation Data Engineer
Experience: 10-15 Years
Location: Bengaluru (Hybrid)
Job Role & Responsibilities
- Build reusable migration accelerators, templates, utilities, libraries, and reference implementations for EDP onboarding.
- Refactor legacy data pipelines, data models, and data products into scalable Lakehouse and Medallion architecture.
- Develop production-ready data pipelines using Databricks, Spark, Python/Scala, SQL, and DBT.
- Define engineering patterns, standards, and best practices for consistent migration and modernization delivery.
- Support data validation, reconciliation, testing, performance optimization, and operational readiness.
Consulting and Advisory
- Act as a technical consultant to business migration teams throughout assessment, planning, and execution.
- Review current-state pipelines and data models, then recommend practical migration and refactoring approaches.
- Facilitate technical workshops, design reviews, and migration planning discussions with business, architects, and engineers.
- Help teams adopt EDP platform capabilities, governance, security, and engineering standards.
Specific Expertise Required
Technical Skills
- Databricks: notebooks, workflows, jobs, Delta Lake, Unity Catalog, and Lakehouse implementation.
- Apache Spark, Python/Scala, and advanced SQL for scalable data engineering and optimization.
- DBT for modeling, transformations, testing, documentation, and deployment practices.
- Modern data architecture: ETL/ELT, batch/streaming, Medallion design, data products, and enterprise data platforms.
- Data formats and modeling: Parquet, JSON, CSV, XML, dimensional modeling, and analytics-ready structures.
AI and Automation Skills
- Ability to use GenAI and agentic AI to accelerate migration, refactoring, code conversion, and documentation.
- Experience or strong understanding of building AI agents, copilots, or automation assistants for engineering teams.
- Prompt engineering, AI-assisted development, and workflow automation to improve delivery productivity.
- Ability to identify migration tasks that can be automated using AI, scripts, reusable tools, or engineering frameworks.