Job Description:
- Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure tools
- Developing scalable and re-usable frameworks for ingesting of assorted data sets
- Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained
- Manage and lead entry level data engineers and provide guidance, support and mentor other members of the team as well as do hands-on work
- Working with event based / streaming technologies to ingest and process data
- Working with other team members for implementing data pipelines and solutions in production using DevOps for deployment/automation
- Working with Azure CI/CD pipelines
- Delivering and presenting technical solutions to business users, technical teams and clients
Requirements:
Strong knowledge of Data Management principles
- Experience in building ETL / data warehouse transformation processes
- Direct experience of building data pipelines using one of the following Azure Data Factory, Azure Synapse, Azure Databricks
- Hands on experience designing and delivering solutions using the Azure Data Analytics platform (Azure AI), Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
- Ability to work seamlessly within a team, react quickly to a fast paced, and rapidly changing environment
- Strong capacity for logical thinking with the ability to understand complex applications
- 5+ years hands on experience using databases
- Excellent written and verbal communication skills
