Data Engineering Consultant
We are Canada’s largest independent information technology services firm, and we are still growing! We are expanding in Atlantic Canada, and we need your skills, enthusiasm, and dedication as part of our team.
We are seeking a highly skilled Data Engineering Consultants to join our dynamic team. In this role, you will be responsible for managing and optimizing the data infrastructure, implementing data governance frameworks, and developing real-time data streaming solutions.
Your future duties and responsibilities:
• Design, develop, and maintain scalable data pipelines using Databricks on Azure.
• Collaborate with data scientists and analysts to implement data solutions that support business objectives.
• Optimize and troubleshoot data processing workflows to ensure high performance and reliability.
• Implement best practices for data security and governance within the Azure environment.
• Work closely with cross-functional teams to integrate data solutions with other systems and applications.
• Stay up-to-date with the latest advancements in Databricks and Azure technologies and apply them to improve our data infrastructure.
Required qualifications to be successful in this role:
• Proven experience with Databricks and Azure, including Spark, Delta Lake, and Azure Data Factory.
• Strong programming skills in Python, Scala, or Java.
• Experience with SQL and NoSQL databases.
• Familiarity with data warehousing concepts and ETL processes.
• Excellent problem-solving skills and attention to detail.
• Ability to work independently and as part of a team in a fast-paced environment.
• Strong communication skills and the ability to collaborate effectively with stakeholders.
Preferred Qualifications:
• Experience with machine learning and AI frameworks.
• Knowledge of CI/CD pipelines and DevOps practices.
• Certification in Azure or Databricks.
Use of the term ‘engineering’ in this job posting refers to the technical sense related to Information Technology (IT) and does not imply that the individual practices engineering or possesses the requisite license as prescribed by the applicable provincial or territorial engineering regulator. We are seeking individuals with expertise in IT engineering-related functions, but licensure from an engineering regulator is not a prerequisite for this position. Engineering is a regulated profession in Canada which is restricted in terms of use of titles and designation.