Design and enable company’s next generation analytics ecosystem by incorporating the latest technologies for big data, analytics, and cognitive science, including machine learning.
Collaborate with other IT professionals, data scientists and clients, to leverage Azure Cloud offerings such as Databricks.
Build systems that are scalable and secure and provide superior user experiences, and promotes automation to democratize big data analytics and machine learning across the enterprise.
Implement large scale data analytics system utilizing Azure Databricks, Azure Data Factory (ADF), and other Open Source and Azure Cloud offerings to enable Business Intelligence (BI), Analytics, Machine Learning and other Cognitive services.
Develop Databricks programs and ADF pipelines to support data and BI engineering requirements.
Redevelop obsolete programs from an on premise to the Azure cloud.
Share best practice expertise through mentoring, pair programming, and training for employees acquiring new skills in Databricks and other Azure offerings.
Define and automate standards, as well as implement policies developed by security, governance and other teams.
Configure analytics platforms and optimize cluster usage.
Build deployment and test pipelines using Azure DevOps.
Write documentation in clear language that can be understood by less technical team members.
5+ years’ data engineering experience working with big data, including expertise designing data models in both column store and relational databases, incorporating disparate data to solve complex business needs.
3+ years’ experience developing in Databricks.
3+ years’ experience with Azure technology at the enterprise level for data engineering.
Expert experience developing in python required, R a plus.
Extensive experience in SQL, including query tuning.