Data Engineer

Job Description:
Data Engineer
Required Technical Skill Set:
Data Bricks, Cognite, Python,OT Operations

Must-Have:
1. Proficiency in Azure Databricks, Cognite Data Platform, SQL Databases
2. Expertise in Data Ingestion from OT Systems like SCADA, PLC, and DCS systems
3. Strong knowledge of data warehousing concepts, ETL (Extract, Transform, Load) processes, and data modeling.
4. Proficiency in Pyspark, and potentially other scripting languages like Python, PowerShell for data manipulation and automation
Good-to-Have:
1. Familiarity of Cognite Data Fusion application
2. Familiarity with cloud platforms like Azure and Cognite Data Platform
3. Deep understanding of industrial protocols, SCADA systems, PLCs, DCS systems, and their associated security considerations.
4. Some experience of Energy/Oil & Gas domain

Responsibility of / Expectations from the Role:
1. Designing and implementing efficient data pipelines to move data from various OT sources (e.g., SCADA, PLC, DCS systems) to data lakes, data warehouses, or other storage systems
2. Strong communication skills to collaborate effectively with other team members, and to explain complex technical concepts to non-technical stakeholders
3. Integrating data from disparate OT systems, often using a variety of protocols and formats, into a unified data model.
4. Ensuring the accuracy, completeness, and consistency of data used for analysis and decision-making.
5. Working closely with IT and OT teams, as well as other stakeholders, to ensure alignment and effective data management practices.
6. Monitoring data pipelines and OT systems for performance issues and security vulnerabilities, and proactively addressing any problems
7. Creating and maintaining documentation for data pipelines, data models, and security procedures