Azure data engineer (Pittsburgh) Job at Cognizant, Pittsburgh, PA

a0hDOGROeVJlZ25yUTBXYkovWUxZcjBHUFE9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Part time, Work experience placement,

Similar Jobs

Mindex

Salesforce CPQ Admin - Remote Job at Mindex

Mindex is a software development company with a rich history of demonstrated software and product development success.For more than 25 years, Mindex has delivered on the needs of a variety of customers when it comes to software developmentand thats a long time. But we...

Vaco by Highspring

Sr. Accountant Job at Vaco by Highspring

 ...Senior Accountant Location: Oakland County Salary Range: $80,000 $110,000 Employment Type: Full-Time About the Role...  ...or related field Minimum 5 years of progressive accounting experience Strong knowledge of GAAP and tax regulations Proficiency... 

Prairie Band, LLC

Family Clinic and Outpatient Nurse Practitioner (NP) Job at Prairie Band, LLC

 ...Glacier Country, Montana. Were seeking an experienced Family Nurse Practitioner (Locum Tenens) to join the dedicated care team at the...  ...extent required pursuant to Section 22-2-4(B). #PBHS #LinkedIN #NP #FNP Requirements: Requirements: ~ Must be certified through... 

Amergis

School Nurse - RN, BSN Job at Amergis

 ...School Registered Nurse oversees the application of thenursing process and the delivery of student care in a school setting. TheSchool RN supports the physical, mental and emotional well-being of studentswithin the school setting. The School RN plays an essential role in... 

SBA Communications Corporation

Tower Technician: Field Construction & Climb Lead Job at SBA Communications Corporation

A leading communications infrastructure company is seeking a Tower Technician to perform construction duties and ensure quality execution...  ...of projects. Candidates should have at least 1 year of tower climbing experience, a high school diploma, and strong physical...