Senior Data Engineer - Databricks

Job Title: Senior AWS Data Engineer

Location: Indianapolis, IN

Role Overview

• Design, develop, and maintain data ingestion pipelines for real-world data (RWD) sources
  
• Build and optimize data models within a Databricks Lakehouse architecture
  
• Ensure data governance, quality, and compliance standards are met
  
• Optimize cost, performance, and resource utilization across AWS and Databricks
  
• Collaborate with business and technical stakeholders to translate requirements into scalable solutions
  
• Provide technical guidance, coding standards, and best practices
  
• Support knowledge sharing through documentation and training sessions
  
• Deliver status updates and progress reports to senior leadership
  

Required Qualifications

• Strong hands-on experience with AWS services (S3, Glue, Lambda, Redshift, etc.)
  
• Proven experience with Databricks (Delta Lake, notebooks, workflows)
  
• Expertise in SQL and Python
  
• Experience building end-to-end ETL/ELT data pipelines
  
• Strong understanding of data modeling techniques
  
• Experience with data governance, quality, and compliance frameworks
  
• Ability to work independently and in a collaborative team environment
  
• Databricks Certification
  
• Experience building and managing CI/CD pipelines
  
• Experience with workflow orchestration tools
  
• Exposure to data observability and lineage tools
  
• Experience with monitoring, logging, and observability frameworks
  

Highly Preferred Skills

• Experience in Healthcare / Pharma domain
  
• Exposure to RWD (Real World Data) or HEOR analytics
  
• Familiarity with Lakehouse architecture / Medallion (Bronze-Silver-Gold) models
  
• Experience optimizing cost and performance in cloud environments
  
• Strong communication and stakeholder management skills