Job Title:  Senior Databricks Developer
Location: Malvern, PA
Duration: 12 Months + Extension
Bill Rate: $81/hour (C2C)
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Summary:
We are seeking a Senior Databricks Developer with 7+ years of Data Engineering experience and deep expertise in Databricks, AWS, PySpark, Python, and SQL. This is a highly technical, hands-on engineering role within a data engineering team focused primarily on building, enhancing, and supporting enterprise data pipelines. The ideal candidate will have extensive experience developing cloud-native data solutions in AWS, implementing Delta Lake and Medallion architectures, and delivering scalable, production-grade data pipelines using Databricks. This is not a consulting role; we are looking for a developer who enjoys building and optimizing data platforms and solving complex data engineering challenges.

Responsibilities:

• Design, develop, and maintain large-scale ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
• Build scalable and highly reliable data ingestion, transformation, and orchestration frameworks within AWS.
• Develop and support Delta Lake implementations and Medallion Architecture (Bronze, Silver, Gold) data pipelines.
• Create and optimize batch and streaming pipelines that process large volumes of structured and unstructured data.
• Leverage AWS services including S3, Glue, Lambda, EMR, IAM, EventBridge, CloudWatch, Kinesis, and Redshift to support end-to-end data workflows.
• Optimize Spark jobs, Delta tables, and Databricks workloads for performance, scalability, and cost efficiency.
• Partner with Data Engineers, Architects, Analytics teams, and business stakeholders to deliver high-quality data solutions.
• Implement data quality controls, monitoring, alerting, and operational support processes.
• Troubleshoot and resolve complex production issues across Databricks and AWS environments.
• Contribute to engineering best practices, code reviews, automation, and platform modernization initiatives.

Required Qualifications:

• 7+ years of Data Engineering experience building enterprise-scale data platforms and pipelines.
• 5+ years of hands-on Databricks development experience in production environments.
• Strong expertise in Python, PySpark, and SQL with a focus on pipeline development.
• Extensive experience developing and supporting data pipelines within AWS.
• Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion Architecture (Bronze, Silver, Gold).
• Hands-on experience with AWS services such as S3, Glue, Lambda, EMR, Redshift, Kinesis, IAM, and CloudWatch.
• Deep understanding of Spark architecture, distributed data processing, performance tuning, partitioning, and optimization.
• Experience building highly scalable batch and streaming data processing solutions.
• Strong understanding of data modeling, data warehousing concepts, and modern data architectures.
• Experience supporting production environments, monitoring data pipelines, and troubleshooting complex data issues.

Preferred Qualifications:

• Experience with Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, and Structured Streaming.
• Experience with Kafka, Airflow, or cloud-native orchestration frameworks.
• Experience implementing CI/CD pipelines using GitHub Actions, Jenkins, or Azure DevOps.
• Infrastructure as Code experience utilizing Terraform or CloudFormation.
• Databricks Certified Data Engineer Associate or Professional certification.
• Experience working within large enterprise AWS environments supporting mission-critical data platforms.