Databricks Architect/Engineer

Full description

• Work on diverse customer-facing Big Data projects, including reference architectures, how-to guides, and MVP development
• Lead end-to-end design, build, and deployment of large-scale big data and AI solutions
• Provide architectural guidance to drive Databricks adoption across business teams
• Design and manage Databricks infrastructure, connectivity, integration patterns, and Unity Catalog migrations
• Manage clusters, networking, platform engineering, and data governance initiatives
• Collaborate closely with platform engineering teams to implement Databricks services effectively
• Evaluate and adopt new Databricks features, including Serverless capabilities
• Design and implement integrations between Databricks and third-party tools such as Collibra and data quality platforms
• Apply AWS administration and architecture expertise to optimize cloud resources and Databricks integrations
• Implement Unity Catalog for strong data governance, security, and lineage tracking
• Promote and implement CI/CD practices for Databricks solution deployments
• Contribute to data mesh architecture by enabling decentralized data ownership and access
• Communicate Databricks analytics capabilities to help teams derive actionable insights

Required Knowledge, Skills, and Abilities

• Strong experience with AWS cloud services and architecture
• Excellent documentation and whiteboarding skills to explain complex concepts
• Deep knowledge of Databricks services, including Unity Catalog and Serverless Databricks
• Proven experience migrating from Databricks Classic to Lakehouse architecture using Delta and/or Delta Live Tables
• Strong collaboration skills with the ability to work across teams and functions