← All jobs
Key Responsibilities Databricks Administration: Manage workspaces, clusters, jobs, security configurations, libraries, and overall platform health. Data Engineering: Build and optimize data pipelines and transformations using PySpark and TSQL. Automation & Development: Create automation and tooling using Python, Bash, and PowerShell. Governance & Security: Implement and automate data governance, access controls, auditability, and compliance best practices. Note: A significant portion of our development focuses on governance and security automation. Solution Architecture: Translate conversational ideas into clear solution architectures; break work into tasks in Planner or Azure DevOps for team execution. Effort Estimation: Estimate level of effort and forecast delivery dates with reasonable accuracy. Documentation: Produce high-quality documentation including workflows, gap analyses, design specs, build docs, test plans, and transition documentation. Testing Oversight: Oversee unit, scenario-based, integrated, capacity, and parallel testing. On-Call Support: Participate in a 247365 on-call rotation to support Databricks, SQL Server, and other relevant applications supported by the team. Upgrades & Monitoring: Support system upgrades, patches, performance tuning, observability, and proactive monitoring. Data Integrity: Maintain accountability for data integrity across applications and pipelines. Change Management: Complete system build and documentation in adherence to departmental guidelines and change management policy. Peer Review: Review teammates work and provide constructive feedback. Communication & Escalation: Communicate effectively with peers, stakeholders, and leadership; provide regular status updates and escalate issues appropriately. Facilitation & Leadership: Drive discussions, mentor accountable leaders, and coordinate resources across initiatives. Training & Mentorship: Provide feedback on training materials; conduct training sessions; develop and mentor other team member. Policies & Vendor Engagement: Help develop department guidelines, standards, procedures, and policies; facilitate vendor reviews and selection processes. Integrations (as applicable): Demonstrate understanding of integration and technical requirements for peripherals/devices that interface with assigned applications. Support Coordination: Coordinate remote and/or on-site support as needed for end-users. Optional Contributions Create complex AI/BI dashboards. Contribute to CI/CD pipelines and integrations using Azure DevOps or GitHub. Apply Terraform for infrastructure as code. Provide MS SQL Server administrative support.
Apply now Share this job
Work setup: Remote