Azure Databricks Engineer
Azure Databricks, SQL, Data Warehouse, Python, PySpark
Description
GSPANN is hiring a Azure Databricks Engineer. As we march ahead on a tremendous growth trajectory, we seek passionate and talented professionals to join our growing family.
Who We Are
GSPANN has been in business for over a decade, with over 2000 employees worldwide, and servicing some of the largest retail, high technology, and manufacturing clients in North America. We provide an environment that enables career growth while still interacting with company leadership.
Visit Why GSPANN for more information.
Location: Hyderabad / Gurugram / Pune
Role Type: Full Time
Published On: 11 September 2024
Experience: 6+ Years
Description
GSPANN is hiring a Azure Databricks Engineer. As we march ahead on a tremendous growth trajectory, we seek passionate and talented professionals to join our growing family.
Role and Responsibilities
- Manage Databricks clusters, including provisioning, configuration, and optimization.
- Manage Databricks users in workspace and services used by them such as notebooks, sql, workflows, mlflow etc.
- Integrate Azure services like Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics with Databricks for a comprehensive data processing solution.
- Write high-performance Spark code for data manipulation and analysis.
- Monitor and troubleshoot Databricks jobs and pipelines for performance and stability.
- Monitor usage of Databricks workspaces and generate reports.
- Collaborate with data analysts, data scientists, and software engineers to design and implement data solutions.
- Stay up-to-date on the latest advancements in Azure and Databricks technologies.
Skills and Experience
- Proven experience of 5 Plus years working with Microsoft Azure and Azure Databricks.
- Good understanding of data warehousing and data lake concepts.
- Experience with Apache Spark for large-scale data processing.
- Proficiency in programming languages like Python (PySpark).
- Experience with SQL and relational databases.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Ability to work independently and as part of a team.
