Databricks Engineer

Hrfy.ai

Location

Remote (India)

Work mode

Remote

Employment

Full Time

Experience

2-2 years

Salary

₹8 - 12 LPA

About the role

The role focuses on data engineering using the Databricks platform. Key areas include designing, developing, and maintaining scalable data pipelines with Apache Spark, implementing ETL processes, and ensuring data quality and integrity. The position requires strong programming skills in Python or Scala and familiarity with major cloud platforms like AWS, Azure, or GCP, along with a solid understanding of data modeling and data warehousing concepts.

Responsibilities

Design, develop, and maintain data pipelines using Databricks and Apache Spark.
Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
Optimize existing data workflows for performance and efficiency.
Implement data quality checks and monitoring to ensure data integrity.
Support data ingestion from various sources, including APIs, databases, and cloud storage.

Full description

Job Title: Databricks Engineer Location: Remote (India) Experience Required: 2 years About the Role: We are seeking a skilled Databricks Engineer with a minimum of 2 years of experience to join our dynamic data engineering team. The ideal candidate will have a strong background in data processing and analytics, leveraging Databricks to build scalable data pipelines and support our data-driven decision-making processes. You will collaborate with data scientists, analysts, and other stakeholders to deliver high-quality data solutions. Key Responsibilities: - Design, develop, and maintain data pipelines using Databricks and Apache Spark. - Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. - Optimize existing data workflows for performance and efficiency. - Implement data quality checks and monitoring to ensure data integrity. - Develop and maintain documentation for data processes and workflows. - Assist in troubleshooting and resolving data-related issues in a timely manner. - Participate in code reviews and contribute to best practices in data engineering. - Stay updated with the latest Databricks features and industry trends to continuously improve our data solutions. - Support data ingestion from various sources, including APIs, databases, and cloud storage. - Provide training and support to team members on Databricks and data engineering practices. Required Skills: - Proficiency in Databricks, Apache Spark, and SQL. - Experience with data modeling and ETL processes. - Strong programming skills in Python or Scala. - Familiarity with cloud platforms (AWS, Azure, or GCP). - Understanding of data warehousing concepts and technologies. - Excellent problem-solving and analytical skills. - Strong communication skills and ability to work collaboratively in a team environment. Good to Have Skills: - Experience with Delta Lake and data lake architecture. - Knowledge of machine learning frameworks and libraries. - Familiarity with data visualization tools (e.g., Tableau, Power BI). - Understanding of DevOps practices and CI/CD pipelines. Qualifications: - Bachelor’s degree in Computer Science, Information Technology, or a related field. - Relevant certifications in Databricks or cloud platforms are a plus. Salary Range: 8 LPA - 12 LPA Employment Type: Full-time Work Mode: Remote