About the Role
We are seeking a Senior Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with expertise in ClickHouse. You will be responsible for designing, building, and maintaining data pipelines and infrastructure, ensuring scalability, performance, and reliability. You will work closely with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
Responsibilities
- Design and implement scalable and efficient data pipelines using ClickHouse.
- Collaborate with data scientists, analysts, and engineers to understand data requirements and deliver solutions.
- Ensure data accuracy, security, and compliance with industry standards.
- Optimize data storage and retrieval processes to enhance performance.
- Develop and maintain data infrastructure, including databases, ETL processes, and data warehouses.
- Monitor and troubleshoot data systems to ensure reliability and availability.
- Implement data governance and quality control measures.
- Provide technical leadership and mentorship to junior team members.
- Stay updated with the latest trends and best practices in data engineering.
- Contribute to the development of data engineering standards and best practices.
- Work with stakeholders to define data strategy and roadmaps.
- Ensure data integrity and consistency across different systems.
- Develop and maintain documentation for data systems and processes.
- Participate in on-call rotations to ensure 24/7 support for data systems.
- Conduct performance tuning and optimization of data queries.
- Implement data backup and recovery solutions.
- Collaborate with IT and operations teams to ensure smooth deployment of data solutions.
- Provide technical support and troubleshooting for data-related issues.
- Develop and maintain data dashboards and reporting tools.
- Ensure compliance with data privacy and security regulations.
- Conduct data audits and assessments to identify areas for improvement.
Nice to Have
- Experience with real-time data processing and streaming technologies.
- Knowledge of data lakes and data mesh architectures.
- Experience with data orchestration tools such as Apache Airflow.
- Familiarity with data cataloging and metadata management tools.
- Experience with data lineage and impact analysis.
- Knowledge of data privacy regulations such as GDPR or CCPA.
- Experience with data encryption and secure data transmission.
- Familiarity with data governance frameworks and standards.
- Experience with data quality assessment and monitoring tools.
- Knowledge of data warehousing solutions such as Snowflake or Redshift.
Compensation
Competitive salary and benefits package.
Work Arrangement
Hybrid work arrangement with flexible hours.
Team
Collaborative and innovative team environment.
What You'll Do
- Design and implement data pipelines and infrastructure using ClickHouse.
- Collaborate with cross-functional teams to understand data requirements.
- Ensure data accuracy, security, and compliance with industry standards.
- Optimize data storage and retrieval processes.
- Develop and maintain data infrastructure, including databases and ETL processes.
- Monitor and troubleshoot data systems to ensure reliability and availability.
- Implement data governance and quality control measures.
- Provide technical leadership and mentorship to junior team members.
- Stay updated with the latest trends and best practices in data engineering.
- Contribute to the development of data engineering standards and best practices.
What You'll Need
- Proven experience as a Senior Data Engineer or similar role.
- Expertise in ClickHouse and other data processing tools.
- Strong knowledge of SQL and NoSQL databases.
- Experience with ETL processes and data warehousing.
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with cloud platforms such as AWS, Azure, or Google Cloud.
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork abilities.
- Experience with data governance and quality control.
- Knowledge of data security and compliance standards.
Nice to Have
- Experience with real-time data processing and streaming technologies.
- Knowledge of data lakes and data mesh architectures.
- Experience with data orchestration tools such as Apache Airflow.
- Familiarity with data cataloging and metadata management tools.
- Experience with data lineage and impact analysis.
- Knowledge of data privacy regulations such as GDPR or CCPA.
- Experience with data encryption and secure data transmission.
- Familiarity with data governance frameworks and standards.
- Experience with data quality assessment and monitoring tools.
- Knowledge of data warehousing solutions such as Snowflake or Redshift.
Our Benefits
- Competitive salary and benefits package.
- Hybrid work arrangement with flexible hours.
- Collaborative and innovative team environment.
- Opportunities for professional development and growth.
- Visa sponsorship available for eligible candidates.
- Comprehensive health and wellness benefits.
- Generous time-off policies and paid holidays.
- Employee assistance programs and support services.
- Tuition reimbursement and professional development opportunities.
- Employee referral bonuses and recognition programs.
Our Culture
- Innovative and collaborative work environment.
- Focus on continuous learning and development.
- Commitment to diversity, equity, and inclusion.
- Supportive and inclusive team culture.
- Opportunities for career growth and advancement.
- Encouragement of work-life balance.
- Recognition and rewards for outstanding performance.
- Open communication and feedback channels.
- Collaborative problem-solving and decision-making.
- Commitment to ethical and responsible business practices.
Visa sponsorship available for eligible candidates.
