Responsibilities
- Evaluate and restructure legacy code originally built by data engineering teams.
- Define and uphold standards for modular, well-organized code development.
- Conduct code reviews and offer constructive input to elevate coding practices.
- Develop and run unit, integration, performance, and regression tests for reliability.
- Ensure backward-compatible updates during migration to newer Python 3 versions.
- Apply SOLID design principles to improve object-oriented code maintainability.
- Utilize Cloudera tools to manage and enhance data workflows.
- Implement distributed computing concepts for efficient data processing.
- Use GitLab CI/CD to automate build and deployment workflows.
- Partner with data engineering units to assess and refine current data pipelines.
- Set up Continuous Integration using Jenkins, Jenkinsfiles/Groovy, and SonarQube.
- Leverage SonarQube for static analysis and code quality monitoring.
- Build, release, and maintain applications using Docker containers.
- Understand deployment approaches involving Virtuozzo and Docker environments.
- Demonstrate understanding of Cloudera platform integration with Python services.
- Interact with Jira API for tracking tasks and managing project workflows.
- Work with Nexus Repository to manage build artifacts and dependencies.
- Diagnose and resolve technical issues while optimizing system performance.
- Manage Python packages using PyPI and pip tools effectively.
- Automate data population across DEV, SIM, and UAT environments.
Team
- Team size: 5
- Structure: team of developers
