GOVX is looking for a Cloud Data Engineer to join our Data team. You will be instrumental in modernizing and transforming our data and reporting capabilities by implementing a new data architecture. You'll be responsible for the day-to-day collection, transportation, maintenance, and access to corporate data assets, working cross-functionally to centralize and standardize data for business reporting, machine learning, AI, and other stakeholders.
What You'll Do
- Support and modernize existing data integrations.
- Craft and maintain efficient, scalable data pipeline architecture.
- Assemble large, complex data sets that meet business requirements.
- Identify and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
- Partner with business analysts, data scientists, and IT teams to translate requirements into scalable data solutions using Fabric and Spark with Delta Lake.
- Develop Spark notebooks for both batch and streaming ETL pipelines leveraging Delta Lake.
- Implement and optimize Delta Lake features including schema enforcement, evolution, and time travel.
- Optimize Delta Lake tables for performance using Z-ordering, compaction, and partitioning.
- Work with the team to ensure clean, meaningful data and greater functionality within our data systems.
- Design processes supporting data transformation, structures, metadata, dependency, and workload management.
What We're Looking For
- Hands-on experience developing, debugging, and optimizing Spark notebooks for ETL and analytics in Microsoft Fabric and Azure.
- Deep expertise in Microsoft Fabric, Dataflows Gen2, and Power BI integration.
- Hands-on experience with Delta Lake table management, including schema evolution, versioning, and data compaction.
- Experience with Data Lakehouse and Medallion Architecture.
- Experience with CI/CD and version control using Git.
- Advanced SQL and NoSQL query authoring; proficiency in Python and Spark scripting.
- Proficiency with object-oriented/object function scripting languages and Metadata-Driven Design using JSON.
- Experience working with streams such as Event Hubs and Event Driven Architectures.
- Experience building, maintaining, and optimizing ‘big data’ pipelines, architectures, and data sets.
- Experience cleaning, testing, and evaluating data quality from a wide variety of sources.
- Knowledge of the Microsoft Power Platform including Copilot Studio and Power Apps.
- Strong collaboration and communication skills with business and technical teams.
Nice to Have
- Bachelor’s degree or equivalent experience.
- 7+ years of proven experience deploying and maintaining always-on data services.
- 2+ years building and maintaining Spark notebook-based pipelines in Microsoft Fabric.
- 2+ years of experience working with Microsoft Fabric.
- 2+ years of experience working with Microsoft Azure.
- 2+ years of experience with Delta Lake.
- 1+ years of experience with SQL Server.
Technical Stack
- Microsoft Fabric, Spark, Delta Lake, Azure, Dataflows Gen2, Power BI, Git, SQL, NoSQL, Python, JSON, Event Hubs, Microsoft Power Platform, Copilot Studio, Power Apps, SQL Server
Team & Environment
You will be part of a Data team under the supervision of the Business Intelligence Manager.
Benefits & Compensation
- Compensation: $120,000 - $140,000 Annually
- Paid Time Off, Paid Sick Leave, and Paid Holidays
- Competitive Medical, Dental, Vision, Short Term Disability, and Life Insurance
- 401(k) plan with discretionary match available
- Flexible Spending Account (FSA) and Health Savings Account (HSA)
- Voluntary benefits including Critical Illness, Group Accident, and Voluntary Life
- Employee Referral Program
- Exposure to a growing ecommerce company
- Discounts on the GOVX website
Work Mode
This is a hybrid position open to candidates located in California, Texas, Washington, Florida, Tennessee, New York, or Colorado.
GOVX is dedicated to supporting current and former members of the United States military, law enforcement, firefighting, medical services, and government personnel. We are an EOE. Veterans/Disabled. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.
