Responsibilities
- Develop new tools, code & services to execute data engineering activities.
- Movement of structured & unstructured data using approved methods.
- Execute data ingestion activities for storing data in a local or enterprise level location.
- View data in its source format.
- Develop code to format data that supports exploration.
- Analyze source data formats & work with Data Scientists & partners to determine the formats & transforms that best meet mission objectives.
- Develop code and tools to provide one-time & on-going data formatting & transformations into enterprise or boutique data models.
- Implement existing ETL code & best practices/standards.
- Develop an ETL Code Transition Plan.
- Develop & deliver documentation for each project including ETL mappings, code use guide, code location & access instructions.
- Facilitate Code Reviews.
- Provide consulting services to support data transport, ingestion, conditioning, access, & management.
Requirements
- Active TS/SCI with Polygraph Clearance
- Ability and willingness to quickly learn a new tool
- Strong communication skills with both your teammates, and your leads
- SQL, Python, Pyspark experience
- Willingness to do development type work when needed (junior level development at best)
- Ability to be a self starter and ask questions when needed
- Comfortable working with and manipulating data in compliance with the offices workflow
- Extract, Transform and Load (ETL) tools and processes
- AWS
- APIs
- Linux
- Geospatial tools/data
Nice to Have
- Palantir Foundry Experience
- Kubeflow
- Experience with OCR and text extraction of PDFs
- Experience with data validation / data quality after ETLing to be sure it’s ready for end users
- Docker
- Jenkins
- Hadoop/Spark
- Kibana
- Kafka
- NiFi
- ElasticSearch
Work Arrangement
On-site — McLean, VA, Chantilly, VA, various field offices throughout Northern VA
Additional Information
- Requires a TS/SCI + Polygraph clearance (acceptable to this customer)
- Work hours are typically quite flexible - roll up your sleeves, get things done, and no one cares much about the specific hours that you work
- Work on this program takes place in McLean, VA and in various field offices throughout Northern VA
- We cannot support remote work
- The work space itself is also quite nice, and there is an excellent cafeteria
