Senior Data Engineer

Job Description

• Position: Senior Data Engineer
• Location: Remote
• Client: Office of the Inspector General (OIG)
• Candidates Required: 1
• Work Authorization: US Citizen

Responsibilities:

• Assist TSD with data products by providing highly skilled and authoritative expertise on data engineering methods and best practices, including code-first development approaches and modern pipeline design patterns.
• Design, implement, and maintain an efficient, secure, stable, and flexible data architecture that supports products and end-users, with all assets managed via source control.
• Design, implement, and maintain ELT/ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning (using SDK V1 and SDK V2)
• Review, maintain, and improve existing architecture and pipelines, including periodic audits to address bottlenecks, deprecated dependencies, and architecture drift.
• Establish quality controls for maintaining all pipelines, and introduce error handling, logging mechanisms, and validation checks.
• Incorporate source control for all pipelines and data analytics codebases to enable iterative code development while ensuring data architecture stability.
• Optimize the ingestion, processing, and storage of a wide variety of datasets and data types, including modern columnar formats such as Parquet.
• Develop self-service capabilities for SBA OIG analysts to query and export data for investigations and audits.
• Coordinate with data scientists to ensure the architecture efficiently supports machine learning algorithms and data pipelines in Azure Machine Learning.
• Develop robust standard operating protocols (SOPs) dictating the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in Azure environment.
• Provide detailed documentation of the data architecture, including data dictionaries, ER diagrams, and pipeline process maps.
• Maintain and expand the environment with additional datasets and services upon request, following a defined intake and testing process prior to production deployment.
• Stay current with emerging AI tools relevant to data engineering and contribute to exploratory efforts evaluating automation and LLM-assisted capabilities.

Qualifications

Required experience for the Data Engineer includes, but is not limited to, the following:

• Five (5) years of hands-on experience in each of the following:

1. Maintaining SQL databases and conducting advanced operations in SQL and T-SQL.
2. Designing, implementing, and maintaining ELT/ETL processes in cloud-based data analytics environments.

• Three (3) years of hands-on experience in each of the following:

1. Working in Azure Synapse and Azure Machine Learning, with the modern data stack. Certifications preferred (DP-203 or equivalent).
2. Manipulating data in Python. Pandas required. PySpark/Polars preferred. Experience developing reusable, modular code preferred.

Preferred experience for the Data Engineer includes the following:

• Implementing pipelines and infrastructure using code-first approaches (Python SDK, CLI, REST APIs, or IaC tooling)
• Implementing source control and CI/CD workflows
• Demonstrated familiarity with AI coding assistants and LLM integration patterns

The background investigation required for this position is at a sensitivity level of public trust.

Additional Information

Integres offers competitive salaries bolstered by a comprehensive benefits package, which provides safety and security for our employees and their families. Our generous benefits package includes

• Healthcare and Insurance: medical, dental, vision, short- and long-term disability protection, basic life and AD&D insurance
• 401(k) Savings Plan
• Accrued Paid Time Off (PTO)
• Employee Recognition and Rewards
• Employee Referral Bonuses

Integres is an equal opportunity affirmative-action employer.  We encourage Minority/Female/Protected Veteran/Disabled applicants to apply.  Integres, LLC proactively fulfills its role as an equal opportunity employer.  We do not discriminate against any employee or applicant for employment because of race, color, sex, religion, age, sexual orientation, gender identity and expression, national origin, marital status, physical or mental disability, status as a Disabled Veteran, Recently Separated Veteran, Active Duty Wartime, or Campaign Badge Veteran, Armed Forces services Medal, or any other characteristic protected by law.

All your information will be kept confidential according to EEO guidelines.