Application Support - Data Engineer (+3 Years Experience)

Valleysoft is a leading IT services provider offering innovative solutions to clients worldwide. As a Data Engineer at Valleysoft, you will be responsible for designing, building, and managing scalable data pipelines and architectures that enable data-driven decision making. You will collaborate with data scientists, analysts, and business stakeholders to ensure the availability, reliability, and quality of data.

Key responsibilities include:

• Develop, construct, test, and maintain data architectures such as databases and large-scale processing systems.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Design and implement ETL/ELT processes to gather and prepare data for analysis.
• Collaborate with cross-functional teams to understand data needs and deliver solutions.
• Monitor and optimize the performance of data systems and pipelines.

Requirements

• Provide second and third-line application support for the enterprise data platform, covering data warehouses, data marts, ETL pipelines, and data integration layers across production and non-production environments.
• Monitor, triage, and resolve incidents related to ETL job failures, data pipeline breaks, data quality anomalies, and performance degradation — ensuring SLA adherence and minimal business disruption.
• Investigate root causes of recurring data issues, pipeline failures, and feed delays; implement permanent fixes and preventive measures through proper change control channels.
• Perform day-to-day operational tasks including job reruns, data corrections, parameter adjustments, and table refreshes in coordination with data owners and business stakeholders.
• Collaborate with data engineering, DBA, middleware, and infrastructure teams to resolve cross-system issues affecting data availability and pipeline integrity.
• Maintain and update ETL code, mappings, workflows, and configuration files in response to source system changes, schema evolution, or business rule updates.
• Support onboarding of new data feeds and sources by assisting in integration testing, environment validation, and go-live stabilization.
• Maintain accurate and up-to-date support documentation: runbooks, known error records, escalation procedures, and post-incident reports.
• Participate in change advisory board (CAB) reviews for data platform changes, providing impact assessments and rollback plans.
• Proactively identify opportunities to improve platform reliability, automate repetitive operational tasks, and reduce manual intervention in pipeline management.

Technical Requirements

• Hands-on experience supporting ETL platforms such as Informatica PowerCenter, IBM DataStage, Talend, or equivalent enterprise integration tools.
• Strong SQL skills (Oracle, SQL Server, or DB2) — capable of writing diagnostic queries, tracing data lineage manually, and performing targeted data fixes under change control.
• Solid understanding of data warehouse architecture: ODS, staging layers, data marts, and EDW — sufficient to trace and isolate issues across pipeline stages.
• Experience with enterprise job scheduling tools such as Control-M, TWS, or Autosys — managing job dependencies, calendars, and failure alerts.
• Familiarity with Unix/Linux shell scripting for log analysis, file transfer monitoring, and lightweight automation of support tasks.
• Understanding of database performance concepts: execution plans, index usage, partition pruning, and statistics — to diagnose slow-running jobs and queries.
• Exposure to data quality monitoring frameworks and the ability to validate pipeline outputs against expected row counts, aggregates, and business rules.
• Experience with ITSM platforms (ServiceNow, Remedy, or equivalent) for incident logging, change requests, and problem management workflows.
• Familiarity with cloud data platform components (Azure Data Factory, AWS Glue, Snowflake) is a plus.

Banking Sector Requirements (Preferred)

• Prior experience supporting data platforms in a bank or financial institution, with exposure to time-sensitive regulatory and financial reporting pipelines.
• Familiarity with banking source systems feeding the data warehouse — core banking, GL, loan origination, cards, and payment systems.
• Understanding of data reconciliation and balancing requirements: ensuring ETL outputs align with source system totals and downstream report figures.
• Experience supporting pipelines that feed regulatory reports (CBE submissions, IFRS 9 staging, AML transaction monitoring) where data accuracy and timeliness are non-negotiable.
• Awareness of data classification and access control requirements for sensitive banking data, including PII handling and audit trail obligations.
• Experience operating within ITIL-aligned support models with strict change control, incident prioritization, and escalation procedures applicable to regulated environments.

Benefits

• Private Health Insurance
• Training & Development