Senior Lead Data Engineer
Virtusa.com
Office
1
Full Time
Senior Lead Data Engineer - (CREQ229174)
* Contribute using our standard tech stack, which is Python, Linux (via WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS cloud infrastructure.
* Participate in the design and implementation of advanced mechanisms for data ingestion and transformation, and mass parallel orchestration of network IO.
* Maintaining Infrastructure as Code and working with infrastructure teams to develop valuable new features for Client's Engineering Platform.
* Model data for a variety of purposes (operational and analytical) in collaboration with your team, other teams, and analytical data customers.
* Contribute to CLIENT's data platform in a way that promotes the development of a data mesh architecture for the entire enterprise.
* Work on indicators of data quality across our estate and improve the quality of our data along key metrics.
* Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.
* Contribute to quality automation in our CI/CD process, optimising for bug-free release cadence and developer delight.
* Maintain good quality documentation (starting with your code) for technical and non-technical audiences.
* Participate in a pairing-first development team, learning skills from your colleagues and giving back to the community by imparting hard-won wisdom and skills learned from past experiences.
* Mentoring and coaching of less experienced team members through design and code reviews
* Participate in a safe and honest workplace culture by participating in open conversations about technical design, implementation and interrogating decisions when appropriate.
* Contribution to test automation through either enhancement to the platform or additional tests
* Assessment of application security to ensure it complies with CLIENT Application Security Standards
* Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards
Skills & Ability
* You are able to grasp, articulate, and advocate for technological solutions in both technical and non-technical settings and intelligently apply agile principles as you do so.
* You are able to interrogate and present data and data analysis and have a willingness to present results of analysis undertaken as a part of your team's work.
* You have demonstrable experience as a senior software developer or engineer with a focus on Python.
* SQL (Postgres, SQL Server, Databricks)
* comfortable working without an ORM.
* familiar with the challenges of working with data *at scale*.
*prefer the command line use of git, bash/zsh, and (optionally) vim.
* An undergraduate degree or demonstrable knowledge and experience of concepts in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or any related discipline would be advantageous.
Description
Key responsibilities and Accountabilities* Contribute using our standard tech stack, which is Python, Linux (via WSL), PostgreSQL, SQL Server, Databricks, and spans both Azure and AWS cloud infrastructure.
* Participate in the design and implementation of advanced mechanisms for data ingestion and transformation, and mass parallel orchestration of network IO.
* Maintaining Infrastructure as Code and working with infrastructure teams to develop valuable new features for Client's Engineering Platform.
* Model data for a variety of purposes (operational and analytical) in collaboration with your team, other teams, and analytical data customers.
* Contribute to CLIENT's data platform in a way that promotes the development of a data mesh architecture for the entire enterprise.
* Work on indicators of data quality across our estate and improve the quality of our data along key metrics.
* Evaluate the quality (completeness and accuracy) of potential new data sources for specific applications.
* Contribute to quality automation in our CI/CD process, optimising for bug-free release cadence and developer delight.
* Maintain good quality documentation (starting with your code) for technical and non-technical audiences.
* Participate in a pairing-first development team, learning skills from your colleagues and giving back to the community by imparting hard-won wisdom and skills learned from past experiences.
* Mentoring and coaching of less experienced team members through design and code reviews
* Participate in a safe and honest workplace culture by participating in open conversations about technical design, implementation and interrogating decisions when appropriate.
* Contribution to test automation through either enhancement to the platform or additional tests
* Assessment of application security to ensure it complies with CLIENT Application Security Standards
* Evaluation of APIs to ensure that they comply with the relevant CLIENT API Standards
Skills & Ability
* You are able to grasp, articulate, and advocate for technological solutions in both technical and non-technical settings and intelligently apply agile principles as you do so.
* You are able to interrogate and present data and data analysis and have a willingness to present results of analysis undertaken as a part of your team's work.
* You have demonstrable experience as a senior software developer or engineer with a focus on Python.
* SQL (Postgres, SQL Server, Databricks)
* comfortable working without an ORM.
* familiar with the challenges of working with data *at scale*.
*prefer the command line use of git, bash/zsh, and (optionally) vim.
* An undergraduate degree or demonstrable knowledge and experience of concepts in Computer Science, Software Engineering, Logic, Applied Statistics, Actuarial Science, Data Science, or any related discipline would be advantageous.