Comptroller - Observability Analyst (Reopen)

Applications must be filed at: https://capps.taleo.net/careersection/ex/jobdetail.ftl?job=00051218

If you have previously applied, there is no need to reapply. Your application is still being considered for this position.

Please note that this job posting can be closed without notice and earlier than the closing date indicated in the posting if a suitable candidate is found. We encourage applicants to submit and complete the application promptly.

Are you ready to grow, make a positive impact? If you are seeking to gain knowledge, build your career, and network among goal-oriented professionals, this is the place for you!

Innovation, collaboration, and a commitment to excellence best describes the culture here at the Comptroller’s Office. We take pride in the work we do serving as Texas' accountant, tax collector, treasurer, and much more! The Comptroller’s office serves virtually every citizen in the state. As Texas’ chief tax collector, accountant, revenue estimator, treasurer and purchasing manager, the agency is responsible for writing the checks and keeping the books for the multi-billion-dollar business of state government.

Our agency workforce is as diverse as the people of Texas we serve. We value our employees and take very seriously our collective commitment to public service.  Personal development opportunities are strongly encouraged through available workshops, teambuilding exercises, and on the job training. We offer flexible scheduling that helps employees maintain a healthy work-life balance. 

Click here to see an inside look at the Texas Comptroller’s office!

ABOUT THE Division

General description

The Texas Comptroller’s Office is hiring an Observability Analyst within the Information Technology Division. This role is responsible for providing advanced support for agency IT systems. Key duties includes building and enhancing our end-to-end monitoring, logging, and performance visibility across complex, hybrid-cloud environments, ensuring the reliability, scalability, and security of our systems through advanced telemetry and insight generation. The position may supervise the work of others. Strong customer service skills and proven expertise in telemetry and observability are essential. The ideal candidate will be a quick learner who can adapt rapidly to shifting priorities in a dynamic, fast-paced environment.

Work Hours

Must be able to work an eight-hour schedule between 7:00 am and 6:00 pm, 40-hour week, Monday – Friday with occasional work schedule variance when necessary. Perform on-call duty on a rotational basis and occasionally work additional hours based on business peaks. Hours may change based on business need.

Teleworking may be considered if certain working conditions are met. The candidate must have permanent residency within the state of Texas and must be able to meet in office requirements, which can vary among different sections (i.e. report to office 1 day/month, 3days/week, etc.). Travel reimbursements are not provided for teleworking.

Minimum Qualifications

• Bachelor's degree.
• Seven (7) years of work experience in open telemetry agents and collector pipelines, observability governance or analysis, information data or security analysis, and/or managing enterprise monitoring and collection applications.
• Seven (7) years of general technology experience in either software development, systems administration, or server administration, configuration, and troubleshooting.
• Five (5) years of experience with programming/scripting for automation in Ansible, Java, Python, BASH scripts, Node.js or similar.

Preferred Qualifications

• Bachelor's degree in information technology, computer engineering, computer information systems, computer science, management information systems, or a related field.
• Experience with the development and execution of enterprise-level strategies and technical direction, including establishing processes and procedures for executing and maintaining technology road maps.
• Experience with deploying to cloud-native environments.
• Experience with web application and web services development and/or deployments.
• Experience with Middleware systems and similar systems in Linux or Windows. 
• Experience in monitoring server and application performance in Splunk, Dynatrace, or similar tools

Substitution

One (1) additional year of the full-time minimum experience in software development, systems administration, or server administration, configuration, and troubleshooting may substitute for thirty semester hours of the required education with a maximum substitution of 120 semester hours (four years).

In this role you will: 

Architecture
• Establishes governance and identifies observability and telemetry types for processing and routing by working with platform and application owners who own the telemetry types at the source to define normalized standards with a focus on data optimization, reduction, and control.
• Designs, implements, and maintains observability tools and frameworks (e.g., Dynatrace, Splunk, OpenTelemetry).
• Drives standardization of metrics, logging, and tracing practices across all environments; ensures alignment with agency requirements and performance SLAs.
• Provides instrument services and applications across cloud (AWS, Azure) and on-prem environments to support root cause analysis and system optimization.
• Coordinate projects across functional agency systems, and other state entities that require planning and scheduling during project development and implementation.
• Designs, develops, and maintains APIs and streaming infrastructure for log collection and analysis and network telemetry projects, with a focus on efficiency and scale.

Operations
• Develops and automates dashboards, alerts, and insights to proactively identify system anomalies and performance bottlenecks.
• Monitors systems, subsystems, and software applications which support the data monitoring section
• Facilitates the setup and coordination of the Agency secure file transfer system for the transfer of agency files internally and externally.
• Evaluates network, telemetry, observability, and system configurations for best practices and risk-based access controls.
• Conducts the training of staff prior to the implementation of systems, and provides advice and guidance during and post implementation
• Recommends to the management systems technology solutions and enterprise-related services, hardware and software standards
• Analyzes new or existing procedures, information systems, or utility programs for efficiency and effectiveness.
• Conducts root cause analysis (RCA) and postmortems to drive continuous improvement in system resilience.
• Provides expertise in incident response processes, including war room participation and rapid diagnostics.
• Troubleshoots complex systems problems related to infrastructure and application information systems.
• Completes projects and performs systems reviews 
• Participates in rotating 24x7 on-call schedule
• Reviews and analyzes business requirements, legislation, project proposals, statements of work, contracts, policies, procedures, and required levels of service, and other requests to determine estimates, accuracy, completeness, necessary system changes, impacts, and risks to agency applications.
• Follows agency procedures for maintaining hardware and software inventory
• Participates in disaster recovery testing

Administration
• Represents the data monitoring section in meetings, audits, and strategic planning forums, translating technical depth into mission-aligned priorities and roadmap outcomes.
• Assists with budget recommendations, including the supporting services, hardware and software
• Research and interprets system requirements to identify, support, and document system interfaces, architectural modifications, and implementation activities.
• Manages complex systems analysis projects involving highly complex design, development, and implementation of programs and applications.
• Assists in advising management and users regarding agency best practices, policies, and procedures for data security, governance, and retention. 
• Mentors junior team members and leads technical working groups focused on telemetry innovation, system health modeling, and AI/ML-based anomaly detection. 
• Provides oversight and coordination of IT vendors.
• Initiates and maintains accurate Change Management workflows
• Creates and maintains accurate documentation of current environments 
• Completes agency mandatory and assigned training courses.

Other
• Performs other related work as assigned.

Maximize Your Earnings!

At the Comptroller's office, we know potential employees are looking for more than just a paycheck. The agency offers a strong benefits package for you and your family. Insurance, retirement plans, and a flexible work schedule are just the start. See our benefits offering

If you are seeking to gain knowledge, build your career, and network among goal-oriented professionals, this is the place for you!

Important Note to the Candidate:

 
Salary is contingent upon qualifications. If you are scheduled for an interview and require any reasonable accommodation in our interview process, please inform the hiring representative who calls you to schedule your interview. Whenever possible, please give the hiring representative sufficient time to consider and respond to your request.

MILITARY PREFERENCE

To receive MILITARY PREFERENCE, you are required to provide proof by attaching the necessary documentation to this job application.

https://hr.sao.texas.gov/CompensationSystem/JobDescriptions
Click on the occupational category for the position.

Additional MOS can be found at the State Auditor’s Office, 

Military Crosswalk Guide.

The Texas Veterans Commission provides helpful employment information. Go to: http://www.texasskillstowork.com, http://www.onetonline.org, https://hr.sao.texas.gov/CompensationSystem/JobDescriptions/ or www.careeronestop.org for assistance translating your military experience and training courses into civilian job terms, qualifications/requirements and skill sets.

Applications submitted through Work in Texas:

Work In Texas (WIT) applicants must complete supplemental questions to be considered for the position. To complete the supplemental questions please go to CAPPS Recruit to register or login and access your profile.

The Comptroller’s Office is proud to be an equal opportunity workplace. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity.

Resumes will not be accepted in lieu of completed applications. The application must contain the necessary information in the work history for the applicant to qualify for the position.

An applicant must be eligible to work in the United States to be hired at the Comptroller's office and must remain eligible, without sponsorship or any assurance of financial or other assistance from this agency, during the term of their employment.