Senior Observability Engineer

Key Responsibilities

• Administer and optimize the Splunk Enterprise platform, including indexers, search heads, forwarders, and data onboarding.
• Build and maintain SPL queries, dashboards, alerts, and scheduled reports to support real-time monitoring and historical analysis.
• Work closely with cross-functional teams to ensure relevant logs, metrics, and events are properly captured and structured for operational use.
• Maintain and improve log pipelines and data parsing workflows using tools like syslog, Cribl, Fluentd, or Logstash.
• Assist in defining and enforcing logging and observability standards across infrastructure and application environments.
• Support onboarding of new data sources, and ensure appropriate indexing and data lifecycle practices.
• Contribute to incident investigations by providing visibility into system behavior and supporting root cause analysis.
• Support efforts to integrate and manage complementary monitoring tools (e.g., Grafana, LogicMonitor, Prometheus, AppDynamics, Dynatrace).
• Document platform configuration, SOPs, dashboards, and knowledge objects to support maintainability and team collaboration.

Qualifications

• 5–8 years of experience in observability, infrastructure monitoring, or SRE roles.
• Minimum 5 years of experience with Splunk Enterprise in an operational or engineering capacity.
• Strong understanding of Splunk architecture, deployment components, and performance tuning techniques.
• Proficiency in writing SPL queries, designing dashboards, and configuring alerts.
• Familiarity with log routing tools such as syslog, Fluentd, Logstash, or Cribl.
• Working knowledge of cloud infrastructure (AWS, Azure, or GCP) and hybrid environments.
• Basic scripting or automation skills (e.g., Python, Bash, Terraform, Ansible).
• Comfortable working collaboratively in cross-functional teams.

Must Have

• Splunk certifications (e.g., Splunk Core Certified Power User, Admin, Architect).
• Experience with metrics and APM tools (e.g., Prometheus, Grafana, OpenTelemetry, Jaeger).