Job Overview (TL;DR):
Seeking a Senior Elastic Stack Engineer and Architect to lead the design, implementation, and maintenance of ELK (ElasticSearch, Logstash, Kibana, Beats) solutions for big data projects in cloud and on-prem environments. The role focuses on data integration, security, performance optimization, and operational support for mission-critical data observability systems. You will be responsible for deploying and managing ELK Stack solutions to help clients derive insights from structured and unstructured data, with a focus on system metrics, logs, and security.

Key Requirements:

Primary Skills:

• ELK Stack (ElasticSearch, Logstash, Kibana, Beats)
• Data ingestion and normalization (Elastic Beats, Logstash)
• System administration for Linux environments
• Query development and relevancy tuning
• Security best practices (Elastic Security, SIEM integration)

Secondary Skills:

• Experience with cloud environments (AWS, Azure, GCP)
• Scripting and automation (Python, Bash, PowerShell)
• Version control systems (GitLab, GitHub)
• Networking and security concepts (TCP/IP, firewalls, NAT)

Tools Used:

• Primary: Elastic Stack (ElasticSearch, Logstash, Kibana, Beats), Elastic Security, SIEM platforms
• Secondary: GitLab, GitHub, Logstash, Filebeat, Metricbeat, Elastic Agent, Bash, Python, AWS, Azure, GCP

Detailed Description:

As a Senior Elastic Stack Engineer and Architect, you will:

• Architect, deploy, and optimize ELK solutions across cloud-based and on-prem Linux environments.
• Lead data onboarding, including parsing, enriching, and normalizing data for integration into ElasticSearch using Beats, Logstash, or custom pipelines.
• Design and maintain indexes, custom dashboards, and queries for monitoring system metrics, logs, and traces.
• Ensure high availability and scalability of Elastic Stack deployments by managing distributed nodes and clusters.
• Implement and maintain security best practices for data observability, including access controls and encryption, while adhering to DoD standards.
• Support operational tasks like troubleshooting, upgrades, and performance tuning of Elastic Stack components.
• Collaborate with multi-disciplinary teams of analysts, engineers, and data scientists to improve the overall mission impact of the observability solutions.

You will be responsible for writing and maintaining documentation for deployment configurations, processes, and troubleshooting procedures, enabling future teams to sustain and improve the ELK solutions.