Senior DevOps Engineer

• Deploy and configure Kafka clusters following best practices for scalability, security, and performance
• Collaborate with cross-functional teams to gather requirements and design Kafka infrastructure to meet data streaming needs
• Manage topics, partitions, replication factors, and broker configurations to ensure efficient data distribution and fault tolerance
• Implement monitoring and alerting solutions to proactively identify and address performance bottlenecks, resource constraints, and anomalies
• Conduct regular performance testing, load testing, and capacity planning to ensure clusters can handle anticipated workloads
• Design and implement high-availability strategies, including failover mechanisms and data replication across multiple data centers or cloud regions
• Develop and maintain disaster recovery plans and procedures to minimize data loss and downtime in the event of failures
• Implement and manage security measures such as encryption, authentication, and authorization to ensure data privacy and compliance with industry standards
• Stay updated on security vulnerabilities and patches, applying necessary updates to maintain a secure Kafka environment
• Diagnose and resolve Kafka-related issues, including performance degradation, data loss, and connectivity problems
• Collaborate with development and data engineering teams to troubleshoot consumer/producer application integration with Kafka
• Maintain thorough documentation of Kafka configurations, deployment processes, and troubleshooting procedures
• Provide training and knowledge sharing sessions to junior team members and other stakeholders
• Collaborate with cross-functional teams, including data engineers, developers, and system administrators, to ensure smooth integration of Kafka into our data ecosystem
• Communicate effectively with stakeholders to provide updates on Kafka performance, maintenance, and improvements
• Provide on-call support for critical issues outside of normal business hours

• Minimum of 5+ years of experience as a Kafka Administrator in production environment
• At least 2+ years of experience with AWS MSK in production environment
• Strong understanding of Kafka architecture, internals, and deployment

• Experience with Terraform
• Experience with CI/CD pipelines and containerization technologies like Docker and Kubernetes
• Familiarity with AWS services
• Excellent problem-solving and analytical skills
• Strong communication and collaboration abilities

Please note: Our intelligent job search engine discovered this job and republished it for your convenience.
Please be aware that the job information may be incorrect or incomplete. The job announcement remains the property of its original publisher. To view the original job and its full details, please visit the job's URL on the owner’s page.