Confluent Kafka Engineer

Job description

Confluent Kafka Engineer - Development, UAT, and Production Support

Join or sign in to find your next job

Join to apply for the Confluent Kafka Engineer - Development, UAT, and Production Support role at Dicetek LLC

Confluent Kafka Engineer - Development, UAT, and Production Support

Join to apply for the Confluent Kafka Engineer - Development, UAT, and Production Support role at Dicetek LLC

Job Title: Confluent Kafka Engineer - Development, UAT, and Production Support
Location: Abu Dhabi
Job Summary
We are seeking a highly skilled Confluent Kafka Engineer to join our dynamic team.

The ideal candidate will be responsible for the development, User Acceptance Testing (UAT), and production support of our Confluent Kafka-based systems.

This role requires deep expertise in Kafka architecture, including Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.

The engineer will collaborate closely with cross-functional teams to ensure the smooth operation of our data streaming services and provide support across various environments.
Experience

• 3+ years of hands-on experience with Apache Kafka and Confluent Kafka in a production environment, including experience with Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.
  
  
• Proven experience in Kafka development, including producer and consumer API, stream processing, and connector development.
  
  
• Experience with Kafka cluster management, including setup, configuration, monitoring, and troubleshooting.
  
  
• Familiarity with distributed systems, microservices architecture, and event-driven design patterns.
  
  
• Experience with cloud platforms (e.g., AWS, Azure) and containerization (Kubernetes) is a plus.
  

Technical Skills

• Proficiency in programming languages such as Java, Python, or Scala.
  
  
• Strong knowledge of Kafka internals, including brokers, zookeepers, topics, partitions, and offsets.
  
  
• Experience with monitoring tools (e.g., Prometheus, Grafana) and logging frameworks (e.g., Log4j, ELK Stack).
  
  
• Proficiency in using Confluent Control Center for monitoring, managing, and optimizing Kafka clusters.
  
  
• Expertise in Kafka Streams for building scalable, fault-tolerant stream processing applications.
  
  
• Experience with KSQLDB for real-time processing and analytics on Kafka topics.
  
  
• Strong understanding of Kafka Connect for integrating Kafka with external data sources and sinks.
  
  
• Understanding of networking, security, and compliance aspects related to Kafka.
  
  
• Familiarity with CI/CD pipelines and automation tools (e.g., Jenkins, GitLab CI).
  

Development
Key Responsibilities:

• Design, develop, and implement Kafka-based solutions to meet business requirements, utilizing Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.
  
  
• Write and maintain high-quality code for Kafka producers, consumers, and stream processing applications.
  
  
• Develop and manage Kafka connectors for seamless integration with external systems, ensuring data consistency and reliability.
  
  
• Utilize Kafka Streams for real-time processing of streaming data, transforming and enriching data as it flows through the pipeline.
  
  
• Employ KSQLDB for stream processing tasks, including real-time analytics and transformations.
  
  
• Collaborate with data engineers, software developers, and DevOps teams to integrate Kafka solutions with existing systems.
  
  
• Ensure all Kafka-based solutions are scalable, secure, and optimized for performance.
  

User Acceptance Testing (UAT)

• Develop and execute UAT plans to validate Kafka solutions, including components such as Kafka Streams, KSQLDB, and Kafka Connect, before deployment to production.
  
  
• Work closely with QA teams to identify and resolve defects during the UAT phase.
  
  
• Ensure all UAT activities comply with the organization's standards and best practices.
  
  
• Provide detailed reports on UAT outcomes and work with developers to implement necessary fixes.
  

Production Support

• Monitor and maintain Kafka clusters, including components like Confluent Control Center, to ensure high availability and reliability of data streaming services.
  
  
• Troubleshoot and resolve issues related to Kafka performance, latency, and data integrity, including issues specific to Kafka Streams, KSQLDB, and Kafka Connect.
  
  
• Perform routine maintenance tasks such as patching, upgrades, and backups for Kafka clusters and associated components.
  
  
• Implement monitoring solutions to proactively identify and mitigate potential production issues, leveraging Confluent Control Center for comprehensive cluster visibility.
  
  
• Provide 24/7 support for production systems, including participation in on-call rotation.
  

Qualifications
Education: Bachelor's degree in Computer Science, Information Technology, or a related field.

Seniority level

• Seniority level

  Not Applicable

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Dicetek LLC by 2x

Get notified about new Production Support Engineer jobs in Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates .

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates AED3,000.00-AED7,000.00 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 9 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 months ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago

1 Full Stack Developer: Node.js Angular React.js

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 day ago

Abu Dhabi Emirate, United Arab Emirates 3 weeks ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates AED12,000.00-AED13,000.00 3 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 2 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 3 hours ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 day ago

Full Stack Java Developer - Banking (m/f/d)

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 19 hours ago

Abu Dhabi Emirate, United Arab Emirates 2 weeks ago

Software Developer ( .NET core, Angular)

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 2 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 20 hours ago

Front-End Developer - React Native and React (Web)

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago

API Developer-Terraform, Azure, and API Gateway for Microservices

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates AED14,000.00-AED15,000.00 3 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates AED14,000.00-AED18,000.00 3 months ago

Frontend Developer (Contractor) - Autonomous Robotics

Abu Dhabi Emirate, United Arab Emirates 2 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 3 hours ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 day ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 day ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago

Abu Dhabi Emirate, United Arab Emirates 2 weeks ago

Abu Dhabi Emirate, United Arab Emirates 3 weeks ago

We're unlocking community knowledge in a new way.

Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

Required Skill Profession

Other General