Date Posted:
20 October, 2025
Industry:
IT Services and IT Consulting
Location:
VAPORVM IT SERVICES DMCC
Job Description:
Role Overview:
We seek a Senior QA Engineer to lead the design, testing, validation, and automation of QA strategies across our Data & AI programs.
The role spans testing data pipelines, AI/ML models, Generative AI (LLM) applications, and multi-agent (Agentic AI) workflows, ensuring solutions meet the highest standards of accuracy, reliability, fairness, security, and compliance.
The ideal candidate will bring hands-on expertise in testing LLM-based applications and Agentic AI systems, along with strong foundations in data quality assurance and AI/ML model validation.
Key Responsibilities:
- Define
and enforce QA frameworks, processes, and standards for data pipelines,
AI/ML models, GenAI/LLM applications, and agent-based AI systems.
- Develop and automate data validation tests (integrity, consistency, lineage, schema drift).
- Design and execute ML model validation tests, including performance, fairness, bias, and reproducibility.
- Implement
automated prompt-response validation for GenAI/LLM applications
(semantic similarity, factuality, hallucination detection).
- Integrate QA processes into CI/CD, MLOps, and LLMOps workflows.
- Develop monitoring and alerting systems for data drift, model decay, and pipeline failures.
- Define, track, and report on AI evaluation metrics.
- Ensure compliance with AI governance, ethics, security, and privacy requirements.
- Document QA processes, maintain reusable test cases, and support audits.
Skills & Qualifications:
- 5+ years of experience in Quality Assurance, with at least 2+ years focused on Data, AI/ML, GenAI, or Agentic AI testing.
- Proven experience testing LLM/GenAI applications (prompt testing, RAG pipelines, grounding validation, hallucination detection).
- Experience testing Agentic AI workflows (multi-agent orchestration, decision-making validation, safety guardrails).
- Strong Python skills and proficiency with testing frameworks (PyTest, unittest).
- Knowledge of ML evaluation metrics (precision, recall, F1, ROC, fairness, bias testing).
- Familiarity with AI governance, compliance, and privacy testing.
- Strong analytical skills, with attention to detail and quality.
- Excellent communication and collaboration skills for cross-functional work.
- Excellent writing/ verbal communication in Arabic/ English