LLM SDET (Software Development Engineer in Test)

Job Description: LLM SDET (Software Development Engineer in Test)

Experience: 3+ years

Budget: ₹80,000 - ₹1,00,000 per month

Role Overview:

We are seeking an experienced LLM SDET to design, develop, and execute test strategies for Large Language Models (LLMs). The ideal candidate will have strong automation skills, a deep understanding of LLM-based applications, and experience in ensuring AI-driven systems' reliability, accuracy, and performance.

Key Responsibilities:

• Develop and implement test automation frameworks for LLM-based applications.
• Design and execute functional, regression, and performance tests for AI models.
• Validate model outputs for accuracy, consistency, and ethical compliance.
• Collaborate with ML engineers and developers to improve test coverage.
• Analyze and report model performance issues and work towards resolutions.
• Optimize test pipelines and workflows for efficiency.
• Ensure seamless API and system integrations with LLM-based applications.

Required Skills & Qualifications:

• 3+ years of experience as an SDET, QA Engineer, or in test automation.
• Strong proficiency in Python, Java, or JavaScript for test automation.
• Experience testing LLM applications, NLP models, or AI-driven systems.
• Hands-on experience with test automation tools (Selenium, PyTest, Robot Framework, etc.).
• Familiarity with API testing (Postman, REST Assured) and performance testing (JMeter, Locust).
• Experience working with ML model evaluation, data validation, and prompt testing.
• Knowledge of cloud environments (AWS, GCP, or Azure) is a plus.
• Strong analytical and problem-solving skills with attention to detail.

Nice to Have:

• Exposure to LLM fine-tuning and model evaluation techniques.
• Experience with CI/CD pipelines and DevOps practices.
• Understanding of ethical AI testing principles.

If you're passionate about AI testing and ensuring the robustness of LLM-based systems, apply now to be part of this exciting short-term opportunity!