Job Area: Operations Group, Operations Group > Program Analyst General Summary: Assists with program development and implementation through managing processes, procedures, and tools that improve efficiencies. A Program Analyst coordinates across teams and monitors timelines, budgets, risks, and priorities to achieve program progress. Typically, a program needing a Program Analyst will be of significant size and will require expertise related to the development of project management mechanisms. Minimum Qualifications: • Associate's degree in Business Administration, Management, Computer Science, Engineering, Computer Science, or related field. OR High School Diploma or equivalent and 2+ years of relevant work experience. Job Title: Program Analyst Sr – Generative AI, Prompt Engineering & Validation Location: Bangalore Experience: 2–6 years Employment Type: Full-time About the Role We are looking for a proactive and detail-oriented Program Analyst to support Generative AI initiatives focused on prompt engineering, feature validation, test execution, output quality evaluation, and program tracking. The role will support internal GenAI platforms and tools, including search, summarization, RAG-based workflows, analytical agents, and domain-specific AI assistants. The ideal candidate should combine strong program coordination skills with a practical understanding of GenAI systems, structured prompt-based testing, multi-source data validation, and clear reporting for engineering and leadership stakeholders. Key Responsibilities 1. GenAI Program Tracking & Coordination Track GenAI feature validation plans, deliverables, owners, dependencies, and closure status across teams. Maintain dashboards, trackers, action item logs, risk registers, and validation status reports. Coordinate with engineering, product, platform, and domain expert teams to drive timely completion of testing and feedback cycles. Consolidate weekly progress updates, blockers, risks, and recommendations for program reviews. Ensure validation documentation, prompt libraries, and issue logs are accurate, current, and accessible. 2. Prompt Engineering & Prompt Test Design Create, refine, and maintain prompts for GenAI features, search workflows, summarization use cases, and analytical agents. Develop zero-shot, few-shot, role-based, multi-turn, edge-case, and negative test prompts. Build reusable prompt templates and feature-wise prompt libraries. Compare prompt variants and document impact on accuracy, relevance, completeness, consistency, and latency. Translate domain expert feedback into improved prompts and updated test scenarios. 3. Feature Validation & Test Execution Design and execute structured test cases for GenAI features, including functional, sanity, regression, negative, and exploratory testing. Validate end-to-end behavior from user prompt to retrieved context, generated response, and final output format. Map feature requirements to prompts, expected outputs, acceptance criteria, and validation evidence. Identify defects, usability gaps, hallucinations, incomplete answers, inconsistent responses, and low-confidence outputs. Support release readiness assessment using clear status indicators such as Go / Watch / Risk. 4. Data, Search & RAG Validation Validate AI outputs against structured, semi-structured, and unstructured data sources such as SharePoint, Confluence, Jira, documents, JSON/XML, and internal knowledge bases. Check search relevance, context quality, source completeness, and data freshness before summarization. Support validation of RAG-based workflows, semantic search, keyword search, global filters, and domain filters. Identify data gaps, indexing issues, retrieval failures, and source-quality problems impacting AI response accuracy. Maintain golden datasets, expected-answer sets, and benchmark queries for repeatable validation. 5. Output Quality Evaluation & Defect Analysis Evaluate GenAI responses for factual correctness, relevance, completeness, clarity, format quality, and consistency across repeated runs. Log defects with clear reproduction steps, prompt used, expected output, actual output, severity, and owner. Perform first-level root cause analysis to distinguish prompt issues, data issues, retrieval issues, model limitations, and product defects. Work with SMEs to validate technical correctness and define acceptance thresholds for complex responses. Track quality improvements over time and report trends in accuracy, hallucination reduction, and feature readiness. 6. Reporting, Documentation & Stakeholder Communication Prepare weekly validation summaries, risk updates, action item trackers, and executive-ready status reports. Document prompt libraries, test suites, validation evidence, known issues, and improvement logs. Facilitate review meetings, capture minutes of meeting, document decisions, and follow up on action items. Communicate clearly with cross-functional stakeholders across engineering, platform, product, and program management teams. Support leadership reviews with concise summaries, dashboards, and recommendations. 7. Automation & Productivity Support Use Excel, Power BI, Jira, Confluence, SharePoint, or similar tools to maintain trackers, dashboards, and reporting views. Support basic scripting or automation for batch prompt execution, response capture, comparison, and reporting where applicable. Help improve validation efficiency through reusable templates, structured checklists, and repeatable test processes. Track productivity metrics such as test coverage, closure rate, defect aging, prompt reuse, and feature readiness. Required Skills & Competencies 2–6 years of experience in program analysis, project coordination, validation, QA, data analysis, or technical operations. Hands-on exposure to Generative AI tools, LLM-based applications, prompt engineering, or AI-assisted workflows. Strong ability to create structured test cases, validation reports, action trackers, and executive summaries. Good understanding of prompt-response behavior, hallucinations, context limitations, and output evaluation methods. Working knowledge of search, summarization, RAG concepts, information retrieval, or knowledge-base validation. Strong Excel skills; Power BI, Jira, Confluence, SharePoint, or dashboarding experience is preferred. Basic Python, SQL, or scripting knowledge is preferred but not mandatory. Excellent written and verbal communication skills with strong attention to detail. Preferred Qualifications Education & Background Bachelor’s degree in Engineering, Computer Science, Electronics, Data Science, Information Systems, or a related discipline. Prior experience in semiconductor, SoC, embedded systems, platform engineering, enterprise AI, or technical program management is preferred. Experience supporting validation of internal tools, AI platforms, search systems, summarization tools, or workflow automation platforms is a plus. Exposure to responsible AI, data privacy, model evaluation, or enterprise GenAI governance is desirable. Success Metrics Completeness and accuracy of GenAI feature validation coverage. Quality, reusability, and effectiveness of prompt test suites. Timely closure of validation defects, risks, action items, and stakeholder feedback. Improvement in output accuracy, relevance, consistency, and hallucination reduction. Clarity and usefulness of weekly reports, dashboards, executive summaries, and readiness recommendations.

About Qualcomm

Qualcomm is a prominent American technology company based in San Diego, California, with a rich history of nearly 40 years in wireless telecommunications and semiconductor technology. Founded in 1985, Qualcomm is known for its pioneering work in Code Division Multiple Access (CDMA) technology, which has significantly influenced mobile communications and the development of modern cellular networks. The company offers a wide range of products, including integrated circuits, system processors, audio components, wireless networking solutions, radio frequency components, sensors, and system software. These products are utilized in various applications such as mobile devices, enterprise networks, automotive connectivity, and smart home systems. Qualcomm serves a diverse customer base, including mobile device manufacturers, network operators, and automotive companies, playing a vital role in the global wireless communication ecosystem.