AI Benchmark
Location
Remote
Job Type
Part-Time
Experience Level
Associate level (1-3 Years)
Salary Range
Not disclosed
Job Description
We are seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reliably expose weaknesses in state-of-the-art (SOTA) AI models (e.g., Claude Opus 5 Max). In this role, you will discover hard-to-surface facts, build precise research tasks around them, test them directly against leading LLMs, and diagnose the exact failure mechanisms in the model's search and reasoning paths. Your work will directly shape AI evaluation frameworks, ensuring next-generation frontier models are tested against rigorous, non-trivial real-world research scenarios. Job Requirements: Core Experience: 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review. Benchmark Familiarity: Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE). Analytical & OSINT Skills: Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases. Model Trajectory Debugging: Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure. Precision Writing: Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys. Job Responsibilities: Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries. Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs. Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies. Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures. Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed (distinguishing genuine model limitations from ambiguous prompts). Pattern Scaling & Review: Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers. Strong Plus (Preferred Qualifications) Bachelor or higher degree from relevant engineering domain is a must for this position. Prior experience specifically designing prompt-injection or reasoning-trap tasks that break SOTA models. Advanced OSINT capabilities or published investigative/academic research demonstrating deep source verification. Niche domain expertise in areas rich with hard-to-find records (e.g., local government filings, regional sports history, legal archives, historical patents). Experience serving as a lead reviewer, benchmark auditor, or quality owner on AI evaluation projects.
About Quickhyre AI
QuickHyre AI is an online job portal and HRMS software platform for hiring and workforce management, trusted by over 10 million active users. It connects freshers and experienced professionals across all career levels with verified job opportunities, while enabling companies to hire, onboard, and manage employees through a single system. Candidates use QuickHyre AI to search and apply for jobs across entry-level, mid-level, and senior roles, including internships and contract positions across tech and non-tech domains. Employers use QuickHyre AI as a hiring platform, applicant tracking system (ATS), and HRMS to post jobs, screen candidates, manage applications, onboard hires, and maintain employee records. Built for startups, growing companies, and enterprises, QuickHyre AI reduces irrelevant applications, improves hiring efficiency, and simplifies HR operations. By combining job discovery, online recruitment, and HR management software, QuickHyre AI supports end-to-end talent acquisition and workforce administration at scale.
Connections
Sai Charan
Senior Developer
Kalpana Sharma
Team Lead
Rahul Patel
Full Stack Developer
Priya Singh
Frontend Developer
Connect with professionals in your network
Skill Match Analysis
??% skills matched (?? of 25 skills)
💡 This is keyword matching for reference only. Your actual match score uses AI semantic analysis.
Login to see your score