Overview

We are looking for a Full-Stack Software Engineer to improve the quality, reliability, and performance of Frank’s AI-powered voice interviews. You will work across backend services, real-time voice agents, infrastructure, observability, and evaluation systems. Your primary responsibility will be to understand how an AI voice interview behaves end to end, establish measurable quality standards, evaluate interviews using quantitative and qualitative metrics, and turn findings into engineering improvements.

Responsibilities:
  • Develop a deep understanding of the complete voice-interview lifecycle, from participant admission and bot greeting through conversation, transcription, recording, and analysis
  • Design repeatable evaluation frameworks for comparing prompts, models, voice pipelines, providers, and infrastructure changes
  • Define, collect, and analyze quantitative metrics, including:
  • Response latency and time to first audio
  • End-of-utterance and STT finalization latency
  • Interruption and false-interruption rates
  • Transcript accuracy and missing or duplicated turns
  • Interview completion, failure, and abandonment rates
  • Provider and pipeline error rates
  • Audio quality, packet loss, jitter, and connection stability
  • Token usage, provider cost, CPU, and memory consumption
  • Evaluate qualitative interview characteristics, including:
  • Question relevance and research-goal coverage
  • Quality and depth of follow-up questions
  • Context retention and conversational coherence
  • Neutrality and avoidance of leading questions
  • Natural turn-taking, pacing, tone, and voice quality
  • Empathy and responsiveness to participants
  • Hallucinations, repetition, awkward transitions, and premature endings
  • Build automated evaluation tools using transcripts, telemetry, recordings, LLM-as-judge
Required Qualifications:
  • Strong software engineering experience with Python and/or TypeScript.
  • Experience building, testing, or evaluating LLM-powered applications.
  • Understanding of real-time voice systems, including STT, LLM, TTS, voice activity detection, endpointing, and interruption handling.
  • Strong analytical skills and experience turning product-quality questions into measurable metrics.
  • Experience with experimentation, statistical analysis, and structured qualitative review.
  • Ability to diagnose problems across application code, external AI providers, media infrastructure, and observability data.
  • Experience writing automated tests and maintaining production-quality systems.
  • Strong written communication and documentation skills.
Benefits:
  • Own meaningful AI features end-to-end.
  • Pragmatic stack with room to choose the right tool (framework-light where it helps).
  • Fast decisions, real users, real impact.
  • Competitive Salary
  • 1-Month Work & Travel Opportunity (per year)
  • Health Insurance or Gym Packages
  • Optional Remote Fridays
  • Paid Day-Offs
  • Specialized Library & Online Courses
  • Weekly Internal Trainings & Knowledge Sharing
  • Mentorship from Founders & Industry Experts
  • Personal Development Plans & Semi-Annual Assessments
  • Access to Global Communities & Industry Events
  • Internal Career Acceleration Program
  • Innovation Challenges
Nice To Have:
  • Experience with LiveKit, WebRTC, or similar real-time media technologies.
  • Experience with Langfuse and OpenTelemetry. Familiarity with Gemini, OpenAI Realtime or OpenRouter.
  • Experience designing LLM-as-judge evaluations and human-annotation rubrics.
  • Knowledge of conversational research, user interviewing, or qualitative research methods.
  • Experience with AWS services such as ECS, CloudWatch, S3, and SQS.
  • Familiarity with NestJS, PostgreSQL, and infrastructure as code.
  • DBT/analytics chops; safety/compliance (SOC-2, GDPR DPIAs).
Skills:
  • Strong analytical skills
  • Experimentation
  • Statistical analysis
  • Structured qualitative review
  • Problem diagnosis
  • Automated testing
  • Written communication
  • Documentation
Technologies:
  • OpenAI
  • Anthropic
  • OpenSearch
  • pgvector
  • Pinecone
  • TypeScript
  • Node
  • Python
  • Terraform
  • CDK
  • LiveKit
  • WebRTC
  • Langfuse
  • OpenTelemetry
  • Gemini
  • OpenAI Realtime
  • OpenRouter
  • AWS
  • ECS
  • CloudWatch
  • S3
  • SQS
  • NestJS
  • PostgreSQL
Note:

✨ Our intelligent job search engine discovered this job and republished it for your convenience.
Please be aware that the job information may be incorrect or incomplete. The job announcement remains the property of its original publisher. To view the original job and its full details, please visit the job's URL on the owner’s page.

Please clearly mention that you have heard of this job opportunity on https://ijob.am.