Mô tả công việc
About Hupo
We are an AI-native start-up building sales enablement products in the banking, financial services, and insurance industry; already trusted by dozens of enterprise customers (including Fortune 500 companies).
Role at a Glance
Title: Production Support Engineer
Team: Engineering, reporting to the Director of Engineering
Employment Type: Full-time
Location: APAC (remote, with a preference for the Philippines, Vietnam, India or Singapore)
Working Hours: APAC business hours, with at least four hours of overlap with the India team
Why This Role Matters
As our client base grows, so does the number of technical issues that require hands-on investigation. We're looking for a Production Support Engineer who can dig into escalated issues, identify root causes, reproduce bugs and ship small, safe fixes through our normal release process.
You'll work closely with Technical Support, which handles initial client responses and communication, and our existing on‑call engineers, who own incident command, mitigation and root‑cause analysis. You'll investigate issues, provide clear technical findings and help resolve routine escalations without pulling product engineers away from their core work.
After your first 60 days, you'll join the existing on‑call rotation. The goal is to build a stronger production support function that resolves technical issues efficiently while allowing our product engineers to focus on building and improving Hupo.
What You'll Do
Investigate and reproduce production issues
- Review logs, traces and session data to identify the causes of escalated client issues, using approved tools and least‑privilege access.
- Reproduce production issues, especially those involving voice, audio, browsers and connectivity, including hands‑on troubleshooting with clients or their IT teams when needed.
- Determine whether an issue is a bug, configuration problem, feature request or user error, and document your findings so Technical Support can communicate them clearly to clients.
Ship small fixes and elevate complex issues
- Implement small, low‑risk fixes through code review and the standard release process, linking each pull request to its corresponding ticket. Direct changes in production are not permitted.
- Involve Engineering or Product when an issue requires a larger fix, providing the identified cause, impact and a suggested approach.
- Keep technical findings and ticket statuses up to date in Jira so Technical Support can provide accurate client updates.
Support incident response
- During incidents, gather evidence, reproduce issues and test potential fixes under the direction of the on‑call engineer.
- Turn the on‑call engineer's root‑cause analysis (RCA) into a clear, client‑ready technical summary for Technical Support.
- Work alongside the on‑call engineer while respecting their ownership of incident command, mitigation and the RCA process.
- Join the existing on‑call rotation after completing your first 60 days.
Own and improve support tooling
- Build and maintain support tooling and integrations, starting with routing tickets from our helpdesk platform (currently Gleap) into Jira using the rules defined by Technical Support.
- Improve the diagnostic tools used by Technical Support and the on‑call team, including log access, session lookup, admin tools and saved queries.
- Analyse ticket trends to identify recurring technical issues and recommend permanent product fixes to Engineering and Product, backed by relevant data.
- Write and maintain technical troubleshooting documentation for the knowledge base.
What Success Looks Like
Within your first few months, you'll:
- Independently investigate and reproduce common production issues, providing clear findings and actionable next steps.
- Resolve routine, low‑risk issues through safe code changes and the standard release process, reducing the number of ordinary escalations reaching product engineers.
- Establish reliable ticket routing and diagnostic workflows between Gleap, Jira and the relevant internal teams.
- Improve the quality of technical information available to Technical Support, enabling faster and more accurate client communication.
- Become familiar with our production systems, troubleshooting workflows and incident response practices, preparing to join the on‑call rotation after your first 60 days.
You'll also be expected to continuously:
- Identify recurring issues and opportunities to prevent problems rather than repeatedly address symptoms.
- Improve support tooling, diagnostics and troubleshooting documentation as the product and client base grow.
- Work closely with Technical Support, the on‑call team, Engineering and Product to resolve issues efficiently.
- Adapt and expand your impact as Hupo's technical support needs evolve.
What We're Looking For
Must‑Haves
- 3–5 years of experience in production support engineering or product support engineering, with hands‑on production debugging experience. This is a mid‑level engineering role, not a helpdesk position.
- Comfortable reading logs and traces, querying MongoDB and SQL databases, and reproducing issues based on user reports.
- Ability to write and ship small, safe fixes in TypeScript or Python, including opening clear pull requests and following standard code review and release processes.
- Working knowledge of web and network fundamentals, including browsers, permissions, HTTP, authentication, firewalls and network whitelisting.
- Experience with Jira and a helpdesk tool, including building integrations using APIs or webhooks.
- Strong data‑handling practices, including using approved tools, respecting access controls and keeping client data within authorised environments.
- Clear written communication skills, with the ability to explain technical causes to non‑engineers and prepare summaries suitable for clients.
- Willingness to work directly with clients' IT and security teams when issues involve their environments.
- Willingness to participate in the on‑call rotation after the initial 60‑day period.
- Ability to meet the background‑check requirements of our banking clients.
Nice‑to‑Haves
- Experience with real‑time audio or video, WebRTC, speech‑to‑text or text‑to‑speech systems.
- Familiarity with SSO and SAML, and LMS or SCORM integrations.
- Experience supporting enterprise clients in banking, financial services or insurance.
- Experience supporting an LLM or AI product in production.
Our Tech Stack
- Languages and services: TypeScript on Node.js (Fastify), Python
- Databases and caching: MongoDB, PostgreSQL and Redis
- Infrastructure and deployment: AWS, EKS, ECS, GitHub Actions and ArgoCD
- Voice: LiveKit and third‑party speech providers
- Identity and authentication: Auth0, SSO and SAML integrations with client identity providers
- Monitoring and diagnostics: Sentry, logs and traces
- Support and issue tracking: Jira and Gleap
Yêu cầu kinh nghiệm
3-5 years production support engineering experience; comfortable with logs, MongoDB, SQL; ability to write TypeScript or Python fixes; web/network fundamentals; Jira and helpdesk tool experience; on-call rotation; background check.