Sr. Staff Software Engineer - AI Chat
Eli Lilly and Company
4 months ago
Remote friendly (Indianapolis, IN)
United States
IT
What You'll Do
- Define the architecture and drive implementation of ChatNow’s conversation routing engine (connects user intent to the right AI agent, model, or workflow).
- Own the agent integration layer on Cortex (design how ChatNow connects to specialized AI agents).
- Build streaming chat interfaces in React and TypeScript with scalable, low-latency services handling thousands of concurrent sessions across multiple AI providers.
- Own features end to end (design, build, test, deploy, monitor) with no hand-offs; operate ChatNow on Cortex and Hangar with LGTM observability.
- Integrate LLM capabilities via Cortex’s model gateway (multi-model routing with fallback and load balancing).
- Implement MCP integrations connecting agents to enterprise data sources and tools.
- Shape how agents are registered, discovered, and invoked on Cortex; make technical decisions across teams; mentor engineers; raise the bar for production AI software.
Basic Qualifications
- 10+ years of professional software engineering experience with demonstrated technical leadership at scale.
- 5+ years in TypeScript/JavaScript, Python, and Go; ability to use AI coding tools proficiently; architecting/building production web apps with React (or equivalent).
- 5+ years designing RESTful or GraphQL APIs at scale.
- Deep hands-on Kubernetes experience (operating, debugging, tuning) plus strong CI/CD and GitOps practices.
- Bachelor’s degree in Computer Science, Software Engineering, or related field.
Preferred Qualifications
- Experience integrating LLM APIs into production (streaming, function calling, tool use).
- Hands-on conversational AI experience (routing, intent classification, context management, or multi-agent orchestration).
- Real-time systems experience (WebSockets, SSE, event-driven architectures, or workflow orchestration).
- Familiarity with MCP, A2A, or emerging AI interoperability protocols; active use of agentic coding tools.
- Track record shipping user-facing products with high reliability and polish.
- Experience setting technical direction across multiple teams/workstreams.
Benefits (if part of job description)
- Company bonus for full-time equivalent employees (dependent on company/individual performance).
- Comprehensive benefits: 401(k), pension, vacation, medical/dental/vision/prescription, flexible benefits (including healthcare/dependent day care FSAs), life insurance/death benefits, certain time off/leave, and well-being benefits (employee assistance, fitness, employee clubs/activities).
Application instructions
- If you require accommodation to submit a resume, complete the workplace accommodation request form: https://careers.lilly.com/us/en/workplace-accommodation
- Define the architecture and drive implementation of ChatNow’s conversation routing engine (connects user intent to the right AI agent, model, or workflow).
- Own the agent integration layer on Cortex (design how ChatNow connects to specialized AI agents).
- Build streaming chat interfaces in React and TypeScript with scalable, low-latency services handling thousands of concurrent sessions across multiple AI providers.
- Own features end to end (design, build, test, deploy, monitor) with no hand-offs; operate ChatNow on Cortex and Hangar with LGTM observability.
- Integrate LLM capabilities via Cortex’s model gateway (multi-model routing with fallback and load balancing).
- Implement MCP integrations connecting agents to enterprise data sources and tools.
- Shape how agents are registered, discovered, and invoked on Cortex; make technical decisions across teams; mentor engineers; raise the bar for production AI software.
Basic Qualifications
- 10+ years of professional software engineering experience with demonstrated technical leadership at scale.
- 5+ years in TypeScript/JavaScript, Python, and Go; ability to use AI coding tools proficiently; architecting/building production web apps with React (or equivalent).
- 5+ years designing RESTful or GraphQL APIs at scale.
- Deep hands-on Kubernetes experience (operating, debugging, tuning) plus strong CI/CD and GitOps practices.
- Bachelor’s degree in Computer Science, Software Engineering, or related field.
Preferred Qualifications
- Experience integrating LLM APIs into production (streaming, function calling, tool use).
- Hands-on conversational AI experience (routing, intent classification, context management, or multi-agent orchestration).
- Real-time systems experience (WebSockets, SSE, event-driven architectures, or workflow orchestration).
- Familiarity with MCP, A2A, or emerging AI interoperability protocols; active use of agentic coding tools.
- Track record shipping user-facing products with high reliability and polish.
- Experience setting technical direction across multiple teams/workstreams.
Benefits (if part of job description)
- Company bonus for full-time equivalent employees (dependent on company/individual performance).
- Comprehensive benefits: 401(k), pension, vacation, medical/dental/vision/prescription, flexible benefits (including healthcare/dependent day care FSAs), life insurance/death benefits, certain time off/leave, and well-being benefits (employee assistance, fitness, employee clubs/activities).
Application instructions
- If you require accommodation to submit a resume, complete the workplace accommodation request form: https://careers.lilly.com/us/en/workplace-accommodation