Senior Manager, Intelligence Systems Lab
Bristol Myers Squibb
2 months ago
Remote friendly (San Diego, CA)
United States
IT
Description
- Develop agentic AI systems using rigorous evaluation science; implement reasoning, planning, memory, and generalization capabilities; build benchmarking methodologies to assess agent performance. Support cross-functional stakeholders in Commercial, Manufacturing, Clinical Development, and Research.
Responsibilities
- Research reasoning/planning/memory/generalization architectures for AI and multi-agent systems; build cognition frameworks (abstraction, goal decomposition, adaptive planning).
- Design and run benchmarking/evaluation experiments; create evaluation datasets, test harnesses, and measurement frameworks; apply experimental design and statistical analysis.
- Engineer and deploy agentic solutions: multi-agent systems (e.g., Strands, LangGraph, DSPy), RAG applications, knowledge graphs, conversational AI, and autonomous workflows.
- Rapid prototype and translate research into production-ready capabilities; document research and technical methods.
- Collaborate cross-functionally; translate AI research into business language; mentor and support best practices in evaluation and research methodology.
Qualifications
- BA/BS required (quantitative: CS, Data Science, Statistics, Math, Cognitive Science, or Engineering). MS preferred.
- 3+ years hands-on experience in data science/ML/AI development or research; pharma/life sciences preferred.
- Python; ML frameworks (Scikit-Learn, TensorFlow, PyTorch); agentic frameworks (Strands, LangGraph, AutoGen, DSPy, CrewAI); LLMs/prompting/RAG; benchmarking & evaluation methods.
- SQL; cloud (AWS/Azure/GCP); Git; experimental design/statistical analysis preferred.
Benefits (explicitly stated)
- Health coverage; wellbeing support; 401(k) and protection benefits; paid time off (including flexible time off and/or vacation depending on location).
- Develop agentic AI systems using rigorous evaluation science; implement reasoning, planning, memory, and generalization capabilities; build benchmarking methodologies to assess agent performance. Support cross-functional stakeholders in Commercial, Manufacturing, Clinical Development, and Research.
Responsibilities
- Research reasoning/planning/memory/generalization architectures for AI and multi-agent systems; build cognition frameworks (abstraction, goal decomposition, adaptive planning).
- Design and run benchmarking/evaluation experiments; create evaluation datasets, test harnesses, and measurement frameworks; apply experimental design and statistical analysis.
- Engineer and deploy agentic solutions: multi-agent systems (e.g., Strands, LangGraph, DSPy), RAG applications, knowledge graphs, conversational AI, and autonomous workflows.
- Rapid prototype and translate research into production-ready capabilities; document research and technical methods.
- Collaborate cross-functionally; translate AI research into business language; mentor and support best practices in evaluation and research methodology.
Qualifications
- BA/BS required (quantitative: CS, Data Science, Statistics, Math, Cognitive Science, or Engineering). MS preferred.
- 3+ years hands-on experience in data science/ML/AI development or research; pharma/life sciences preferred.
- Python; ML frameworks (Scikit-Learn, TensorFlow, PyTorch); agentic frameworks (Strands, LangGraph, AutoGen, DSPy, CrewAI); LLMs/prompting/RAG; benchmarking & evaluation methods.
- SQL; cloud (AWS/Azure/GCP); Git; experimental design/statistical analysis preferred.
Benefits (explicitly stated)
- Health coverage; wellbeing support; 401(k) and protection benefits; paid time off (including flexible time off and/or vacation depending on location).