Director - Cloud Platforms & Agent Infrastructure

Role Summary

The Senior Director, Cloud Platforms & Agent Infrastructure will lead technical architecture and platform engineering for Data Foundry's cloud-native infrastructure, building scalable, secure, and agent-ready systems that enable both human scientists and autonomous AI agents to accelerate molecule discovery. Reporting to the Head of Architecture4Insight, this role designs and operates the computational infrastructure, APIs, and workflow automation that support real-time data access, analytic method deployment, and agent-driven experimentation. The role also provides production-ready platforms with governance and metadata standards integrated across Data Foundry pillars. Location: San Diego, CA; San Francisco, CA; Louisville, CO; Indianapolis, IN; Boston, MA.

Responsibilities

• Lead the design and operation of Data Foundry’s cloud-native infrastructure (AWS, Azure, and/or GCP), including infrastructure-as-code, containerization, orchestration, and self-service platforms that empower discovery scientists to deploy tools and run analyses at scale.
• Partner closely with the Frontier AI group to build agent-ready infrastructure—including APIs with structured error handling, versioning, and audit trails; Model Context Protocol (MCP) servers; and real-time data streaming architectures—that enable autonomous AI agents to access data, invoke analytical methods, and control laboratory systems.
• Ensure Architecture4Insight serves as the enabling technical foundation across all Data Foundry pillars: deploying Methods4Insight’s analytical models as production APIs, integrating Scale4Insight’s real-time instrument data streams, and embedding Preparedness4Insight’s governance and metadata standards into platform design.
• Design and implement workflow orchestration systems (Prefect, Airflow, Nextflow, WDL) that automate data pipelines, enable closed-loop experimentation, and provide reusable workflow templates for scientists.
• Build data access and integration layers providing unified interfaces to diverse data sources (LIMS, ELNs, instruments, databases), with caching, indexing, and query optimization for interactive analytics and agent queries.
• Lead platform engineering to build shared services, CI/CD automation, API gateways, and microservices architectures used across Data Foundry, establishing software engineering best practices and modern development standards.
• Collaborate with Tech@Lilly Product Engineering to define pathways for transitioning prototypes to enterprise-managed production systems, establish SLAs, and represent Data Foundry in enterprise architecture forums.
• Build, mentor, and help develop a high-performing platform engineering team of architects, software engineers, and DevOps specialists, fostering a culture of technical excellence and continuous learning.
• Develop a multi-year roadmap for cloud platforms and infrastructure aligned with Data Foundry strategy, monitoring emerging technologies and establishing KPIs for platform performance, cost efficiency, and scientist satisfaction.

Qualifications

• Required: MS/Ph.D. in Computer Science, Bioinformatics, Computational Biology, or related field, OR MS with equivalent experience in platform engineering and scientific software.
• Required: 6+ years of experience in cloud platform architecture, scientific software engineering, or research informatics, with at least 3 years in pharmaceutical, biotechnology, or life sciences industry.
• Preferred: Proven track record leading platform engineering or infrastructure teams (6–10+ professionals) delivering production systems.
• Preferred: Deep expertise in cloud-native architecture on AWS, Azure, or GCP including infrastructure-as-code, containerization, orchestration, and security.
• Preferred: Strong software engineering skills with experience in multiple programming languages and modern development practices.
• Preferred: Experience building APIs, microservices, and integration layers for complex scientific or technical systems.
• Preferred: Demonstrated ability to translate scientific requirements into technical architecture and scalable platform solutions.
• Preferred: Excellent communication skills with ability to explain technical concepts to scientists, collaborate with IT organizations, and influence stakeholders.
• Preferred: Experience with AI agent infrastructure, autonomous system platforms, or building APIs that AI/ML systems invoke programmatically.
• Preferred: Hands-on experience with Model Context Protocol (MCP) servers, LangChain, or similar frameworks for AI agent tool integration.
• Preferred: Deep understanding of workflow orchestration tools (Nextflow, Prefect, Airflow, WDL, Snakemake) and experience building automated scientific pipelines.
• Preferred: Experience with genomics, proteomics, or bioinformatics platforms and understanding of scientific data characteristics.
• Preferred: Familiarity with LIMS, ELN systems, and laboratory automation data integration patterns.
• Preferred: Track record of founding or building platforms from ground up, not just maintaining existing systems.
• Preferred: Experience with data governance, semantic modeling, ontologies, and metadata frameworks in scientific contexts.
• Preferred: Strong mentorship capabilities with passion for developing engineering talent and building collaborative teams.
• Preferred: Entrepreneurial mindset with ability to operate in ambiguous, fast-paced research environments.

Additional Requirements

• Physical Demands: The physical demands of this job are consistent with an office environment.
• Travel: Travel (~15–20%) to collaborate with teams across multiple Lilly sites and occasional external conferences.