Events Lead

Early engineer (#2) for the AI team at Couchbase, building foundational infrastructure for supporting AI workloads. • Implemented high-throughput, multi-modal batch processing offering for supporting structured and unstructured data extraction with in-flight vectorization. • Developed a unified mount point over fsspec for supporting data ingestion for remote stores such as S3, GCS, Azure and more. • Fine-tuned Phi 2 and Llama 3 for query rewrites, achieving a 11% cost improvement per query. • Achieved 34% median improvement in recall metrics by introducing RRF, rank aggregation and statistical post-filtering for vector workloads. • Re-architected Airflow based infrastructure to AWS step-functions based serverless deployment, unlocking usage-based scaling and fine grained metering.

Software Engineer

With over 75k downloads, Wizdom is amongst the Top 10 Grossing Educational Apps in India and in the Top 100 in US, UK and Australia.
• Conceptualized pivot, gathered functional requirements and drafted the product roadmap for the MVP
• Conducted market research, experimented with marketplace seeding and refined the GTM based on ICP.
• Oversaw the MVP Pilot and outreach campaign and assisted the founding team to onboard users post pivot.
• Addressed issues pertaining to media distribution rights and reduced product compliance overhead.
• Assisted the founding team during fundraise and explored a plausible exit, prior to pivot.
• Refined the deck and helped the company prepare to secure a seed round post-pivot.

Founder's Office

Product and Investor Relations

- Implemented a multi-modal batch processing for Unstructured Data Service with in-flight vectorization
- Developed a unified mount point to support ingestion from 6+ remote object stores including S3, GCS, etc.
- Fine-tuned Phi-2 and Llama-3 for query rewrites, achieving upto 11% cost improvement per query
- Built ColQwen2 based retriever, delivering 0.94 nDCG, a 34% recall improvement over baselines
- Migrated Airflow to AWS Step Functions enabling usage-based metering for batch processing workloads
- Extended Couchbase Analytics Service to support analytical queries over unstructured data using vision language models

Software Engineer, R&D Core

- Designed & Implemented public and internal control-plane APIs for AI agent workloads in Capella (CQI)
- Enabled support for privately hosted models and third-party providers such as AWS Bedrock and Azure OpenAI for Capella-hosted agents
- Built the core networking infrastructure for CQI agents, enabling secure and fault-tolerant connectivity to Couchbase clusters
- Implemented secrets management for AI agent workloads, handling storage, rotation, and management of credentials used across agents

Software Engineer II

- Led development of AI Gateway, unifying serving and governance for public and privately hosted models
- Migrated serving infra from vLLM to NVIDIA NIM, adding support for 28+ models via model catalog
- Implemented context-aware semantic caching with RBACs, reducing median TAT by 21% for near duplicate queries
- Implemented 7 routing policies, reduced TTFT by 58% by preserving KV cache locality during inference
- Reduced pre-fill latency with KV cache reuse, added support for larger context windows via KV cache offloading
- Implemented policy-based moderation and added safe-guards against jailbreaks and prompt-injections
- Introduced distributed tracing and added support for fallbacks and failovers during model inference
- Led E2E technical onboarding of 3 engineers - 2 SDE2s and 1 Lead SWE to AI data-plane team

Software Engineer, Model Service

• Led development of AI Gateway as core IC, delivering a common interface to public and private model deployments for key enterprise customers. • Introduced support for autoscaling, fallbacks, and failovers ensuring high-availability across distributed model replicas. • Led end to end migration from vLLM to NVIDIA NIM, extending support to over 17 models via Model Catalog. • Added over 7 routing policies such as KV-cache-aware, prefix-aware routing and more, reducing TTFT by 58% • Implemented context-aware semantic caching resulting in 21% reduction in median turn around time for near-duplicate inference requests. • Designed fault-tolerant batch processing with durable execution, continuous batching and checkpointing, ensuring no starvation of real-time requests. • Migrated secret management to fine-grained, user-scoped tokens, retiring legacy Couchbase cluster credentials. • Introduced support for moderation and policy based guardrails, with safe-guard against jailbreaks and prompt-in

Building Userplane.io | Loom for Customer Support

github

Know about Nayan Kumar's work experience, education, bootcamps, certifications, and much more. Nayan Kumar