Fieldwork - UX research agent

Scaling Human Insights: Building Fieldwork with Gemini Live 🧭🎙️

Solving the UX Research Bottleneck with 24/7 AI-Native Moderation

UX research is the heartbeat of product development, but it has a scaling problem. High-quality moderated research traditionally requires hundreds of manual hours for guide creation, participant scheduling, and live moderation. This bottleneck often forces teams to choose between speed and depth.

Fieldwork was built to eliminate that choice. By leveraging the power of Gemini 2.5 and Google Cloud, Fieldwork provides an AI-native collaborator that helps you design research and a 24/7 moderator that conducts natural, voice-first interviews with participants anywhere in the world.

How Fieldwork Works

Fieldwork isn't just a survey tool; it's a bi-directional research partner.

1. Guided Collaboration: Use Gemini 2.5 Flash to transform your research goals and target audience into a professional, tailored interview guide in seconds.

2. Multimodal Interaction: Deploy a research agent that "hears," "speaks," and "understands" nuance. Participants engage in a natural conversation powered by the Vertex AI Multimodal Live API.

3. Real-Time Adaptability: The agent doesn't just read questions; it probes deeper, identifies overlapping content to smartly skip redundant questions, and handles multimodal inputs like multiple-choice selections when precision is needed.

4. Instant Synthesis: Transcripts are captured in real-time, and video recordings are securely persisted to Google Cloud Storage, allowing for instant AI-powered synthesis of insights.

The Tech Stack: Powered by Google AI & Cloud

Fieldwork is built as a cloud-native application designed for high performance and secure scaling:

1. The Brain: Gemini 2.5 Flash & Gemini Live

We chose Gemini 2.5 Flash for its exceptional speed and reasoning.

• Real-time Voice: We utilize the gemini-live-2.5-flash-native-audio model. This allows for zero-latency, bi-directional voice interactions that feel like talking to a human researcher.

• Node.js SDK: Built using the Google Cloud Vertex AI SDK (@google-cloud/vertexai), we leverage the WebSocket-based Multimodal Live API to stream audio chunks (16kHz in / 24kHz out) for the most natural experience.

2. The Infrastructure: Google Cloud Platform (GCP)

• Cloud Run: Fieldwork is hosted on Cloud Run, providing a scalable, containerized environment. We use Application Default Credentials (ADC) and IAM service accounts to manage secure, keyless access to Vertex AI.

• Vertex AI Token Bridge: To maintain security, we built a secure backend bridge that provides the frontend with short-lived OAuth2 tokens for direct, low-latency WebSocket connections to Vertex AI.

• Cloud Storage (GCS): Participant videos and large-scale interview data are persisted to GCS buckets using signed URLs, ensuring data is both secure and highly available for downstream analysis.

The Future of Research

Fieldwork demonstrates that AI-native agents can participate in the research process not as a replacement for human researchers, but as a force multiplier. By handling the logistics and moderation of discovery studies at scale, Fieldwork allows researchers to focus on what matters most: deciphering the "Why" behind human behavior.

Note: I created this piece of content for the purposes of entering the Gemini Live Agent Challenge hackathon.
#GeminiLiveAgentChallenge #GoogleCloud #GeminiAI #UXResearch