Editorial illustration for POLAR builds multimodal knowledge graph for semantic and episodic memory
POLAR builds multimodal knowledge graph for semantic and...
A robot that genuinely knows you is still science fiction. But the first step toward that isn't more raw compute, it's a functional memory.
Most multimodal AI agents are goldfish. They see your coffee mug, you tell them it's yours, and by the next interaction it's a forgotten object. POLAR is an attempt to give these models a scrapbook.
It builds a knowledge graph from past interactions, stitching together two kinds of memory. Semantic memory holds static facts about your world, like your favorite mug or your dog's name. Episodic memory logs embodied experiences, like the robot's own path through the kitchen to find that mug last Tuesday.
POLAR organizes prior interactions into a multimodal knowledge graph that captures semantic memory for personalized context and visual concepts, and episodic memory for embodied experiences such as agent trajectories.
The results are specific and telling. When an agent has to connect information from several past chats, or make a logical jump based on your history, POLAR's structured memory provides a clear edge. It turns a series of isolated commands into a context.
This isn't about making a slightly smarter single reply. It's about building a thread. The promise is an agent whose understanding doesn't reset when a conversation ends, but accumulates.
Slowly, awkwardly, like any real relationship.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives MEMO trains a memory model on new knowledge with two roles, no LLM changes
From Archives GEM framework casts LLM data curation as hyperspherical variational problem
Common Questions Answered
What problem does POLAR solve for multimodal AI agents?
POLAR addresses the memory limitation of most multimodal AI agents, which forget information between interactions like a goldfish. The system builds a knowledge graph from past interactions to create persistent memory, allowing agents to retain and reference information across multiple conversations rather than starting fresh each time.
How does POLAR combine semantic and episodic memory?
POLAR stitches together two types of memory: semantic memory that holds static facts about your world, and episodic memory that captures specific events and interactions. This dual-memory approach enables the AI agent to both understand general information about you and recall specific past experiences.
What advantage does POLAR's structured memory provide over traditional AI agents?
POLAR's structured memory gives agents a clear edge when they need to connect information from multiple past conversations or make logical inferences based on user history. Rather than treating each command as isolated, POLAR turns a series of interactions into meaningful context that accumulates over time, similar to how real relationships develop.
How does POLAR change the way AI agents understand user interactions?
Instead of resetting an agent's understanding when a conversation ends, POLAR enables continuous accumulation of knowledge about the user across sessions. This transforms the agent from providing slightly smarter single replies into building an ongoing thread of understanding that persists and grows with each interaction.
Further Reading
- AriGraph: Learning Knowledge Graph World Models with Episodic Memory — arXiv
- AriGraph: Learning Knowledge Graph World Models with Episodic Memory — IJCAI 2025
- Time-Aware Knowledge Graphs for Episodic Memory — Substack
- Episodic Memory Graph | GraphRAG — GraphRAG
- Learning through experience: Episodic memory representation for multimodal agents — OpenReview
