Editorial illustration for Remote uses LangChain and LangGraph to AI-onboard thousands, notes token bloat
Remote uses LangChain and LangGraph to AI-onboard thousands, notes token bloat
Remote’s engineering team set out to automate the onboarding of thousands of new clients, stitching together LangChain and the newer LangGraph framework to power a conversational assistant that can navigate payroll, tax forms and local labor rules. The goal was clear: shave weeks off a process that traditionally required legal review and manual data entry. Early tests showed the workflow could pull data from HRIS systems, query tax APIs and generate compliance-checked contracts—all without a human typing a single line.
Yet the prototype quickly ran into a hidden cost. Every time the AI-driven agent reached out to an external tool, the response was routed back through the language model, inflating the prompt with layers of intermediate output. Those extra tokens accumulated faster than the team anticipated, threatening to push the request beyond the model’s context window.
When a platform whose reputation hinges on precision and regulatory adherence hits that ceiling, the stakes rise sharply.
As Anthropic engineers have pointed out, when agents call tools directly, every intermediate result flows through the model, which can add tens of thousands of tokens per call and even exceed the context limit. For a global employment platform like Remote, where accuracy and compliance are non-negoti…
As Anthropic engineers have pointed out, when agents call tools directly, every intermediate result flows through the model, which can add tens of thousands of tokens per call and even exceed the context limit. For a global employment platform like Remote, where accuracy and compliance are non-negotiable, these constraints made it clear that a different approach to large-scale data migrations was necessary. The Solution: Let the models reason, let code execute Remote's Code Execution Agent separates the "thinking" from the "doing." Instead of forcing the LLM to ingest all the data, it uses LangChain's tool-calling interface to decide what steps to take, then writes and runs real Python code to transform the data. Anthropic's research on code execution shows why this hybrid design works: by letting agents run code in a sandbox, tool definitions and intermediate results stay outside the context window.
Can AI truly replace manual onboarding? Remote thinks so, at least for now. By wiring LangChain and LangGraph into its pipeline, the company has automated the ingestion of thousands of spreadsheets and SQL dumps, turning raw payroll data into structured records without human intervention.
Yet, as Anthropic engineers warned, each tool call pushes intermediate results back through the language model, inflating token usage by tens of thousands per request. In practice, those extra tokens sometimes breach the model’s context window, forcing truncation or additional calls. Because Remote’s business hinges on precise compliance, any loss of detail could be costly.
The team acknowledges the trade-off, but it's unclear whether the current architecture can sustain growth without further token-management strategies. Future iterations may need to batch calls or prune intermediate data, but the article stops short of outlining concrete solutions. For now, the experiment demonstrates both promise and limitation, highlighting the delicate balance between automation speed and the strict accuracy demands of global payroll.
Further Reading
- Papers with Code - Latest NLP Research - Papers with Code
- Hugging Face Daily Papers - Hugging Face
- ArXiv CS.CL (Computation and Language) - ArXiv
Related Reading
Open SourceUK PM vows action on Grok's deepfake scandal, Starmer condemns X
Open SourceGPT-5 helps mathematicians offload tedious tasks, says Timothy Gowers
Open SourceIndia proposes licensing and royalty rules for AI training by Google, OpenAI
From ArchivesMistral Wins Rs 100 Cr Radar Subsystem Order for Low-Altitude UAV Surveillance
From Archives