How LangExtract Uses URLs and Text Lists for Data Extraction with LLMs
When I first saw a tool that could grab text straight from a URL, I wondered if it would matter to anyone fiddling with large language models. Most open-source projects still want you to hand them raw strings or files you’ve scraped yourself, so there’s a bit of a gap. LangExtract seems to try to plug that hole by letting you feed one of three things: a single text block, a list of snippets, or even a web address - think Gutenberg or any other site that serves plain text.
After the content lands, you write a plain-language prompt that tells the model what to pull out, and you can throw in a few example pairs to show the desired format. Those three pieces - source, instruction, examples - make up the core of the library. Getting a feel for them is probably the first thing you’ll want to do before deciding if LangExtract fits your workflow, and the quote below spells them out in plain terms.
The key arguments are: text_or_documents: your input text, a list of texts, or a URL string (LangExtract can fetch and process text from Gutenberg or another URL). prompt_description: the extraction instructions as a simple string. examples: a list of ExampleData that demonstrate the expected output.
The key arguments are: text_or_documents : Your input text, or a list of texts, or even a URL string (LangExtract can fetch and process text from a Gutenberg or other URL).prompt_description : The extraction instructions (a string).examples : A list ofExampleData that illustrate the desired output.model_id : The identifier of the LLM to use (e.g."gemini-2.5-flash" for Google Gemini Flash, or an Ollama model like"gemma2:2b" , or an OpenAI model like"gpt-4o" ).- Other optional parameters: extraction_passes (to re-run extraction for higher recall on long texts),max_workers (to do parallel processing on chunks),fence_output ,use_schema_constraints , etc.
LangExtract gives you a simple way to pull structured data out of messy text. You can hand it a raw string, a list of documents, or even a URL, it will fetch the page, even from places like Project Gutenberg, without you writing extra code. The workflow is basically: write a short prompt that tells the model what you want, add a few example rows that show the output shape, and let the LLM run over the input.
The result comes back in the same format you demonstrated. Because the code is open source, anyone can peek under the hood or tweak it for a very specific job, which is nice for folks who don’t trust opaque services. That said, the whole thing leans on the model’s ability to follow the prompt exactly; vague wording often produces spotty extracts.
I’m not sure how well it handles huge corpora or highly technical papers - performance might drop off. For someone just trying to get a quick, flexible extraction tool, LangExtract looks promising, but its sturdiness in heavy-duty use still needs testing.
More in Open Source
Anthropic CEO Dario Amodei calls AGI a marketing term, echoing Altman's view
The AI community has grown weary of a word that shows up on every press release and investor deck.
Xiaomi launches MiMo‑V2‑Flash AI model: 150 t/s, USD 0.1‑USD 0.3 per million tokens
Xiaomi’s latest foray into generative AI arrives as an open‑source model named MiMo‑V2‑Flash, a...
OpenAI signs USD 10 bn deal to use Amazon Trainium3 chips, 4.4× faster compute
OpenAI has just inked a ten‑billion‑dollar agreement to run its models on Amazon’s newest Trainium3...
Zepto Cafe uses MCP to parse text orders, Playwright runs the clicks
Zepto Cafe has taken a modest step toward turning plain‑text requests into a full‑fledged ordering...
Larian CEO denies AI cuts, says generative AI hasn't boosted efficiency
Why does the Larian story matter now? The studio’s founder‑CEO, Swen Vincke, has been under fire...
Common Questions Answered
What types of inputs can LangExtract accept?
LangExtract can accept a single block of raw text, a list of text snippets, or a URL string that points to an online source such as Project Gutenberg. This flexibility lets users feed unstructured data directly without pre‑scraping.
How does LangExtract process a URL that points to a Gutenberg source?
When a Gutenberg URL is provided, LangExtract fetches the web page, extracts the textual content, and passes it to the selected LLM for extraction. The tool handles the fetching and cleaning internally, removing the need for separate plumbing.
What is the purpose of the 'prompt_description' parameter in LangExtract?
The 'prompt_description' supplies plain‑language instructions that tell the LLM what information to extract from the input text. By describing the desired output format, it guides the model to produce structured data that matches the examples.
Which model identifiers can be used with LangExtract and how are they specified?
LangExtract supports identifiers for various LLM providers, such as "gemini-2.5-flash" for Google Gemini Flash, "gemma2:2b" for an Ollama model, or "gpt-4o" for an OpenAI model. The identifier is passed via the 'model_id' field in the request payload.