Editing Portal:Artificial Intelligence (section)

=== Local models, Ollama, vLLM, and OpenAI-compatible APIs ===

Modern AI work increasingly includes local or self-hosted models. Perl can participate through HTTP-based interfaces to model servers such as Ollama, LM Studio, llama.cpp servers, SGLang, and vLLM. These systems often expose OpenAI-compatible or provider-compatible APIs, which means Perl can use ordinary HTTP and JSON libraries or higher-level CPAN modules.

Langertha’s Ollama engine documentation describes Ollama as a way to run large language models locally and notes support for chat, embeddings, streaming, MCP tool calling, and an OpenAI-compatible API.<ref>MetaCPAN, “Langertha::Engine::Ollama,” https://metacpan.org/pod/Langertha::Engine::Ollama, accessed July 3, 2026.</ref> Langertha’s vLLM engine documentation describes vLLM as a high-throughput inference engine for large language models that exposes an OpenAI-compatible API.<ref>MetaCPAN, “Langertha::Engine::vLLM,” https://metacpan.org/pod/Langertha::Engine::vLLM, accessed July 3, 2026.</ref>

This matters for Perl because local model support gives Perl programs an AI path that does not always require sending data to a commercial API. For organizations with privacy, cost, latency, or offline requirements, a Perl script or service can call a model running on local hardware and integrate the result into existing Perl workflows.