Jan: Open-Source Local AI Desktop App and API

What Jan is

Jan is an open-source desktop application for running large language models on your own computer. Its README calls it an open-source ChatGPT replacement and says the goal is to bring the best of open-source AI into an easy-to-use product, so that you can download and run LLMs with full control and privacy. The current documentation positions it more broadly, as a replacement for both Claude and ChatGPT that comes with its own foundation models and an application ecosystem running entirely on your hardware.

In practice Jan is three things in one install. It is a chat app with a model hub, conversation threads, projects and custom assistants. It is a local inference runtime built on llama.cpp, with an experimental MLX engine on Apple Silicon. And it is a local, OpenAI-compatible API server plus a command-line tool, so other programs, including coding agents, can use the models you have downloaded. Cloud providers can be added next to local models, and the app lets you switch between them per conversation.

The project is maintained by Menlo Research, built on the Tauri desktop framework with Rust as its primary language on GitHub, and licensed under Apache-2.0. The LICENSE file adds a request, not a condition, that attribution appear in user-facing documentation where appropriate. Jan is aimed at developers and privacy-conscious users who want a polished front end for local models, teams that need an offline-capable assistant, and anyone who wants a local endpoint that existing OpenAI-style clients can talk to.

How it works

The desktop app and its data folder

Jan is a Tauri application: a native shell written in Rust hosting a web front end. Everything it produces lives in a local data folder: downloaded models, conversation threads, settings and logs. The installation guides give the default locations as ~/Library/Application Support/Jan/data on macOS and ~/.local/share/Jan/data on Linux (resolved through XDG_DATA_HOME), with the per-user configuration that records the data folder's location under ~/.config/Jan/. The desktop app and the CLI share this folder, so a model downloaded through either is available to both.

Local inference engines

Local models run through llama.cpp, which Jan acknowledges in its README alongside Tauri and Scalar. They use the GGUF format and consume your machine's RAM, VRAM and CPU, so the docs advise choosing models that match the hardware guidance for your platform. Since v0.8.5, according to the engine documentation, llama.cpp is compiled into Jan and runs in a supervised worker process called jan-llama-worker. There is no separate llama-server to download and no backend to pick: the engine updates with the app, loads and unloads models on demand, and restarts if it stops unexpectedly when you send the next message. Each installer ships the GPU backends for its platform; on Windows and Linux x64 that means NVIDIA through CUDA 13 and AMD, Intel Arc and NVIDIA through Vulkan, with a setting to choose which API to use when a GPU is reachable through both.

The bundled engine cannot be swapped. If you need a llama.cpp version or backend that Jan does not ship, the documented route is to run your own llama-server and add it as a custom endpoint. On Apple Silicon there is also an MLX engine, which the docs label experimental: it needs macOS 14 or later, cannot do embeddings yet, does not surface reasoning output separately, and fails to load some newer architectures, in which case the GGUF version is the suggested fallback.

The API server and the CLI

Inside the app, a local API server exposes loaded models through an OpenAI-compatible interface, by default at http://127.0.0.1:1337/v1. The jan CLI, available since Jan 0.7.8 in March 2026, is separate: jan serve loads a model and exposes it at localhost:6767/v1, and jan launch starts a model and then launches an agent such as Claude Code or OpenClaw with the environment variables already pointing at it. The CLI documentation notes that since Jan 0.8.0, llama.cpp serving runs in a router mode under a single process, and per-model settings come from a preset file the desktop app generates.

Key features

• Model hub with fit hints. The Hub, backed by Hugging Face, marks each model as Fits, May be slow or Won't fit for your hardware and groups quantisations as Small, Balanced or Large. The docs say no data is downloaded to work out fit.
• Hugging Face import. Paste a model ID into the Hub search, use "Use this model" then Jan on a Hugging Face model page, or import local GGUF files. Gated models need a Hugging Face token in settings.
• Built-in foundation models. Jan downloads a default model on first launch and offers its own models for coding, general instruction following and vision-language tasks.
• Cloud providers. OpenAI, Anthropic, Google, Groq, Mistral, OpenRouter and others can be added with an API key, plus any OpenAI- or Anthropic-compatible custom endpoint.
• OpenAI-compatible local server. Configurable host, port, API prefix, API key, trusted hosts, request timeout, CORS and verbose logging, with an option to execute MCP tools on the server side.
• Model Context Protocol. Jan acts as an MCP host, with per-call approval shown inline in the chat and an optional smart routing mode that loads tools from only the relevant servers each turn.
• Web search and fetch. The quickstart says built-in tools can search the web and fetch pages without extra setup, with the option of an Exa MCP server instead.
• Projects, assistants and Cowork. Projects group conversations with shared instructions and files; assistants are custom set-ups tailored to how you work; Cowork, marked as a preview, works through a task on its own in a sandbox or on your project.
• CLI for agents. jan launch wires Claude Code or OpenClaw to a local model, and jan threads lists, reads and deletes conversations saved by the desktop app.

Getting started

The quickest route is a desktop installer. The README links builds for Windows, a universal macOS DMG, a Debian package and an AppImage for Linux, plus Microsoft Store and Flathub listings. On Linux, the documentation recommends Flatpak because it runs on any distribution and updates through Flathub:

flatpak install flathub ai.jan.Jan
flatpak run ai.jan.Jan

If the Flathub remote is not set up yet:

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo

After installation, the quickstart describes a short path: Jan downloads its default foundation model on first launch, and once that finishes you can start chatting. More models come from the Hub in the left sidebar, and cloud providers are added under Settings, Model Providers.

To use Jan from other programs, open Settings, Local API Server and start the server. The logs show JAN API listening at http://127.0.0.1:1337 when it is ready, and the documentation's test request looks like this, with your own model ID and whatever API key you configured:

curl http://127.0.0.1:1337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer secret-key-123" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "messages": [{"role": "user", "content": "Tell me a joke."}]
  }'

The CLI is installed automatically the first time you launch the desktop app, into /usr/local/bin/jan when that directory is writable and ~/.local/bin/jan otherwise, so check that the directory is on your PATH. From there, a single command picks an agent, sets up a model and connects the two:

jan launch
jan launch claude --model janhq/Jan-code-4b-gguf
jan serve janhq/Jan-code-4b-gguf --detach

Building from source needs Node.js 20 or later, Yarn 4.5.3 or later, Make and a Rust toolchain for Tauri; on Apple Silicon you also need Apple's Metal toolchain. The README's one-step path is:

git clone https://github.com/janhq/jan
cd jan
make dev

On Windows the README says to run make dev from Git Bash, with the Visual Studio 2022 Build Tools, LLVM, Ninja and CMake on the PATH, and the CUDA Toolkit only for CUDA engine variants selected through JAN_ENGINE_VARIANT.

Use cases

A private desktop assistant

The default experience is a chat app whose models run locally, with no account and no API key. For drafting, summarising or coding help on material that should not leave the machine, that is the main draw, and the Hub's fit indicators help pick a model the hardware can actually run.

A local endpoint for existing tools

Because the server speaks the OpenAI chat completions format, scripts and applications written against that API can point at Jan instead by changing the base URL. The docs describe it as a drop-in replacement for cloud APIs when building private, offline-capable applications, and the server can be exposed to other devices on the LAN once an API key is set.

Running coding agents on local models

The CLI's launch command exists for this: it starts a model and then runs Claude Code or OpenClaw with the API base URL and key exported for it. The documentation also has integration guides for Claude Code, OpenClaw and MCP servers. How well this works depends on the model; the MCP documentation warns that not every model is good at tool calling.

Mixing local and frontier models

Teams that sometimes need a large hosted model can add providers next to local models and choose per conversation, keeping sensitive work on local models and sending other tasks to a cloud provider.

How it compares

Jan sits in the same space as other desktop front ends for local models, and the closest technical relative is llama.cpp itself, which Jan uses as its engine. Running llama-server directly gives you full control over the build, flags and backend, while Jan trades some of that control for a packaged engine, a model hub, a chat interface, MCP support and provider management. The documentation is explicit that the bundled engine cannot be replaced, and that anyone who needs a specific llama.cpp build should run it themselves and connect it as a custom endpoint, which is a reasonable division of labour.

Against hosted assistants such as ChatGPT or Claude, which Jan names as the products it replaces, the trade-off is the usual one for local inference: privacy, offline use and no per-token cost, against the quality ceiling of models that fit in your RAM and VRAM. Jan narrows the gap by letting you add those same hosted models as providers, but at that point messages go to the provider, as the privacy page points out.

Things to know before adopting

• Hardware sets the limits. The macOS guide suggests 8 GB of memory for models up to about 3B parameters, 16 GB for about 7B and 32 GB for about 13B, depending on quantisation. The Linux guide lists an AVX2-capable CPU, 8 GB of RAM (16 GB recommended), 6 GB of VRAM and 10 GB of free storage as minimums.
• Intel Macs are not supported. The macOS guide says Jan runs natively on Apple Silicon only and requires macOS 13.6 or later. MLX additionally needs macOS 14.
• Update paths differ on Linux. Flatpak updates through Flathub with the in-app updater disabled, the AppImage uses Jan's built-in updater, and the .deb is not updated by the app at all; you reinstall the new package by hand.
• Analytics are a first-launch choice. The privacy page says you choose tracking preferences at first launch and can change them later. When allowed, Jan counts active users and retention against a random ID, using PostHog's EU hosting, and says it does not collect chats, prompts, files or model choices.
• Cloud providers see your messages. Local models keep data on the machine; cloud models need to see your messages to work, as the docs put it, so their own data policies apply.
• Review the server's exposure settings. By default the API server listens only on 127.0.0.1 and needs no key. Jan refuses to bind to 0.0.0.0 without an API key, but CORS is enabled by default and a trusted-hosts value of * lets a web page from any origin call the API, so leave those settings tight unless a browser client needs them.
• MCP tools are powerful. The MCP guide calls granting models access to tools a significant security consideration and mentions prompt injection. Calls are approved one at a time by default, but an "Allow All MCP Tool Permissions" switch pre-approves everything. Many MCP servers also need Node.js or Python installed.
• Some CLI flags are currently ignored. In router mode, --ctx-size, --n-gpu-layers, --threads and --fit are accepted but have no effect; inference settings come from the desktop app. jan threads delete permanently deletes a conversation.
• Two default ports. The app's API server uses 1337, while jan serve and jan launch default to 6767. Scripts written for one will not find the other.
• Fast-moving 0.x releases. Releases are still 0.8.x and features such as Cowork and MLX are labelled preview or experimental. Parts of the documentation lag the latest release, so check the changelog when something behaves differently from the docs.
• Not everything under the Jan name is the open-source app. The documentation also lists Jan Web, a hosted chat service at chat.jan.ai, and a Jan Platform marked as coming soon; this write-up covers the Apache-2.0 desktop app and CLI in the repository.

Project activity

As of November 2026 the janhq/jan repository has about 45,500 stars on GitHub and around 3,150 forks. It was created in August 2023, its primary language is Rust and it is licensed under Apache-2.0, with the GitHub API reporting the licence as unrecognised because of the extra attribution note in the LICENSE file. The latest release at the time of writing is v0.8.8, published on 28 October 2026, and the repository was pushed to on the day of writing. Contributors are asked to sign their commits, and support runs through GitHub issues and the project's Discord.

The source is at github.com/janhq/jan, and downloads, documentation and the changelog are at jan.ai.

[![Featured on TechLogHub](https://techloghub.com/badge-neutral.svg)](https://techloghub.com/open-source/jan-local-ai?utm_source=techloghub_badge&utm_medium=referral&utm_campaign=readme)