LocalAI is a free, open-source alternative to OpenAI (Anthropic, etc.), functioning as a drop-in replacement REST API for local inferencing. It allows you to run LLMs, generate images, and produce audio, all locally or on-premises with consumer-grade hardware, supporting multiple model families and architectures.

Using the Bash Installer

Install LocalAI easily using the bash installer with the following command:

  curl https://localai.io/install.sh | sh

For a full list of options, refer to the Installer Options documentation.

Binaries can also be manually downloaded.

Using Container Images or Kubernetes

LocalAI is available as a container image compatible with various container engines such as Docker, Podman, and Kubernetes. Container images are published on quay.io and Docker Hub.

For detailed instructions, see Using container images. For Kubernetes deployment, see Run with Kubernetes.

Running LocalAI with All-in-One (AIO) Images

Already have a model file? Skip to Run models manually.

LocalAI’s All-in-One (AIO) images are pre-configured with a set of models and backends to fully leverage almost all the features of LocalAI. If pre-configured models are not required, you can use the standard images.

These images are available for both CPU and GPU environments. AIO images are designed for ease of use and require no additional configuration.

It is recommended to use AIO images if you prefer not to configure the models manually or via the web interface. For running specific models, refer to the manual method.

The AIO images come pre-configured with the following features:

  • Text to Speech (TTS)
  • Speech to Text
  • Function calling
  • Large Language Models (LLM) for text generation
  • Image generation
  • Embedding server

For instructions on using AIO images, see Using container images.

What’s Next?

There is much more to explore with LocalAI! You can run any model from Hugging Face, perform video generation, and also voice cloning. For a comprehensive overview, check out the features section.

Explore additional resources and community contributions:

Edit this page

Last updated 22 Jun 2024, 11:58 +0200 . history