Run language, image and audio models on your own AWS instance or Google Cloud VM with LocalAI, the open-source engine with an OpenAI-compatible API. The pre-configured image runs on CPU or GPU, protects the web UI and the API with your own API key, and comes with a model ready to chat with.
LocalAI is an open-source, MIT-licensed AI engine that runs language, vision, voice and image models on your own hardware, with no GPU required. Its OpenAI-compatible API means code written for the OpenAI SDK works with a change of base URL, and its web UI lets you chat with models and install new ones from a gallery. It suits teams that want text, image and audio models behind one private API.
Code written for the OpenAI SDK works with a change of base URL.
Chat, embeddings, image generation, speech-to-text and text-to-speech.
Runs on CPU; an NVIDIA GPU makes it faster.
Install models from the gallery or import them from Hugging Face.
Chat with models and manage them in the browser.
Model Context Protocol support and built-in agents with tool use.
The image is ready to use the moment the deployment finishes: LocalAI is installed, a model is loaded, and access is locked behind the API key you choose at launch.
On AWS, t3a.xlarge runs LocalAI on CPU and g4dn.xlarge adds an NVIDIA T4. On Google Cloud, e2-standard-4 runs on CPU and the g2 types add an NVIDIA L4. GPUs often need a quota increase first.
Launch the CloudFormation stack from AWS Marketplace, or fill in the Google Cloud Marketplace deployment form, with your domain and an API key of your choice.
Add an A record for the server's public IP. SSL is set up automatically when it can be, and a certificate script on the server covers the rest.
Sign in to the web UI with your API key, then point any OpenAI client at https://your-domain/v1.
An AMI on AWS Marketplace, launched on a CPU or GPU instance in your own AWS account with a CloudFormation stack.
Launch on AWS 
The same engine as a Google Cloud Marketplace image, deployed on a CPU or NVIDIA L4 GPU VM in your own GCP project.
Launch on Google Cloud The same image runs on both. Start on CPU to evaluate LocalAI or run small models, and move to a GPU when response times matter.
| Cloud | Machine type | Hardware | Good for |
|---|---|---|---|
| AWS | t3a.xlarge | 4 vCPUs, 16 GB RAM | Trying LocalAI, small chat models, embeddings |
| AWS | g4dn.xlarge | 4 vCPUs, 16 GB RAM, 1× NVIDIA T4 with 16 GB | 7B to 8B chat models, image generation, speech |
| Google Cloud | e2-standard-4 | 4 vCPUs, 16 GB RAM | Trying LocalAI, light use, embeddings, batch jobs |
| Google Cloud | g2-standard-8 | 8 vCPUs, 32 GB RAM, 1× NVIDIA L4 with 24 GB | Interactive chat, 7B to 14B models, image generation, speech |
GPU instances use the On-Demand G and VT vCPU quota on AWS and the NVIDIA L4 GPU quota on Google Cloud, and both are often 0 in new accounts.
| LocalAI by Meetrix | OpenAI API | |
|---|---|---|
| Hosting | A server in your own AWS account or GCP project | OpenAI's platform |
| Pricing model | Billed by the hour, no per-token fee | Mostly per token, with separate input and output rates |
| Models | Open models from the LocalAI gallery or Hugging Face | OpenAI's own models |
| Data | Prompts and outputs stay on your server | Not used for training by default; abuse monitoring logs kept up to 30 days |
| Modalities | Text, embeddings, images, speech-to-text, text-to-speech | Text, embeddings, images, speech-to-text, text-to-speech, realtime voice |
LocalAI speaks the OpenAI API, so moving code between the two is a change of base URL and API key.
An open-source, MIT-licensed engine for running AI models on your own hardware. It serves text, embedding, image and audio models through an OpenAI-compatible API, and includes a web UI for chatting with models and installing new ones.
LocalAI itself is free, open-source software under the MIT licence. You pay for the instance or VM and any charges shown on the Marketplace listing, but there is no per-token fee. The Google Cloud listing includes a 5-day free trial with up to USD 50 in licence credit.
No. The same image runs on CPU and GPU. On AWS, t3a.xlarge is the recommended CPU size and g4dn.xlarge adds an NVIDIA T4 with 16 GB of GPU memory. On Google Cloud, e2-standard-4 produces about 6-7 tokens per second with the pre-installed model, and the g2 types add an NVIDIA L4 with 24 GB for about 65 tokens per second.
qwen3-4b-instruct is pre-installed, so you can chat with it as soon as you sign in. Install more from the gallery under Models in the web UI, choosing sizes that fit your instance.
Open the DashboardUrl from the CloudFormation Outputs tab on AWS, or your domain on Google Cloud, and enter the API key you set at launch. The same key is the bearer token for API requests.
On AWS, LocalAI sets up SSL automatically when your domain is hosted on Route 53. On Google Cloud, it issues SSL on first boot if your domain already points at the VM. Otherwise, point an A record at the public IP and run the certificate script on the server, as the developer guides show.
When a new version is on AWS or Google Cloud Marketplace, back up any models and configuration you need, delete the old deployment and launch the new version.
Both speak the same API, so code moves between them with a change of base URL and key. LocalAI runs open models on a server in your own AWS account or GCP project, billed by the hour, while the OpenAI API runs OpenAI's models on its platform and bills mostly per token.
We build and run self-hosted AI infrastructure for teams every day, from choosing between CPU and GPU to picking models and scaling. Tell us what you need.
Contact us