Large language models
Large language models: run an LLM on your own servers.
A large language model, or LLM, is the model behind today’s AI assistants and agents. NeueCode runs open LLMs on your own GPUs, so the model works for your organization without your data leaving it.
Free for 30 days · Runs on your own server · No credit card
What is a large language model?
A large language model is a model trained on a very large amount of text to predict what comes next. That one ability is enough to answer questions, summarize, translate, write code and follow instructions.
Some LLMs are closed: you reach them only through their maker’s service. Others are open-weight: the model itself is published, so you can run it on your own hardware. An open-weight model on your own server is what makes private AI possible.
The words you will meet
| Open-weight model | A model whose files are published, so you can run it on your own hardware. |
|---|---|
| Context window | How much text the model can consider at once. |
| GPU memory (VRAM) | What decides whether a model fits on a card, and how many people it serves at the same time. |
| Local LLM | An LLM run on your own hardware instead of a provider’s. |
How NeueCode runs LLMs
Open models, on your own GPUs, with everything an organization needs around them.
Open models, your choice
NeueCode is not tied to one model. An administrator chooses from a variety of open AI models.
Fitted to your GPU
The installer checks the server and installs the models that fit its card. One server serves several models at once.
Supporting models included
Every server also gets a vision model that reads images and scanned documents, an embedding model that finds things by meaning, and a dictation model.
More than a model
Given tools and a goal, the model plans a task, does it and reports back, with a person approving the actions that matter.
Size it before you buy
The GPU VRAM calculator shows how much GPU memory an open model needs and how many people it serves at once.
Larger than one server
A model too large for any one server runs as one copy split across your NeueCode server and its GPU servers, over a private link between them.
You can check
- The installer checks the server first and says plainly if something is missing.
- The minimum is one NVIDIA card with 24 GB of memory.
Questions about large language models
- What is a large language model (LLM)?
- A large language model is an AI model trained on a very large amount of text to predict what comes next, which lets it answer, summarize, translate and write code.
- What is the difference between an open and a closed LLM?
- A closed LLM is reached only through its maker’s service. An open-weight LLM is published, so you can run it on your own hardware and keep your data there.
- How much GPU memory does an LLM need?
- It depends on the size of the model and on how many people use it at once. The GPU VRAM calculator gives the figure for each open model.
- What if an LLM is too large for one server?
- NeueCode can run it as one copy split across several servers. Over an ordinary network link it answers more slowly than on one server; a fast private link between the servers lets them share every layer of the model.
- Can an LLM run without the internet?
- Yes. NeueCode needs the internet only during the install; afterwards the models run on the server itself.
- Which LLM does NeueCode use?
- NeueCode is not tied to one model. An administrator chooses from a variety of open models that run on your own GPUs.
Read next
Your IT team has it running in about an hour, on one server.