Install with Docker
Pre-requisites
Ensure that your system meets the following requirements:
- Linux or WSL2 Docker
- Latest Docker Engine and Docker Compose
To enable GPU support, you will need:
nvidia-drivernvidia-docker2
note
Run Jan in Docker
You can run Jan in Docker with two methods:
- Run Jan in CPU mode
- Run Jan in GPU mode
- CPU
- GPU
To run Jan in Docker CPU mode, by using the following code:
# cpu mode with default file system
docker compose --profile cpu-fs up -d
# cpu mode with S3 file system
docker compose --profile cpu-s3fs up -d
To run Jan in Docker CPU mode, follow the steps below:
- Check CUDA compatibility with your NVIDIA driver by running nvidia-smi and check the CUDA version in the output as shown below:
nvidia-smi
# Output
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
- Visit NVIDIA NGC Catalog and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
- Update the
Dockerfile.gpuline number 5 with the latest minor version of the image tag from step 2 (e.g. changeFROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS basetoFROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base) - Run Jan in GPU mode by using the following command:
# GPU mode with default file system
docker compose --profile gpu-fs up -d
# GPU mode with S3 file system
docker compose --profile gpu-s3fs up -d
Docker Compose Profile and Environment
The available Docker Compose profile and the environment variables listed below:
Docker Compose Profile
| Profile | Description |
|---|---|
| cpu-fs | Run Jan in CPU mode with default file system |
| cpu-s3fs | Run Jan in CPU mode with S3 file system |
| gpu-fs | Run Jan in GPU mode with default file system |
| gpu-s3fs | Run Jan in GPU mode with S3 file system |
Environment Variables
| Environment Variable | Description |
|---|---|
| S3_BUCKET_NAME | S3 bucket name - leave blank for default file system |
| AWS_ACCESS_KEY_ID | AWS access key ID - leave blank for default file system |
| AWS_SECRET_ACCESS_KEY | AWS secret access key - leave blank for default file system |
| AWS_ENDPOINT | AWS endpoint URL - leave blank for default file system |
| AWS_REGION | AWS region - leave blank for default file system |
| API_BASE_URL | Jan Server URL, please modify it as your public ip address or domain name default http://localhost:1377 |
warning
If you are stuck in a broken build, go to the Broken Build section of Common Errors.