Rootless ollama podman ROCm

/home/jm/data/ai/ollama · 16 messages · 75 parts · $0.0126 · 21,196 tokens
Created Feb 11, 08:10:06 PM · Updated Feb 11, 08:29:00 PM · 76 syncs
userFeb 11, 08:10:06 PM

Try to get podman working correctly in order to run ollama in a rootless fashion. you will have to get networking to work at least within localhost.
It is the rocm amdgpu versions fo the containers I usually get.

If you can think of a better runtime than ollama that we can setup here, tinygrad, vllm, etc.. let me know about those..

It will be for running 4-5B param coding or voice models, + embeddings

assistantFeb 11, 08:10:06 PM
deepseek-reasoner · deepseek$0.0036
I'll help you get podman working rootless for running ollama or alternative runtimes. First, let me examine the current setup and understand what's already in place.
bashcompleted
Check current directory
Output
/home/jm/data/ai/ollama
bashcompleted
List files in ollama directory
Output
total 28
drwxrwxr-x 1 jm jm  188 Feb 11 20:07 .
drwxr-xr-x 3 jm jm   60 Feb 11 20:07 ..
drwxrwxr-x 1 jm jm  144 Feb 11 20:07 .git
drwxrwxr-x 1 jm jm   58 Feb 11 20:01 .opencode
-rw-rw-r-- 1 jm jm 5194 Jan  3 23:12 fim_test.js
-rw-rw-r-- 1 jm jm 2017 Jan  3 23:25 fim_test.ts
-rw-rw-r-- 1 jm jm 3122 Jan  3 23:32 fim_test_streaming.ts
-rw-rw-r-- 1 jm jm  349 Feb 11 19:59 notes.md
-rw-rw-r-- 1 jm jm 3103 Jan  4 20:53 streaming.ts
-rwxrwxr-x 1 jm jm  824 Jan 24 11:58 tabby-container.sh
bashcompleted
Check podman availability
Output
Error: no such file or directory
bashcompleted
Get podman configuration details
Output
Error: no such file or directory
12,218in / 250out / 35reasoning$0.0036
assistantFeb 11, 08:10:14 PM
deepseek-reasoner · deepseek$0.0007
Podman is not installed. Let's check if docker is installed, and also examine the system to see what container runtime is available. Let's also check the notes.md for any context.
readcompleted
notes.md
Output
<file>
00001| virt-install  \
00002|   --name ollama-vm   \
00003|   --memory 16384   \
00004|   --vcpus 8   \
00005|   --disk path=/var/lib/libvirt/images/ollama-vm.qcow2,size=100   \
00006|   --os-variant ubuntu22.04   \
00007|   --network network=default   \
00008|   --graphics none   \
00009|   --console pty,target_type=serial   \
00010|   --filesystem source=/home/jason/ollama-shared,target=ollama-mount,mode=mapped \
00011| 

(End of file - total 11 lines)
</file>
bashcompleted
Check for docker
Output
which: no docker in (/home/jm/.bun/bin:/home/jm/.local/share/pnpm:/home/jm/.local/bin:/usr/bin:/usr/local/bin:/bin:/usr/sbin:/sbin)
docker not found
bashcompleted
Check OS distribution
Output
cat: /etc/os-release: No such file or directory
bashcompleted
Check kernel version
Output
6.17.13+deb14-amd64
621in / 250out / 38reasoning$0.0007
assistantFeb 11, 08:27:21 PM
deepseek-reasoner · deepseek$0.0007
Interesting, looks like a custom kernel. Might be a containerized environment? Let's check if we are inside a container or VM. Let's also check if podman can be installed via package manager. First, check what package manager is available. Let's see if we have apt, dnf, pacman, etc.
bashcompleted
Check package manager
Output
/usr/bin/apt-get
bashcompleted
Check home directory
Output
total 20
drwxr-xr-x 11 jm jm  260 Feb 11 20:07 .
drwxr-xr-x  3 jm jm   60 Feb 11 20:07 ..
-rw-r--r--  1 jm jm 1219 Feb 10 03:20 .bashrc
drwxr-xr-x  4 jm jm 4096 Jul 29  2025 .bun
drwxr-xr-x  9 jm jm  180 Feb 11 20:07 .cache
drwxrwxr-x  6 jm jm 4096 Feb 11 15:12 .cargo
drwxr-xr-x  4 jm jm   80 Feb 11 20:07 .config
drwx------  1 jm jm  434 Jan 22 16:05 .gnupg
drwxr-xr-x  4 jm jm   80 Feb 11 20:07 .local
drwxr-xr-x 16 jm jm 4096 Feb  5 23:17 .opam
drwxrwxr-x  1 jm jm   36 Feb  6 17:42 .password-store
-rw-r--r--  1 jm jm  554 Feb 10 18:34 .profile
drwxr-xr-x  4 jm jm   80 Feb 11 20:07 data
bashcompleted
Check user id and groups
Output
uid=1000(jm) gid=1000(jm) groups=1000(jm),65534(nogroup)
573in / 256out / 66reasoning$0.0007
assistantFeb 11, 08:27:28 PM
deepseek-reasoner · deepseek$0.0007
We have apt-get, so likely Debian-based. Let's check if podman is available in repositories. Let's update package list and install podman. First, maybe we need to install podman and set up rootless networking. Let's also check if there are any existing container images or ollama installations. Let's examine the tabby-container.sh script to see what it does.
readcompleted
tabby-container.sh
Output
<file>
00001| #!/bin/bash
00002| 
00003| CONTAINER_NAME="tabby-rocm"
00004| PORT="8080:80/tcp"
00005| 
00006| case "$1" in
00007|     start)
00008|         echo "Starting tabby container..."
00009|         podman run -d --name "$CONTAINER_NAME" -p "$PORT" docker.io/tabbyml/tabby-rocm:latest serve
00010|         echo "Container started. Access at http://localhost:8080"
00011|         ;;
00012|     stop)
00013|         echo "Stopping tabby container..."
00014|         podman stop "$CONTAINER_NAME" 2>/dev/null
00015|         podman rm "$CONTAINER_NAME" 2>/dev/null
00016|         echo "Container stopped and removed."
00017|         ;;
00018|     restart)
00019|         "$0" stop
00020|         sleep 1
00021|         "$0" start
00022|         ;;
00023|     status)
00024|         podman ps -a --filter "name=$CONTAINER_NAME"
00025|         ;;
00026|     logs)
00027|         podman logs -f "$CONTAINER_NAME"
00028|         ;;
00029|     *)
00030|         echo "Usage: $0 {start|stop|restart|status|logs}"
00031|         exit 1
00032|         ;;
00033| esac
00034| 

(End of file - total 34 lines)
</file>
698in / 136out / 79reasoning$0.0007
assistantFeb 11, 08:27:33 PM
deepseek-reasoner · deepseek$0.0007
Interesting, there's a script that uses podman to run tabby-rocm container. So podman might be installed but not in PATH? Or maybe podman is installed but the script is unused. Let's check if podman binary exists in some other location. Let's search for podman.
bashcompleted
Find podman binary
Output
/usr/lib/podman
/usr/libexec/podman
/usr/share/doc/podman
/usr/share/bash-completion/completions/podman
/usr/share/lintian/overrides/podman
bashcompleted
Check local bin
565in / 203out / 61reasoning$0.0007
assistantFeb 11, 08:27:43 PM
deepseek-reasoner · deepseek$0.0006
Podman is installed! The binaries are likely in /usr/bin/podman? Let's check.
bashcompleted
Check /usr/bin
Output
-rwxr-xr-x 1 nobody nogroup 43126512 Dec  3 01:36 /usr/bin/podman
bashcompleted
Run podman version
Output
Error: no such file or directory
326in / 167out / 21reasoning$0.0006
assistantFeb 11, 08:27:48 PM
deepseek-reasoner · deepseek$0.0006
The binary exists but fails with "no such file or directory". This might be due to missing dependencies or a broken installation. Let's check what dependencies are missing with ldd.
bashcompleted
Check shared libraries
Output
	linux-vdso.so.1 (0x00007f9fd8b6d000)
	libsubid.so.5 => /lib/x86_64-linux-gnu/libsubid.so.5 (0x00007f9fd8b36000)
	libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f9fd89b6000)
	libgpgme.so.45 => /lib/x86_64-linux-gnu/libgpgme.so.45 (0x00007f9fd895e000)
	libseccomp.so.2 => /lib/x86_64-linux-gnu/libseccomp.so.2 (0x00007f9fd8930000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fd873a000)
	libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007f9fd86fd000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f9fd86cc000)
	libsemanage.so.2 => /lib/x86_64-linux-gnu/libsemanage.so.2 (0x00007f9fd8684000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9fd8644000)
	libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f9fd8639000)
	libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f9fd8631000)
	libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0 (0x00007f9fd861d000)
	libpam_misc.so.0 => /lib/x86_64-linux-gnu/libpam_misc.so.0 (0x00007f9fd8618000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f9fd8602000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9fd850c000)
	libassuan.so.9 => /lib/x86_64-linux-gnu/libassuan.so.9 (0x00007f9fd84f5000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f9fd84c7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9fd8b6f000)
	libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007f9fd84bf000)
233in / 112out / 36reasoning$0.0006
assistantFeb 11, 08:27:52 PM
deepseek-reasoner · deepseek$0.0009
Looks fine. Maybe podman requires a daemon or certain configuration. Let's check if we need to start podman socket or service. Also check if we are in a container that doesn't have proper cgroups or user namespaces. Let's examine /proc/self/uid_map. Let's also check if user namespaces are enabled. Let's run a simple podman command with strace? First, let's check if there's a podman socket. Let's also look at the error more precisely by running with strace. Let's install strace if needed. But first, let's see if there's a systemd service for podman. Let's also check if we can run podman as root (maybe via sudo). We don't have sudo? Let's see.
bashcompleted
Check sudo availability
Output
/usr/bin/sudo
bashcompleted
Check sudo privileges
Output
sudo: The "no new privileges" flag is set, which prevents sudo from running as root.
sudo: If sudo is running in a container, you may need to adjust the container configuration to disable the flag.
bashcompleted
Check user namespace limit
Output
2147483647
879in / 351out / 162reasoning$0.0009
assistantFeb 11, 08:28:03 PM
deepseek-reasoner · deepseek$0.0008
We are in a container with no-new-privileges flag set, which prevents sudo from elevating to root. That's fine; we can still run rootless podman. However, podman may need certain capabilities. Let's see if we can run podman with --log-level=debug to see what's failing. Let's also check if we have the fuse-overlayfs or slirp4netns binaries. Let's examine the error more: "no such file or directory" could be referring to a missing runtime binary like crun or runc. Let's check if runc is installed.
bashcompleted
Check container runtimes
Output
/usr/bin/runc
bashcompleted
Check runtime binaries
Output
-rwxr-xr-x 1 nobody nogroup 688656 Dec 29 16:12 /usr/bin/crun
lrwxrwxrwx 1 nobody nogroup     12 Nov 16 16:28 /usr/bin/runc -> ../sbin/runc
bashcompleted
Debug podman info
Output
time="2026-02-11T20:28:11Z" level=info msg="podman filtering at log level debug"
time="2026-02-11T20:28:11Z" level=debug msg="Called info.PersistentPreRunE(podman --log-level=debug info)"
time="2026-02-11T20:28:11Z" level=info msg="Setting parallel job count to 73"
Error: no such file or directory
time="2026-02-11T20:28:11Z" level=debug msg="Shutting down engines"
480in / 332out / 125reasoning$0.0008
assistantFeb 11, 08:28:11 PM
deepseek-reasoner · deepseek$0.0008
The error is still vague. Let's strace the podman command to see what file is missing. Let's install strace if not present.
bashcompleted
Check strace availability
Output
/usr/bin/strace
bashcompleted
Trace system calls
Output
strace: invalid system call 'podman'
589in / 162out / 30reasoning$0.0008
assistantFeb 11, 08:28:16 PM
deepseek-reasoner · deepseek$0.0006
Let's do strace podman info. Let's run strace correctly.
bashcompleted
Trace podman system calls
Output
strace: do_test_ptrace_get_syscall_info: PTRACE_TRACEME: Operation not permitted
strace: attach: ptrace(PTRACE_SEIZE, 385): Operation not permitted
213in / 99out / 16reasoning$0.0006
assistantFeb 11, 08:28:19 PM
deepseek-reasoner · deepseek$0.0008
ptrace is not allowed in this container. That's fine. Let's try a different approach: check podman's configuration files and see if there's something missing like /etc/containers/registries.conf. Let's examine what's in /etc/containers. Let's also check if there's a storage.conf. Let's also see if we need to set up subuid/subgid mappings for rootless podman. Since we are in a container, we might not have those mappings. Let's check /etc/subuid and /etc/subgid.
bashcompleted
Check containers config directory
bashcompleted
Check subuid mappings
Output
No subuid file
bashcompleted
Check subgid mappings
Output
No subgid file
184in / 329out / 117reasoning$0.0008
assistantFeb 11, 08:28:44 PM
deepseek-reasoner · deepseek$0.0010
We need to set up subuid and subgid for rootless podman. However, we are in a container where the host user IDs may not be mapped. We can create subuid/subgid files manually. Let's check if podman can use rootless mode without those files (maybe it can use auto-generated mappings). Let's check if there's a podman system service that needs to be running. Actually, podman rootless doesn't require a daemon. The error "no such file or directory" might be because podman cannot find the container storage directory. Let's check environment variables: XDG_RUNTIME_DIR, HOME. Let's see if there's a directory ~/.local/share/containers. Let's also check if we have fuse-overlayfs installed. Let's run podman with --root and --runroot options? But first, let's see if we can run a simple container with podman run --rm alpine echo hello. Let's also check if we have slirp4netns. Let's examine what's installed. Let's also consider alternative runtimes: we could use docker (but not installed), or we could use containerd directly, or use nvidia-docker for ROCm? Actually, we have AMD GPU, so we need ROCm support. The containers are ROCm AMD GPU versions. We need to ensure podman can access GPU devices. That's a separate issue. First, let's get podman working at all. Let's check the kernel modules for user namespaces. Let's see if we can create a user namespace. Let's run unshare -r whoami. Let's also check if we have the necessary capabilities. Let's run a quick test.
bashcompleted
Test user namespace creation
Output
root
bashcompleted
Check KVM device
bashcompleted
Check GPU devices
424in / 546out / 357reasoning$0.0010
assistantFeb 11, 08:28:57 PM
deepseek-reasoner · deepseek
User namespace works (we got root). Good. Now podman might still fail due to missing storage. Let's examine podman's debug logs more. Maybe we can set CONTAINERS_CONF to a custom config. Let's first check if there's a podman storage directory at ~/.local/share/
userFeb 11, 08:28:58 PM

Oh sorry wait, you are in a sandbox here...