I've got a wrapper around tmux for an audience of one. I can operate claude code, codex, opencode, or just a shell, from any of my devices to any of my devices (via tailscale) or more commonly, operate it on my exe.dev server.
I often continue a session on my phone, sometimes with voice. I have buttons for viewing files or following links the agent has referenced, extracted from the stream of text, and I have some buttons for exactly the git stuff I need. I have a button to toggle between yolo mode and normal.
Basically, very simple UI for everything I actually use, easy to use on a phone - and maybe more importantly, no UI for anything I don't personally use. Also all my machines have the repo for the uh, harness-harness, so I just open the tab for it if I need some changes and prompt them into existence and get the changes live.
All this is great, except it enables me to work every waking hour of my life. That part might be bad.
Would love to learn more about your setup as I put something fairly similar for myself and looking for ways to improve it even further.
My goal is to have agents work independently from any of my devices (hence VM setup) and to access any session at any time.
My setup so far:
A simple VM on Google Cloud. Always on. Repo lives in there. Claude Code, Codex, Gemini installed as CLIs.
The agents run inside terminal sessions on the VM, managed by zmx (small Zig project, keeps a raw PTY around). Session keeps running regardless of what’s connected.
From the Mac, transport is Eternal Terminal on port 2022. ET is like SSH but reconnects silently. I never see the disconnect.
From the phone, transport is mosh. UDP based, roams gracefully (cellular to wifi and back). Client is Moshi (iOS), renders the terminal and manages tmux sessions well. Moshi needs alternate screen for scrollback, so on the phone path zmx is wrapped in tmux. The tmux is just an adapter for the phone client, not the persistence layer.
The point: both transports attach to the same zmx session. Same scrollback, same prompt, same agent state. Laptop, phone, both at once, all clients of the same long running shell on the VM.
The annoying part was scrollback: duplicated history, lack of native scroll, lagging. zmx avoids the alternate screen (tmux uses it by default, which makes scrolling agent output awkward). Combined with Claude’s --no-flicker flag, the terminal’s own scrollback ends up actually usable.
I wanted to avoid manually switching scroll modes and re-attaching. Moshi sees all my current sessions on the Mac, I just tap and swipe between them. Awesome.
A few shell functions hide the layering on the Mac with a few commands and keyboard shortcuts.
Do you have a way to use voice on your smartphone?
I use a whisper wrapper (also "built for one") for Windows and connect through SSH but it only works well enough because of the laptop's ARC graphics card. I wish I could do that when connecting to SSH from my smartphone.
FUTO has a superb Android keyboard with on-device voice transcription using a fine-tuned Whisper model, which is quite performant for me (and actually does fairly impressive text autocorrect and prediction too): https://keyboard.futo.org/
They also do a pure play voice input keyboard, if that's your jam or you want to use it with other keuboards: https://voiceinput.futo.org/
Yeah, two ways, lol. I have a big mic button on the bottom that I hold to speak, which uses the browser speech api. edit: I'm reading that that's local in safari but remote in chrome.
But I also type into an input field rather than the terminal and can use the mic on the phone keyboard (iOS).
I often continue a session on my phone, sometimes with voice. I have buttons for viewing files or following links the agent has referenced, extracted from the stream of text, and I have some buttons for exactly the git stuff I need. I have a button to toggle between yolo mode and normal.
Basically, very simple UI for everything I actually use, easy to use on a phone - and maybe more importantly, no UI for anything I don't personally use. Also all my machines have the repo for the uh, harness-harness, so I just open the tab for it if I need some changes and prompt them into existence and get the changes live.
All this is great, except it enables me to work every waking hour of my life. That part might be bad.