That's pretty much exactly what tmate does (it uses tmux under the hood). However, I was looking at how to do this myself and couldn't figure out how to share my session and only allow people into that, rather than the whole system, so thanks!