It was something with Ubuntu server 22.04. I have it running on a 20.04 instance now and I can connect to everything internally!