Think it would be enough to just add ssh agent to the server?