I don't care if it is common sense or reasonable to you, to have the four freedoms for every software I use (remotely or localy). I want the four freedoms for every software I use, no matter if it runs on my computer or the computer of someone else. I just matters that I am a user and don't have the four freedoms.
That would be invasion of privacy, you should not have the right to modify the version of telnet that runs on my machine, no matter how much you'd like that.
What does make sense, is for you to create your own server that runs like mine. This is why the GNU Affero license exists.