Q:

ssh client problem: Connection reset by peer

I'm having a really annoying problem on my Ubuntu laptop.
I noticed it today, after upgrading to Ubuntu 11.04, although I'm not entirely sure this is the cause as I played with my ssh keys a few days ago.
The problem is, whenever I try to ssh to ANY host I get the following error:
Read from socket failed: Connection reset by peer

running with -vvv gives the following output:
OpenSSH_5.8p1 Debian-1ubuntu3, OpenSSL 0.9.8o 01 Jun 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to hostname [10.0.0.2] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 1.99, remote software version OpenSSH_4.2
debug1: match: OpenSSH_4.2 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.8p1 Debian-1ubuntu3
debug2: fd 3 setting O_NONBLOCK
debug3: load_hostkeys: loading entries for host "hostname" from file "/root/.ssh/known_hosts"
debug3: load_hostkeys: loaded 0 keys
debug1: SSH2_MSG_KEXINIT sent
Read from socket failed: Connection reset by peer

My /etc/ssh/ssh_config:
Host *
    SendEnv LANG LC_*
    HashKnownHosts yes
    GSSAPIAuthentication no
    GSSAPIDelegateCredentials no

I can connect to my laptop from any other server via ssh, and I can also ssh localhost from my laptop successfully.
I can connect to all these other server from other laptops, and I don't see anything in the logs of the other servers regarding my failed attempt.
I tried to stop iptables, didn't help.
I tried several tricks I could find online with my /etc/ssh/ssh_config, but I was unsuccessful in solving the problem...
Any ideas?

Edit:
This is the log from one of the hosts I try to connect to:
May  1 19:15:23 localhost sshd[2845]: debug1: Forked child 2847.
May  1 19:15:23 localhost sshd[2845]: debug3: send_rexec_state: entering fd = 8 config len 577
May  1 19:15:23 localhost sshd[2845]: debug3: ssh_msg_send: type 0
May  1 19:15:23 localhost sshd[2845]: debug3: send_rexec_state: done
May  1 19:15:23 localhost sshd[2847]: debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8
May  1 19:15:23 localhost sshd[2847]: debug1: inetd sockets after dupping: 3, 3
May  1 19:15:23 localhost sshd[2847]: Connection from 10.0.0.7 port 55747
May  1 19:15:23 localhost sshd[2847]: debug1: Client protocol version 2.0; client software version OpenSSH_5.8p1 Debian-1ubuntu3
May  1 19:15:23 localhost sshd[2847]: debug1: match: OpenSSH_5.8p1 Debian-1ubuntu3 pat OpenSSH*
May  1 19:15:23 localhost sshd[2847]: debug1: Enabling compatibility mode for protocol 2.0
May  1 19:15:23 localhost sshd[2847]: debug1: Local version string SSH-2.0-OpenSSH_5.3
May  1 19:15:23 localhost sshd[2847]: debug2: fd 3 setting O_NONBLOCK
May  1 19:15:23 localhost sshd[2847]: debug2: Network child is on pid 2848
May  1 19:15:23 localhost sshd[2847]: debug3: preauth child monitor started
May  1 19:15:23 localhost sshd[2847]: debug3: mm_request_receive entering
May  1 19:15:23 localhost sshd[2848]: debug3: privsep user:group 74:74
May  1 19:15:23 localhost sshd[2848]: debug1: permanently_set_uid: 74/74
May  1 19:15:23 localhost sshd[2848]: debug1: list_hostkey_types: ssh-rsa,ssh-dss
May  1 19:15:23 localhost sshd[2848]: debug1: SSH2_MSG_KEXINIT sent
May  1 19:15:23 localhost sshd[2848]: debug3: Wrote 784 bytes for a total of 805
May  1 19:15:23 localhost sshd[2848]: fatal: Read from socket failed: Connection reset by peer

A:

That's a tough-to-debug one in openssh, It seems to happen only from particular clients to particular servers. 

Cause?
I didn't get to the root cause. My best finding is that the connection packets are too big for the server to handle, and the connection is reset.
Workarounds: limit the packet size. Two alternatives:

Limit cipher list length by using '-c ' in the ssh command line, e.g. '-c aes256-ctr'
Limit HostKeyAlgorithms list by adding to ~/.ssh/config:

HostKeyAlgorithms       ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss

URLs:

Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/708493
My thread on openssh-unix-dev list: http://lists.mindrot.org/pipermail/openssh-unix-dev/2011-February/029361.html

Affected versions:
AFAIK it started with 5.7p1. Downgrading to 5.5p1 solves the problem. However, on machines that do NOT have this problem, 5.7p1, 5.8p1 work perfectly.
Thus my assumption is that it has to do with an innocent library call that was added on 5.7p1 to a 3rd party lib, which is broken only on certain environments. Crazy assumption to a crazy bug.

