Q:

Socket not connected exception after calling EndAccept and accessing RemoteEndPoint

When accessing the RemoteEndPoint property of an accepted socket on the server, a SocketException: 'The socket is not connected' is sometimes thrown. 
This happens occasionally (like once in 100 connection attempts) and only when there is a significant latency over the network (e.g. 10ms+, wifi network). Seems to be some kind of race condition but I'm not sure what can be done to prevent this.
Server code:
this.Socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
this.Socket.Bind(new IPEndPoint(IPAddress.Parse("0.0.0.0"), port));
this.Socket.Listen(0);
this.Socket.BeginAccept(OnSocketAccepted, null);

private void OnSocketAccepted(IAsyncResult ar)
{
    var socket = Socket.EndAccept(ar);
    var endPoint = socket.RemoteEndPoint; // SocketException: The socket is not connected
}

Client code:
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.Connect(ip, port);
// times out after a couple of seconds if the exception happened on the server

Server is running on Mono 6.4.0.198 on ARMv7 rev4 Raspberry Pi 3b+
Client is running on Windows 10 .NET framework 4.6.1 console app
I thought EndAccept basically guaranteed that the socket would be in a connected state and the RemoteEndPoint would be set. I've gone through the whole msdn EndAccept and RemoteEndPoint documentation and can't find any obvious mistakes or misunderstandings.

A:

The Socket can close after you have called EndAccept() and before you have queried RemoteEndPoint.  It can also close before your callback calls EndAccept(), this will also throw a Socket Exception.
There a a number of race conditions that can happen in here, particularly if your system is slow.  You can check Socket.Connected but even that isn't guaranteed.
The correct approach is a Try/Catch and gracefully exit or throw an unrecoverable socket.
e.g.,
        catch (SocketException ex)
        #region SocketException
        {
            /// Some SocketExceptions we manage quietly.  Others we surface as critical errors.
            if ((SocketError)ex.ErrorCode == SocketError.AddressAlreadyInUse)
            {
                Log.Error("Endpoint failed to start. IPEndPoint {ipEndPoint} already in use.", this.localIPEndPoint.ToString());
                // Execution will continue after finally{}
            }
            else
            {
                Log.Error(ex, "Endpoint has ended with an unrecoverable socket error:", Name);
                throw;
            }
        }

