Q:

HTTPS proxy tunneling with the ssl module

I'd like to manually (using the socket and ssl modules) make an HTTPS request through a proxy which itself uses HTTPS.
I can perform the initial CONNECT exchange just fine:
import ssl, socket

PROXY_ADDR = ("proxy-addr", 443)
CONNECT = "CONNECT example.com:443 HTTP/1.1\r\n\r\n"

sock = socket.create_connection(PROXY_ADDR)
sock = ssl.wrap_socket(sock)
sock.sendall(CONNECT)
s = ""
while s[-4:] != "\r\n\r\n":
    s += sock.recv(1)
print repr(s)

The above code prints HTTP/1.1 200 Connection established plus some headers, which is what I expect.  So now I should be ready to make the request, e.g.
sock.sendall("GET / HTTP/1.1\r\n\r\n")

but the above code returns
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />
Instead use the HTTPS scheme to access this URL, please.<br />
</body></html>

This makes sense too, since I still need to do an SSL handshake with the example.com server to which I'm tunneling.  However, if instead of immediately sending the GET request I say
sock = ssl.wrap_socket(sock)

to do the handshake with the remote server, then I get an exception:
Traceback (most recent call last):
  File "so_test.py", line 18, in <module>
    ssl.wrap_socket(sock)
  File "/usr/lib/python2.6/ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "/usr/lib/python2.6/ssl.py", line 118, in __init__
    self.do_handshake()
  File "/usr/lib/python2.6/ssl.py", line 293, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

So how can I do the SSL handshake with the remote example.com server?
EDIT: I'm pretty sure that no additional data is available before my second call to wrap_socket because calling sock.recv(1) blocks indefinitely.

A:

This should work if the CONNECT string is rewritten as follows:
CONNECT = "CONNECT %s:%s HTTP/1.0\r\nConnection: close\r\n\r\n" % (server, port)

Not sure why this works, but maybe it has something to do with the proxy I'm using. Here's an example code:
from OpenSSL import SSL
import socket

def verify_cb(conn, cert, errun, depth, ok):
        return True

server = 'mail.google.com'
port = 443
PROXY_ADDR = ("proxy.example.com", 3128)
CONNECT = "CONNECT %s:%s HTTP/1.0\r\nConnection: close\r\n\r\n" % (server, port)

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(PROXY_ADDR)
s.send(CONNECT)
print s.recv(4096)      

ctx = SSL.Context(SSL.SSLv23_METHOD)
ctx.set_verify(SSL.VERIFY_PEER, verify_cb)
ss = SSL.Connection(ctx, s)

ss.set_connect_state()
ss.do_handshake()
cert = ss.get_peer_certificate()
print cert.get_subject()
ss.shutdown()
ss.close()

Note how the socket is first opened and then open socket placed in SSL context. Then I manually initialize SSL handshake. And output:

HTTP/1.1 200 Connection established
<X509Name object '/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com'>

It's based on pyOpenSSL because I needed to fetch invalid certificates too and Python built-in ssl module will always try to verify the certificate if it's received.

A:

Judging from the API of the OpenSSL and GnuTLS library, stacking a SSLSocket onto a SSLSocket is actually not straightforwardly possible as they provide special read/write functions to implement the encryption, which they are not able to use themselves when wrapping a pre-existing SSLSocket.
The error is therefore caused by the inner SSLSocket directly reading from the system socket and not from the outer SSLSocket. This ends in sending data not belonging to the outer SSL session, which ends badly and for sure never returns a valid ServerHello.
Concluding from that, I would say there is no simple way to implement what you (and actually myself) would like to accomplish.

