Working on a web crawler (again), except now I've had a lot more time to hone my skills.  The largest bottleneck in my program, as is to be expected, is the actual downloading of the data, so I'd love some feedback on improving this section specifically.The Response Object is simply a container for an optional JSoup Document (generated by buildDocument(HttpURLConnection, String, String)), the URI queried, the SHA-256 digest of the data, the content type, the size of the data, the connection speed, and the date fetched.isInfiniteStream(HttpURLConnection) uses crude heuristics to try to figure out if the stream is infinite.  I can post it if people are interested, but it's not very exciting.BrokenConnectionExceptions are handled up the chain and are thrown when a previously accessible connection is no longer accessible.protected Response download(HttpURLConnection connection, InputStream stream)        throws NoSuchAlgorithmException, URISyntaxException, BrokenConnectionException{    MessageDigest digest = MessageDigest.getInstance(SHA-256);    if(isInfiniteStream(connection))        return new Response(null, connection.getURL().toURI(), digest.digest(),                connection.getContentType(), -1, 0, new Date());    if(Runtime.getRuntime().maxMemory() < connection.getContentLengthLong())        return new Response(null, connection.getURL().toURI(), digest.digest(),                connection.getContentType(), connection.getContentLengthLong(), 0, new Date());    StringBuilder content;    long size = 0;    long duration = System.currentTimeMillis();    try{        content = new StringBuilder(Math.max(connection.getContentLength(), stream.available()));        try(BufferedReader in = new BufferedReader(new InputStreamReader(new DigestInputStream(stream, digest)),                8192)){            int read;            for(char[] buffer = new char[8192]; (read = in.read(buffer)) != -1; size += read){                if(Thread.interrupted()){                    InterruptedIOException iIOE =                            new InterruptedIOException(Interrupted while reading stream into buffer);                    iIOE.bytesTransferred = (int)Math.min(size, Integer.MAX_VALUE);                    throw iIOE;                }                content.append(buffer, 0, read);            }        }catch(InterruptedIOException e){            throw new BrokenConnectionException(Connection interrupted by Thread#interrupt()., e);        }catch(IOException e){            throw new BrokenConnectionException(Failure while reading stream into buffer., e);        }    }catch(IOException e){        throw new BrokenConnectionException(Failure while getting stream availability., e);    }    duration = System.currentTimeMillis() - duration;    long speed = Math.round((double) size / ((double) duration / 1000));    String type = connection.getContentType().trim().toLowerCase().intern();    Document document = buildDocument(connection, content.toString(), type);    return new Response(document, connection.getURL().toURI(), digest.digest(),            type, size, speed, new Date());}I tried using Java 8 Streams, but things like BufferedReader#lines() were giving me inaccurate byte counts, and performance is actually higher when recycling a char[].  I'm not sure about the 8KB buffer size; is this optimal, and if not, what is?
Download the contents of an HttpURLConnection while generating metadata
java;performance;io