<commit_before>=pod

=head1 NAME

OSSL_HTTP_REQ_CTX,
OSSL_HTTP_REQ_CTX_new,
OSSL_HTTP_REQ_CTX_free,
OSSL_HTTP_REQ_CTX_set_request_line,
OSSL_HTTP_REQ_CTX_add1_header,
OSSL_HTTP_REQ_CTX_i2d,
OSSL_HTTP_REQ_CTX_nbio,
OSSL_HTTP_REQ_CTX_sendreq_d2i,
OSSL_HTTP_REQ_CTX_get0_mem_bio,
OSSL_HTTP_REQ_CTX_set_max_response_length
- HTTP request functions

=head1 SYNOPSIS

 #include <openssl/http.h>

 typedef struct ossl_http_req_ctx_st OSSL_HTTP_REQ_CTX;

 OSSL_HTTP_REQ_CTX *OSSL_HTTP_REQ_CTX_new(BIO *wbio, BIO *rbio,
                                          int method_POST, int maxline,
                                          unsigned long max_resp_len,
                                          int timeout,
                                          const char *expected_content_type,
                                          int expect_asn1);
 void OSSL_HTTP_REQ_CTX_free(OSSL_HTTP_REQ_CTX *rctx);

 int OSSL_HTTP_REQ_CTX_set_request_line(OSSL_HTTP_REQ_CTX *rctx,
                                        const char *server, const char *port,
                                        const char *path);
 int OSSL_HTTP_REQ_CTX_add1_header(OSSL_HTTP_REQ_CTX *rctx,
                                   const char *name, const char *value);

 int OSSL_HTTP_REQ_CTX_i2d(OSSL_HTTP_REQ_CTX *rctx, const char *content_type,
                           const ASN1_ITEM *it, ASN1_VALUE *req);
 int OSSL_HTTP_REQ_CTX_nbio(OSSL_HTTP_REQ_CTX *rctx);
 ASN1_VALUE *OSSL_HTTP_REQ_CTX_sendreq_d2i(OSSL_HTTP_REQ_CTX *rctx,
                                           const ASN1_ITEM *it);

 BIO *OSSL_HTTP_REQ_CTX_get0_mem_bio(OSSL_HTTP_REQ_CTX *rctx);
 void OSSL_HTTP_REQ_CTX_set_max_response_length(OSSL_HTTP_REQ_CTX *rctx,
                                                unsigned long len);

=head1 DESCRIPTION

B<OSSL_HTTP_REQ_CTX> is a context structure for an HTTP request, used to
collect all the necessary data to perform that request.

This file documents low-level HTTP functions rarely used directly.  High-level
HTTP client functions like L<OSSL_HTTP_get(3)> and L<OSSL_HTTP_transfer(3)>
should be preferred.

OSSL_HTTP_REQ_CTX_new() allocates a new HTTP request context structure,
which gets populated with the B<BIO> to send the request to (I<wbio>),
the B<BIO> to read the response from (I<rbio>, which may be equal to I<wbio>),
the request method (I<method_POST>, which may be 1 to indicate that the C<POST>
method is to be used, or 0 to indicate that the C<GET> method is to be used),
the maximum expected response header length (I<max_resp_len>,
where any zero or less indicates the default of 4KiB),
a response timeout measure in seconds (I<timeout>,
where 0 indicates no timeout, i.e., waiting indefinitely),
the expected MIME content type of the response (I<expected_content_type>,
which may be NULL for no expectation),
and a flag indicating that the response is expected to be
a DER encoded ASN.1 structure (I<expect_asn1>).
The allocated context structure is also populated with an internal allocated
memory B<BIO>, which collects the HTTP request and additional headers as text.
The returned context should only be used for a single HTTP request/response.

OSSL_HTTP_REQ_CTX_free() frees up the HTTP request context I<rctx>.
The I<wbio> and I<rbio> are not free'd and it is up to the application
to do so.

OSSL_HTTP_REQ_CTX_set_request_line() adds the HTTP request line to the context.
The request method itself becomes C<GET> or C<POST> depending on the value
of I<method_POST> in the OSSL_HTTP_REQ_CTX_new() call.  I<server> and I<port>
may be set to indicate a proxy server and port that the request should go
through, otherwise they should be left NULL.  I<path> is the HTTP request path;
if left NULL, C</> is used.

OSSL_HTTP_REQ_CTX_add1_header() adds header I<name> with value I<value> to the
context I<rctx>. It can be called more than once to add multiple headers.
For example, to add a C<Host> header for C<example.com> you would call:

 OSSL_HTTP_REQ_CTX_add1_header(ctx, "Host", "example.com");

OSSL_HTTP_REQ_CTX_i2d() finalizes the HTTP request context by adding the DER
encoding of I<req>, using the ASN.1 template I<it> to do the encoding.  The
HTTP header C<Content-Length> is automatically filled out, and if
I<content_type> isn't NULL, the HTTP header C<Content-Type> is also added with
its content as value.  All of this ends up in the internal memory B<BIO>.
This requires that the request type be C<POST>,
i.e., that I<method_POST> is 1 in the OSSL_HTTP_REQ_CTX_new() call.

OSSL_HTTP_REQ_CTX_nbio() attempts the exchange of request and response via HTTP,
using the I<rbio> and I<wbio> that were given in the OSSL_HTTP_REQ_CTX_new()
call.  When successful, the contents of the internal memory B<BIO> is replaced
with the contents of the HTTP response, without the response headers.
It may need to be called again if its result is -1, which indicates
L<BIO_should_retry(3)>.  In such a case it is advisable to sleep a little in
between to prevent a busy loop.

OSSL_HTTP_REQ_CTX_sendreq_d2i() calls OSSL_HTTP_REQ_CTX_nbio(), possibly
several times until a timeout is reached, and DER decodes the received
response using the ASN.1 template I<it>.

OSSL_HTTP_REQ_CTX_set_max_response_length() sets the maximum response length
for I<rctx> to I<len>. If the response exceeds this length an error occurs.
If not set a default value of 100k is used.

OSSL_HTTP_REQ_CTX_get0_mem_bio() returns the internal memory B<BIO>.  This can
be used to affect the HTTP request text.  I<Use with caution!>

=head1 WARNINGS

The server's response may be unexpected if the hostname that was used to
create the I<wbio>, any C<Host> header, and the host specified in the
request URL do not match.

Many of these functions must be called in a certain order.

First, the HTTP request context must be allocated:
OSSL_HTTP_REQ_CTX_new().

Then, the HTTP request must be prepared with request data:

=over 4

=item 1.

Calling OSSL_HTTP_REQ_CTX_set_request_line().  This must be done exactly once.

=item 2.

Adding extra headers with OSSL_HTTP_REQ_CTX_add1_header().  This is optional.

=item 3.

Add C<POST> data with OSSL_HTTP_REQ_CTX_i2d().  This may only be done if
I<method_POST> was 1 in the OSSL_HTTP_REQ_CTX_new() call, and must be done
exactly once in that case.

=back

When the request context is fully prepared, the HTTP exchange may be performed
with OSSL_HTTP_REQ_CTX_nbio() or OSSL_HTTP_REQ_CTX_sendreq_d2i().

Furthermore, all calls of OSSL_HTTP_REQ_CTX_set_request_line() and
OSSL_HTTP_REQ_CTX_add1_header() must be done before any call to
int OSSL_HTTP_REQ_CTX_nbio() or OSSL_HTTP_REQ_CTX_sendreq_d2i().

=head1 RETURN VALUES

OSSL_HTTP_REQ_CTX_new() returns a pointer to a B<OSSL_HTTP_REQ_CTX>, or NULL
on error.
<commit_msg>OSSL_HTTP_REQ_CTX.pod: minor addition and remove redundant paragraph<commit_after>=pod

=head1 NAME

OSSL_HTTP_REQ_CTX,
OSSL_HTTP_REQ_CTX_new,
OSSL_HTTP_REQ_CTX_free,
OSSL_HTTP_REQ_CTX_set_request_line,
OSSL_HTTP_REQ_CTX_add1_header,
OSSL_HTTP_REQ_CTX_i2d,
OSSL_HTTP_REQ_CTX_nbio,
OSSL_HTTP_REQ_CTX_sendreq_d2i,
OSSL_HTTP_REQ_CTX_get0_mem_bio,
OSSL_HTTP_REQ_CTX_set_max_response_length
- HTTP request functions

=head1 SYNOPSIS

 #include <openssl/http.h>

 typedef struct ossl_http_req_ctx_st OSSL_HTTP_REQ_CTX;

 OSSL_HTTP_REQ_CTX *OSSL_HTTP_REQ_CTX_new(BIO *wbio, BIO *rbio,
                                          int method_POST, int maxline,
                                          unsigned long max_resp_len,
                                          int timeout,
                                          const char *expected_content_type,
                                          int expect_asn1);
 void OSSL_HTTP_REQ_CTX_free(OSSL_HTTP_REQ_CTX *rctx);

 int OSSL_HTTP_REQ_CTX_set_request_line(OSSL_HTTP_REQ_CTX *rctx,
                                        const char *server, const char *port,
                                        const char *path);
 int OSSL_HTTP_REQ_CTX_add1_header(OSSL_HTTP_REQ_CTX *rctx,
                                   const char *name, const char *value);

 int OSSL_HTTP_REQ_CTX_i2d(OSSL_HTTP_REQ_CTX *rctx, const char *content_type,
                           const ASN1_ITEM *it, ASN1_VALUE *req);
 int OSSL_HTTP_REQ_CTX_nbio(OSSL_HTTP_REQ_CTX *rctx);
 ASN1_VALUE *OSSL_HTTP_REQ_CTX_sendreq_d2i(OSSL_HTTP_REQ_CTX *rctx,
                                           const ASN1_ITEM *it);

 BIO *OSSL_HTTP_REQ_CTX_get0_mem_bio(OSSL_HTTP_REQ_CTX *rctx);
 void OSSL_HTTP_REQ_CTX_set_max_response_length(OSSL_HTTP_REQ_CTX *rctx,
                                                unsigned long len);

=head1 DESCRIPTION

B<OSSL_HTTP_REQ_CTX> is a context structure for an HTTP request, used to
collect all the necessary data to perform that request.

This file documents low-level HTTP functions rarely used directly.  High-level
HTTP client functions like L<OSSL_HTTP_get(3)> and L<OSSL_HTTP_transfer(3)>
should be preferred.

OSSL_HTTP_REQ_CTX_new() allocates a new HTTP request context structure,
which gets populated with the B<BIO> to send the request to (I<wbio>),
the B<BIO> to read the response from (I<rbio>, which may be equal to I<wbio>),
the request method (I<method_POST>, which may be 1 to indicate that the C<POST>
method is to be used, or 0 to indicate that the C<GET> method is to be used),
the maximum expected response header length (I<max_resp_len>,
where any zero or less indicates the default of 4KiB),
a response timeout measure in seconds (I<timeout>,
where 0 indicates no timeout, i.e., waiting indefinitely),
the expected MIME content type of the response (I<expected_content_type>,
which may be NULL for no expectation),
and a flag indicating that the response is expected to be
a DER encoded ASN.1 structure (I<expect_asn1>).
The allocated context structure is also populated with an internal allocated
memory B<BIO>, which collects the HTTP request and additional headers as text.
The returned context should only be used for a single HTTP request/response.

OSSL_HTTP_REQ_CTX_free() frees up the HTTP request context I<rctx>.
The I<wbio> and I<rbio> are not free'd and it is up to the application
to do so.

OSSL_HTTP_REQ_CTX_set_request_line() adds the HTTP request line to the context.
The request method itself becomes C<GET> or C<POST> depending on the value
of I<method_POST> in the OSSL_HTTP_REQ_CTX_new() call.  I<server> and I<port>
may be set to indicate a proxy server and port that the request should go
through, otherwise they should be left NULL.  I<path> is the HTTP request path;
if left NULL, C</> is used.

OSSL_HTTP_REQ_CTX_add1_header() adds header I<name> with value I<value> to the
context I<rctx>. It can be called more than once to add multiple headers.
For example, to add a C<Host> header for C<example.com> you would call:

 OSSL_HTTP_REQ_CTX_add1_header(ctx, "Host", "example.com");

OSSL_HTTP_REQ_CTX_i2d() finalizes the HTTP request context by adding the DER
encoding of I<req>, using the ASN.1 template I<it> to do the encoding.  The
HTTP header C<Content-Length> is automatically filled out, and if
I<content_type> isn't NULL, the HTTP header C<Content-Type> is also added with
its content as value.  All of this ends up in the internal memory B<BIO>.
This requires that the request type be C<POST>,
i.e., that I<method_POST> is 1 in the OSSL_HTTP_REQ_CTX_new() call.

OSSL_HTTP_REQ_CTX_nbio() attempts the exchange of request and response via HTTP,
using the I<rbio> and I<wbio> that were given in the OSSL_HTTP_REQ_CTX_new()
call.  When successful, the contents of the internal memory B<BIO> is replaced
with the contents of the HTTP response, without the response headers.
It may need to be called again if its result is -1, which indicates
L<BIO_should_retry(3)>.  In such a case it is advisable to sleep a little in
between to prevent a busy loop.

OSSL_HTTP_REQ_CTX_sendreq_d2i() calls OSSL_HTTP_REQ_CTX_nbio(), possibly
several times until a timeout is reached, and DER decodes the received
response using the ASN.1 template I<it>.

OSSL_HTTP_REQ_CTX_set_max_response_length() sets the maximum response length
for I<rctx> to I<len>. If the response exceeds this length an error occurs.
If not set a default value of 100k is used.

OSSL_HTTP_REQ_CTX_get0_mem_bio() returns the internal memory B<BIO>.  This can
be used to affect the HTTP request text.  I<Use with caution!>

=head1 WARNINGS

The server's response may be unexpected if the hostname that was used to
create the I<wbio>, any C<Host> header, and the host specified in the
request URL do not match.

Many of these functions must be called in a certain order.

First, the HTTP request context must be allocated:
OSSL_HTTP_REQ_CTX_new().

Then, the HTTP request must be prepared with request data:

=over 4

=item 1.

Calling OSSL_HTTP_REQ_CTX_set_request_line().  This must be done exactly once.

=item 2.

Adding extra headers with OSSL_HTTP_REQ_CTX_add1_header().
This is optional and may be done multiple times with different names.

=item 3.

Add C<POST> data with OSSL_HTTP_REQ_CTX_i2d().  This may only be done if
I<method_POST> was 1 in the OSSL_HTTP_REQ_CTX_new() call, and must be done
exactly once in that case.

=back

When the request context is fully prepared, the HTTP exchange may be performed
with OSSL_HTTP_REQ_CTX_nbio() or OSSL_HTTP_REQ_CTX_sendreq_d2i().

=head1 RETURN VALUES

OSSL_HTTP_REQ_CTX_new() returns a pointer to a B<OSSL_HTTP_REQ_CTX>, or NULL
on error.

{'fraction_non_alphanumeric': 0.05732626707471822, 'fraction_numerical': 0.007165783384339777, 'mean_word_length': 3.8578680203045685, 'pattern_counts': {'":': 0, '<': 111, '<?xml version=': 0, '>': 111, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 34, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5692547', 'n_tokens_mistral': 4259, 'n_tokens_neox': 4157, 'n_words': 1621}