UNLOZE/sm1.13-ripext
9
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

first commit
CI / Build (clang, clang++, ubuntu-22.04, linux, 3.10, master, x86,x86_64) (push) Canceled after 0s
Details
CI / Build (clang, clang++, windows-2022, windows, 3.10, master, x86,x86_64) (push) Canceled after 0s
Details
CI / Release (push) Canceled after 0s
Details

Browse Source
This commit is contained in:
jenz
2026-09-13 22:18:11 +01:00
commit 08c03ac533
6701 changed files with 1577314 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nghttp2/doc/sources/building-android-binary.rst
+155
View File
@@ -0,0 +1,155 @@
Building Android binary
=======================
In this article, we briefly describe how to build Android binary using
`Android NDK <https://developer.android.com/ndk/index.html>`_
cross-compiler on Debian Linux.
The easiest way to build android binary is use Dockerfile.android.
See Dockerfile.android for more details. If you cannot use
Dockerfile.android for whatever reason, continue to read the rest of
this article.
We offer ``android-config`` and ``android-make`` scripts to make the
build easier. To make these script work, NDK toolchain must be
installed in the following way. First, let us introduce
``ANDROID_HOME`` environment variable. We need to install toolchain
under ``$ANDROID_HOME/toolchain``. An user can freely choose the path
for ``ANDROID_HOME``. For example, to install toolchain under
``$ANDROID_HOME/toolchain``, do this in the the directory where NDK is
unpacked:
.. code-block:: text
$ build/tools/make_standalone_toolchain.py \
--arch arm --api 16 --stl gnustl \
--install-dir $ANDROID_HOME/toolchain
The API level (``--api``) is not important here because we don't use
Android specific C/C++ API.
The dependent libraries, such as OpenSSL, libev, and c-ares should be
built with the toolchain and installed under
``$ANDROID_HOME/usr/local``. We recommend to build these libraries as
static library to make the deployment easier. libxml2 support is
currently disabled.
Although zlib comes with Android NDK, it seems not to be a part of
public API, so we have to built it for our own. That also provides us
proper .pc file as a bonus.
Before running ``android-config`` and ``android-make``,
``ANDROID_HOME`` environment variable must be set to point to the
correct path. Also add ``$ANDROID_HOME/toolchain/bin`` to ``PATH``:
.. code-block:: text
$ export PATH=$PATH:$ANDROID_HOME/toolchain/bin
To configure OpenSSL, use the following script:
.. code-block:: sh
#!/bin/sh
if [ -z "$ANDROID_HOME" ]; then
echo 'No $ANDROID_HOME specified.'
exit 1
fi
PREFIX=$ANDROID_HOME/usr/local
TOOLCHAIN=$ANDROID_HOME/toolchain
PATH=$TOOLCHAIN/bin:$PATH
export CROSS_COMPILE=$TOOLCHAIN/bin/arm-linux-androideabi-
./Configure --prefix=$PREFIX android
And run ``make install_sw`` to build and install without
documentation.
We cannot compile libev without modification. Apply `this patch
<https://gist.github.com/tatsuhiro-t/48c45f08950f587180ed>`_ before
configuring libev. This patch is for libev-4.19. After applying the
patch, to configure libev, use the following script:
.. code-block:: sh
#!/bin/sh
if [ -z "$ANDROID_HOME" ]; then
echo 'No $ANDROID_HOME specified.'
exit 1
fi
PREFIX=$ANDROID_HOME/usr/local
TOOLCHAIN=$ANDROID_HOME/toolchain
PATH=$TOOLCHAIN/bin:$PATH
./configure \
--host=arm-linux-androideabi \
--build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \
--prefix=$PREFIX \
--disable-shared \
--enable-static \
CPPFLAGS=-I$PREFIX/include \
LDFLAGS=-L$PREFIX/lib
And run ``make install`` to build and install.
To configure c-ares, use the following script:
.. code-block:: sh
#!/bin/sh -e
if [ -z "$ANDROID_HOME" ]; then
echo 'No $ANDROID_HOME specified.'
exit 1
fi
PREFIX=$ANDROID_HOME/usr/local
TOOLCHAIN=$ANDROID_HOME/toolchain
PATH=$TOOLCHAIN/bin:$PATH
./configure \
--host=arm-linux-androideabi \
--build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \
--prefix=$PREFIX \
--disable-shared
To configure zlib, use the following script:
.. code-block:: sh
#!/bin/sh -e
if [ -z "$ANDROID_HOME" ]; then
echo 'No $ANDROID_HOME specified.'
exit 1
fi
PREFIX=$ANDROID_HOME/usr/local
TOOLCHAIN=$ANDROID_HOME/toolchain
PATH=$TOOLCHAIN/bin:$PATH
HOST=arm-linux-androideabi
CC=$HOST-gcc \
AR=$HOST-ar \
LD=$HOST-ld \
RANLIB=$HOST-ranlib \
STRIP=$HOST-strip \
./configure \
--prefix=$PREFIX \
--libdir=$PREFIX/lib \
--includedir=$PREFIX/include \
--static
And run ``make install`` to build and install.
After prerequisite libraries are prepared, run ``android-config`` and
then ``android-make`` to compile nghttp2 source files.
If all went well, application binaries, such as nghttpx, are created
under src directory. Strip debugging information from the binary
using the following command:
.. code-block:: text
$ arm-linux-androideabi-strip src/nghttpx
nghttp2/doc/sources/contribute.rst
+56
View File
@@ -0,0 +1,56 @@
Contribution Guidelines
=======================
[This text was composed based on 1.2. License section of curl/libcurl
project.]
When contributing with code, you agree to put your changes and new
code under the same license nghttp2 is already using unless stated and
agreed otherwise.
When changing existing source code, you do not alter the copyright of
the original file(s). The copyright will still be owned by the
original creator(s) or those who have been assigned copyright by the
original author(s).
By submitting a patch to the nghttp2 project, you are assumed to have
the right to the code and to be allowed by your employer or whatever
to hand over that patch/code to us. We will credit you for your
changes as far as possible, to give credit but also to keep a trace
back to who made what changes. Please always provide us with your
full real name when contributing!
Coding style
------------
We use clang-format to format source code consistently. The
clang-format configuration file .clang-format is located at the root
directory. Since clang-format produces slightly different results
between versions, we currently use clang-format-10.
To detect any violation to the coding style, we recommend to setup git
pre-commit hook to check coding style of the changes you introduced.
The pre-commit file is located at the root directory. Copy it under
.git/hooks and make sure that it is executable. The pre-commit script
uses clang-format-diff.py to detect any style errors. If it is not in
your PATH or it exists under different name (e.g.,
clang-format-diff-10 in debian), either add it to PATH variable or add
git option ``clangformatdiff.binary`` to point to the script.
For emacs users, integrating clang-format to emacs is very easy.
clang-format.el should come with clang distribution. If it is not
found, download it from `here
<https://llvm.org/svn/llvm-project/cfe/trunk/tools/clang-format/clang-format.el>`_.
And add these lines to your .emacs file:
.. code-block:: lisp
;; From
;; https://code.google.com/p/chromium/wiki/Emacs#Use_Google's_C++_style!
(load "/<path/to>/clang-format.el")
(add-hook 'c-mode-common-hook
(function (lambda () (local-set-key (kbd "TAB")
'clang-format-region))))
You can find other editor integration in
http://clang.llvm.org/docs/ClangFormat.html.
nghttp2/doc/sources/h2load-howto.rst
+133
View File
@@ -0,0 +1,133 @@
.. program:: h2load
h2load - HTTP/2 benchmarking tool - HOW-TO
==========================================
:doc:`h2load.1` is benchmarking tool for HTTP/2 and HTTP/1.1. It
supports SSL/TLS and clear text for all supported protocols.
Compiling from source
---------------------
h2load is compiled alongside nghttp2 and requires that the
``--enable-app`` flag is passed to ``./configure`` and `required
dependencies <https://github.com/nghttp2/nghttp2#requirements>`_ are
available during compilation. For details on compiling, see `nghttp2:
Building from Git
<https://github.com/nghttp2/nghttp2#building-from-git>`_.
Basic Usage
-----------
In order to set benchmark settings, specify following 3 options.
:option:`-n`
The number of total requests. Default: 1
:option:`-c`
The number of concurrent clients. Default: 1
:option:`-m`
The max concurrent streams to issue per client. Default: 1
For SSL/TLS connection, the protocol will be negotiated via ALPN/NPN.
You can set specific protocols in :option:`--npn-list` option. For
cleartext connection, the default protocol is HTTP/2. To change the
protocol in cleartext connection, use :option:`--no-tls-proto` option.
For convenience, :option:`--h1` option forces HTTP/1.1 for both
cleartext and SSL/TLS connections.
Here is a command-line to perform benchmark to URI \https://localhost
using total 100000 requests, 100 concurrent clients and 10 max
concurrent streams:
.. code-block:: text
$ h2load -n100000 -c100 -m10 https://localhost
The benchmarking result looks like this:
.. code-block:: text
finished in 7.08s, 141164.80 req/s, 555.33MB/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4125025824 bytes total, 11023424 bytes headers (space savings 93.07%), 4096000000 bytes data
min max mean sd +/- sd
time for request: 15.31ms 146.85ms 69.78ms 9.26ms 92.43%
time for connect: 1.08ms 25.04ms 10.71ms 9.80ms 64.00%
time to 1st byte: 25.36ms 184.96ms 79.11ms 53.97ms 78.00%
req/s (client) : 1412.04 1447.84 1426.52 10.57 63.00%
See the h2load manual page :ref:`h2load-1-output` section for the
explanation of the above numbers.
Timing-based load-testing
-------------------------
As of v1.26.0, h2load supports timing-based load-testing. This method
performs load-testing in terms of a given duration instead of a
pre-defined number of requests. The new option :option:`--duration`
specifies how long the load-testing takes. For example,
``--duration=10`` makes h2load perform load-testing against a server
for 10 seconds. You can also specify a “warming-up” period with
:option:`--warm-up-time`. If :option:`--duration` is used,
:option:`-n` option is ignored.
The following command performs load-testing for 10 seconds after 5
seconds warming up period:
.. code-block:: text
$ h2load -c100 -m100 --duration=10 --warm-up-time=5 https://localhost
Flow Control
------------
HTTP/2 has flow control and it may affect benchmarking results. By
default, h2load uses large enough flow control window, which
effectively disables flow control. To adjust receiver flow control
window size, there are following options:
:option:`-w`
Sets the stream level initial window size to
(2**<N>)-1.
:option:`-W`
Sets the connection level initial window size to
(2**<N>)-1.
Multi-Threading
---------------
Sometimes benchmarking client itself becomes a bottleneck. To remedy
this situation, use :option:`-t` option to specify the number of native
thread to use.
:option:`-t`
The number of native threads. Default: 1
Selecting protocol for clear text
---------------------------------
By default, if \http:// URI is given, HTTP/2 protocol is used. To
change the protocol to use for clear text, use :option:`-p` option.
Multiple URIs
-------------
If multiple URIs are specified, they are used in round robin manner.
.. note::
Please note that h2load uses scheme, host and port in the first URI
and ignores those parts in the rest of the URIs.
UNIX domain socket
------------------
To request against UNIX domain socket, use :option:`--base-uri`, and
specify ``unix:`` followed by the path to UNIX domain socket. For
example, if UNIX domain socket is ``/tmp/nghttpx.sock``, use
``--base-uri=unix:/tmp/nghttpx.sock``. h2load uses scheme, host and
port in the first URI in command-line or input file.
nghttp2/doc/sources/index.rst
+55
View File
@@ -0,0 +1,55 @@
.. nghttp2 documentation master file, created by
sphinx-quickstart on Sun Mar 11 22:57:49 2012.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
nghttp2 - HTTP/2 C Library
============================
This is an implementation of Hypertext Transfer Protocol version 2.
The project is hosted at `github.com/nghttp2/nghttp2
<https://github.com/nghttp2/nghttp2>`_.
Contents:
.. toctree::
:maxdepth: 2
package_README
contribute
security
building-android-binary
tutorial-client
tutorial-server
tutorial-hpack
nghttp.1
nghttpd.1
nghttpx.1
h2load.1
nghttpx-howto
h2load-howto
programmers-guide
apiref
libnghttp2_asio
python-apiref
nghttp2.h
nghttp2ver.h
asio_http2_server.h
asio_http2_client.h
asio_http2.h
Source <https://github.com/nghttp2/nghttp2>
Issues <https://github.com/nghttp2/nghttp2/issues>
nghttp2.org <https://nghttp2.org/>
Released Versions
=================
https://github.com/nghttp2/nghttp2/releases
Resources
---------
* HTTP/2 https://tools.ietf.org/html/rfc7540
* HPACK https://tools.ietf.org/html/rfc7541
* HTTP Alternative Services https://tools.ietf.org/html/rfc7838
nghttp2/doc/sources/libnghttp2_asio.rst
+433
View File
@@ -0,0 +1,433 @@
libnghttp2_asio: High level HTTP/2 C++ library
==============================================
libnghttp2_asio is C++ library built on top of libnghttp2 and provides
high level abstraction API to build HTTP/2 applications. It depends
on Boost::ASIO library and OpenSSL. Currently libnghttp2_asio
provides server and client side API.
libnghttp2_asio is not built by default. Use ``--enable-asio-lib``
configure flag to build libnghttp2_asio. The required Boost libraries
are:
* Boost::Asio
* Boost::System
* Boost::Thread
We have 3 header files for this library:
* :doc:`asio_http2_server.h`
* :doc:`asio_http2_client.h`
* :doc:`asio_http2.h`
asio_http2.h is included from the other two files.
To build a program with libnghttp2_asio, link to the following
libraries::
-lnghttp2_asio -lboost_system
If ``boost::asio::ssl`` is used in application code, OpenSSL is also
required in link line::
-lnghttp2_asio -lboost_system -lssl -lcrypto
Server API
----------
To use server API, first include following header file:
.. code-block:: cpp
#include <nghttp2/asio_http2_server.h>
Also take a look at that header file :doc:`asio_http2_server.h`.
Server API is designed to build HTTP/2 server very easily to utilize
C++11 anonymous function and closure. The bare minimum example of
HTTP/2 server looks like this:
.. code-block:: cpp
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::server;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
http2 server;
server.handle("/", [](const request &req, const response &res) {
res.write_head(200);
res.end("hello, world\n");
});
if (server.listen_and_serve(ec, "localhost", "3000")) {
std::cerr << "error: " << ec.message() << std::endl;
}
}
First we instantiate ``nghttp2::asio_http2::server::http2`` object.
``nghttp2::asio_http2::server::http2::handle`` function registers
pattern and its handler function. In this example, we register "/" as
pattern, which matches all requests. Then call
``nghttp2::asio_http2::server::http2::listen_and_serve`` function with
address and port to listen to.
The ``req`` and ``res`` represent HTTP request and response
respectively. ``nghttp2::asio_http2_::server::response::write_head``
constructs HTTP response header fields. The first argument is HTTP
status code, in the above example, which is 200. The second argument,
which is omitted in the above example, is additional header fields to
send.
``nghttp2::asio_http2::server::response::end`` sends response body.
In the above example, we send string "hello, world".
The life time of req and res object ends after the callback set by
``nghttp2::asio_http2::server::response::on_close`` function.
Application must not use those objects after this call.
Serving static files and enabling SSL/TLS
+++++++++++++++++++++++++++++++++++++++++
In this example, we serve a couple of static files and also enable
SSL/TLS.
.. code-block:: cpp
#include <nghttp2/asio_http2_server.h>
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::server;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
boost::asio::ssl::context tls(boost::asio::ssl::context::sslv23);
tls.use_private_key_file("server.key", boost::asio::ssl::context::pem);
tls.use_certificate_chain_file("server.crt");
configure_tls_context_easy(ec, tls);
http2 server;
server.handle("/index.html", [](const request &req, const response &res) {
res.write_head(200);
res.end(file_generator("index.html"));
});
if (server.listen_and_serve(ec, tls, "localhost", "3000")) {
std::cerr << "error: " << ec.message() << std::endl;
}
}
We first create ``boost::asio::ssl::context`` object and set path to
private key file and certificate file.
``nghttp2::asio_http2::server::configure_tls_context_easy`` function
configures SSL/TLS context object for HTTP/2 server use, including NPN
callbacks.
In the above example, if request path is "/index.html", we serve
index.html file in the current working directory.
``nghttp2::asio_http2::server::response::end`` has overload to take
function of type ``nghttp2::asio_http2::generator_cb`` and application
pass its implementation to generate response body. For the
convenience, libnghttp2_asio library provides
``nghttp2::asio_http2::file_generator`` function to generate function
to server static file. If other resource is requested, server
automatically responds with 404 status code.
Server push
+++++++++++
Server push is also supported.
.. code-block:: cpp
#include <nghttp2/asio_http2_server.h>
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::server;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
boost::asio::ssl::context tls(boost::asio::ssl::context::sslv23);
tls.use_private_key_file("server.key", boost::asio::ssl::context::pem);
tls.use_certificate_chain_file("server.crt");
configure_tls_context_easy(ec, tls);
http2 server;
std::string style_css = "h1 { color: green; }";
server.handle("/", [&style_css](const request &req, const response &res) {
boost::system::error_code ec;
auto push = res.push(ec, "GET", "/style.css");
push->write_head(200);
push->end(style_css);
res.write_head(200);
res.end(R"(
<!DOCTYPE html><html lang="en">
<title>HTTP/2 FTW</title><body>
<link href="/style.css" rel="stylesheet" type="text/css">
<h1>This should be green</h1>
</body></html>
)");
});
server.handle("/style.css",
[&style_css](const request &req, const response &res) {
res.write_head(200);
res.end(style_css);
});
if (server.listen_and_serve(ec, tls, "localhost", "3000")) {
std::cerr << "error: " << ec.message() << std::endl;
}
}
When client requested any resource other than "/style.css", we push
"/style.css". To push resource, call
``nghttp2::asio_http2::server::response::push`` function with desired
method and path. It returns another response object and use its
functions to send push response.
Enable multi-threading
++++++++++++++++++++++
Enabling multi-threading is very easy. Just call
``nghttp2::asio_http2::server::http2::num_threads`` function with the
desired number of threads:
.. code-block:: cpp
http2 server;
// Use 4 native threads
server.num_threads(4);
Client API
----------
To use client API, first include following header file:
.. code-block:: cpp
#include <nghttp2/asio_http2_client.h>
Also take a look at that header file :doc:`asio_http2_client.h`.
Here is the sample client code to access HTTP/2 server and print out
response header fields and response body to the console screen:
.. code-block:: cpp
#include <iostream>
#include <nghttp2/asio_http2_client.h>
using boost::asio::ip::tcp;
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::client;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
boost::asio::io_service io_service;
// connect to localhost:3000
session sess(io_service, "localhost", "3000");
sess.on_connect([&sess](tcp::resolver::iterator endpoint_it) {
boost::system::error_code ec;
auto req = sess.submit(ec, "GET", "http://localhost:3000/");
req->on_response([](const response &res) {
// print status code and response header fields.
std::cerr << "HTTP/2 " << res.status_code() << std::endl;
for (auto &kv : res.header()) {
std::cerr << kv.first << ": " << kv.second.value << "\n";
}
std::cerr << std::endl;
res.on_data([](const uint8_t *data, std::size_t len) {
std::cerr.write(reinterpret_cast<const char *>(data), len);
std::cerr << std::endl;
});
});
req->on_close([&sess](uint32_t error_code) {
// shutdown session after first request was done.
sess.shutdown();
});
});
sess.on_error([](const boost::system::error_code &ec) {
std::cerr << "error: " << ec.message() << std::endl;
});
io_service.run();
}
``nghttp2::asio_http2::client::session`` object takes
``boost::asio::io_service`` object and remote server address. When
connection is made, the callback function passed to
``nghttp2::asio_http2::client::on_connect`` is invoked with connected
address as its parameter. After this callback call, use
``nghttp2::asio_http2::session::submit`` to send request to the
server. You can submit multiple requests at once without waiting for
the completion of previous request.
The life time of req and res object ends after the callback set by
``nghttp2::asio_http2::server::request::on_close`` function.
Application must not use those objects after this call.
Normally, client does not stop even after all requests are done unless
connection is lost. To stop client, call
``nghttp2::asio_http2::server::session::shutdown()``.
Receive server push and enable SSL/TLS
++++++++++++++++++++++++++++++++++++++
.. code-block:: cpp
#include <iostream>
#include <nghttp2/asio_http2_client.h>
using boost::asio::ip::tcp;
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::client;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
boost::asio::io_service io_service;
boost::asio::ssl::context tls(boost::asio::ssl::context::sslv23);
tls.set_default_verify_paths();
// disabled to make development easier...
// tls_ctx.set_verify_mode(boost::asio::ssl::verify_peer);
configure_tls_context(ec, tls);
// connect to localhost:3000
session sess(io_service, tls, "localhost", "3000");
sess.on_connect([&sess](tcp::resolver::iterator endpoint_it) {
boost::system::error_code ec;
auto req = sess.submit(ec, "GET", "http://localhost:3000/");
req->on_response([&sess](const response &res) {
std::cerr << "response received!" << std::endl;
res.on_data([&sess](const uint8_t *data, std::size_t len) {
std::cerr.write(reinterpret_cast<const char *>(data), len);
std::cerr << std::endl;
});
});
req->on_push([](const request &push) {
std::cerr << "push request received!" << std::endl;
push.on_response([](const response &res) {
std::cerr << "push response received!" << std::endl;
res.on_data([](const uint8_t *data, std::size_t len) {
std::cerr.write(reinterpret_cast<const char *>(data), len);
std::cerr << std::endl;
});
});
});
});
sess.on_error([](const boost::system::error_code &ec) {
std::cerr << "error: " << ec.message() << std::endl;
});
io_service.run();
}
The above sample code demonstrates how to enable SSL/TLS and receive
server push. Currently,
``nghttp2::asio_http2::client::configure_tls_context`` function setups
NPN callbacks for SSL/TLS context for HTTP/2 use.
To receive server push, use
``nghttp2::asio_http2::client::request::on_push`` function to set
callback function which is invoked when server push request is
arrived. The callback function takes
``nghttp2::asio_http2::client::request`` object, which contains the
pushed request. To get server push response, set callback using
``nghttp2::asio_http2::client::request::on_response``.
As stated in the previous section, client does not stop automatically
as long as HTTP/2 session is fine and connection is alive. We don't
call ``nghttp2::asio_http2::client::session::shutdown`` in this
example, so the program does not terminate after all responses are
received. Hit Ctrl-C to terminate the program.
Multiple concurrent requests
++++++++++++++++++++++++++++
.. code-block:: cpp
#include <iostream>
#include <nghttp2/asio_http2_client.h>
using boost::asio::ip::tcp;
using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::client;
int main(int argc, char *argv[]) {
boost::system::error_code ec;
boost::asio::io_service io_service;
// connect to localhost:3000
session sess(io_service, "localhost", "3000");
sess.on_connect([&sess](tcp::resolver::iterator endpoint_it) {
boost::system::error_code ec;
auto printer = [](const response &res) {
res.on_data([](const uint8_t *data, std::size_t len) {
std::cerr.write(reinterpret_cast<const char *>(data), len);
std::cerr << std::endl;
});
};
std::size_t num = 3;
auto count = std::make_shared<int>(num);
for (std::size_t i = 0; i < num; ++i) {
auto req = sess.submit(ec, "GET",
"http://localhost:3000/" + std::to_string(i + 1));
req->on_response(printer);
req->on_close([&sess, count](uint32_t error_code) {
if (--*count == 0) {
// shutdown session after |num| requests were done.
sess.shutdown();
}
});
}
});
sess.on_error([](const boost::system::error_code &ec) {
std::cerr << "error: " << ec.message() << std::endl;
});
io_service.run();
}
Here is the sample to send 3 requests at once. Depending on the
server settings, these requests are processed out-of-order. In this
example, we have a trick to shutdown session after all requests were
done. We made ``count`` object which is shared pointer to int and is
initialized to 3. On each request closure (the invocation of the
callback set by ``nghttp2::asio_http2::client::request::on_close``),
we decrement the count. If count becomes 0, we are sure that all
requests have been done and initiate shutdown.
nghttp2/doc/sources/nghttpx-howto.rst
+581
View File
@@ -0,0 +1,581 @@
.. program:: nghttpx
nghttpx - HTTP/2 proxy - HOW-TO
===============================
:doc:`nghttpx.1` is a proxy translating protocols between HTTP/2 and
other protocols (e.g., HTTP/1). It operates in several modes and each
mode may require additional programs to work with. This article
describes each operation mode and explains the intended use-cases. It
also covers some useful options later.
Default mode
------------
If nghttpx is invoked without :option:`--http2-proxy`, it operates in
default mode. In this mode, it works as reverse proxy (gateway) for
both HTTP/2 and HTTP/1 clients to backend servers. This is also known
as "HTTP/2 router".
By default, frontend connection is encrypted using SSL/TLS. So
server's private key and certificate must be supplied to the command
line (or through configuration file). In this case, the frontend
protocol selection will be done via ALPN or NPN.
To turn off encryption on frontend connection, use ``no-tls`` keyword
in :option:`--frontend` option. HTTP/2 and HTTP/1 are available on
the frontend, and an HTTP/1 connection can be upgraded to HTTP/2 using
HTTP Upgrade. Starting HTTP/2 connection by sending HTTP/2 connection
preface is also supported.
nghttpx can listen on multiple frontend addresses. This is achieved
by using multiple :option:`--frontend` options. For each frontend
address, TLS can be enabled or disabled.
By default, backend connections are not encrypted. To enable TLS
encryption on backend connections, use ``tls`` keyword in
:option:`--backend` option. Using patterns and ``proto`` keyword in
:option:`--backend` option, backend application protocol can be
specified per host/request path pattern. It means that you can use
both HTTP/2 and HTTP/1 in backend connections at the same time. Note
that default backend protocol is HTTP/1.1. To use HTTP/2 in backend,
you have to specify ``h2`` in ``proto`` keyword in :option:`--backend`
explicitly.
The backend is supposed to be a Web server. For example, to make
nghttpx listen to encrypted HTTP/2 requests at port 8443, and a
backend Web server is configured to listen to HTTP requests at port
8080 on the same host, run nghttpx command-line like this:
.. code-block:: text
$ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
Then an HTTP/2 enabled client can access the nghttpx server using HTTP/2. For
example, you can send a GET request using nghttp:
.. code-block:: text
$ nghttp -nv https://localhost:8443/
HTTP/2 proxy mode
-----------------
If nghttpx is invoked with :option:`--http2-proxy` (or its shorthand
:option:`-s`) option, it operates in HTTP/2 proxy mode. The supported
protocols in frontend and backend connections are the same as in `default
mode`_. The difference is that this mode acts like a forward proxy and
assumes the backend is an HTTP proxy server (e.g., Squid, Apache Traffic
Server). HTTP/1 requests must include an absolute URI in request line.
By default, the frontend connection is encrypted. So this mode is
also called secure proxy.
To turn off encryption on the frontend connection, use ``no-tls`` keyword
in :option:`--frontend` option.
The backend must be an HTTP proxy server. nghttpx supports multiple
backend server addresses. It translates incoming requests to HTTP
request to backend server. The backend server performs real proxy
work for each request, for example, dispatching requests to the origin
server and caching contents.
The backend connection is not encrypted by default. To enable
encryption, use ``tls`` keyword in :option:`--backend` option. The
default backend protocol is HTTP/1.1. To use HTTP/2 in backend
connection, use :option:`--backend` option, and specify ``h2`` in
``proto`` keyword explicitly.
For example, to make nghttpx listen to encrypted HTTP/2 requests at
port 8443, and a backend HTTP proxy server is configured to listen to
HTTP/1 requests at port 8080 on the same host, run nghttpx command-line
like this:
.. code-block:: text
$ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
At the time of this writing, Firefox 41 and Chromium v46 can use
nghttpx as HTTP/2 proxy.
To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has to
create proxy.pac script file like this:
.. code-block:: javascript
function FindProxyForURL(url, host) {
return "HTTPS SERVERADDR:PORT";
}
``SERVERADDR`` and ``PORT`` is the hostname/address and port of the
machine nghttpx is running. Please note that both Firefox and
Chromium require valid certificate for secure proxy.
For Firefox, open Preference window and select Advanced then click
Network tab. Clicking Connection Settings button will show the
dialog. Select "Automatic proxy configuration URL" and enter the path
to proxy.pac file, something like this:
.. code-block:: text
file:///path/to/proxy.pac
For Chromium, use following command-line:
.. code-block:: text
$ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
As HTTP/1 proxy server, Squid may work as out-of-box. Traffic server
requires to be configured as forward proxy. Here is the minimum
configuration items to edit:
.. code-block:: text
CONFIG proxy.config.reverse_proxy.enabled INT 0
CONFIG proxy.config.url_remap.remap_required INT 0
Consult Traffic server `documentation
<http://trafficserver.readthedocs.org/en/latest/admin-guide/configuration/transparent-forward-proxying.en.html>`_
to know how to configure traffic server as forward proxy and its
security implications.
ALPN support
------------
ALPN support requires OpenSSL >= 1.0.2.
Disable frontend SSL/TLS
------------------------
The frontend connections are encrypted with SSL/TLS by default. To
turn off SSL/TLS, use ``no-tls`` keyword in :option:`--frontend`
option. If this option is used, the private key and certificate are
not required to run nghttpx.
Enable backend SSL/TLS
----------------------
The backend connections are not encrypted by default. To enable
SSL/TLS encryption, use ``tls`` keyword in :option:`--backend` option.
Enable SSL/TLS on memcached connection
--------------------------------------
By default, memcached connection is not encrypted. To enable
encryption, use ``tls`` keyword in
:option:`--tls-ticket-key-memcached` for TLS ticket key, and
:option:`--tls-session-cache-memcached` for TLS session cache.
Specifying additional server certificates
-----------------------------------------
nghttpx accepts additional server private key and certificate pairs
using :option:`--subcert` option. It can be used multiple times.
Specifying additional CA certificate
------------------------------------
By default, nghttpx tries to read CA certificate from system. But
depending on the system you use, this may fail or is not supported.
To specify CA certificate manually, use :option:`--cacert` option.
The specified file must be PEM format and can contain multiple
certificates.
By default, nghttpx validates server's certificate. If you want to
turn off this validation, knowing this is really insecure and what you
are doing, you can use :option:`--insecure` option to disable
certificate validation.
Read/write rate limit
---------------------
nghttpx supports transfer rate limiting on frontend connections. You
can do rate limit per frontend connection for reading and writing
individually.
To perform rate limit for reading, use :option:`--read-rate` and
:option:`--read-burst` options. For writing, use
:option:`--write-rate` and :option:`--write-burst`.
Please note that rate limit is performed on top of TCP and nothing to
do with HTTP/2 flow control.
Rewriting location header field
-------------------------------
nghttpx automatically rewrites location response header field if the
following all conditions satisfy:
* In the default mode (:option:`--http2-proxy` is not used)
* :option:`--no-location-rewrite` is not used
* URI in location header field is an absolute URI
* URI in location header field includes non empty host component.
* host (without port) in URI in location header field must match the
host appearing in ``:authority`` or ``host`` header field.
When rewrite happens, URI scheme is replaced with the ones used in
frontend, and authority is replaced with which appears in
``:authority``, or ``host`` request header field. ``:authority``
header field has precedence over ``host``.
Hot swapping
------------
nghttpx supports hot swapping using signals. The hot swapping in
nghttpx is multi step process. First send USR2 signal to nghttpx
process. It will do fork and execute new executable, using same
command-line arguments and environment variables.
As of nghttpx version 1.20.0, that is all you have to do. The new
master process sends QUIT signal to the original process, when it is
ready to serve requests, to shut it down gracefully.
For earlier versions of nghttpx, you have to do one more thing. At
this point, both current and new processes can accept requests. To
gracefully shutdown current process, send QUIT signal to current
nghttpx process. When all existing frontend connections are done, the
current process will exit. At this point, only new nghttpx process
exists and serves incoming requests.
If you want to just reload configuration file without executing new
binary, send SIGHUP to nghttpx master process.
Re-opening log files
--------------------
When rotating log files, it is desirable to re-open log files after
log rotation daemon renamed existing log files. To tell nghttpx to
re-open log files, send USR1 signal to nghttpx process. It will
re-open files specified by :option:`--accesslog-file` and
:option:`--errorlog-file` options.
Multiple frontend addresses
---------------------------
nghttpx can listen on multiple frontend addresses. To specify them,
just use :option:`--frontend` (or its shorthand :option:`-f`) option
repeatedly. TLS can be enabled or disabled per frontend address
basis. For example, to listen on port 443 with TLS enabled, and on
port 80 without TLS:
.. code-block:: text
frontend=*,443
frontend=*,80;no-tls
Multiple backend addresses
--------------------------
nghttpx supports multiple backend addresses. To specify them, just
use :option:`--backend` (or its shorthand :option:`-b`) option
repeatedly. For example, to use ``192.168.0.10:8080`` and
``192.168.0.11:8080``, use command-line like this:
``-b192.168.0.10,8080 -b192.168.0.11,8080``. In configuration file,
this looks like:
.. code-block:: text
backend=192.168.0.10,8080
backend=192.168.0.11,8008
nghttpx can route request to different backend according to request
host and path. For example, to route request destined to host
``doc.example.com`` to backend server ``docserv:3000``, you can write
like so:
.. code-block:: text
backend=docserv,3000;doc.example.com/
When you write this option in command-line, you should enclose
argument with single or double quotes, since the character ``;`` has a
special meaning in shell.
To route, request to request path ``/foo`` to backend server
``[::1]:8080``, you can write like so:
.. code-block:: text
backend=::1,8080;/foo
If the last character of path pattern is ``/``, all request paths
which start with that pattern match:
.. code-block:: text
backend=::1,8080;/bar/
The request path ``/bar/buzz`` matches the ``/bar/``.
You can use ``*`` at the end of the path pattern to make it wildcard
pattern. ``*`` must match at least one character:
.. code-block:: text
backend=::1,8080;/sample*
The request path ``/sample1/foo`` matches the ``/sample*`` pattern.
Of course, you can specify both host and request path at the same
time:
.. code-block:: text
backend=192.168.0.10,8080;example.com/foo
We can use ``*`` in the left most position of host to achieve wildcard
suffix match. If ``*`` is the left most character, then the remaining
string should match the request host suffix. ``*`` must match at
least one character. For example, ``*.example.com`` matches
``www.example.com`` and ``dev.example.com``, and does not match
``example.com`` and ``nghttp2.org``. The exact match (without ``*``)
always takes precedence over wildcard match.
One important thing you have to remember is that we have to specify
default routing pattern for so called "catch all" pattern. To write
"catch all" pattern, just specify backend server address, without
pattern.
Usually, host is the value of ``Host`` header field. In HTTP/2, the
value of ``:authority`` pseudo header field is used.
When you write multiple backend addresses sharing the same routing
pattern, they are used as load balancing. For example, to use 2
servers ``serv1:3000`` and ``serv2:3000`` for request host
``example.com`` and path ``/myservice``, you can write like so:
.. code-block:: text
backend=serv1,3000;example.com/myservice
backend=serv2,3000;example.com/myservice
You can also specify backend application protocol in
:option:`--backend` option using ``proto`` keyword after pattern.
Utilizing this allows ngttpx to route certain request to HTTP/2, other
requests to HTTP/1. For example, to route requests to ``/ws/`` in
backend HTTP/1.1 connection, and use backend HTTP/2 for other
requests, do this:
.. code-block:: text
backend=serv1,3000;/;proto=h2
backend=serv1,3000;/ws/;proto=http/1.1
The default backend protocol is HTTP/1.1.
TLS can be enabled per pattern basis:
.. code-block:: text
backend=serv1,8443;/;proto=h2;tls
backend=serv2,8080;/ws/;proto=http/1.1
In the above case, connection to serv1 will be encrypted by TLS. On
the other hand, connection to serv2 will not be encrypted by TLS.
Dynamic hostname lookup
-----------------------
By default, nghttpx performs backend hostname lookup at start up, or
configuration reload, and keeps using them in its entire session. To
make nghttpx perform hostname lookup dynamically, use ``dns``
parameter in :option:`--backend` option, like so:
.. code-block:: text
backend=foo.example.com,80;;dns
nghttpx will cache resolved addresses for certain period of time. To
change this cache period, use :option:`--dns-cache-timeout`.
Enable PROXY protocol
---------------------
PROXY protocol can be enabled per frontend. In order to enable PROXY
protocol, use ``proxyproto`` parameter in :option:`--frontend` option,
like so:
.. code-block:: text
frontend=*,443;proxyproto
nghttpx supports both PROXY protocol v1 and v2. AF_UNIX in PROXY
protocol version 2 is ignored.
Session affinity
----------------
Two kinds of session affinity are available: client IP, and HTTP
Cookie.
To enable client IP based affinity, specify ``affinity=ip`` parameter
in :option:`--backend` option. If PROXY protocol is enabled, then an
address obtained from PROXY protocol is taken into consideration.
To enable HTTP Cookie based affinity, specify ``affinity=cookie``
parameter, and specify a name of cookie in ``affinity-cookie-name``
parameter. Optionally, a Path attribute can be specified in
``affinity-cookie-path`` parameter:
.. code-block:: text
backend=127.0.0.1,3000;;affinity=cookie;affinity-cookie-name=nghttpxlb;affinity-cookie-path=/
Secure attribute of cookie is set if client connection is protected by
TLS.
PSK cipher suites
-----------------
nghttpx supports pre-shared key (PSK) cipher suites for both frontend
and backend TLS connections. For frontend connection, use
:option:`--psk-secrets` option to specify a file which contains PSK
identity and secrets. The format of the file is
``<identity>:<hex-secret>``, where ``<identity>`` is PSK identity, and
``<hex-secret>`` is PSK secret in hex, like so:
.. code-block:: text
client1:9567800e065e078085c241d54a01c6c3f24b3bab71a606600f4c6ad2c134f3b9
client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99
nghttpx server accepts any of the identity and secret pairs in the
file. The default cipher suite list does not contain PSK cipher
suites. In order to use PSK, PSK cipher suite must be enabled by
using :option:`--ciphers` option. The desired PSK cipher suite may be
listed in `HTTP/2 cipher black list
<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use
such PSK cipher suite with HTTP/2, disable HTTP/2 cipher black list by
using :option:`--no-http2-cipher-black-list` option. But you should
understand its implications.
At the time of writing, even if only PSK cipher suites are specified
in :option:`--ciphers` option, certificate and private key are still
required.
For backend connection, use :option:`--client-psk-secrets` option to
specify a file which contains single PSK identity and secret. The
format is the same as the file used by :option:`--psk-secrets`
described above, but only first identity and secret pair is solely
used, like so:
.. code-block:: text
client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99
The default cipher suite list does not contain PSK cipher suites. In
order to use PSK, PSK cipher suite must be enabled by using
:option:`--client-ciphers` option. The desired PSK cipher suite may
be listed in `HTTP/2 cipher black list
<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use
such PSK cipher suite with HTTP/2, disable HTTP/2 cipher black list by
using :option:`--client-no-http2-cipher-black-list` option. But you
should understand its implications.
TLSv1.3
-------
As of nghttpx v1.34.0, if it is built with OpenSSL 1.1.1 or later, it
supports TLSv1.3. 0-RTT data is supported, but by default its
processing is postponed until TLS handshake completes to mitigate
replay attack. This costs extra round trip and reduces effectiveness
of 0-RTT data. :option:`--tls-no-postpone-early-data` makes nghttpx
not wait for handshake to complete before forwarding request included
in 0-RTT to get full potential of 0-RTT data. In this case, nghttpx
adds ``Early-Data: 1`` header field when forwarding a request to a
backend server. All backend servers should recognize this header
field and understand that there is a risk for replay attack. See `RFC
8470 <https://tools.ietf.org/html/rfc8470>`_ for ``Early-Data`` header
field.
nghttpx disables anti replay protection provided by OpenSSL. The anti
replay protection of OpenSSL requires that a resumed request must hit
the same server which generates the session ticket. Therefore it
might not work nicely in a deployment where there are multiple nghttpx
instances sharing ticket encryption keys via memcached.
Because TLSv1.3 completely changes the semantics of cipher suite
naming scheme and structure, nghttpx provides the new option
:option:`--tls13-ciphers` and :option:`--tls13-client-ciphers` to
change preferred cipher list for TLSv1.3.
WebSockets over HTTP/2
----------------------
nghttpx supports `RFC 8441 <https://tools.ietf.org/html/rfc8441>`_
Bootstrapping WebSockets with HTTP/2 for both frontend and backend
connections. This feature is enabled by default and no configuration
is required.
Migration from nghttpx v1.18.x or earlier
-----------------------------------------
As of nghttpx v1.19.0, :option:`--ciphers` option only changes cipher
list for frontend TLS connection. In order to change cipher list for
backend connection, use :option:`--client-ciphers` option.
Similarly, :option:`--no-http2-cipher-black-list` option only disables
HTTP/2 cipher black list for frontend connection. In order to disable
HTTP/2 cipher black list for backend connection, use
:option:`--client-no-http2-cipher-black-list` option.
``--accept-proxy-protocol`` option was deprecated. Instead, use
``proxyproto`` parameter in :option:`--frontend` option to enable
PROXY protocol support per frontend.
Migration from nghttpx v1.8.0 or earlier
----------------------------------------
As of nghttpx 1.9.0, ``--frontend-no-tls`` and ``--backend-no-tls``
have been removed.
To disable encryption on frontend connection, use ``no-tls`` keyword
in :option:`--frontend` potion:
.. code-block:: text
frontend=*,3000;no-tls
The TLS encryption is now disabled on backend connection in all modes
by default. To enable encryption on backend connection, use ``tls``
keyword in :option:`--backend` option:
.. code-block:: text
backend=127.0.0.1,8080;tls
As of nghttpx 1.9.0, ``--http2-bridge``, ``--client`` and
``--client-proxy`` options have been removed. These functionality can
be used using combinations of options.
Use following option instead of ``--http2-bridge``:
.. code-block:: text
backend=<ADDR>,<PORT>;;proto=h2;tls
Use following options instead of ``--client``:
.. code-block:: text
frontend=<ADDR>,<PORT>;no-tls
backend=<ADDR>,<PORT>;;proto=h2;tls
Use following options instead of ``--client-proxy``:
.. code-block:: text
http2-proxy=yes
frontend=<ADDR>,<PORT>;no-tls
backend=<ADDR>,<PORT>;;proto=h2;tls
We also removed ``--backend-http2-connections-per-worker`` option. It
was present because previously the number of backend h2 connection was
statically configured, and defaulted to 1. Now the number of backend
h2 connection is increased on demand. We know the maximum number of
concurrent streams per connection. When we push as many request as
the maximum concurrency to the one connection, we create another new
connection so that we can distribute load and avoid delay the request
processing. This is done automatically without any configuration.
nghttp2/doc/sources/python-apiref.rst
+439
View File
@@ -0,0 +1,439 @@
Python API Reference
====================
.. py:module:: nghttp2
nghttp2 offers some high level Python API to C library. The bindings
currently provide HPACK compressor and decompressor classes and HTTP/2
server class.
The extension module is called ``nghttp2``.
``make`` will build the bindings. The target Python version is
determined by configure script. If the detected Python version is not
what you expect, specify a path to Python executable in ``PYTHON``
variable as an argument to configure script (e.g., ``./configure
PYTHON=/usr/bin/python3.8``).
HPACK API
---------
.. py:class:: HDDeflater(hd_table_bufsize_max=DEFLATE_MAX_HEADER_TABLE_SIZE)
This class is used to perform header compression. The
*hd_table_bufsize_max* limits the usage of header table in the
given amount of bytes. The default value is
:py:data:`DEFLATE_MAX_HEADER_TABLE_SIZE`. This is necessary
because the deflater and inflater share the same amount of header
table and the inflater decides that number. The deflater may not
want to use all header table size because of limited memory
availability. In that case, *hd_table_bufsize_max* can be used to
cap the upper limit of table size whatever the header table size is
chosen by the inflater.
.. py:method:: deflate(headers)
Deflates the *headers*. The *headers* must be sequence of tuple
of name/value pair, which are byte strings (not unicode string).
This method returns the deflated header block in byte string.
Raises the exception if any error occurs.
.. py:method:: set_no_refset(no_refset)
Tells the deflater not to use reference set if *no_refset* is
evaluated to ``True``. If that happens, on each subsequent
invocation of :py:meth:`deflate()`, deflater will clear up
refersent set.
.. py:method:: change_table_size(hd_table_bufsize_max)
Changes header table size to *hd_table_bufsize_max* byte. if
*hd_table_bufsize_max* is strictly larger than
``hd_table_bufsize_max`` given in constructor,
``hd_table_bufsize_max`` is used as header table size instead.
Raises the exception if any error occurs.
.. py:method:: get_hd_table()
Returns copy of current dynamic header table.
The following example shows how to deflate header name/value pairs:
.. code-block:: python
import binascii, nghttp2
deflater = nghttp2.HDDeflater()
res = deflater.deflate([(b'foo', b'bar'),
(b'baz', b'buz')])
print(binascii.b2a_hex(res))
.. py:class:: HDInflater()
This class is used to perform header decompression.
.. py:method:: inflate(data)
Inflates the deflated header block *data*. The *data* must be
byte string.
Raises the exception if any error occurs.
.. py:method:: change_table_size(hd_table_bufsize_max)
Changes header table size to *hd_table_bufsize_max* byte.
Raises the exception if any error occurs.
.. py:method:: get_hd_table()
Returns copy of current dynamic header table.
The following example shows how to inflate deflated header block:
.. code-block:: python
deflater = nghttp2.HDDeflater()
data = deflater.deflate([(b'foo', b'bar'),
(b'baz', b'buz')])
inflater = nghttp2.HDInflater()
hdrs = inflater.inflate(data)
print(hdrs)
.. py:function:: print_hd_table(hdtable)
Convenient function to print *hdtable* to the standard output. The
*hdtable* is the one retrieved by
:py:meth:`HDDeflater.get_hd_table()` or
:py:meth:`HDInflater.get_hd_table()`. This function does not work
if header name/value cannot be decoded using UTF-8 encoding.
In output, ``s=N`` means the entry occupies ``N`` bytes in header
table. If ``r=y``, then the entry is in the reference set.
.. py:data:: DEFAULT_HEADER_TABLE_SIZE
The default header table size, which is 4096 as per HTTP/2
specification.
.. py:data:: DEFLATE_MAX_HEADER_TABLE_SIZE
The default header table size for deflater. The initial value
is 4096.
HTTP/2 servers
--------------
.. note::
We use :py:mod:`asyncio` for HTTP/2 server classes, and ALPN.
Therefore, Python 3.8 or later is required to use these objects.
To explicitly configure nghttp2 build to use Python 3.8, specify
the ``PYTHON`` variable to the path to Python 3.8 executable when
invoking configure script like this:
.. code-block:: text
$ ./configure PYTHON=/usr/bin/python3.8
.. py:class:: HTTP2Server(address, RequestHandlerClass, ssl=None)
This class builds on top of the :py:mod:`asyncio` event loop. On
construction, *RequestHandlerClass* must be given, which must be a
subclass of :py:class:`BaseRequestHandler` class.
The *address* must be a tuple of hostname/IP address and port to
bind. If hostname/IP address is ``None``, all interfaces are
assumed.
To enable SSL/TLS, specify instance of :py:class:`ssl.SSLContext`
in *ssl*. Before passing *ssl* to
:py:func:`BaseEventLoop.create_server`, ALPN protocol identifiers
are set using :py:meth:`ssl.SSLContext.set_npn_protocols`.
To disable SSL/TLS, omit *ssl* or specify ``None``.
.. py:method:: serve_forever()
Runs server and processes incoming requests forever.
.. py:class:: BaseRequestHandler(http2, stream_id)
The class is used to handle the single HTTP/2 stream. By default,
it does not nothing. It must be subclassed to handle each event
callback method.
The first callback method invoked is :py:meth:`on_headers()`. It is
called when HEADERS frame, which includes request header fields, is
arrived.
If request has request body, :py:meth:`on_data()` is invoked for
each chunk of received data chunk.
When whole request is received, :py:meth:`on_request_done()` is
invoked.
When stream is closed, :py:meth:`on_close()` is called.
The application can send response using :py:meth:`send_response()`
method. It can be used in :py:meth:`on_headers()`,
:py:meth:`on_data()` or :py:meth:`on_request_done()`.
The application can push resource using :py:meth:`push()` method.
It must be used before :py:meth:`send_response()` call.
A :py:class:`BaseRequestHandler` has the following instance
variables:
.. py:attribute:: client_address
Contains a tuple of the form ``(host, port)`` referring to the
client's address.
.. py:attribute:: stream_id
Stream ID of this stream
.. py:attribute:: scheme
Scheme of the request URI. This is a value of ``:scheme``
header field.
.. py:attribute:: method
Method of this stream. This is a value of ``:method`` header
field.
.. py:attribute:: host
This is a value of ``:authority`` or ``host`` header field.
.. py:attribute:: path
This is a value of ``:path`` header field.
.. py:attribute:: headers
Request header fields.
A :py:class:`BaseRequestHandler` has the following methods:
.. py:method:: on_headers()
Called when request HEADERS is arrived. By default, this method
does nothing.
.. py:method:: on_data(data)
Called when a chunk of request body *data* is arrived. This
method will be called multiple times until all data are
received. By default, this method does nothing.
.. py:method:: on_request_done()
Called when whole request was received. By default, this method
does nothing.
.. py:method:: on_close(error_code)
Called when stream is about to close. The *error_code*
indicates the reason of closure. If it is ``0``, the stream is
going to close without error.
.. py:method:: send_response(status=200, headers=None, body=None)
Send response. The *status* is HTTP status code. The *headers*
is additional response headers. The *:status* header field will
be appended by the library. The *body* is the response body.
It could be ``None`` if response body is empty. Or it must be
instance of either ``str``, ``bytes``, :py:class:`io.IOBase` or
callable, called body generator, which takes one parameter,
size. The body generator generates response body. It can pause
generation of response so that it can wait for slow backend data
generation. When invoked, it should return tuple, byte string
at most size length and flag. The flag is either
:py:data:`DATA_OK`, :py:data:`DATA_EOF` or
:py:data:`DATA_DEFERRED`. For non-empty byte string and it is
not the last chunk of response, :py:data:`DATA_OK` must be
returned as flag. If this is the last chunk of the response
(byte string could be ``None``), :py:data:`DATA_EOF` must be
returned as flag. If there is no data available right now, but
additional data are anticipated, return tuple (``None``,
:py:data:`DATA_DEFERRED`). When data arrived, call
:py:meth:`resume()` and restart response body transmission.
Only the body generator can pause response body generation;
instance of :py:class:`io.IOBase` must not block.
If instance of ``str`` is specified as *body*, it will be
encoded using UTF-8.
The *headers* is a list of tuple of the form ``(name,
value)``. The ``name`` and ``value`` can be either byte string
or unicode string. In the latter case, they will be encoded
using UTF-8.
Raises the exception if any error occurs.
.. py:method:: push(path, method='GET', request_headers=None, status=200, headers=None, body=None)
Push a specified resource. The *path* is a path portion of
request URI for this resource. The *method* is a method to
access this resource. The *request_headers* is additional
request headers to access this resource. The ``:scheme``,
``:method``, ``:authority`` and ``:path`` are appended by the
library. The ``:scheme`` and ``:authority`` are inherited from
request header fields of the associated stream.
The *status* is HTTP status code. The *headers* is additional
response headers. The ``:status`` header field is appended by
the library. The *body* is the response body. It has the same
semantics of *body* parameter of :py:meth:`send_response()`.
The headers and request_headers are a list of tuple of the form
``(name, value)``. The ``name`` and ``value`` can be either byte
string or unicode string. In the latter case, they will be
encoded using UTF-8.
Returns an instance of ``RequestHandlerClass`` specified in
:py:class:`HTTP2Server` constructor for the pushed resource.
Raises the exception if any error occurs.
.. py:method:: resume()
Signals the restarting of response body transmission paused by
``DATA_DEFERRED`` from the body generator (see
:py:meth:`send_response()` about the body generator). It is not
an error calling this method while response body transmission is
not paused.
.. py:data:: DATA_OK
``DATA_OK`` indicates non empty data is generated from body generator.
.. py:data:: DATA_EOF
``DATA_EOF`` indicates the end of response body.
.. py:data:: DATA_DEFERRED
``DATA_DEFERRED`` indicates that data are not available right now
and response should be paused.
The following example illustrates :py:class:`HTTP2Server` and
:py:class:`BaseRequestHandler` usage:
.. code-block:: python
#!/usr/bin/env python3
import io, ssl
import nghttp2
class Handler(nghttp2.BaseRequestHandler):
def on_headers(self):
self.push(path='/css/style.css',
request_headers = [('content-type', 'text/css')],
status=200,
body='body{margin:0;}')
self.send_response(status=200,
headers = [('content-type', 'text/plain')],
body=io.BytesIO(b'nghttp2-python FTW'))
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ctx.options = ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
ctx.load_cert_chain('server.crt', 'server.key')
# give None to ssl to make the server non-SSL/TLS
server = nghttp2.HTTP2Server(('127.0.0.1', 8443), Handler, ssl=ctx)
server.serve_forever()
The following example illustrates HTTP/2 server using asynchronous
response body generation. This is simplified reverse proxy:
.. code-block:: python
#!/usr/bin/env python3
import ssl
import os
import urllib
import asyncio
import io
import nghttp2
@asyncio.coroutine
def get_http_header(handler, url):
url = urllib.parse.urlsplit(url)
ssl = url.scheme == 'https'
if url.port == None:
if url.scheme == 'https':
port = 443
else:
port = 80
else:
port = url.port
connect = asyncio.open_connection(url.hostname, port, ssl=ssl)
reader, writer = yield from connect
req = 'GET {path} HTTP/1.0\r\n\r\n'.format(path=url.path or '/')
writer.write(req.encode('utf-8'))
# skip response header fields
while True:
line = yield from reader.readline()
line = line.rstrip()
if not line:
break
# read body
while True:
b = yield from reader.read(4096)
if not b:
break
handler.buf.write(b)
writer.close()
handler.buf.seek(0)
handler.eof = True
handler.resume()
class Body:
def __init__(self, handler):
self.handler = handler
self.handler.eof = False
self.handler.buf = io.BytesIO()
def generate(self, n):
buf = self.handler.buf
data = buf.read1(n)
if not data and not self.handler.eof:
return None, nghttp2.DATA_DEFERRED
return data, nghttp2.DATA_EOF if self.handler.eof else nghttp2.DATA_OK
class Handler(nghttp2.BaseRequestHandler):
def on_headers(self):
body = Body(self)
asyncio.async(get_http_header(
self, 'http://localhost' + self.path.decode('utf-8')))
self.send_response(status=200, body=body.generate)
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ctx.options = ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
ctx.load_cert_chain('server.crt', 'server.key')
server = nghttp2.HTTP2Server(('127.0.0.1', 8443), Handler, ssl=ctx)
server.serve_forever()
nghttp2/doc/sources/security.rst
+38
View File
@@ -0,0 +1,38 @@
Security Process
================
If you find a vulnerability in our software, please send the email to
"tatsuhiro.t at gmail dot com" about its details instead of submitting
issues on github issue page. It is a standard practice not to
disclose vulnerability information publicly until a fixed version is
released, or mitigation is worked out. In the future, we may setup a
dedicated mail address for this purpose.
If we identify that the reported issue is really a vulnerability, we
open a new security advisory draft using `GitHub security feature
<https://github.com/nghttp2/nghttp2/security>`_ and discuss the
mitigation and bug fixes there. The fixes are committed to the
private repository.
We write the security advisory and get CVE number from GitHub
privately. We also discuss the disclosure date to the public.
We make a new release with the fix at the same time when the
vulnerability is disclosed to public.
At least 7 days before the public disclosure date, we will post
security advisory (which includes all the details of the vulnerability
and the possible mitigation strategies) and the patches to fix the
issue to `distros@openwall
<https://oss-security.openwall.org/wiki/mailing-lists/distros>`_
mailing list. We also open a new issue on `nghttp2 issue tracker
<https://github.com/nghttp2/nghttp2/issues>`_ which notifies that the
upcoming release will have a security fix. The ``SECURITY`` label is
attached to this kind of issue.
Before few hours of new release, we merge the fixes to the master
branch (and/or a release branch if necessary) and make a new release.
Security advisory is disclosed on GitHub. We also post the
vulnerability information to `oss-security
<https://oss-security.openwall.org/wiki/mailing-lists/oss-security>`_
mailing list.
nghttp2/doc/sources/tutorial-client.rst
+498
View File
@@ -0,0 +1,498 @@
Tutorial: HTTP/2 client
=========================
In this tutorial, we are going to write a very primitive HTTP/2
client. The complete source code, `libevent-client.c`_, is attached at
the end of this page. It also resides in the examples directory in
the archive or repository.
This simple client takes a single HTTPS URI and retrieves the resource
at the URI. The synopsis is:
.. code-block:: text
$ libevent-client HTTPS_URI
We use libevent in this tutorial to handle networking I/O. Please
note that nghttp2 itself does not depend on libevent.
The client starts with some libevent and OpenSSL setup in the
``main()`` and ``run()`` functions. This setup isn't specific to
nghttp2, but one thing you should look at is setup of the NPN
callback. The NPN callback is used by the client to select the next
application protocol over TLS. In this tutorial, we use the
`nghttp2_select_next_protocol()` helper function to select the HTTP/2
protocol the library supports::
static int select_next_proto_cb(SSL *ssl _U_, unsigned char **out,
unsigned char *outlen, const unsigned char *in,
unsigned int inlen, void *arg _U_) {
if (nghttp2_select_next_protocol(out, outlen, in, inlen) <= 0) {
errx(1, "Server did not advertise " NGHTTP2_PROTO_VERSION_ID);
}
return SSL_TLSEXT_ERR_OK;
}
If you are following TLS related RFC, you know that NPN is not the
standardized way to negotiate HTTP/2. NPN itself is not event
published as RFC. The standard way to negotiate HTTP/2 is ALPN,
Application-Layer Protocol Negotiation Extension, defined in `RFC 7301
<https://tools.ietf.org/html/rfc7301>`_. The one caveat of ALPN is
that OpenSSL >= 1.0.2 is required. We use macro to enable/disable
ALPN support depending on OpenSSL version. OpenSSL's ALPN
implementation does not require callback function like the above. But
we have to instruct OpenSSL SSL_CTX to use ALPN, which we'll talk
about soon.
The callback is added to the SSL_CTX object using
``SSL_CTX_set_next_proto_select_cb()``::
static SSL_CTX *create_ssl_ctx(void) {
SSL_CTX *ssl_ctx;
ssl_ctx = SSL_CTX_new(SSLv23_client_method());
if (!ssl_ctx) {
errx(1, "Could not create SSL/TLS context: %s",
ERR_error_string(ERR_get_error(), NULL));
}
SSL_CTX_set_options(ssl_ctx,
SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 |
SSL_OP_NO_COMPRESSION |
SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
SSL_CTX_set_next_proto_select_cb(ssl_ctx, select_next_proto_cb, NULL);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
SSL_CTX_set_alpn_protos(ssl_ctx, (const unsigned char *)"\x02h2", 3);
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
return ssl_ctx;
}
Here we see ``SSL_CTX_get_alpn_protos()`` function call. We instructs
OpenSSL to notify the server that we support h2, ALPN identifier for
HTTP/2.
The example client defines a couple of structs:
We define and use a ``http2_session_data`` structure to store data
related to the HTTP/2 session::
typedef struct {
nghttp2_session *session;
struct evdns_base *dnsbase;
struct bufferevent *bev;
http2_stream_data *stream_data;
} http2_session_data;
Since this program only handles one URI, it uses only one stream. We
store the single stream's data in a ``http2_stream_data`` structure
and the ``stream_data`` points to it. The ``http2_stream_data``
structure is defined as follows::
typedef struct {
/* The NULL-terminated URI string to retrieve. */
const char *uri;
/* Parsed result of the |uri| */
struct http_parser_url *u;
/* The authority portion of the |uri|, not NULL-terminated */
char *authority;
/* The path portion of the |uri|, including query, not
NULL-terminated */
char *path;
/* The length of the |authority| */
size_t authoritylen;
/* The length of the |path| */
size_t pathlen;
/* The stream ID of this stream */
int32_t stream_id;
} http2_stream_data;
We create and initialize these structures in
``create_http2_session_data()`` and ``create_http2_stream_data()``
respectively.
``initiate_connection()`` is called to start the connection to the
remote server. It's defined as::
static void initiate_connection(struct event_base *evbase, SSL_CTX *ssl_ctx,
const char *host, uint16_t port,
http2_session_data *session_data) {
int rv;
struct bufferevent *bev;
SSL *ssl;
ssl = create_ssl(ssl_ctx);
bev = bufferevent_openssl_socket_new(
evbase, -1, ssl, BUFFEREVENT_SSL_CONNECTING,
BEV_OPT_DEFER_CALLBACKS | BEV_OPT_CLOSE_ON_FREE);
bufferevent_enable(bev, EV_READ | EV_WRITE);
bufferevent_setcb(bev, readcb, writecb, eventcb, session_data);
rv = bufferevent_socket_connect_hostname(bev, session_data->dnsbase,
AF_UNSPEC, host, port);
if (rv != 0) {
errx(1, "Could not connect to the remote host %s", host);
}
session_data->bev = bev;
}
``initiate_connection()`` creates a bufferevent for the connection and
sets up three callbacks: ``readcb``, ``writecb``, and ``eventcb``.
The ``eventcb()`` is invoked by the libevent event loop when an event
(e.g. connection has been established, timeout, etc.) occurs on the
underlying network socket::
static void eventcb(struct bufferevent *bev, short events, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
if (events & BEV_EVENT_CONNECTED) {
int fd = bufferevent_getfd(bev);
int val = 1;
const unsigned char *alpn = NULL;
unsigned int alpnlen = 0;
SSL *ssl;
fprintf(stderr, "Connected\n");
ssl = bufferevent_openssl_get_ssl(session_data->bev);
SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
if (alpn == NULL) {
SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
}
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
fprintf(stderr, "h2 is not negotiated\n");
delete_http2_session_data(session_data);
return;
}
setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *)&val, sizeof(val));
initialize_nghttp2_session(session_data);
send_client_connection_header(session_data);
submit_request(session_data);
if (session_send(session_data) != 0) {
delete_http2_session_data(session_data);
}
return;
}
if (events & BEV_EVENT_EOF) {
warnx("Disconnected from the remote host");
} else if (events & BEV_EVENT_ERROR) {
warnx("Network error");
} else if (events & BEV_EVENT_TIMEOUT) {
warnx("Timeout");
}
delete_http2_session_data(session_data);
}
Here we validate that HTTP/2 is negotiated, and if not, drop
connection.
For ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and ``BEV_EVENT_TIMEOUT``
events, we just simply tear down the connection.
The ``BEV_EVENT_CONNECTED`` event is invoked when the SSL/TLS
handshake has completed successfully. After this we're ready to begin
communicating via HTTP/2.
The ``initialize_nghttp2_session()`` function initializes the nghttp2
session object and several callbacks::
static void initialize_nghttp2_session(http2_session_data *session_data) {
nghttp2_session_callbacks *callbacks;
nghttp2_session_callbacks_new(&callbacks);
nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks,
on_frame_recv_callback);
nghttp2_session_callbacks_set_on_data_chunk_recv_callback(
callbacks, on_data_chunk_recv_callback);
nghttp2_session_callbacks_set_on_stream_close_callback(
callbacks, on_stream_close_callback);
nghttp2_session_callbacks_set_on_header_callback(callbacks,
on_header_callback);
nghttp2_session_callbacks_set_on_begin_headers_callback(
callbacks, on_begin_headers_callback);
nghttp2_session_client_new(&session_data->session, callbacks, session_data);
nghttp2_session_callbacks_del(callbacks);
}
Since we are creating a client, we use `nghttp2_session_client_new()`
to initialize the nghttp2 session object. The callbacks setup are
explained later.
The `delete_http2_session_data()` function destroys ``session_data``
and frees its bufferevent, so the underlying connection is closed. It
also calls `nghttp2_session_del()` to delete the nghttp2 session
object.
A HTTP/2 connection begins by sending the client connection preface,
which is a 24 byte magic byte string (:macro:`NGHTTP2_CLIENT_MAGIC`),
followed by a SETTINGS frame. The 24 byte magic string is sent
automatically by nghttp2. We send the SETTINGS frame in
``send_client_connection_header()``::
static void send_client_connection_header(http2_session_data *session_data) {
nghttp2_settings_entry iv[1] = {
{NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}};
int rv;
/* client 24 bytes magic string will be sent by nghttp2 library */
rv = nghttp2_submit_settings(session_data->session, NGHTTP2_FLAG_NONE, iv,
ARRLEN(iv));
if (rv != 0) {
errx(1, "Could not submit SETTINGS: %s", nghttp2_strerror(rv));
}
}
Here we specify SETTINGS_MAX_CONCURRENT_STREAMS as 100. This is not
needed for this tiny example program, it just demonstrates use of the
SETTINGS frame. To queue the SETTINGS frame for transmission, we call
`nghttp2_submit_settings()`. Note that `nghttp2_submit_settings()`
only queues the frame for transmission, and doesn't actually send it.
All ``nghttp2_submit_*()`` family functions have this property. To
actually send the frame, `nghttp2_session_send()` has to be called,
which is described (and called) later.
After the transmission of the client connection header, we enqueue the
HTTP request in the ``submit_request()`` function::
static void submit_request(http2_session_data *session_data) {
int32_t stream_id;
http2_stream_data *stream_data = session_data->stream_data;
const char *uri = stream_data->uri;
const struct http_parser_url *u = stream_data->u;
nghttp2_nv hdrs[] = {
MAKE_NV2(":method", "GET"),
MAKE_NV(":scheme", &uri[u->field_data[UF_SCHEMA].off],
u->field_data[UF_SCHEMA].len),
MAKE_NV(":authority", stream_data->authority, stream_data->authoritylen),
MAKE_NV(":path", stream_data->path, stream_data->pathlen)};
fprintf(stderr, "Request headers:\n");
print_headers(stderr, hdrs, ARRLEN(hdrs));
stream_id = nghttp2_submit_request(session_data->session, NULL, hdrs,
ARRLEN(hdrs), NULL, stream_data);
if (stream_id < 0) {
errx(1, "Could not submit HTTP request: %s", nghttp2_strerror(stream_id));
}
stream_data->stream_id = stream_id;
}
We build the HTTP request header fields in ``hdrs``, which is an array
of :type:`nghttp2_nv`. There are four header fields to be sent:
``:method``, ``:scheme``, ``:authority``, and ``:path``. To queue the
HTTP request, we call `nghttp2_submit_request()`. The ``stream_data``
is passed via the *stream_user_data* parameter, which is helpfully
later passed back to callback functions.
`nghttp2_submit_request()` returns the newly assigned stream ID for
the request.
The next bufferevent callback is ``readcb()``, which is invoked when
data is available to read from the bufferevent input buffer::
static void readcb(struct bufferevent *bev, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
ssize_t readlen;
struct evbuffer *input = bufferevent_get_input(bev);
size_t datalen = evbuffer_get_length(input);
unsigned char *data = evbuffer_pullup(input, -1);
readlen = nghttp2_session_mem_recv(session_data->session, data, datalen);
if (readlen < 0) {
warnx("Fatal error: %s", nghttp2_strerror((int)readlen));
delete_http2_session_data(session_data);
return;
}
if (evbuffer_drain(input, (size_t)readlen) != 0) {
warnx("Fatal error: evbuffer_drain failed");
delete_http2_session_data(session_data);
return;
}
if (session_send(session_data) != 0) {
delete_http2_session_data(session_data);
return;
}
}
In this function we feed all unprocessed, received data to the nghttp2
session object using the `nghttp2_session_mem_recv()` function.
`nghttp2_session_mem_recv()` processes the received data and may
invoke nghttp2 callbacks and queue frames for transmission. Since
there may be pending frames for transmission, we call immediately
``session_send()`` to send them. ``session_send()`` is defined as
follows::
static int session_send(http2_session_data *session_data) {
int rv;
rv = nghttp2_session_send(session_data->session);
if (rv != 0) {
warnx("Fatal error: %s", nghttp2_strerror(rv));
return -1;
}
return 0;
}
The `nghttp2_session_send()` function serializes pending frames into
wire format and calls the ``send_callback()`` function to send them.
``send_callback()`` has type :type:`nghttp2_send_callback` and is
defined as::
static ssize_t send_callback(nghttp2_session *session _U_, const uint8_t *data,
size_t length, int flags _U_, void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
struct bufferevent *bev = session_data->bev;
bufferevent_write(bev, data, length);
return (ssize_t)length;
}
Since we use bufferevent to abstract network I/O, we just write the
data to the bufferevent object. Note that `nghttp2_session_send()`
continues to write all frames queued so far. If we were writing the
data to the non-blocking socket directly using the ``write()`` system
call, we'd soon receive an ``EAGAIN`` or ``EWOULDBLOCK`` error, since
sockets have a limited send buffer. If that happens, it's possible to
return :macro:`NGHTTP2_ERR_WOULDBLOCK` to signal the nghttp2 library
to stop sending further data. When writing to a bufferevent, you
should regulate the amount of data written, to avoid possible huge
memory consumption. In this example client however we don't implement
a limit. To see how to regulate the amount of buffered data, see the
``send_callback()`` in the server tutorial.
The third bufferevent callback is ``writecb()``, which is invoked when
all data written in the bufferevent output buffer has been sent::
static void writecb(struct bufferevent *bev _U_, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
if (nghttp2_session_want_read(session_data->session) == 0 &&
nghttp2_session_want_write(session_data->session) == 0 &&
evbuffer_get_length(bufferevent_get_output(session_data->bev)) == 0) {
delete_http2_session_data(session_data);
}
}
As described earlier, we just write off all data in `send_callback()`,
so there is no data to write in this function. All we have to do is
check if the connection should be dropped or not. The nghttp2 session
object keeps track of reception and transmission of GOAWAY frames and
other error conditions. Using this information, the nghttp2 session
object can state whether the connection should be dropped or not.
More specifically, when both `nghttp2_session_want_read()` and
`nghttp2_session_want_write()` return 0, the connection is no-longer
required and can be closed. Since we're using bufferevent and its
deferred callback option, the bufferevent output buffer may still
contain pending data when the ``writecb()`` is called. To handle this
situation, we also check whether the output buffer is empty or not. If
all of these conditions are met, then we drop the connection.
Now let's look at the remaining nghttp2 callbacks setup in the
``initialize_nghttp2_setup()`` function.
A server responds to the request by first sending a HEADERS frame.
The HEADERS frame consists of response header name/value pairs, and
the ``on_header_callback()`` is called for each name/value pair::
static int on_header_callback(nghttp2_session *session _U_,
const nghttp2_frame *frame, const uint8_t *name,
size_t namelen, const uint8_t *value,
size_t valuelen, uint8_t flags _U_,
void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
switch (frame->hd.type) {
case NGHTTP2_HEADERS:
if (frame->headers.cat == NGHTTP2_HCAT_RESPONSE &&
session_data->stream_data->stream_id == frame->hd.stream_id) {
/* Print response headers for the initiated request. */
print_header(stderr, name, namelen, value, valuelen);
break;
}
}
return 0;
}
In this tutorial, we just print the name/value pairs on stderr.
After the HEADERS frame has been fully received (and thus all response
header name/value pairs have been received), the
``on_frame_recv_callback()`` function is called::
static int on_frame_recv_callback(nghttp2_session *session _U_,
const nghttp2_frame *frame, void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
switch (frame->hd.type) {
case NGHTTP2_HEADERS:
if (frame->headers.cat == NGHTTP2_HCAT_RESPONSE &&
session_data->stream_data->stream_id == frame->hd.stream_id) {
fprintf(stderr, "All headers received\n");
}
break;
}
return 0;
}
``on_frame_recv_callback()`` is called for other frame types too.
In this tutorial, we are just interested in the HTTP response HEADERS
frame. We check the frame type and its category (it should be
:macro:`NGHTTP2_HCAT_RESPONSE` for HTTP response HEADERS). We also
check its stream ID.
Next, zero or more DATA frames can be received. The
``on_data_chunk_recv_callback()`` function is invoked when a chunk of
data is received from the remote peer::
static int on_data_chunk_recv_callback(nghttp2_session *session _U_,
uint8_t flags _U_, int32_t stream_id,
const uint8_t *data, size_t len,
void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
if (session_data->stream_data->stream_id == stream_id) {
fwrite(data, len, 1, stdout);
}
return 0;
}
In our case, a chunk of data is HTTP response body. After checking the
stream ID, we just write the received data to stdout. Note the output
in the terminal may be corrupted if the response body contains some
binary data.
The ``on_stream_close_callback()`` function is invoked when the stream
is about to close::
static int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
nghttp2_error_code error_code,
void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
int rv;
if (session_data->stream_data->stream_id == stream_id) {
fprintf(stderr, "Stream %d closed with error_code=%d\n", stream_id,
error_code);
rv = nghttp2_session_terminate_session(session, NGHTTP2_NO_ERROR);
if (rv != 0) {
return NGHTTP2_ERR_CALLBACK_FAILURE;
}
}
return 0;
}
If the stream ID matches the one we initiated, it means that its
stream is going to be closed. Since we have finished receiving
resource we wanted (or the stream was reset by RST_STREAM from the
remote peer), we call `nghttp2_session_terminate_session()` to
commence closure of the HTTP/2 session gracefully. If you have
some data associated for the stream to be closed, you may delete it
here.
nghttp2/doc/sources/tutorial-hpack.rst
+126
View File
@@ -0,0 +1,126 @@
Tutorial: HPACK API
===================
In this tutorial, we describe basic use of nghttp2's HPACK API. We
briefly describe the APIs for deflating and inflating header fields.
The full example of using these APIs, `deflate.c`_, is attached at the
end of this page. It also resides in the examples directory in the
archive or repository.
Deflating (encoding) headers
----------------------------
First we need to initialize a :type:`nghttp2_hd_deflater` object using
the `nghttp2_hd_deflate_new()` function::
int nghttp2_hd_deflate_new(nghttp2_hd_deflater **deflater_ptr,
size_t deflate_hd_table_bufsize_max);
This function allocates a :type:`nghttp2_hd_deflater` object,
initializes it, and assigns its pointer to ``*deflater_ptr``. The
*deflate_hd_table_bufsize_max* is the upper bound of header table size
the deflater will use. This will limit the memory usage by the
deflater object for the dynamic header table. If in doubt, just
specify 4096 here, which is the default upper bound of dynamic header
table buffer size.
To encode header fields, use the `nghttp2_hd_deflate_hd()` function::
ssize_t nghttp2_hd_deflate_hd(nghttp2_hd_deflater *deflater,
uint8_t *buf, size_t buflen,
const nghttp2_nv *nva, size_t nvlen);
The *deflater* is the deflater object initialized by
`nghttp2_hd_deflate_new()` described above. The encoded byte string is
written to the buffer *buf*, which has length *buflen*. The *nva* is
a pointer to an array of headers fields, each of type
:type:`nghttp2_nv`. *nvlen* is the number of header fields which
*nva* contains.
It is important to initialize and assign all members of
:type:`nghttp2_nv`. For security sensitive header fields (such as
cookies), set the :macro:`NGHTTP2_NV_FLAG_NO_INDEX` flag in
:member:`nghttp2_nv.flags`. Setting this flag prevents recovery of
sensitive header fields by compression based attacks: This is achieved
by not inserting the header field into the dynamic header table.
`nghttp2_hd_deflate_hd()` processes all headers given in *nva*. The
*nva* must include all request or response header fields to be sent in
one HEADERS (or optionally following (multiple) CONTINUATION
frame(s)). The *buf* must have enough space to store the encoded
result, otherwise the function will fail. To estimate the upper bound
of the encoded result length, use `nghttp2_hd_deflate_bound()`::
size_t nghttp2_hd_deflate_bound(nghttp2_hd_deflater *deflater,
const nghttp2_nv *nva, size_t nvlen);
Pass this function the same parameters (*deflater*, *nva*, and
*nvlen*) which will be passed to `nghttp2_hd_deflate_hd()`.
Subsequent calls to `nghttp2_hd_deflate_hd()` will use the current
encoder state and perform differential encoding, which yields HPAC's
fundamental compression gain.
If `nghttp2_hd_deflate_hd()` fails, the failure is fatal and any
further calls with the same deflater object will fail. Thus it's very
important to use `nghttp2_hd_deflate_bound()` to determine the
required size of the output buffer.
To delete a :type:`nghttp2_hd_deflater` object, use the
`nghttp2_hd_deflate_del()` function.
Inflating (decoding) headers
----------------------------
A :type:`nghttp2_hd_inflater` object is used to inflate compressed
header data. To initialize the object, use
`nghttp2_hd_inflate_new()`::
int nghttp2_hd_inflate_new(nghttp2_hd_inflater **inflater_ptr);
To inflate header data, use `nghttp2_hd_inflate_hd2()`::
ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater,
nghttp2_nv *nv_out, int *inflate_flags,
const uint8_t *in, size_t inlen,
int in_final);
`nghttp2_hd_inflate_hd2()` reads a stream of bytes and outputs a
single header field at a time. Multiple calls are normally required to
read a full stream of bytes and output all of the header fields.
The *inflater* is the inflater object initialized above. The *nv_out*
is a pointer to a :type:`nghttp2_nv` into which one header field may
be stored. The *in* is a pointer to input data, and *inlen* is its
length. The caller is not required to specify the whole deflated
header data via *in* at once: Instead it can call this function
multiple times as additional data bytes become available. If
*in_final* is nonzero, it tells the function that the passed data is
the final sequence of deflated header data.
The *inflate_flags* is an output parameter; on success the function
sets it to a bitset of flags. It will be described later.
This function returns when each header field is inflated. When this
happens, the function sets the :macro:`NGHTTP2_HD_INFLATE_EMIT` flag
in *inflate_flags*, and a header field is stored in *nv_out*. The
return value indicates the number of bytes read from *in* processed so
far, which may be less than *inlen*. The caller should call the
function repeatedly until all bytes are processed. Processed bytes
should be removed from *in*, and *inlen* should be adjusted
appropriately.
If *in_final* is nonzero and all given data was processed, the
function sets the :macro:`NGHTTP2_HD_INFLATE_FINAL` flag in
*inflate_flags*. When you see this flag set, call the
`nghttp2_hd_inflate_end_headers()` function.
If *in_final* is zero and the :macro:`NGHTTP2_HD_INFLATE_EMIT` flag is
not set, it indicates that all given data was processed. The caller
is required to pass additional data.
Example usage of `nghttp2_hd_inflate_hd2()` is shown in the
`inflate_header_block()` function in `deflate.c`_.
Finally, to delete a :type:`nghttp2_hd_inflater` object, use
`nghttp2_hd_inflate_del()`.
nghttp2/doc/sources/tutorial-server.rst
+625
View File
@@ -0,0 +1,625 @@
Tutorial: HTTP/2 server
=========================
In this tutorial, we are going to write a single-threaded, event-based
HTTP/2 web server, which supports HTTPS only. It can handle concurrent
multiple requests, but only the GET method is supported. The complete
source code, `libevent-server.c`_, is attached at the end of this
page. The source also resides in the examples directory in the
archive or repository.
This simple server takes 3 arguments: The port number to listen on,
the path to your SSL/TLS private key file, and the path to your
certificate file. The synopsis is:
.. code-block:: text
$ libevent-server PORT /path/to/server.key /path/to/server.crt
We use libevent in this tutorial to handle networking I/O. Please
note that nghttp2 itself does not depend on libevent.
The server starts with some libevent and OpenSSL setup in the
``main()`` and ``run()`` functions. This setup isn't specific to
nghttp2, but one thing you should look at is setup of the NPN
callback. The NPN callback is used by the server to advertise which
application protocols the server supports to a client. In this
example program, when creating the ``SSL_CTX`` object, we store the
application protocol name in the wire format of NPN in a statically
allocated buffer. This is safe because we only create one ``SSL_CTX``
object in the program's entire lifetime.
If you are following TLS related RFC, you know that NPN is not the
standardized way to negotiate HTTP/2. NPN itself is not even
published as RFC. The standard way to negotiate HTTP/2 is ALPN,
Application-Layer Protocol Negotiation Extension, defined in `RFC 7301
<https://tools.ietf.org/html/rfc7301>`_. The one caveat of ALPN is
that OpenSSL >= 1.0.2 is required. We use macro to enable/disable
ALPN support depending on OpenSSL version. In ALPN, client sends the
list of supported application protocols, and server selects one of
them. We provide the callback for it::
static unsigned char next_proto_list[256];
static size_t next_proto_list_len;
static int next_proto_cb(SSL *s _U_, const unsigned char **data,
unsigned int *len, void *arg _U_) {
*data = next_proto_list;
*len = (unsigned int)next_proto_list_len;
return SSL_TLSEXT_ERR_OK;
}
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
static int alpn_select_proto_cb(SSL *ssl _U_, const unsigned char **out,
unsigned char *outlen, const unsigned char *in,
unsigned int inlen, void *arg _U_) {
int rv;
rv = nghttp2_select_next_protocol((unsigned char **)out, outlen, in, inlen);
if (rv != 1) {
return SSL_TLSEXT_ERR_NOACK;
}
return SSL_TLSEXT_ERR_OK;
}
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
static SSL_CTX *create_ssl_ctx(const char *key_file, const char *cert_file) {
SSL_CTX *ssl_ctx;
EC_KEY *ecdh;
ssl_ctx = SSL_CTX_new(SSLv23_server_method());
...
next_proto_list[0] = NGHTTP2_PROTO_VERSION_ID_LEN;
memcpy(&next_proto_list[1], NGHTTP2_PROTO_VERSION_ID,
NGHTTP2_PROTO_VERSION_ID_LEN);
next_proto_list_len = 1 + NGHTTP2_PROTO_VERSION_ID_LEN;
SSL_CTX_set_next_protos_advertised_cb(ssl_ctx, next_proto_cb, NULL);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
SSL_CTX_set_alpn_select_cb(ssl_ctx, alpn_select_proto_cb, NULL);
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
return ssl_ctx;
}
The wire format of NPN is a sequence of length prefixed strings, with
exactly one byte used to specify the length of each protocol
identifier. In this tutorial, we advertise the specific HTTP/2
protocol version the current nghttp2 library supports, which is
exported in the identifier :macro:`NGHTTP2_PROTO_VERSION_ID`. The
``next_proto_cb()`` function is the server-side NPN callback. In the
OpenSSL implementation, we just assign the pointer to the NPN buffers
we filled in earlier. The NPN callback function is set to the
``SSL_CTX`` object using ``SSL_CTX_set_next_protos_advertised_cb()``.
In ``alpn_select_proto_cb()``, we use `nghttp2_select_next_protocol()`
to select application protocol. The `nghttp2_select_next_protocol()`
returns 1 only if it selected h2 (ALPN identifier for HTTP/2), and out
parameters were assigned accordingly.
Next, let's take a look at the main structures used by the example
application:
We use the ``app_context`` structure to store application-wide data::
struct app_context {
SSL_CTX *ssl_ctx;
struct event_base *evbase;
};
We use the ``http2_session_data`` structure to store session-level
(which corresponds to one HTTP/2 connection) data::
typedef struct http2_session_data {
struct http2_stream_data root;
struct bufferevent *bev;
app_context *app_ctx;
nghttp2_session *session;
char *client_addr;
} http2_session_data;
We use the ``http2_stream_data`` structure to store stream-level data::
typedef struct http2_stream_data {
struct http2_stream_data *prev, *next;
char *request_path;
int32_t stream_id;
int fd;
} http2_stream_data;
A single HTTP/2 session can have multiple streams. To manage them, we
use a doubly linked list: The first element of this list is pointed
to by the ``root->next`` in ``http2_session_data``. Initially,
``root->next`` is ``NULL``.
libevent's bufferevent structure is used to perform network I/O, with
the pointer to the bufferevent stored in the ``http2_session_data``
structure. Note that the bufferevent object is kept in
``http2_session_data`` and not in ``http2_stream_data``. This is
because ``http2_stream_data`` is just a logical stream multiplexed
over the single connection managed by the bufferevent in
``http2_session_data``.
We first create a listener object to accept incoming connections.
libevent's ``struct evconnlistener`` is used for this purpose::
static void start_listen(struct event_base *evbase, const char *service,
app_context *app_ctx) {
int rv;
struct addrinfo hints;
struct addrinfo *res, *rp;
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE;
#ifdef AI_ADDRCONFIG
hints.ai_flags |= AI_ADDRCONFIG;
#endif /* AI_ADDRCONFIG */
rv = getaddrinfo(NULL, service, &hints, &res);
if (rv != 0) {
errx(1, NULL);
}
for (rp = res; rp; rp = rp->ai_next) {
struct evconnlistener *listener;
listener = evconnlistener_new_bind(
evbase, acceptcb, app_ctx, LEV_OPT_CLOSE_ON_FREE | LEV_OPT_REUSEABLE,
16, rp->ai_addr, (int)rp->ai_addrlen);
if (listener) {
freeaddrinfo(res);
return;
}
}
errx(1, "Could not start listener");
}
We specify the ``acceptcb`` callback, which is called when a new connection is
accepted::
static void acceptcb(struct evconnlistener *listener _U_, int fd,
struct sockaddr *addr, int addrlen, void *arg) {
app_context *app_ctx = (app_context *)arg;
http2_session_data *session_data;
session_data = create_http2_session_data(app_ctx, fd, addr, addrlen);
bufferevent_setcb(session_data->bev, readcb, writecb, eventcb, session_data);
}
Here we create the ``http2_session_data`` object. The connection's
bufferevent is initialized at the same time. We specify three
callbacks for the bufferevent: ``readcb``, ``writecb``, and
``eventcb``.
The ``eventcb()`` callback is invoked by the libevent event loop when an event
(e.g. connection has been established, timeout, etc.) occurs on the
underlying network socket::
static void eventcb(struct bufferevent *bev _U_, short events, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
if (events & BEV_EVENT_CONNECTED) {
const unsigned char *alpn = NULL;
unsigned int alpnlen = 0;
SSL *ssl;
fprintf(stderr, "%s connected\n", session_data->client_addr);
ssl = bufferevent_openssl_get_ssl(session_data->bev);
SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
if (alpn == NULL) {
SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
}
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
fprintf(stderr, "%s h2 is not negotiated\n", session_data->client_addr);
delete_http2_session_data(session_data);
return;
}
initialize_nghttp2_session(session_data);
if (send_server_connection_header(session_data) != 0 ||
session_send(session_data) != 0) {
delete_http2_session_data(session_data);
return;
}
return;
}
if (events & BEV_EVENT_EOF) {
fprintf(stderr, "%s EOF\n", session_data->client_addr);
} else if (events & BEV_EVENT_ERROR) {
fprintf(stderr, "%s network error\n", session_data->client_addr);
} else if (events & BEV_EVENT_TIMEOUT) {
fprintf(stderr, "%s timeout\n", session_data->client_addr);
}
delete_http2_session_data(session_data);
}
Here we validate that HTTP/2 is negotiated, and if not, drop
connection.
For the ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and
``BEV_EVENT_TIMEOUT`` events, we just simply tear down the connection.
The ``delete_http2_session_data()`` function destroys the
``http2_session_data`` object and its associated bufferevent member.
As a result, the underlying connection is closed.
The
``BEV_EVENT_CONNECTED`` event is invoked when SSL/TLS handshake has
completed successfully. After this we are ready to begin communicating
via HTTP/2.
The ``initialize_nghttp2_session()`` function initializes the nghttp2
session object and several callbacks::
static void initialize_nghttp2_session(http2_session_data *session_data) {
nghttp2_session_callbacks *callbacks;
nghttp2_session_callbacks_new(&callbacks);
nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks,
on_frame_recv_callback);
nghttp2_session_callbacks_set_on_stream_close_callback(
callbacks, on_stream_close_callback);
nghttp2_session_callbacks_set_on_header_callback(callbacks,
on_header_callback);
nghttp2_session_callbacks_set_on_begin_headers_callback(
callbacks, on_begin_headers_callback);
nghttp2_session_server_new(&session_data->session, callbacks, session_data);
nghttp2_session_callbacks_del(callbacks);
}
Since we are creating a server, we use `nghttp2_session_server_new()`
to initialize the nghttp2 session object. We also setup 5 callbacks
for the nghttp2 session, these are explained later.
The server now begins by sending the server connection preface, which
always consists of a SETTINGS frame.
``send_server_connection_header()`` configures and submits it::
static int send_server_connection_header(http2_session_data *session_data) {
nghttp2_settings_entry iv[1] = {
{NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}};
int rv;
rv = nghttp2_submit_settings(session_data->session, NGHTTP2_FLAG_NONE, iv,
ARRLEN(iv));
if (rv != 0) {
warnx("Fatal error: %s", nghttp2_strerror(rv));
return -1;
}
return 0;
}
In the example SETTINGS frame we've set
SETTINGS_MAX_CONCURRENT_STREAMS to 100. `nghttp2_submit_settings()`
is used to queue the frame for transmission, but note it only queues
the frame for transmission, and doesn't actually send it. All
functions in the ``nghttp2_submit_*()`` family have this property. To
actually send the frame, `nghttp2_session_send()` should be used, as
described later.
Since bufferevent may buffer more than the first 24 bytes from the client, we
have to process them here since libevent won't invoke callback functions for
this pending data. To process the received data, we call the
``session_recv()`` function::
static int session_recv(http2_session_data *session_data) {
ssize_t readlen;
struct evbuffer *input = bufferevent_get_input(session_data->bev);
size_t datalen = evbuffer_get_length(input);
unsigned char *data = evbuffer_pullup(input, -1);
readlen = nghttp2_session_mem_recv(session_data->session, data, datalen);
if (readlen < 0) {
warnx("Fatal error: %s", nghttp2_strerror((int)readlen));
return -1;
}
if (evbuffer_drain(input, (size_t)readlen) != 0) {
warnx("Fatal error: evbuffer_drain failed");
return -1;
}
if (session_send(session_data) != 0) {
return -1;
}
return 0;
}
In this function, we feed all unprocessed but already received data to
the nghttp2 session object using the `nghttp2_session_mem_recv()`
function. The `nghttp2_session_mem_recv()` function processes the data
and may both invoke the previously setup callbacks and also queue
outgoing frames. To send any pending outgoing frames, we immediately
call ``session_send()``.
The ``session_send()`` function is defined as follows::
static int session_send(http2_session_data *session_data) {
int rv;
rv = nghttp2_session_send(session_data->session);
if (rv != 0) {
warnx("Fatal error: %s", nghttp2_strerror(rv));
return -1;
}
return 0;
}
The `nghttp2_session_send()` function serializes the frame into wire
format and calls the ``send_callback()``, which is of type
:type:`nghttp2_send_callback`. The ``send_callback()`` is defined as
follows::
static ssize_t send_callback(nghttp2_session *session _U_, const uint8_t *data,
size_t length, int flags _U_, void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
struct bufferevent *bev = session_data->bev;
/* Avoid excessive buffering in server side. */
if (evbuffer_get_length(bufferevent_get_output(session_data->bev)) >=
OUTPUT_WOULDBLOCK_THRESHOLD) {
return NGHTTP2_ERR_WOULDBLOCK;
}
bufferevent_write(bev, data, length);
return (ssize_t)length;
}
Since we use bufferevent to abstract network I/O, we just write the
data to the bufferevent object. Note that `nghttp2_session_send()`
continues to write all frames queued so far. If we were writing the
data to a non-blocking socket directly using the ``write()`` system
call in the ``send_callback()``, we'd soon receive an ``EAGAIN`` or
``EWOULDBLOCK`` error since sockets have a limited send buffer. If
that happens, it's possible to return :macro:`NGHTTP2_ERR_WOULDBLOCK`
to signal the nghttp2 library to stop sending further data. But here,
when writing to the bufferevent, we have to regulate the amount data
to buffered ourselves to avoid using huge amounts of memory. To
achieve this, we check the size of the output buffer and if it reaches
more than or equal to ``OUTPUT_WOULDBLOCK_THRESHOLD`` bytes, we stop
writing data and return :macro:`NGHTTP2_ERR_WOULDBLOCK`.
The next bufferevent callback is ``readcb()``, which is invoked when
data is available to read in the bufferevent input buffer::
static void readcb(struct bufferevent *bev _U_, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
if (session_recv(session_data) != 0) {
delete_http2_session_data(session_data);
return;
}
}
In this function, we just call ``session_recv()`` to process incoming
data.
The third bufferevent callback is ``writecb()``, which is invoked when all
data in the bufferevent output buffer has been sent::
static void writecb(struct bufferevent *bev, void *ptr) {
http2_session_data *session_data = (http2_session_data *)ptr;
if (evbuffer_get_length(bufferevent_get_output(bev)) > 0) {
return;
}
if (nghttp2_session_want_read(session_data->session) == 0 &&
nghttp2_session_want_write(session_data->session) == 0) {
delete_http2_session_data(session_data);
return;
}
if (session_send(session_data) != 0) {
delete_http2_session_data(session_data);
return;
}
}
First we check whether we should drop the connection or not. The
nghttp2 session object keeps track of reception and transmission of
GOAWAY frames and other error conditions as well. Using this
information, the nghttp2 session object can state whether the
connection should be dropped or not. More specifically, if both
`nghttp2_session_want_read()` and `nghttp2_session_want_write()`
return 0, the connection is no-longer required and can be closed.
Since we are using bufferevent and its deferred callback option, the
bufferevent output buffer may still contain pending data when the
``writecb()`` is called. To handle this, we check whether the output
buffer is empty or not. If all of these conditions are met, we drop
connection.
Otherwise, we call ``session_send()`` to process the pending output
data. Remember that in ``send_callback()``, we must not write all data to
bufferevent to avoid excessive buffering. We continue processing pending data
when the output buffer becomes empty.
We have already described the nghttp2 callback ``send_callback()``. Let's
learn about the remaining nghttp2 callbacks setup in
``initialize_nghttp2_setup()`` function.
The ``on_begin_headers_callback()`` function is invoked when the reception of
a header block in HEADERS or PUSH_PROMISE frame is started::
static int on_begin_headers_callback(nghttp2_session *session,
const nghttp2_frame *frame,
void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
http2_stream_data *stream_data;
if (frame->hd.type != NGHTTP2_HEADERS ||
frame->headers.cat != NGHTTP2_HCAT_REQUEST) {
return 0;
}
stream_data = create_http2_stream_data(session_data, frame->hd.stream_id);
nghttp2_session_set_stream_user_data(session, frame->hd.stream_id,
stream_data);
return 0;
}
We are only interested in the HEADERS frame in this function. Since
the HEADERS frame has several roles in the HTTP/2 protocol, we check
that it is a request HEADERS, which opens new stream. If the frame is
a request HEADERS, we create a ``http2_stream_data`` object to store
the stream related data. We associate the created
``http2_stream_data`` object with the stream in the nghttp2 session
object using `nghttp2_set_stream_user_data()`. The
``http2_stream_data`` object can later be easily retrieved from the
stream, without searching through the doubly linked list.
In this example server, we want to serve files relative to the current working
directory in which the program was invoked. Each header name/value pair is
emitted via ``on_header_callback`` function, which is called after
``on_begin_headers_callback()``::
static int on_header_callback(nghttp2_session *session,
const nghttp2_frame *frame, const uint8_t *name,
size_t namelen, const uint8_t *value,
size_t valuelen, uint8_t flags _U_,
void *user_data _U_) {
http2_stream_data *stream_data;
const char PATH[] = ":path";
switch (frame->hd.type) {
case NGHTTP2_HEADERS:
if (frame->headers.cat != NGHTTP2_HCAT_REQUEST) {
break;
}
stream_data =
nghttp2_session_get_stream_user_data(session, frame->hd.stream_id);
if (!stream_data || stream_data->request_path) {
break;
}
if (namelen == sizeof(PATH) - 1 && memcmp(PATH, name, namelen) == 0) {
size_t j;
for (j = 0; j < valuelen && value[j] != '?'; ++j)
;
stream_data->request_path = percent_decode(value, j);
}
break;
}
return 0;
}
We search for the ``:path`` header field among the request headers and
store the requested path in the ``http2_stream_data`` object. In this
example program, we ignore the ``:method`` header field and always
treat the request as a GET request.
The ``on_frame_recv_callback()`` function is invoked when a frame is
fully received::
static int on_frame_recv_callback(nghttp2_session *session,
const nghttp2_frame *frame, void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
http2_stream_data *stream_data;
switch (frame->hd.type) {
case NGHTTP2_DATA:
case NGHTTP2_HEADERS:
/* Check that the client request has finished */
if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
stream_data =
nghttp2_session_get_stream_user_data(session, frame->hd.stream_id);
/* For DATA and HEADERS frame, this callback may be called after
on_stream_close_callback. Check that stream still alive. */
if (!stream_data) {
return 0;
}
return on_request_recv(session, session_data, stream_data);
}
break;
default:
break;
}
return 0;
}
First we retrieve the ``http2_stream_data`` object associated with the
stream in ``on_begin_headers_callback()`` using
`nghttp2_session_get_stream_user_data()`. If the requested path
cannot be served for some reason (e.g. file is not found), we send a
404 response using ``error_reply()``. Otherwise, we open
the requested file and send its content. We send the header field
``:status`` as a single response header.
Sending the file content is performed by the ``send_response()`` function::
static int send_response(nghttp2_session *session, int32_t stream_id,
nghttp2_nv *nva, size_t nvlen, int fd) {
int rv;
nghttp2_data_provider data_prd;
data_prd.source.fd = fd;
data_prd.read_callback = file_read_callback;
rv = nghttp2_submit_response(session, stream_id, nva, nvlen, &data_prd);
if (rv != 0) {
warnx("Fatal error: %s", nghttp2_strerror(rv));
return -1;
}
return 0;
}
nghttp2 uses the :type:`nghttp2_data_provider` structure to send the
entity body to the remote peer. The ``source`` member of this
structure is a union, which can be either a void pointer or an int
(which is intended to be used as file descriptor). In this example
server, we use it as a file descriptor. We also set the
``file_read_callback()`` callback function to read the contents of the
file::
static ssize_t file_read_callback(nghttp2_session *session _U_,
int32_t stream_id _U_, uint8_t *buf,
size_t length, uint32_t *data_flags,
nghttp2_data_source *source,
void *user_data _U_) {
int fd = source->fd;
ssize_t r;
while ((r = read(fd, buf, length)) == -1 && errno == EINTR)
;
if (r == -1) {
return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
}
if (r == 0) {
*data_flags |= NGHTTP2_DATA_FLAG_EOF;
}
return r;
}
If an error occurs while reading the file, we return
:macro:`NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE`. This tells the
library to send RST_STREAM to the stream. When all data has been
read, the :macro:`NGHTTP2_DATA_FLAG_EOF` flag is set to signal nghttp2
that we have finished reading the file.
The `nghttp2_submit_response()` function is used to send the response to the
remote peer.
The ``on_stream_close_callback()`` function is invoked when the stream
is about to close::
static int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
uint32_t error_code _U_, void *user_data) {
http2_session_data *session_data = (http2_session_data *)user_data;
http2_stream_data *stream_data;
stream_data = nghttp2_session_get_stream_user_data(session, stream_id);
if (!stream_data) {
return 0;
}
remove_stream(session_data, stream_data);
delete_http2_stream_data(stream_data);
return 0;
}
Lastly, we destroy the ``http2_stream_data`` object in this function,
since the stream is about to close and we no longer need the object.