Apache HTTP Server — Multi-process architecture
Introduction
The Apache HTTP Server/Apache Web Server is a multi-process system that uses multiple processes to handle the incoming connections. This blog post provides details about how the requests are distributed between the multiple processes. It also shows how Apache scales in response to increase in the concurrency of the incoming requests, provided it has been properly configured.
At its core, the Apache Web Server listens to incoming TCP connections, accepts them and responds to the HTTP requests that are sent over the TCP connection. Different platforms — like Unix, Solaris, Windows have different abilities/features and interfaces for performing these operations such as — asynchronously waiting for TCP connections, accepting TCP connections, support for thread-safe operations. The underlying network stack is inherently different. Hence, Apache ships with a selection of modules called Multi Processing Modules (MPM) that can be chosen at runtime based on the system on which Apache is expected to run.
If Apache is running on the Windows, mpm_winnt can be chosen.For an Apache Web Server running on Linux Operating systems — there is a choice between event, worker and prefork MPMs.
A website that need a great deal of scalability can choose to use a threaded MPM like worker or event, while sites requiring stability or compatibility with older software can use a prefork.
In this post, I would be writing about the Event MPM.
Processes in an Apache Web Server that uses Event MPM
The Apache WebServer is started using the binary httpd. When the httpd binary is invoked, a process is spawned. This process is called as the parent process in the Apache Web Server terminology as it spawns multiple other processes called child processes. The number of child processes that the parent process spawns in controlled by the StartServers configuration directive in the WebServer’s configuration file.
The parent process reads the configuration file and identifies all the Listen directives. For each Listen directive the parent process creates the socket, binds them to the required IP and port. The parent process also executes the listen() API on the socket. Post this, the child processes are created. As a result, the child processes inherit the listen socket/s. A thread in the child processes called the listener thread in the child processes registers the listen socket/s using the epoll_wait API, which causes the listener thread to wait for incoming TCP connections. The parent process does not register for incoming connections using the epoll API. As a result, the parent process does not handle any incoming TCP connections.
The parent process is single threaded. The child processes that are spawned have ThreadsPerChild + 1 number of threads. ThreadsPerChild is a directive that is specified in the server’s configuration. This directive configures the number of worker threads in the child processes. The worker threads are different from the listener thread. These threads are responsible for processing the request. It is within these threads that the code of different WebServer modules is run. These threads perform tasks such as decrypting the request body (in case TLS protocols are used), parsing the request and identifying the HTTP request headers and the HTTP request body, authentication, executing actions to provide the HTTP response, encrypting the response, caching the response, compressing the response etc and sending the response back to the client. Hence, every child process will ThreadsPerChild worker threads and an additional listener thread.
When an Apache Web Server is started, it can therefore handle StartServers * ThreadsPerChild number of concurrent connections. As the number of concurrent connections increases, the parent process spawns additional child processes upto a maximum called ServerLimit. Hence, the maximum concurrent connections that can be handled is ServerLimit*ThreadsPerChild. This value is also specified as MaxRequestWorkers.
Let us look at the following configuration
<IfModule mpm_event_module>
StartServers 3
MinSpareThreads 75
MaxSpareThreads 250
ThreadsPerChild 25
MaxRequestWorkers 400
MaxConnectionsPerChild 0
</IfModule>Next, looking at the ps listing, we see that there are 4 httpd process. Of these 4, 1 is the parent process and the remaining 3 are worker processes.
brkaran+ 1948 1372 0 12:54 ? 00:00:00 ./httpd -k start
brkaran+ 1949 1948 0 12:54 ? 00:00:00 ./httpd -k start
brkaran+ 1950 1948 0 12:54 ? 00:00:00 ./httpd -k start
brkaran+ 1951 1948 0 12:54 ? 00:00:00 ./httpd -k startUsing the pstree output, we can see that each process starts up with 25 threads. We can also see that process with PID 1948 is the parent process and processes with PIDs 1949, 1950 and 1951 are its children.
httpd(1948)-+-httpd(1949)-+-{httpd}(2007)
| |-{httpd}(2008)
| |-{httpd}(2009)
| |-{httpd}(2010)
| |-{httpd}(2011)
| |-{httpd}(2012)
| |-{httpd}(2013)
| |-{httpd}(2014)
| |-{httpd}(2015)
| |-{httpd}(2016)
| |-{httpd}(2017)
| |-{httpd}(2018)
| |-{httpd}(2019)
| |-{httpd}(2020)
| |-{httpd}(2021)
| |-{httpd}(2022)
| |-{httpd}(2023)
| |-{httpd}(2024)
| |-{httpd}(2025)
| |-{httpd}(2026)
| |-{httpd}(2027)
| |-{httpd}(2028)
| |-{httpd}(2029)
| |-{httpd}(2030)
| |-{httpd}(2031)
| `-{httpd}(2032)
|-httpd(1950)-+-{httpd}(1981)
| |-{httpd}(1982)
| |-{httpd}(1983)
| |-{httpd}(1984)
| |-{httpd}(1985)
| |-{httpd}(1986)
| |-{httpd}(1987)
| |-{httpd}(1988)
| |-{httpd}(1989)
| |-{httpd}(1990)
| |-{httpd}(1991)
| |-{httpd}(1992)
| |-{httpd}(1993)
| |-{httpd}(1994)
| |-{httpd}(1995)
| |-{httpd}(1996)
| |-{httpd}(1997)
| |-{httpd}(1998)
| |-{httpd}(1999)
| |-{httpd}(2000)
| |-{httpd}(2001)
| |-{httpd}(2002)
| |-{httpd}(2003)
| |-{httpd}(2004)
| |-{httpd}(2005)
| `-{httpd}(2006)
`-httpd(1951)-+-{httpd}(1955)
|-{httpd}(1956)
|-{httpd}(1957)
|-{httpd}(1958)
|-{httpd}(1959)
|-{httpd}(1960)
|-{httpd}(1961)
|-{httpd}(1962)
|-{httpd}(1963)
|-{httpd}(1964)
|-{httpd}(1965)
|-{httpd}(1966)
|-{httpd}(1967)
|-{httpd}(1968)
|-{httpd}(1969)
|-{httpd}(1970)
|-{httpd}(1971)
|-{httpd}(1972)
|-{httpd}(1973)
|-{httpd}(1974)
|-{httpd}(1975)
|-{httpd}(1976)
|-{httpd}(1977)
|-{httpd}(1978)
|-{httpd}(1979)
`-{httpd}(1980)
As discussed above, Apache Web Server starts with 3 processes each having 25 threads each. This Web Server can handle 75 concurrent connections. This is the initial state of the Web Server. This Web server can scale upto handling 200 concurrent connections as MaxRequestWorkers is set to 200. It would require 8 processes with 25 threads each to reach 200 concurrent connections.
Let us know look at how the scaling from 3 processes to 8 process takes place as the number of concurrent connections increase.
Apache HTTPD parent and child processes communicate with each other using a data structure called Scoreboard that is placed in a shared memory segment on Linux OSes. The scoreboard is a matrix containing MaxRequestWorkers number of slots. The worker threads in each of the processes update the scoreboard with the state they are in.
These states are as follows:
#define SERVER_DEAD 0
#define SERVER_STARTING 1 /* Server Starting up */
#define SERVER_READY 2 /* Waiting for connection (or accept() lock) */
#define SERVER_BUSY_READ 3 /* Reading a client request */
#define SERVER_BUSY_WRITE 4 /* Processing a client request */
#define SERVER_BUSY_KEEPALIVE 5 /* Waiting for more requests via keepalive */
#define SERVER_BUSY_LOG 6 /* Logging the request */
#define SERVER_BUSY_DNS 7 /* Looking up a hostname */
#define SERVER_CLOSING 8 /* Closing the connection */
#define SERVER_GRACEFUL 9 /* server is gracefully finishing request */
#define SERVER_IDLE_KILL 10 /* Server is cleaning up idle children. */
#define SERVER_NUM_STATUS 11 /* number of status settings */Threads that are in SERVER_STARTING or SERVER_READY are considered as idle threads. The parent process performs periodic check to ensure that the number of idle threads are equal to or greater than MinSpareThreads. In case the number of idle threads falls below the threshold of MinSpareThreads, the parent process creates a new child process, provided that there are free slots in the scoreboard.
Hence, the parent process is responsible for monitoring the scoreboard, thereby monitoring the state of the system and creating new worker threads to match the configured concurrency.
The parent process also ensure that the idle threads do not exceed MaxSpareThreads. If this situation occurs, it will gracefully terminate the child processes so that the number of idle threads is within MaxSpareThreads.
