Enhance Apache and Lighttpd on Centos Linux: increasing file descriptors limit

Centos Linux servers typically have the maximum file descriptors at 1024. Other Linux distributions may have the same settings, but what is this file descriptor and how to increase the value? Please use linux man pages to get further details of the commands in this document. These are information for educational purposes.


Some web servers that have to work harder, may need to multitask more than others. This is when there will be warnings in /var/log/messages related to

can't have more connections than fds/2:  1024 1024

or Apache error logs showing

ulimit:error setting limit(Operation not permitted)


Another situation is when a software runs into an infite loop of process, such that is increase resource usage until the server comes to a complete stand still. Causing no other users to have access to the server. On a production server, this would be a disaster. What is really affected by these limits? Its the application performances such as database and web servers that are under heavy usage. Typical linux terminal users wont really need to concern with the limits.


View limits
Users can view file limits with the command;

$ ulimit 
$ ulimit -aH

Value of unlimited is common for development servers.

To view limit for current BASH open files of the user;

$ ulimit -n

To view max user processes

$ ulimit -Hn
$ ulimit -u

To view limits of any running process, identify the process ID ($PID) then run command;
$ grep 'open files' /proc/{$PID}/limits

Replace {$PID} with the PID to be checked.

Any user can reduce their file descriptor limit. This is useful to test problematic applications. Example to reduce to 1024.

$ ulimit -u 1024


Configuration file sysctl
This sysctl.conf and files in /etc/sysctl.d is an interface that allows you to make changes to a running Linux kernel. This file is extensively configured to harden a production Centos Linux server. Example, to reboot the linux after a kernel panic, the following values is used;

kernel.panic=10

With /etc/sysctl.conf you can configure various Linux networking and system settings. To view current values;

$ sudo sysctl -A
$ sudo sysctl fs.file-max
fs.file-max = 994255

Centos Linux

Check the current value of the file descriptor

$ ulimit -aHS
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 39253
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

To view system configured default limit on file descriptors.

$ cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     4096
root       soft    nproc     unlimited


Here, the plan is to increase limits to 65530.

Change the hard and soft limits in configuration file.

Edit /etc/security/limits.conf with these values. The * refers to any user.

* soft nproc 65535
* hard nproc 65535
* soft nofile 65535
* hard nofile 65535

In cases where the fs.file-max is below the 65535 value, then it can be set by editing /etc/sysctl.conf

fs.filemax = 65535

In both cases above, apply changes with the command;

$ sudo sysctl -p

Lighttpd

Edit /etc/lighttpd/lighttpd.conf

server.max-fds = 2048

Apache Httpd

The Centos, this document referring is with httpd version 2.4.6 (CentOS) and is using httpd compiled as preforked (non threaded).

Edit the file /etc/httpd/conf/httpd.conf

<IfModule prefork.c>
StartServers       4
MinSpareServers    3
MaxSpareServers   10
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  10000
</IfModule>

Refer details at Apache.org

Selinux

When selinux is in mode Enforcing, it needs to allow changes. Check the status with

$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          permissive
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31

$ getsebool httpd_setrlimit
httpd_setrlimit --> off

Temporary turn it on, then later view the status again;

$ setsebool httpd_setrlimit on

To make this change permanent;

$ sudo setsebool -P httpd_setrlimit on

MYSQL database

Edit /usr/lib/systemd/system/mysqld.service

LimitNOFILE=65535
LimitNPROC=65535

Edit /etc/my.cnf

table_open_cache=65535
open_files_limit=65535


Apply the changes;

$ sudo systemctl daemon-reload
$ sudo systemctl restart mysqld.service