Install Tensorflow and Keras on Centos 7 Linux

Tensorflow (available in github) provides a version that utilise the GPU of a server to perform much more heavier computations. Tensorflow version 1.6 onwards are optimimsed to use AVX instructions that is not available in older CPUs. These versions are installed via pip, uses the AVX instruction set at compile time and require CPU that support the AVX instruction set. This instruction set is supported from the second generation of Intel Core CPUs (codenamed SandyBridge).

Pre-requisite: 

Centos 7 Linux updated

Installed Python (see previous posting)

Created a project directory called tensorflow

Default steps are to install Tensorflow 2.x (see tensorflow), but server environments that does not support AVX, they need to install Tensorflow 1.5 (see article and blog). To determine if AVX support is available, run the following command and look for AVX or AVX2.

more /proc/cpuinfo | grep flags

My output shows no AVX.

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm constant_tsc rep_good nopl eagerfpu pni monitor ssse3 lahf_lm

Optional:

Configure the project directory for python:

python3 -m venv my_env

Step 1: Install Tensorflow (Or tensorflow-gpu)

cd tensorflow

source my_env/bin/activate

Install Tensorflow application, then print its version

pip install --upgrade tensorflow

Or for server that doesn't have AVX support,

pip install tensorflow==1.15

pip show tensorflow

Step 2: Test Tensorflow install

Create a script file hello.py with following contents;

import tensorflow as tf

hello = tf.constant('Hello, World')

session = tf.Session()

print( session.run(hello) )

Run the script

python hello.py

Step 3: Install Keras

pip install --upgrade scikit-learn pillow

pip install --upgrade keras keras-utils

Or for server that doesn't have AVX support,

pip install --upgrade scikit-learn pillow

pip install keras==2.1.6

pip show keras

Configuration file will be at ~/.keras/keras.conf

{

    "floatx": "float32",

    "epsilon": 1e-07,

    "backend": "tensorflow",

    "image_data_format": "channels_last"

}

Test install by editing hello.py

import tensorflow as tf

from tensorflow import keras

hello = tf.constant('Hello, World')

session = tf.Session()

print( session.run(hello) )

The above instructions are meant to provide the platform prior to my learning chatbot at site and colab.

Install Python on Centos 7 Linux

Python is a programming language that can be used for:

1. Back-end web application
2. Desktop application
3. Processing huge amount of data
4. Control a computer

Example of a basic Python command:

print("Hello, world!")

Install Python

Pip is a tool to manage packages/libraries within Python. On Centos 7, the default pip is version 9.0.3 and it should be upgraded once python is installed.

Following are notes to install Python 3 on Centos 7 Linux. All these are at the command prompt.

sudo yum install python3  python3-devel

Check the installed version

python3 -V

Create directory for python projects

mkdir projects

cd projects

python3 -m venv my_env

Initiate the python project environment

source my_env/bin/activate

Create your first programme in a file called hello.py and save the following line.

print("Hello, World")

Run the programme.

python hello.py

Upgrade pip package manager.

pip install --upgrade pip

Once done with Python, leave the project environment.

deactivate

Extra pip example 

Install setuptools to install applications from source

pip install --upgrade setuptools wheel

Install Tensorflow application, then print its version

pip install --upgrade tensorflow

pip show tensorflow

Postgresql 10: Create database and user

Postgresql provides creation of database schemas with a specific role or user assigned to it. This is great where there will be a few users who need to access one or another database on the same server.

Default user created by Postgresql is named postgres, that can be used to create any database schema and user roles. For this tutorial, Postgresql is installed on Centos Linux 7. To start postgresql client and access the default user, at the terminal type;

sudo -u postgres psql

Steps to create the database are as follows;

1. Create the database. In this case we call it student_management.
2. Create user and assign password for login. We call out user student and password is 123456
3. Assign the access to the database. We give full privileges to read, write.
4. Enable the user to login.
5. Exit from client (optional)

CREATE DATABASE student_management;
CREATE ROLE student WITH PASSWORD '123456';
GRANT ALL PRIVILEGES ON DATABASE student_management TO student;
ALTER ROLE student WITH LOGIN;

Next, test by login with that user account and password.

There is an alternative approach for a dedicated database server that require users to have their own databases. This approach doesn't require you to use the psql client. It is useful when you plan to have scripts that auto generate the process.

sudo -u postgres createdb developer
sudo -u postgres createuser --interactive
sudo adduser developer
sudo passwd developer

The user "developer" can login and get connected to the default database with the same name as the user. In this case the database name is "developer".

Login as the user.

sudo -i -u developer psql

Laravel: Getting started

Notes on getting started with Laravel, on Centos Linux 6,7,8.

In order to get started with Laravel, there are required software to run composer and laravel that needs to be installed. This includes;

1. Web server
2. PHP
3. Composer
4. Node JS

Step 1. Install PHP 7.3 or newer.

If you havent installed PHP, follow notes from previous post.
After installing PHP, install all the regular extensions. Ensure PHP is working with the web browser.

sudo yum install php-cli php-zip wget unzip  php-dom php-posix php-shmop  php-xsl php-xmlrender

Step 2. Web server.

In this case, I am using Apache HTTPD. Install this and test that the php is working.

sudo yum install httpd

Step 3. Composer

Download the installer.

php -r "copy('https://getcomposer.org/installer', 'composer-setup.php' ");
sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer

Step 4. Node JS

sudo yum install nodejs

You are ready to work with Laravel projects. Here are links to the remainder tutorials
Part 1, Part 2,

Install Postgresql 10 database on Centos 7

Installation and configuration of Postgresql 10 on Centos 7

Installation

Centos Linux 7 by default installs Postgresql version 9. These are summary of steps in order to install Postgresql version 10, the following steps can be taken.

1. Yum manages the installation of software, and utilises a list of repositories of there to locate its repository. It is highly advised to update existing software with yum before proceeding with software installations. Add the additional repository site must be added for Postgresql 10 from postgresql.org
2. Install the version of Postgresql database client software that is needed to open the database.
3. Install the version of Postgresql database server software. This will hold the database files and controlling software.
4. Initialise Postgresql database files. This will create the username "postgres" where it is configured to run all the client commands.
5. Start Postgresql database server.
6. Test the server by requesting info.
7. Configure Centos 7 to automatically start at boot up.

At a terminal, execute the commands

# yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# yum install postgresql10
# yum install postgresql10-server
# /usr/pgsql-10/bin/postgresql-10-setup initdb
# systemctl start postgresql-10
# /usr/pgsql-10/bin/postgres -V
# systemctl enable postgresql-10

Using the client

For those who wish to use the postgres clients, here are commands to get started.

Login as user "postgres" and use the Postgresql client.

# sudo -u postgres psql

Once in psql, here are several commands to browse the database;

Display server version (the Uppercase is a matter of good practice)
SELECT version();

List database schema
\l

Switch database schema
\c database_name

Describe a table
\d table_name

Get the last command
\g

Show history of commands
\s

Help with a command
\h command_name

Exit Postgresql client
\q

Remote access and firewall

In development databases, many users may require to access the database from remote computers. On the default Centos 7, with firewall running this require some configuration. The steps involved;

1. Add httpd and postgresql service rule to the firewall. The httpd is an additional web service I am demonstrating here.
2. Restart firewall
3. Configure postgresql to listen from incoming networks, or all networks. Replace the line listen_addresses = 'localhost'
4. Allow user authentication from incoming networks, or all networks. Replace the line with 127.0.0.1/32
5. Restart postgresql

At a terminal, execute the commands

# sudo firewall-cmd --add-port=8080/tcp --permanent

# sudo firewall-cmd --add-service=postgresql --permanent

# sudo firewall-cmd --reload

# sudo vi /var/lib/pgsql/10/data/postgresql.conf

listen_addresses = '*'

# sudo vi /var/lib/pgsql/10/data/pg_hba.conf

host    all             all             0.0.0.0/0            md5

# sudo systemctl restart postgresql-10

Laravel 6 Step by Step Tutorial Part 2

Laravel 6 provides an advanced framework for programmers to quickly create enterprise on a wide number of PHP platforms. Part 2 of this Laravel guide can be found at Laravel 6 Part 2

In order to proceed with part 2 of the tutorial, please complete part 1 as posted in
First of Laravel 6 step by step guide.

The tutorial can be followed by having your copy of PHP 7.3 onwards and a database. Notes are provided for MySQL and Postgresql database.

This demonstrates how to implement data relationships of has one. In Laravel this is implemented with the function belongsTo. The idea is to maintain a list of articles where each have one author. That author could essentially have written many articles.

Install MongoDB database on Centos 7

MongoDB is a noSQL database and is available on Centos Linux 7. Ensure that SElinux is only on permissive.

For cases SElinux is enforcing, additional steps need to be taken which is outside of this installation note.

Step 1: Configure the repository

Create the file /etc/yum.repos.d/mongodb-org.repo

[mongodb-org-4.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc

Step 2: Install and start

sudo yum install -y mongodb-org
sudo systemctl start mongod

If all is running good, then allow it to start at boot time.

sudo systemctl enable mongod

Step 3: Verify

mongo
db.version()
exit

Further configuration can be done with the file /etc/mongod.conf

Allow database through the firewall

sudo firewall-cmd --zone=public --add-port=27017/tcp --permanent
sudo firewall-cmd --reload

Those with Selinux in mode Enforcing, allow the default mongodb port.

sudo semanage port -a -t mongod_port_t -p tcp 27017

Install Mariadb database on Centos 7

MariaDB is a replacement for Oracle Mysql Community edition.

Installation on Centos 7 is as follows;

Step 1: Install

sudo yum install mariadb-server mariadb

Step 2: Configure and assign database root password

sudo systemctl start mariadb
sudo mysql_secure_installation

sudo systemctl enable mariadb

Step 3: Verify

mysql -u root -p

Configuration of the database can be done with the files at
/etc/my.cnf
/etc/my.cnf.d/*

To allow access of the database through the firewall.

sudo firewall-cmd --permanent --add-service=mysql

Howto install PHP 7 on Centos 7

The Centos Linux 7 comes with php version 5.4.16. It is possible to install php version 7 by adding an additional rpm repository. Lets see how its done to install php 7.3 (which is the version I have chosen).

Step 1: Add Remi repository for php 7.3

sudo yum install yum-utils
sudo yum install epel-release
sudo yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
sudo yum-config-manager --enable remi-php73

Step 2: Install php

sudo yum install php php-common php-opcache php-mcrypt php-cli php-gd php-curl php-json php-mysql php-mbstring

Step 3: Verify 

php --version

For the web server to accept php, it needs to be restarted.

Specialised development environment will require additional php extensions. For example laravel framework could require;

opcache, bz2, calendar, ctype, curl, dom, exif, fileinfo, ftp, gd, gettext, iconv, mysqlnd, pdo, phar, posix, shmop, simpleexml, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, xml, xmlwriter, xsl, mcrypt, mysqli, pdo_mysqli, wddx, xmlrender, zip.

and the list continues.

Android Studio 3 Installation on MS Windows 10

Android Studio is provided by Google for development of Android applications. Android Studio or AS is available for MS Windows, Linux and Mac OSX.

Following are steps to get started with a fresh MS Windows 10 machine.

Step 1. Download Android Studio

Download from https://developer.android.com/studio/index.html

and follow the installation process.

Step 2. Install Android Studio

Double click and run the downloaded EXE file. Accept the default selections and click "Next" until its installed. An internet connection is required to complete the installation.

Once installation is done, create a first application and test the emulator. In this example, I have choosen min API 23. You will be prompted by MS Windows to allow OpenJDK through firewall. Allow and proceed. Have to consider, the emulator require 1.5Gb additional disk space.

Step 3. Start and create an emulator.

In Android Studio, click "Tools" ->"AVD Manager" ->"Create Virtual Device"

Select "Phone", "Pixel 2". Click "Clone Device". This allows us to keep the original configuration intact and change what ever is required. Click Next.

Click Download for "Nougat" with API 25. Read and Accept the agreements. Click "Next" then "Finish"

First of Laravel 6 step by step tutorial

Just completed the first part of the step by step tutorial.

Feel free to check it out at https://tboxmy.blogspot.com/p/laravel-6-crud-step-by-step.html

This will complement the Laravel notes found on this blog.

Remove User From MySQL

Basic database security involves having a password and user being assigned to  specific database, tables and their roles. This is also known as privileges in MySQL and Maria DB. In previous post, its shown how to list user permissions in MySQL database. But how do you remove that user?

The term MySQL in this post will refer to also MariaDb unless specifically mentioned otherwise.

The process to remove a user involves (1) to identify what privileges the user have, then (2)to revoke privileges and finally (3)remove the user from MySQL. In this tutorial, using the command line, a user with the name patrick is to be removed from the database quartz.

REVOKE statement

Firstly, here is the syntax;

REVOKE
    priv_type [(column_list)]
      [, priv_type [(column_list)]] ...
    ON [object_type] priv_level
    FROM user [, user] ...

REVOKE ALL PRIVILEGES, GRANT OPTION
    FROM user [, user] ...

REVOKE PROXY ON user
    FROM user [, user] ...

The REVOKE statement enables system administrators to revoke privileges
from MySQL accounts.

When the read_only system variable is enabled, REVOKE requires the
SUPER privilege in addition to any other required privileges described
in the following discussion.

Each account name uses the format described in
http://dev.mysql.com/doc/refman/5.7/en/account-names.html. For example:

REVOKE INSERT ON *.* FROM 'jeffrey'@'localhost';

If you specify only the user name part of the account name, a host name
part of '%' is used.

For details on the levels at which privileges exist, the permissible
priv_type, priv_level, and object_type values, and the syntax for
specifying users and passwords, see [HELP GRANT]

To use the first REVOKE syntax, you must have the GRANT OPTION
privilege, and you must have the privileges that you are revoking.

To revoke all privileges, use the second syntax, which drops all global, database, table, column, and routine privileges for the named
user or users:

REVOKE ALL PRIVILEGES, GRANT OPTION FROM user [, user] ...

To use this REVOKE syntax, you must have the global CREATE USER
privilege or the UPDATE privilege for the mysql database.

URL: http://dev.mysql.com/doc/refman/5.7/en/revoke.html

Step 1 : Identify existing user privileges

Before any revoke, it is a practise to verify the user privileges.

> SHOW GRANTS FOR patrick@'localhost';

If @'localhost' is not used, the default will be @'%' which will look like this;

> SHOW GRANTS FOR patrick@'localhost';
+----------------------------------------------------------------------------+
| Grants for patrick@localhost                                                            |
+----------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'patrick'@'localhost'
| GRANT ALL PRIVILEGES ON `quartz`.* TO 'patrick'@'localhost' WITH GRANT OPTION     |
+----------------------------------------------------------------------------+
4 rows in set (0.00 sec)

Step 2 : Revoke(remove) user privilege

I have not mentioned use of roles, but there are options to manage large number of users by roles. Could be another topic. The REVOKE only works at the next user login. Which means, a user that is currently connected online will still have the privilege unchanged.

To revoke privileges of user for a specific task like INSERT;

> REVOKE INSERT ON  'quartz'.'*' FROM patrick@'localhost';

If there are many users to be removed at the same time, just append with a comma separator.

> REVOKE INSERT ON  'quartz'.'*' FROM patrick@'localhost', username2, username3;

All privileges of a user is removed with the REVOKE ALL PRIVILEGES, GRANT OPTION.

If the user did not have any privileges to be revoked, and ERROR 1269 is returned.

> REVOKE ALL PRIVILEGES, GRANT OPTION FROM patrick;
> SHOW GRANTS FOR patrick@'localhost';
+----------------------------------------------------------------------------+
| Grants for patrick@localhost                                |
+----------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'patrick'@'localhost'
+----------------------------------------------------------------------------+
4 rows in set (0.00 sec)

Step 3. Remove(delete) user

The last step is to remove or delete the user.

> DROP USER patrick@'localhost';

The user is now safely removed from MySQL along with its privileges.

Practise

Here is an exercise which should reinforce the skills and knowledge from this tutorial. The user Patrick is assigned to retrieve data from a database named fruit_shop. Patrick will only be working on this database for 1 day and his user access is to be removed from the database once his task is done.

1. Login (as root) to MySQL and create a database named fruit_shop.
2. Create and apply a user named patrick who can only user the commands for SELECT and SHOW VIEW, with access from any host, using the password Banana1234 
3. Display list of user privileges on the system.
4. Remove all privileges for patrick.
5. Display list of user privileges on the system.
6. Remove user patrick.
7. Display list of user privileges on the system.

Display code in Blogger.com

This is how I display programme code and command line instructions on blogger.com.

<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; 
  font-family: "arial"; font-size: 12px;
  height: auto; line-height: 20px; overflow: auto; padding: 0px; 
  text-align: left; width: 99%;">
<code style="color: black; word-wrap: normal;">

Tutorial on File with Selinux Enforcing - part 2

Continuation from Part 1 post.

Apache HTTPD Not Allowed to Access Folder

It is possible to quickly implement security policy on a folder to allow application the required security. In this example, I am having the webserver Apache httpd to write log to a non standard folder /var/www/log. On a SELinux enabled Linux, permission is configured to allow read and write only for folders in /var/www/html.

On a Centos Linux with SELinux enabled, the HTPD server at start will show Failed to start The Apache HTTPD Server.

$ sudo systemctl stop httpd
$ sudo systemctl start httpd
$ sudo journalctl -xe

-- Unit httpd.service has begun starting up.
Jul 04 15:43:54 localdomain systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Jul 04 15:43:54 localdomain kill[76313]: kill: cannot find process ""
Jul 04 15:43:54 localdomain systemd[1]: httpd.service: control process exited, code=exited status=1
Jul 04 15:43:54 localdomain systemd[1]: Failed to start The Apache HTTP Server.
-- Subject: Unit httpd.service has failed

The error dd not specifically mention its of SELinux nature, but disabling SELinux, the error is gone. Following are the permissions of the authorised folder /var/log and our custom folder.

The security context type should be var_log_t to allow system logging.

The tool chcon can be used to copy security context from the original folder as follows;

$ sudo chcon --reference /var/log /var/www/log

Restart the HTTPD server without errors.

Quick Command Reference

To complete this tutorial here are a few common commands and brief notes from man pages;

1. audit2allow 
• generate SELinux policy allow/dontaudit rules from logs of denied operations
2. audit2why
•  translates SELinux audit messages into a description of why the access was denied (audit2allow -w)
3. ausearch
• ausearch  is  a  tool  that  can  query the audit daemon logs based for events based on different search criteria.  The  ausearch  utility  can also  take  input  from stdin as long as the input is the raw log data. Each commandline option given forms an "and"  statement.  For  example, searching  with  -m  and  -ui  means  return  events that have both the requested type and match the user id given.
4. getenforce
• reports  whether  SELinux is enforcing, permissive, or disabled.
5. getsebool
•  reports  where  a  particular SELinux boolean or all SELinux booleans are on or off.
6. journalctl
•  to query the contents of the systemd(1) journal as written by systemd-journald.service
7. ls -Z
•   Display  security context so it fits on most displays.  Displays only mode, user, group, security context and file name.
8. ps auxZ
•  report a snapshot of the current processes. Z displays security context.
9. restorecon
•  primarily used to set the security context (extended attributes) on one or more files.
•  It can also be run at any other time to correct inconsistent labels, to add  support  for newly-installed policy or, by using the -n option, to passively check whether the file contexts are all set as  specified  by the active policy (default behavior).
10. semanage
•  SELinux Policy Management tool
11. semodule
•  manage SELinux policy modules, including installing, upgrading, listing and removing modules.
12. sestatus
•  get the status of a system running SELinux. It displays data about whether SELinux is enabled or disabled, location of key  directories, and the loaded policy with its status
13. setenforce
• Use Enforcing or 1 to put SELinux in enforcing mode.
• Use Permissive or 0 to put SELinux in permissive mode.
14. setsebool 
• sets  the current state of a particular SELinux boolean or a list of booleans to a given value. The value may be 1 or true or on  to enable the boolean, or 0 or false or off to disable it.
• If the -P option is given, all pending values are written to the policy file on disk. So they will be persistent across reboots.

The default configuration file is found in;
/etc/selinux/config

Tutorial on File with Selinux Enforcing - part 1

While deploying an application to a production Centos Linux 7 server, there were some problems relating to not being able to read folder. Even when it was set to full read and write by everyone (or in Linux terms 777). I won't go into details of Selinux security as its available in modern Linux systems, but just matters related to allowing access of the restricted folder.

Note: The title looks grammatically wrong, but the term is "enforcing" is correctly used here.
These commands are here for reference to my guided tutorials.

Problem

In a basic Linux system, each file and folder (or directory) is allowed the access by the owner, group or others. And this access is further controlled into permissions of being able to read(r), write(w) and executable(x). I will refer to the term file to include folders unless mentioned otherwise. Users and programs can change owners and permission of files, if they have the permission to do so. The system typically divides the users into 2 groups;

1. root = Linux Administrator
2. user = Every other user who is not an administrator

If everyone and every program can change their own settings, eventually there will be a chance of creating security hole. Common approaches for security includes (1) restrict user to certain file permissions (use of groups are common) and (2)processes are further restricted in a "chroot" jail. Even with all this, the root user will have trouble controlling breaches due to user file permissions as these users still can change their file permission to allow everyone.

There are 2 approaches to change SELinux policies, here I will be looking at compiling the policy as a module.

SELinux basics

A set of file policies is introduced in Linux where every file is denied and a series of exceptions to access files is built is what makes SELinux work. All logs are by default to /var/log/audit/audit.log and where it may be in any one of the following modes;

1. Enforcing - policies are enforced
2. Permissive - policies are logged
3. Disabled 

View the SELinux status with commands such as;

• sestatus
• getenforce 

Try these Example:

Enable SELinux

 $ setenforce 1
 $ getenforce 
 Enforcing

Disable SELinux but still continue logging

 $ setenforce 0
 $ getenforce
 Permissive

These policies are defined for files and processes (yes, running programmes) and each are known as "security context". These can be of scontext (source context) or tcontext (target context). Here are ways you can display security context of;

1. Files. ls -Z
2. Processes. ps auxZ

Example.

A security context have the format

user : role : type : mls


Where
system_u:object_r:httpd_sys_content_t:s0

• user - run as user (e.g. system_u)
• role - run as this role (e.g. object_r)
• type - domain type to run (e.g. httpd_dyd_content_t)
• mls - multi-level security, its hidden if not in use. (Lowest security level with no compartments e.g. s0, s0.c0.c5 to show )

The tool semanage allows configuration of the security. Example, to display list of user for security context;

Security contexts as stored as modules in /etc/selinux/targeted/active/modules.

Try these examples;
Compare the different security context results for folder /bin, /home, /media, /etc, /usr, /var, /tmp and /sbin;

 $ ls -Z /
system_u:object_r:bin_t:s0       bin -> usr/bin
system_u:object_r:etc_t:s0       etc
system_u:object_r:home_root_t:s0 home
system_u:object_r:mnt_t:s0       media
system_u:object_r:bin_t:s0       sbin -> usr/sbin
system_u:object_r:tmp_t:s0       tmp
system_u:object_r:usr_t:s0       usr
system_u:object_r:var_t:s0       var

Pre-requisites

Install policy makefile, audit2allow and audit2why

$ sudo yum install policycoreutils-python
$ sudo yum install policycoreutils-devel

The audit2allow will retrive data from /var/log/audit/audit.log and list existing rules with

$ sudo audit2allow -a

Step 1. Investigation

In this example, the application nrpe is not running as it should on a SELinux enabled system. I could start the nrpe process but the output did not work as expected. The tool ausearch and journalctl  are used to list the messages during start of a process, where the command is systemctl start nrpe.

Its found that the folder could not be open for reading. Listing with the command ls -l showed that it has the correct permission but when ls -Z, then it is seen that there isn't enough access. The process nrpe is by the user nrpe, not nagios. The audit2why is then used to translate the problem into an easier to understand term.

$ sudo ausearch -m avc -c nrpe

$ sudo ausearch -m AVC,USER_AVC,SELINUX_ERR -ts today

$ sudo journalctl -xe

$ ls -l
$ ls -Z

$ ps aux | grep nrpe

$ sudo grep "nrpe" /var/log/audit/audit.log |audit2why

The file localhost.cfg cannot be open for reading

scontext=system_u:system_r:nrpe_t:s0 tcontext=unconfined_u:object_r:nagios_etc_t:s0

The folder nrpe.d cannot be accessed

scontext=system_u:system_r:nrpe_t:s0 tcontext=unconfined_u:object_r:nagios_etc_t:s0 

Based on what I have covered before, it can be seen that each have a source context and a target context with their security as user:role:type:mls defined.

Check what context does these nrep and nagios have

$ sudo semanage fcontext -l |grep nagios
/etc/nagios(/.*)?                                  all files          system_u:object_r:nagios_etc_t:s0
/etc/nagios/nrpe\.cfg                              regular file       system_u:object_r:nrpe_etc_t:s0

The problem is, nrpe is an application that listens to requires from a remote server to execute Nagios plugins. At NRPE start up, it needs to access customised configuration files located at /etc/nagios/nrpe.d folder.

NRPE, poor thing doesn't have access to sub-folders of /etc/nagios but nagios does.

Step 2: Action

Get the tool audit2allow to generate the required policy.

$ sudo audit2allow -i /var/log/audit/audit.log

#============= nrpe_t ==============
allow nrpe_t nagios_etc_t:dir read;
allow nrpe_t nagios_etc_t:file { getattr open read };


Looks like this will allow nrpe to access nagios_etc context type. Generate the required policy and view the policy in local.te

$ sudo audit2allow -i /var/log/audit/audit.log -M local
$ cat local.te

Install the generate local.pp as a module

$ sudo semodule -i local.pp

Check the security policy again and notice that nrpe_t now have access to same type as nagios_etc_t

Stop, and start NRPE process. Check with the log indicate that the error is gone.

$ systemctl stop nrpe
$ systemctl start nrpe
$ journalctl -xe



References:
https://wiki.centos.org/HowTos/SELinux#head-faa96b3fdd922004cdb988c1989e56191c257c01
https://opensource.com/article/18/7/sysadmin-guide-selinux

Enhance Apache and Lighttpd on Centos Linux: increasing file descriptors limit

Centos Linux servers typically have the maximum file descriptors at 1024. Other Linux distributions may have the same settings, but what is this file descriptor and how to increase the value? Please use linux man pages to get further details of the commands in this document. These are information for educational purposes.


Some web servers that have to work harder, may need to multitask more than others. This is when there will be warnings in /var/log/messages related to

can't have more connections than fds/2:  1024 1024

or Apache error logs showing

ulimit:error setting limit(Operation not permitted)


Another situation is when a software runs into an infite loop of process, such that is increase resource usage until the server comes to a complete stand still. Causing no other users to have access to the server. On a production server, this would be a disaster. What is really affected by these limits? Its the application performances such as database and web servers that are under heavy usage. Typical linux terminal users wont really need to concern with the limits.


View limits
Users can view file limits with the command;

$ ulimit 
$ ulimit -aH

Value of unlimited is common for development servers.

To view limit for current BASH open files of the user;

$ ulimit -n

To view max user processes

$ ulimit -Hn
$ ulimit -u

To view limits of any running process, identify the process ID ($PID) then run command;
$ grep 'open files' /proc/{$PID}/limits

Replace {$PID} with the PID to be checked.

Any user can reduce their file descriptor limit. This is useful to test problematic applications. Example to reduce to 1024.

$ ulimit -u 1024


Configuration file sysctl
This sysctl.conf and files in /etc/sysctl.d is an interface that allows you to make changes to a running Linux kernel. This file is extensively configured to harden a production Centos Linux server. Example, to reboot the linux after a kernel panic, the following values is used;

kernel.panic=10

With /etc/sysctl.conf you can configure various Linux networking and system settings. To view current values;

$ sudo sysctl -A
$ sudo sysctl fs.file-max
fs.file-max = 994255

Centos Linux

Check the current value of the file descriptor

$ ulimit -aHS
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 39253
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

To view system configured default limit on file descriptors.

$ cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     4096
root       soft    nproc     unlimited


Here, the plan is to increase limits to 65530.

Change the hard and soft limits in configuration file.

Edit /etc/security/limits.conf with these values. The * refers to any user.

* soft nproc 65535
* hard nproc 65535
* soft nofile 65535
* hard nofile 65535

In cases where the fs.file-max is below the 65535 value, then it can be set by editing /etc/sysctl.conf

fs.filemax = 65535

In both cases above, apply changes with the command;

$ sudo sysctl -p

Lighttpd

Edit /etc/lighttpd/lighttpd.conf

server.max-fds = 2048

Apache Httpd

The Centos, this document referring is with httpd version 2.4.6 (CentOS) and is using httpd compiled as preforked (non threaded).

Edit the file /etc/httpd/conf/httpd.conf

<IfModule prefork.c>
StartServers       4
MinSpareServers    3
MaxSpareServers   10
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  10000
</IfModule>

Refer details at Apache.org

Selinux

When selinux is in mode Enforcing, it needs to allow changes. Check the status with

$ sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          permissive
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31

$ getsebool httpd_setrlimit
httpd_setrlimit --> off

Temporary turn it on, then later view the status again;

$ setsebool httpd_setrlimit on

To make this change permanent;

$ sudo setsebool -P httpd_setrlimit on

MYSQL database

Edit /usr/lib/systemd/system/mysqld.service

LimitNOFILE=65535
LimitNPROC=65535

Edit /etc/my.cnf

table_open_cache=65535
open_files_limit=65535


Apply the changes;

$ sudo systemctl daemon-reload
$ sudo systemctl restart mysqld.service

Show user permission to MySQL and MariaDb databases

User permission levels in MySQL (and MariaDb) can be drilled down into many layers. From database permission, table permission, column permission and routine permission. There could be more that I don't know of. Its great to have greater control over who have access to the database;

How do I show user permission just at database level? First step is to know where such information is stored, followed by how to retrieve the information.

Access MySQL login as admin OR root.

mysql.user and mysql.db Tables

Have a look at the following tables, as they contain the relevant information

mysql> describe * from mysql.user;
mysql> describe * from mysql.db;

Listing users

Let's say, you want to see which user to host have access

mysql> select user, host from mysql.user;

+------------+-------------+
| user       | host        |
+------------+-------------+
| oc_admin   | %           |
| auditor    | 10.1.20.204 |
| auditor    | 10.1.20.244 |
| root       | 10.1.20.44  |
| tbox       | localhost   |
| mysql.sys  | localhost   |
| oc_admin   | localhost   |
| pentaho    | localhost   |
| root       | localhost   |
+------------+-------------+

List grants for users

mysql> show grants;

+---------------------------------------------------------------------+
| Grants for root@localhost                                           |
+---------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION |
| GRANT PROXY ON ''@'' TO 'root'@'localhost' WITH GRANT OPTION        |
+---------------------------------------------------------------------+

List User Access to Databases, Tables and Columns

mysql> SELECT Host, Db, User from mysql.db

There are separate tables for each database, table or column specific access. Go thru and see what you need in the following tables.

mysql> SELECT Host, Db, User from mysql.db
mysql> SELECT * FROM mysql.db;
mysql> SELECT * FROM mysql.tables_priv;
mysql> SELECT * FROM mysql.columns_priv;

Git and undo last commit

Updating a remote git repository would mean all other users having a same source code. Sometimes, a local commit may be done and its found that there are changes that should not be push to the remote. The solution, remove not needed files then move to the previous commit on the local machine.

The command is
$ git reset --hard HEAD~1

Unstage any other files with the command
$ git reset

Howto install mcrypt for PHP 7.2

PHP extensions have always utilised reuse of applications to provide extra functionality to developers. One such is the use of mycrypt. As of PHP 7.1, the mcrypt is deprecated and totally removed in PHP 7.2. Unfortunately for those with application that rely on mcrypt, this means the end of the road. In my case, development of a Laravel application had dependencies on mcrypt when doing an update with the command;

$ composer update

The mcrypt is a replacement of Unix/Linux crypt( ) that encrypt files. The PHP extension is php-mcrypt, and this is used to interface between PHP application and the mcrypt utility.

However, there is a solution that use mcrypt found in the PHP Extension Community Library (PECL). In this example, its tested on

CentOS 7.5 
Apache 2.4.6 (CentOS)
PHP 7.2.5 (sp.repo.webtatic.com)

Step 1. Install mcrypt utility.

$ pecl search mcrypt
WARNING: channel "pecl.php.net" has updated its protocols, use "pecl channel-update pecl.php.net" to update
Retrieving data...0%
.Matched packages, channel pecl.php.net:
=======================================
Package       Stable/(Latest)  Local
mcrypt        1.0.1 (snapshot)       Bindings for the libmcrypt library
mcrypt_filter 0.1.0 (beta)           Applies mcrypt symmetric encryption using stream filters


Install mcrypt and its development library then update the pecl.

$ sudo pecl channel-update pecl.php.net
$ sudo yum install libmcrypt
$ sudo yum install libmcrypt-devel

Step 2. Install php-devel (or php72w-devel if you use webtatic repo)

$ sudo yum install php72w-devel

Step 3. Install mcrypt extension

Either specify specifically a channel or use default
$ sudo /usr/bin/pecl install channel://pecl.php.net/mcrypt-1.0.1
or
$ sudo /usr/bin/pecl install mcrypt-1.0.1

Create the file /etc/php.d/mcrypt.ini with the contents
; Enable mcrypt extension module from pecl
extension=mcrypt.so

Restart Web server.

Note: If for some strange reason php still doesn't detect mcrypt extension, I would give a reboot.

Done.

Laravel, Swiftmailer and Uncaught Swift_TransportException

Writing PHP applications that deliver email notification helps to build a versatile system that uses the most basic communication media on the Internet. The Email.

Getting the Email out would require knowledge of the Email server in use to deliver 1 or thousands of email notifications. Common Email use SMTP, sendmail, postfix or a custom Transport implementation for corporate use. There are security and reliability concerns when passing to any Email server. Some of which include protection from email header injection attacks, MIME compliant, HTML/multipart emails, inline or embedded images.

Swift Mailer helps to handle most of these things. Why reinvent the wheel? In order to use it, the application should support PHP 7.0 or higher. Installation through Composer simplifies the whole process.

$ composer require "swiftmailer/swiftmailer:^6.0"

More details are found at Swift Mailer.

Purpose

Back to the original reason of this post. On a recent Laravel application, tested was carried out and checklist done on Centos Linux development server However when deployed to production Centos Linux server, email notification produced following error;

PHP Fatal error: Uncaught Swift_TransportException: Connection could not be established with host EMAILSERVER_NAME[Permission denied #13]

The full error is shown below.

[Thu Aug 30 14:35:54.225637 2018] [php7:error] [pid 18234] [client IPADDRESS:32852] PHP Fatal error: Uncaught Swift_TransportException: Connection could not be established with host EMAILSERVER_NAME[Permission denied #13] in /var/www/tboxmy/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:277\nStack trace:\n#0 /var/www/tboxmy/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(62): Swift_Transport_StreamBuffer->_establishSocketConnection()\n#1 /var/www/tboxmy/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(113): Swift_Transport_StreamBuffer->initialize(Array)\n#2 /var/www/tboxmy/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Mailer.php(79): Swift_Transport_AbstractSmtpTransport->start()\n#3 /var/www/tboxmy/public/info/testemail.php(21): Swift_Mailer->send(Object(Swift_Message))\n#4 {main}\n thrown in /var/www/tboxmy/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php on line 277 

 Solution

I am sure there are other possible ways to get this error, but in my case it was SELINUX. On the development selinux was on permissive and on production it was enforcing (quite strictly).

The next time this happens; here are the steps to take.

Determine selinux status and the booleans.

$ sestatus
. . .
$ getsebool httpd_can_sendmail
httpd_can_sendmail --> off
$ getsebool httpd_can_network_connect
httpd_can_network_connect --> off

Turn the permission to ON.

$ sudo setsebool -P httpd_can_sendmail 1
$ sudo setsebool -P httpd_can_network_connect 1

Restart the web server.
Done.