Using Andrew Mallett’s Linux Host Security course on Pluralsight. These are the commands and observations from the Ubuntu lab, including a few settings changed just to see what they do.

procfs

Virtual file system that is mounted through to proc. Mostly read only, but some can be tuned, written to. We can inspect with: grep '^proc' /proc/self/mounts find -L /etc /proc -maxdepth 1 -samefile /proc/self/mounts

chad@ubuntu:~/$ find -L /etc /proc -maxdepth 1 -samefile /proc/self/mounts
/etc/mtab
/proc/mounts

More information: man 5 proc

Directory layout of the proc file system: sudo apt install tree tree -L 1 /proc/sys

chad@ubuntu:~/chadduffey.github.io$ tree -L 1 /proc/sys
/proc/sys
+-- abi
+-- debug
+-- dev
+-- fs
+-- kernel
+-- net
+-- user
\-- vm

8 directories, 0 files

sysctl command

sysctl reads and writes the kernel settings exposed under /proc/sys. It doesn’t manage all of procfs. We can read those settings as files too, but sysctl gives us a convenient interface.

The demonstration we worked through was with the NIS ‘domainname’: cat /proc/sys/kernel/domainname

We could write it with: echo "dropbearsec" | sudo tee /proc/sys/kernel/domainname but we should write it with: sudo sysctl -w kernel.domainname='dropbearsec'

sysctl -w only changes the running kernel. To keep a value after reboot, add it to the distribution’s sysctl configuration, usually a .conf file under /etc/sysctl.d/. Apply a particular file with sudo sysctl -p /etc/sysctl.d/example.conf, then read the value back. Don’t confuse changing it now with saving it for the next boot.

We can see the persistant, per file configurations:

chad@ubuntu:~/$ ls /etc/sysctl.*
/etc/sysctl.conf

/etc/sysctl.d:
10-console-messages.conf   10-magic-sysrq.conf       99-sysctl.conf
10-ipv6-privacy.conf       10-network-security.conf  protect-links.conf
10-kernel-hardening.conf   10-ptrace.conf            README.sysctl
10-link-restrictions.conf  10-zeropage.conf

To see all the keys and values: sysctl -a

To filter: sysctl -ar domainname

ASLR

PIE means Position Independent Executable. It lets the main executable be loaded at a randomized base; shared libraries can still be randomized when the executable itself isn’t PIE.

We can check the ASLR configuration with:

sysctl -ar randomize

We can see it in action with:

chad@ubuntu:~/chadduffey.github.io$ ldd /bin/bash
	linux-vdso.so.1 (0x00007ffdca12b000)
	libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f749ed59000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f749ed53000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f749eb62000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f749eec4000)
chad@ubuntu:~/chadduffey.github.io$ ldd /bin/bash
	linux-vdso.so.1 (0x00007ffdffdd2000)
	libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f22a667f000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f22a6679000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f22a6488000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f22a67ea000)

In the first example we can see: linux-vdso.so.1 (0x00007ffdca12b000) Second: linux-vdso.so.1 (0x00007ffdffdd2000)

For a lab where we want to compare with randomization disabled, save the current setting first:

saved_aslr=$(sysctl -n kernel.randomize_va_space)
sudo sysctl -w kernel.randomize_va_space=0

This affects new processes across the host, so keep it to the debugging VM. Put the original value back when finished:

sudo sysctl -w kernel.randomize_va_space="$saved_aslr"

The common values are 0 (off), 1 (randomize mappings, stack and VDSO, plus PIE code) and 2 (also randomize the heap). Kernel documentation.

Disabling ping as an example

sysctl -ar icmp

To set this:

chad@ubuntu:~/chadduffey.github.io$ sudo sysctl -w net.ipv4.icmp_echo_ignore_all=1
net.ipv4.icmp_echo_ignore_all = 1
chad@ubuntu:~/chadduffey.github.io$ ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
^C
--- 127.0.0.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3057ms

chad@ubuntu:~/chadduffey.github.io$ sudo sysctl -w net.ipv4.icmp_echo_ignore_all=0
net.ipv4.icmp_echo_ignore_all = 0
chad@ubuntu:~/chadduffey.github.io$ ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.095 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.047 ms
^C
--- 127.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1019ms
rtt min/avg/max/mdev = 0.047/0.071/0.095/0.024 ms

If we did want the permanent change:

Create a file: eg: sudo vim /etc/sysctl.d/60-icmp-block.conf Content: net.ipv4.icmp_echo_ignore_all=1 This will take effect on system boot, or we could read it in like we did in the previous example. Be careful with ICMP changes on production systems; blocking ping can make troubleshooting harder without meaningfully protecting the host by itself.

Monitoring Ports and Services

systemd can let us see running services.

systemctl list-units --type service --state running

To take it further and disable the service and also stop it now: sudo systemctl disable atd --now

We could check into removing the service by finding out which package it belongs to: dpkg -S $(which atd) sudo apt purge at is how we’d do it, but we might not want to. Let’s see what apt says: apt show ubuntu-server would show us that we wouldn’t want to do that. sudo systemctl mask atd would mask the service instead so that we couldn’t accidentally start something we don’t want, but can’t remove.

In terms of ports, netstat is mostly obsolete and we should move to ss.

ss -ntl is the netstat-ish view. ss -l '( sport = :ssh )' (spaces matter)

Maybe we want SSH listening on IPv4 only. AddressFamily inet in sshd_config makes that intention explicit. Commented examples aren’t active settings. Check the effective configuration with sudo sshd -T, validate changes with sudo sshd -t, then reload the distribution’s SSH service and inspect the listeners with ss.

That is only a good idea if the host and network design are intentionally IPv4-only. Accidentally ignoring IPv6 is a common way to create monitoring and firewall blind spots; deliberately disabling or deliberately supporting it are both better than forgetting it exists.

Chroot Jails

Changing the root directory seen by a process. This can restrict its view of files, but chroot alone isn’t a complete sandbox: it doesn’t isolate the network or remove privileges. The chroot manual explicitly calls out that limitation.

/usr/sbin/chroot

We might configure a service to run in a chroot jail. DNS for example often has all the required binaries and only those binaries in the jail. We can use ldd to investigate the files required for each application we want to make available.

After we configure a chroot directory with the required binaries we could restrict a SSH user called User1 like this:

sudo useradd -s /bin/bash user1
sudo passwd user1
sudo vim /etc/ssh/sshd_config

Add these lines to sshd_config (the service commands don’t go in the file):

Match User user1
    ChrootDirectory /var/chroot
    DisableForwarding yes

Every component of the chroot path must be owned by root and not writable by other users or groups. An interactive shell also needs its libraries and device files inside the jail. For a file-transfer-only account, ForceCommand internal-sftp avoids most of that setup. OpenSSH’s requirements.

Run sudo sshd -t before reloading SSH. Keep the existing admin session open while testing a second login; locking ourselves out doesn’t improve the lab much.

Limiting Access to Resources

We use a “pam” module - pam_limits.so

We can see this via:

grep -F pam_limits.so /etc/pam.d/*

shows:

chad@ubuntu:~/$ grep -F pam_limits.so /etc/pam.d/*
/etc/pam.d/cron:session    required   pam_limits.so
/etc/pam.d/gdm-autologin:session required        pam_limits.so
/etc/pam.d/gdm-fingerprint:session required        pam_limits.so
/etc/pam.d/gdm-launch-environment:session required        pam_limits.so
/etc/pam.d/gdm-password:session required        pam_limits.so
/etc/pam.d/login:session    required   pam_limits.so
/etc/pam.d/runuser:session		required	pam_limits.so
/etc/pam.d/su:session    required   pam_limits.so
/etc/pam.d/systemd-user:session  required pam_limits.so

ulimit -a ulimit -u

chad@ubuntu:~/$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 15419
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 15419
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

In the output we can see that it shows how to get more information on each item. -u for example, show the max user processes:

chad@ubuntu:~/$ ulimit -u
15419

We can edit this for everyone in the file: sudo vim /etc/security/limits.conf

These limits apply to sessions using pam_limits; they aren’t automatically a limit on every systemd service. For a service, look at its unit settings, such as TasksMax= and MemoryMax=, and check the running unit’s values. systemd resource controls.

Reset local password with Grub

At the GRUB menu hit e (to edit) That’ll show the boot parameters. Scroll down to the linux line. ctrl+e to get to the end of the line where we add init=/bin/bash, then boot to the bash root shell. This assumes we can edit the boot entry and access the root filesystem; it doesn’t bypass disk encryption. Then we mount the disk as RW: mount -o remount,rw / Then passwd user to change the password of the account we want. We should then move the disk back to read only to reduce the chance of corruption when we power it off: mount -o remount,ro /

GRUB can require authentication to edit boot entries. grub-mkpasswd-pbkdf2 generates a password hash, not an encrypted password. Configure it using the distribution’s supported GRUB custom configuration and regenerate the boot configuration as required. It won’t stop someone reading an unencrypted disk from another machine. GRUB authentication.

Auditing

tail /var/log/syslog tail /var/log/auth.log

We can see things, but it might not be the level of detail we need.

sudo apt install -y auditd audispd-plugins

Then we can search the audit records by event type:

ausearch -m ADD_USER --start recent

Installing auditd doesn’t record every command or file access automatically. The events available depend on the applications, kernel audit configuration and loaded rules. sudo auditctl -l shows the rules currently loaded; use a known action to check that the event you need is actually recorded. Auditctl reference.

References