Improve the error log message when Nsjail fails to write to the `/sys/fs/cgroup/cgroup.subtree_control` file when it attempts to setup the cgroupv2 configuration.
The previous message looked like this:
```
[E][2023-05-28T21:52:56+0000][8807] writeBufToFile():105 Couldn't write '7' bytes to file '/sys/fs/cgroup/cgroup.subtree_control' (fd='4'): Device or resource busy
[E][2023-05-28T21:52:56+0000][8807] enableCgroupSubtree():95 Could not apply '+memory' to cgroup.subtree_control in '/sys/fs/cgroup'. If you are running in Docker, nsjail MUST be the root process to use cgroups.
[E][2023-05-28T21:52:56+0000][8807] main():354 Couldn't setup parent cgroup (cgroupv2)
```
It could have been confusing because the nsjail may have already been running as real root with full capabilities, e.g., when the user ran the container with the `--privileged --user 0:0` flags. In such a case, the issue is that Docker enters new pid, uts, network, ipc, mount and cgroup namespaces (but not user or time namespaces, fwiw) and I believe that if you do so after the cgroupv2 filesystem is mounted, the root of its filesystem hierarchy will start to render only a subtree, or, generally a limited view of the cgroup.
This can be seen below. On the host, we can see the cgroup sub-hierarchies and the `cgroup.subtree_control` shows us the controllers properly:
```
# ls /sys/fs/cgroup/
cgroup.controllers cgroup.threads dev-mqueue.mount memory.numa_stat system.slice
cgroup.max.depth cpu.pressure init.scope memory.pressure user.slice
cgroup.max.descendants cpuset.cpus.effective io.cost.model memory.stat
cgroup.procs cpuset.mems.effective io.cost.qos sys-fs-fuse-connections.mount
cgroup.stat cpu.stat io.pressure sys-kernel-config.mount
cgroup.subtree_control dev-hugepages.mount io.stat sys-kernel-debug.mount
# cat /sys/fs/cgroup/cgroup.subtree_control
cpuset cpu io memory hugetlb pids rdma
```
However, even in a privileged container, we can't see the same:
```
# sudo docker run --rm -it --privileged nsjail ls /sys/fs/cgroup
cgroup.controllers cpuset.cpus memory.events.local
cgroup.events cpuset.cpus.effective memory.high
cgroup.freeze cpuset.cpus.partition memory.low
cgroup.kill cpuset.mems memory.max
cgroup.max.depth cpuset.mems.effective memory.min
cgroup.max.descendants hugetlb.2MB.current memory.numa_stat
cgroup.procs hugetlb.2MB.events memory.oom.group
cgroup.stat hugetlb.2MB.events.local memory.pressure
cgroup.subtree_control hugetlb.2MB.max memory.stat
cgroup.threads hugetlb.2MB.rsvd.current memory.swap.current
cgroup.type hugetlb.2MB.rsvd.max memory.swap.events
cpu.idle io.latency memory.swap.high
cpu.max io.max memory.swap.max
cpu.max.burst io.pressure pids.current
cpu.pressure io.stat pids.events
cpu.stat io.weight pids.max
cpu.weight memory.current rdma.current
cpu.weight.nice memory.events rdma.max
# sudo docker run --rm -it --privileged nsjail cat /sys/fs/cgroup/cgroup.subtree_control
#
```
Of course, the namespaces itself can be seen by comparing them like this:
```
// HOST
# ls -la /proc/self/ns
total 0
dr-x--x--x 2 root root 0 May 28 22:17 .
dr-xr-xr-x 9 root root 0 May 28 22:17 ..
lrwxrwxrwx 1 root root 0 May 28 22:17 cgroup -> 'cgroup:[4026531835]'
lrwxrwxrwx 1 root root 0 May 28 22:17 ipc -> 'ipc:[4026531839]'
lrwxrwxrwx 1 root root 0 May 28 22:17 mnt -> 'mnt:[4026531841]'
lrwxrwxrwx 1 root root 0 May 28 22:17 net -> 'net:[4026531840]'
lrwxrwxrwx 1 root root 0 May 28 22:17 pid -> 'pid:[4026531836]'
lrwxrwxrwx 1 root root 0 May 28 22:17 pid_for_children -> 'pid:[4026531836]'
lrwxrwxrwx 1 root root 0 May 28 22:17 time -> 'time:[4026531834]'
lrwxrwxrwx 1 root root 0 May 28 22:17 time_for_children -> 'time:[4026531834]'
lrwxrwxrwx 1 root root 0 May 28 22:17 user -> 'user:[4026531837]'
lrwxrwxrwx 1 root root 0 May 28 22:17 uts -> 'uts:[4026531838]'
// CONTAINER
# sudo docker run --rm -it --privileged nsjail ls -la /proc/self/ns
total 0
dr-x--x--x 2 user user 0 May 28 22:17 .
dr-xr-xr-x 9 user user 0 May 28 22:17 ..
lrwxrwxrwx 1 user user 0 May 28 22:17 cgroup -> 'cgroup:[4026532381]'
lrwxrwxrwx 1 user user 0 May 28 22:17 ipc -> 'ipc:[4026532317]'
lrwxrwxrwx 1 user user 0 May 28 22:17 mnt -> 'mnt:[4026532315]'
lrwxrwxrwx 1 user user 0 May 28 22:17 net -> 'net:[4026532319]'
lrwxrwxrwx 1 user user 0 May 28 22:17 pid -> 'pid:[4026532318]'
lrwxrwxrwx 1 user user 0 May 28 22:17 pid_for_children -> 'pid:[4026532318]'
lrwxrwxrwx 1 user user 0 May 28 22:17 time -> 'time:[4026531834]'
lrwxrwxrwx 1 user user 0 May 28 22:17 time_for_children -> 'time:[4026531834]'
lrwxrwxrwx 1 user user 0 May 28 22:17 user -> 'user:[4026531837]'
lrwxrwxrwx 1 user user 0 May 28 22:17 uts -> 'uts:[4026532316]'
```
Anyway, passing `--cgroupns=host` solves this problem, which can be seen below:
```
# ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 root root 0 May 28 22:18 cgroup -> cgroup:[4026531835]
# sudo docker run --rm -it --cgroupns=host --privileged nsjail ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 user user 0 May 28 22:19 cgroup -> 'cgroup:[4026531835]'
# sudo docker run --rm -it --privileged nsjail ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 user user 0 May 28 22:19 cgroup -> 'cgroup:[4026532381]'
```
For new mounts if MNT_LOCK_READONLY is locked on the visible mnt
mount_too_revealing will fail and the whole mount will fail.
Those mounts need to be created with the readonly flag set.
This commit adds extra setup when cgroupsv2 is enabled. In particular,
we make sure that the root namespace has setup cgroup.subtree_control
with the controllers we need.
If the necessary controller are not listed, we have to move all
processes out of the root namespace before we can change this
(the 'no internal processes' rule:
https://unix.stackexchange.com/a/713343). Currently we only
handle the case where the nsjail process is the only process in
the cgroup. It seems like this would be relatively rare, but since
nsjail is frequently the root process in a Docker container (e.g.
for hosting CTF challenges), I think this case is common enough to
make it worth implementing.
This also adds `--detect_cgroupv2`, which will attempt to detect
whether `--cgroupv2_mount` is a valid cgroupv2 mount, and if so
it will set `use_cgroupv2`. This is useful in containerized
environments where you may not know the kernel version ahead of time.
References:
https://github.com/redpwn/jail/blob/master/internal/cgroup/cgroup2.go
Currently, we always kill children by sending them a SIGKILL signal in
case we've got a fatal signal. This is rather inflexible and forbids
some usecases where e.g. child process listen for specific signals to
shut down gracefully.
Add a new command configuration `--forward_signals` that allows the user
to opt-in to forwarding fatal signals to the child process.
`subproc::killAndReapAll()` is always killing the child process with the
SIGKILL signal. We're about to make this configurable though so that we
may optionally forward signals received by nsjail to the child process.
Add a new parameter to `killAndReapAll()` to prepare for this change.