Commit Graph

12 Commits

Author SHA1 Message Date
d7e29edd02 fix: force clear stream state introduced by rdbuf 2024-03-13 19:05:30 +08:00
d573c588b0 feat: add memory related stats 2024-03-13 17:58:33 +08:00
25ba8d7076 feat: add dump_stats support 2024-03-13 16:35:47 +08:00
Robert Swiecki
3d0bbf6927 cgroups2: make a function declaration-less by moving it earlier 2023-10-04 08:28:18 +02:00
Disconnect3d
f7265e0690
cgroup2.cc: improve note about using Docker
Improve the error log message when Nsjail fails to write to the `/sys/fs/cgroup/cgroup.subtree_control` file when it attempts to setup the cgroupv2 configuration.

The previous message looked like this:

```
[E][2023-05-28T21:52:56+0000][8807] writeBufToFile():105 Couldn't write '7' bytes to file '/sys/fs/cgroup/cgroup.subtree_control' (fd='4'): Device or resource busy
[E][2023-05-28T21:52:56+0000][8807] enableCgroupSubtree():95 Could not apply '+memory' to cgroup.subtree_control in '/sys/fs/cgroup'. If you are running in Docker, nsjail MUST be the root process to use cgroups.
[E][2023-05-28T21:52:56+0000][8807] main():354 Couldn't setup parent cgroup (cgroupv2)
```

It could have been confusing because the nsjail may have already been running as real root with full capabilities, e.g., when the user ran the container with the `--privileged --user 0:0` flags. In such a case, the issue is that Docker enters new pid, uts, network, ipc, mount and cgroup namespaces (but not user or time namespaces, fwiw) and I believe that if you do so after the cgroupv2 filesystem is mounted, the root of its filesystem hierarchy will start to render only a subtree, or, generally a limited view of the cgroup.

This can be seen below. On the host, we can see the cgroup sub-hierarchies and the `cgroup.subtree_control` shows us the controllers properly:

```
# ls /sys/fs/cgroup/
cgroup.controllers      cgroup.threads         dev-mqueue.mount  memory.numa_stat               system.slice
cgroup.max.depth        cpu.pressure           init.scope        memory.pressure                user.slice
cgroup.max.descendants  cpuset.cpus.effective  io.cost.model     memory.stat
cgroup.procs            cpuset.mems.effective  io.cost.qos       sys-fs-fuse-connections.mount
cgroup.stat             cpu.stat               io.pressure       sys-kernel-config.mount
cgroup.subtree_control  dev-hugepages.mount    io.stat           sys-kernel-debug.mount

# cat /sys/fs/cgroup/cgroup.subtree_control 
cpuset cpu io memory hugetlb pids rdma
```

However, even in a privileged container, we can't see the same:

```
# sudo docker run --rm -it --privileged nsjail ls /sys/fs/cgroup
cgroup.controllers	cpuset.cpus		  memory.events.local
cgroup.events		cpuset.cpus.effective	  memory.high
cgroup.freeze		cpuset.cpus.partition	  memory.low
cgroup.kill		cpuset.mems		  memory.max
cgroup.max.depth	cpuset.mems.effective	  memory.min
cgroup.max.descendants	hugetlb.2MB.current	  memory.numa_stat
cgroup.procs		hugetlb.2MB.events	  memory.oom.group
cgroup.stat		hugetlb.2MB.events.local  memory.pressure
cgroup.subtree_control	hugetlb.2MB.max		  memory.stat
cgroup.threads		hugetlb.2MB.rsvd.current  memory.swap.current
cgroup.type		hugetlb.2MB.rsvd.max	  memory.swap.events
cpu.idle		io.latency		  memory.swap.high
cpu.max			io.max			  memory.swap.max
cpu.max.burst		io.pressure		  pids.current
cpu.pressure		io.stat			  pids.events
cpu.stat		io.weight		  pids.max
cpu.weight		memory.current		  rdma.current
cpu.weight.nice		memory.events		  rdma.max

# sudo docker run --rm -it --privileged nsjail cat /sys/fs/cgroup/cgroup.subtree_control
 
# 
```

Of course, the namespaces itself can be seen by comparing them like this:

```
// HOST
# ls -la /proc/self/ns
total 0
dr-x--x--x 2 root root 0 May 28 22:17 .
dr-xr-xr-x 9 root root 0 May 28 22:17 ..
lrwxrwxrwx 1 root root 0 May 28 22:17 cgroup -> 'cgroup:[4026531835]'
lrwxrwxrwx 1 root root 0 May 28 22:17 ipc -> 'ipc:[4026531839]'
lrwxrwxrwx 1 root root 0 May 28 22:17 mnt -> 'mnt:[4026531841]'
lrwxrwxrwx 1 root root 0 May 28 22:17 net -> 'net:[4026531840]'
lrwxrwxrwx 1 root root 0 May 28 22:17 pid -> 'pid:[4026531836]'
lrwxrwxrwx 1 root root 0 May 28 22:17 pid_for_children -> 'pid:[4026531836]'
lrwxrwxrwx 1 root root 0 May 28 22:17 time -> 'time:[4026531834]'
lrwxrwxrwx 1 root root 0 May 28 22:17 time_for_children -> 'time:[4026531834]'
lrwxrwxrwx 1 root root 0 May 28 22:17 user -> 'user:[4026531837]'
lrwxrwxrwx 1 root root 0 May 28 22:17 uts -> 'uts:[4026531838]'

// CONTAINER
# sudo docker run --rm -it --privileged nsjail ls -la /proc/self/ns
total 0
dr-x--x--x 2 user user 0 May 28 22:17 .
dr-xr-xr-x 9 user user 0 May 28 22:17 ..
lrwxrwxrwx 1 user user 0 May 28 22:17 cgroup -> 'cgroup:[4026532381]'
lrwxrwxrwx 1 user user 0 May 28 22:17 ipc -> 'ipc:[4026532317]'
lrwxrwxrwx 1 user user 0 May 28 22:17 mnt -> 'mnt:[4026532315]'
lrwxrwxrwx 1 user user 0 May 28 22:17 net -> 'net:[4026532319]'
lrwxrwxrwx 1 user user 0 May 28 22:17 pid -> 'pid:[4026532318]'
lrwxrwxrwx 1 user user 0 May 28 22:17 pid_for_children -> 'pid:[4026532318]'
lrwxrwxrwx 1 user user 0 May 28 22:17 time -> 'time:[4026531834]'
lrwxrwxrwx 1 user user 0 May 28 22:17 time_for_children -> 'time:[4026531834]'
lrwxrwxrwx 1 user user 0 May 28 22:17 user -> 'user:[4026531837]'
lrwxrwxrwx 1 user user 0 May 28 22:17 uts -> 'uts:[4026532316]'
```

Anyway, passing `--cgroupns=host` solves this problem, which can be seen below:

```
# ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 root root 0 May 28 22:18 cgroup -> cgroup:[4026531835]

# sudo docker run --rm -it --cgroupns=host --privileged nsjail ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 user user 0 May 28 22:19 cgroup -> 'cgroup:[4026531835]'

# sudo docker run --rm -it --privileged nsjail ls -la /proc/self/ns | grep cgroup
lrwxrwxrwx 1 user user 0 May 28 22:19 cgroup -> 'cgroup:[4026532381]'
```
2023-05-29 00:19:31 +02:00
Robert Swiecki
f2dc5966e3 all: unify comments on /**/ 2022-11-22 22:19:05 +01:00
Robert Swiecki
cc4245d23a make indent depend + style of comments 2022-11-22 22:15:01 +01:00
Andrew Haberlandt
12df56b9f1 Setup cgroup.subtree_control controllers when necessary in cgroupsv2
This commit adds extra setup when cgroupsv2 is enabled. In particular,
we make sure that the root namespace has setup cgroup.subtree_control
with the controllers we need.

If the necessary controller are not listed, we have to move all
processes out of the root namespace before we can change this
(the 'no internal processes' rule:
https://unix.stackexchange.com/a/713343). Currently we only
handle the case where the nsjail process is the only process in
the cgroup. It seems like this would be relatively rare, but since
nsjail is frequently the root process in a Docker container (e.g.
for hosting CTF challenges), I think this case is common enough to
make it worth implementing.

This also adds `--detect_cgroupv2`, which will attempt to detect
whether `--cgroupv2_mount` is a valid cgroupv2 mount, and if so
it will set `use_cgroupv2`. This is useful in containerized
environments where you may not know the kernel version ahead of time.

References:
https://github.com/redpwn/jail/blob/master/internal/cgroup/cgroup2.go
2022-11-17 17:09:40 -05:00
Philip
bf93e8a25d cgroup2: use cgroup_mem_swap_max and cgroup_mem_memsw_max 2021-11-01 10:28:41 +01:00
Philip
5a8ad82311 cgroup2: support cgroup_mem_memsw_max 2021-11-01 10:28:41 +01:00
Robert Swiecki
0b1d5ac039 cgroup-code: remove some spaces to make code more consistent 2019-08-04 09:54:38 +02:00
Jay Lees
08f62b6f76 [cgroup-v2] support cgroup v2 for mem, cpu and pids 2019-07-26 07:02:17 -07:00